Back to all stories

Testing Spark applications to improve quality

  • Adevinta Life
  • Tech

Developers at Adevinta Spain wanted to run unit, integration and end-to-end tests on Spark applications. Learn how they implemented this.

A useful read for big data developers, get into the detail of how the team at Adevinta Spain implements unit, integration and end-to-end tests on Spark applications as they build a Data Platform. The full article reveals a typical project layout, how to use Spark sessions, what the various test functions enable and how this improves the quality of the code and the CI pipelines.

Highlights include:

  • Layout of a Spark project
  • How to isolate different Spark tests
  • Details of the test implementation code
  • The idea behind SharedSparkSessionHelper
  • Links to real-life Adevinta Spain examples

Related stories

Discover all stories

International Day of Persons with Disabilities – Initiatives across Adevinta

Read more about International Day of Persons with Disabilities – Initiatives across Adevinta

How we celebrated Black History Month in Adevinta

Read more about How we celebrated Black History Month in Adevinta

Adevinta for Everyone: who are we?

Read more about Adevinta for Everyone: who are we?