Location selector
www.adevinta.com
Location selector Location selector

Taming data workflows with a move to Athena

  • twitter
  • linkedin
  • facebook
  • email

Find out how leboncoin shifted from Apache Kafka to Athena to better leverage high volumes of data.

After many years of using Apache Kafka, the tech teams at leboncoin needed to rethink workflows in order to be able to analyse, transform and consolidate events in “offline” data storage. Learn how they went from a tedious, manual process to a fully automated workflow, using AWS S3/Athena/Spark, minimising the amount of human input required.

Highlights include:

  • A deep dive into our team’s journey from the Kafka “hot” to S3/Athena “cold” storage workflow
  • How they programmed a solution to fully automate an error-prone archiving process
  • The technical and organisational challenges of ensuring consistency
  • Examples of connector configurations, base configuration and error handling
  • How monitoring and alerting keeps our datalake “fresh” for our data scientists, analysts and data services

Read the full article from leboncoin tech blog

More resources

Discover our media resources, brand assets, guidelines, photos and more