Senior Data Engineer (Autonomous Vehicles Data)

Senior Data Engineer (Autonomous Vehicles Data) At RSB Automotive Consulting, we work with engineers and technology teams developing advanced mobility solutions. We are currently looking for a Senior Data Engineer to join a long-term project focused on data processing and analytics for Autonomous Vehicle (AV) development. In this role, you will work with large-scale sensor datasets collected from test vehicles and support teams building simulation environments and machine learning pipelines used to validate safety-critical AV systems. Project context: data pipelines and analytics supporting Autonomous Vehicle (AV) development Tech stack: Cloud, Python, SQL, Spark / PySpark, Databricks Data scale: up to ~1 TB of sensor data per hour (cameras, LiDAR, radar) Focus: time-series data, advanced analytics, simulation-ready datasets, and ML data preparation Work mode: remote (any location), with daily overlap with the US team Start: ASAP Your responsibilities Analyze large volumes of sensor and time-series data from autonomous vehicle test fleets Develop advanced SQL, Python, and PySpark queries to filter, transform, validate, and aggregate datasets Design and maintain ETL and data processing pipelines handling large-scale structured and semi-structured data Support and troubleshoot distributed data workflows and pipeline operations Monitor and optimize orchestration pipelines (e.g. Airflow, Argo Workflows, or similar technologies) Identify and extract data suitable for AV simulation scenarios and ML training pipelines Support the discovery of rare or complex driving situations (e.g. unusual traffic scenarios, hard braking events, edge cases) Investigate data inconsistencies, pipeline failures, and performance bottlenecks Develop scripts and internal tools supporting data mining, debugging, and operational automation Collaborate with engineers working across backend, infrastructure, and autonomous driving technology teams What we’re looking for Strong software engineering background Advanced SQL skills with experience writing complex queries Advanced Python programming Understanding of distributed data processing and large-scale data workflows Experience working with cloud-based data platforms Experience with workflow orchestration tools such as Airflow, Argo Workflows, or similar Understanding of infrastructure concepts including storage systems, microservices, and pipeline architecture Experience working with notebooks and analytical workflows Familiarity with troubleshooting and operational support of production data pipelines Understanding of data preparation for machine learning workflows Nice to have Hands-on experience with Spark / PySpark Experience working with Databricks Experience in advanced data analytics and time-series analysis Experience supporting analytics, simulation, or ML-related data pipelines Understanding of Autonomous Vehicle development context and real-world edge cases Degree in Computer Science or related field Project context You will work with sensor data generated by autonomous vehicle test fleets equipped with multiple cameras, LiDAR, and radar systems. These platforms generate extremely large datasets used to build and validate simulation environments supporting autonomous driving algorithms. The role focuses on transforming raw sensor data into structured, simulation-ready datasets used by engineering and research teams working on safety-critical AV features, including obstacle detection, path planning, and complex traffic scenarios. If you are interested in working with large-scale data systems and real-world autonomous driving datasets, we would be happy to connect. Apply To This Job