Data management in the development of autonomous driving functions

A distinction must be made between the collection of data during the operation of the vehicle, i.e. regularly purchased and registered vehicles, and development vehicles.

“Standard data”
In general, the type and amount of data collected varies from OEMOEM – Original Equipment Manufacturer. More to OEMOEM – Original Equipment Manufacturer. More. In recent years, most OEMs have started to collect basic values such as location, odometer, speed, RPM, diagnostic trouble codes (DTCs), and so on. This currently collects around 50 megabytes of data per month per vehicle from “standard data”.

Development data
One of the biggest challenges in the development of autonomous vehicles is data collection and data management. Today’s test prototypes typically house between four and six cameras and one to five LiDAR sensors, which continuously produce complex data sets. An average Level 3 test vehicle produces about 150 TB of data per day (the equivalent of 38,400 two-hour HD movies or 30 million songs). A fleet of 10 to 20 research vehicles collects 1.5 Peta Byte of data per day.
It would take more than 150 days to transfer it via a 5G5G is short for “fifth generation”: 5G is set to be as much as 100 times faster than 4G. 5G networks will have greater bandwidth, giving faster download speeds. 5G is the fifth generation technology standard for cellular networks. More network to the research labs. A real hurdle for the development of production-ready technologies for autonomous driving. Typical validation programmes for AV neural networks require a minimum of 20 petabytes of training data and about 70% of development time costs managing, labelling and processing that data, and transforming it into something to use just to train the neural networks.
The solution: physical data transfer for fast availability
In concrete terms, this means that all camera and sensor data are stored on a drive in the boot of the research vehicle. After the test drive, the portable drive is moved to where the data is needed next for further analysis: for example, on the company’s own private cloud server or at a storage-as-a-service cloud provider.

In autonomous driving, it is above all the software that plays a decisive role. The artificial intelligence (AI) of an autonomous vehicle must be able to recognise situations in the shortest possible time and make the right decision. In order to “train” the AIAI – artificial intelligence More in the best possible way for different traffic situations, large amounts of data are necessary, as well as good data management.

Examples, how to handle the BIg Data:

Data-driven development — BMW collects millions and millions of anonymised driving data from test vehicles in its test fleet for this purpose, as well as information from thousands of production vehicles since December 2019 – of course with the consent of the customers.

Reprocessing: The recorded journeys from reality are stored once and can then be virtually run through again and again with new software versions. By the middle of 2021, more than 500 million test kilometres will have been driven.

Mobileye: The ADASADAS – Advanced Driver Assistance System – supports the driver with the handling of the automotive. They are often used to increase the safety, efficiency or the comfort of the ride. The systems use sensors to record the driving situation, process the collected information with the aid of computer technology and provide the driver with visual, acoustic or haptic feedback. In some cases, they intervene autonomously or semi-autonomously in the control and operation of the vehicle, for example by operating the accelerator, brake, signaling or steering. This can extend all the way to fully autonom… More specialist Mobileye now has over one billion test kilometres for high-precision map material on its account, with eight million more being added every day. By 2024, this could be one billion kilometres within 24 hours.

With “KIsSME“, the Karlsruhe Institute of Technology (KIT) has launched a research project with the objective of reducing the mountains of data that accumulate during testing in order to save storage space, electricity and evaluation effort.This is how Big Data becomes Smart Data: AIAI – artificial intelligence More can already be used during data acquisition to identify certain interesting driving scenarios in data. In so-called smart layer architectures, i.e. data management structures through which information is sent, intelligent algorithms decide whether data is allowed through or filtered out.

Tesla launches supercomputer: Project Dojo.
Dojo is an AIAI – artificial intelligence More supercomputer that has a total performance of about 1.1 exaflops. In addtion: Tesla has announced that from now on it will dispense with radar sensors in its vehicles and enable autonomous driving based solely on camera data.

Cariad relies on machine learning and edge computing. Data Driven Development is already the focus of many development activities. In future, autonomous Volkswagen models will also calculate directly in the vehicle via Edge Computing. (Edge Computing, in contrast to Cloud Computing, refers to decentralised data processing).

Virtual test kilometres have long been the standard for testing autonomous vehicles in order to reduce costs and effort to a minimum.