Generative AI for autonomous driving: DXC-Luxoft’s new tool uses AI in automotive testing

When was the last time you had a near miss in your car? Can you describe what the situation on the roads was like at the time? Sure you can. But can you create a machine-readable description of the same traffic situation in an automotive-grade format? That’s a whole different ball game.

Creating such a description requires deep domain-specific knowledge, but this description is exactly what’s needed for virtual validation of an autonomous driving system (ADS). Luckily, generative AI — with the right software application — can now be used to develop simulation-executable scenarios from a natural language description.

In this story, we will express our findings and highlight some challenges we encountered during the assessment, as well as how we overcame them.  

To achieve homologation for an ADS, car manufacturers must ensure that their systems function effectively across diverse road environments — including specific pre-defined conditions deduced from the operational design domain (ODD) specification of the ADS. This necessitates significant efforts in testing, validation and verification — often referred to as the “billion-mile challenge” in autonomous driving (AD).

Not all of those test miles can be driven in the real world: Effort, cost, time and environmental reasons all set a natural limit. So, a scalable virtual validation approach is needed to achieve  trustworthy testing levels, and this virtual validation approach requires detailed descriptions of the diverse road environments.

The devil lies in the details when it comes to creating a consistent and machine-readable description of a traffic scenario. Even a simple traffic situation depends on so many individual granularities and parameters that creating a concise description is very complicated — and creating a description for a complex traffic scenario is even more so.

Consider a conversation with your best friend explaining a dangerous situation you faced on the freeway earlier that day. You would probably tell them where, when and how it happened — in general. Plus, you’d include your version of what the traffic was like and how the other driver(s) behaved. This conversation would likely imply things that were not specifically mentioned, about traffic lanes, vehicle dynamics and the like. 

Yet, even if you were able to explicitly account for these conditions, the description would still be a far cry from a simulation executable example. And the actual creation of such scenario descriptions is tedious and repetitive. Indeed, to create autonomous vehicle solutions, carmakers need to test against hundreds of thousands of scenarios (specified in textual form) and identify crucial combinations or gaps to ensure a sophisticated enough scenario coverage of the ODD¹.

On top of that, interdisciplinary teams have to work together for scenario specification — testing and debugging to avoid ambiguities. These activities are time consuming, will likely cause delays in product development and require deep domain knowledge. All this at a time when well-qualified experts are difficult to find.

To address the process of scenario specification and enable alignment across teams and organizations, the industry has accepted standards all developed by ASAM e.V.’s cross-industry initiative, such as ASAM OpenDRIVE®, ASAM OpenSCENARIO® DSL and ASAM OpenSCENARIO® XML.

And to manage the lack of qualified experts, large language models (LLMs) and generative AI are now mature enough to be used as a universal, scalable extension of labor forces.

In automotive, a generative AI tool can be used to speed up scenario-based testing: Instead of teaching deep domain knowledge to hundreds of people and training them to use safety and simulation tools, just develop a generative AI automotive software application that creates executable scenarios from general descriptions. With that, even non-experts can transform their abstract scenarios (specified in natural language) into executable scenarios for simulation (written in machine-readable form).

A generative scenario AI application has to be extensible, multilingual and flexible to use. Then it can help in every step of the scenario creation, from starting with a single AD function up to a full AD system test complete with verification and digital homologation.

Such an AI does not exist as an out-of-the-box solution. Developing and training such an AI still requires some effort combined with the necessary domain knowledge. The good news is, it’s a one-time effort: Once the system is set up and trained, it can provide you with a constant flow of scenario descriptions — while the system still continues to learn.

DXC-Luxoft has created scenᴀʀɪ.Lux, an easy-to-use generative AI-powered chatbot application based on state-of-the-art (LLMs. scenARI.Lux harnesses Luxoft’s deep expertise in the fields of generative AI, driving-scenario description technologies and standards. ASAM OpenDRIVE® and ASAM OpenSCENARIO® were chosen for the target file formats, because DXC Luxoft — an active member of and contributor to ASAM — supports and advocates the use of standards and open source as a way to foster industry-wide collaboration without additional license costs.

The chatbot application has been designed and built in a way that it can support various LLMs. This is because different foundation models require different steps for implementing a fully-fledged solution. The whole application consists of different bots; and a dedicated agent application orchestrates the bots to get from a human-readable textual description to an OpenDRIVE or OpenSCENARIO XML file.

The LLMs as foundation models (FMs) already handle a vast amount of data, tuning billions of parameters. With additional knowledge about output formats incorporated in the FM, the data pool gets even bigger. This would usually lead to latency and slow system behavior.

For improving answer quality, scenᴀʀɪ.Lux uses Retrieval-Augmented Generation (RAG) and generation skills to optimize the bot’s response time and quality. RAG feeds in the OpenX standards’ definitions as an additional authoritative knowledge base in the process. With this, scenᴀʀɪ.Lux is able to gain accurate responses within short response times and in a cost-effective manner.

With a publicly available foundation model trained to understand human language and equipped with XML support, we use manually created scenarios (and optionally clients' scenarios) for fine-tuning the foundation model of scenᴀʀɪ.Lux. The result is an automotive generative AI that is not just another tool, but your knowledgeable assistant. 

Such an autonomous driving AI toolchain can generate endless AD-relevant scenario combinations while lowering costs and reducing the need for domain expertise. These scenarios are the starting point for virtual validation of AD algorithms. In addition, DXC-Luxoft’s end-to-end validation toolchain — based on our collaboration with MicroVision — offers real-word scenario abstraction. 

Combine the two sources, and you will never run out of scenarios again. 

_____________________________________

¹ Parts of this area are also examined in the publicly funded research project just better DATA, where DXC-Luxoft is contributing as a consortium member.