AI-based code generation tools: Are they ready for automotive software development?

There has been a lot of noise in the last couple of years about Generative AI, especially regarding how it can benefit the work of millions of software developers. To understand the real benefits and help organizations make the right choice between the many products appearing on the market, we’ve carried out an assessment on a variety of code generation tools. We conducted automotive-specific coding challenges and found a significant acceleration of development tasks. We also discovered the strengths of individual code generation tools. 

In this story, we will express our findings and highlight some challenges we encountered during the assessment, as well as how we overcame them.  

In the rapidly evolving landscape of automotive, the significance of software development has surpassed its traditional role to become the cornerstone of innovation for software-defined vehicles. With the rise of this paradigm, the automotive industry sits at a pivotal stage where lines of code define our cars’ features, performance and capability to self-repair and update. This paradigm shift not only underlines the relevance of software in enhancing vehicle performance, vehicle safety and user experience, but also emphasizes the need to optimize the entire software development lifecycle (SDLC) in an industry known for its cost sensitivity. 

Within the SDLC, a significant — if not major — amount of time and resources is invested in the implementation phase. The generative AI tools for software development like GitHub Copilot, Amazon CodeWhisperer, Starcoder and Tabnine hold the promise of revolutionizing the automotive landscape by streamlining workflows, enhancing productivity and ultimately reducing costs. 

However, the adoption of such tools introduces a unique and new set of challenges: Understanding which tool delivers the best performance in terms of coding acceleration (in a complex and very domain-specific development environment such as automotive) while ensuring a smooth integration and superior compatibility with development paradigms, standards, processes, methods and tools.  Paramount challenges are addressing privacy, functional safety, intellectual property protection and cybersecurity concerns, and maintaining regulatory compliance. 

In this article, we investigate the quickly evolving landscape of generative AI tools for software development, in relation to their application in the automotive industry. We present a comprehensive, rigorous, efficient, methodology that allows automakers to compare and benchmark generative AI tools for software development in operational, automotive-specific conditions, and to make informed decisions when defining the trajectory of their generative AI transformation journey.  

As the automotive industry embraces GenAI tools for software development, there is a fascinating battle unfolding between open source software and tech giants. Open-source AI projects have gained popularity for their collaborative nature and accessibility — these projects allow developers to harness the power of AI without being tied to a specific vendor or technology stack. In the automotive industry, open-source AI initiatives have contributed to advancements in autonomous driving, computer vision and natural language processing.  

On the other hand, tech giants like Google, Microsoft and Tesla are developing proprietary GenAI tools tailored to various industry applications, including automotive. These tools often come with the backing of vast resources and expertise, promising cutting-edge solutions. However, they also raise concerns about vendor lock-in, cost and data privacy.  

To make an informed decision, automakers should carefully weigh the benefits of open-source solutions which offer flexibility and community-driven innovation, against the potential advantages of tech giants' offerings that may come with enhanced integration capabilities and support. 

The advent of GenAI is ushering in a new era in software development within the automotive industry. By harnessing the power of GenAI, software engineers can now automate and expedite a wide array of tasks, from code generation and debugging to documentation writing. This accelerates the development process and enables developers to focus on more complex, creative aspects of their work. However, the integration of GenAI necessitates a reevaluation of quality control and ethical considerations, as the code and content generated by AI must meet stringent industry standards and adhere to privacy and security regulations. The role of software developers is shifting toward a more supervisory and creative one, overseeing AI-generated outputs and ensuring they align with the automotive sector's rigorous safety and performance requirements. 

For OEMs and Tier 1s, in the automotive industry, AI represents a transformative shift in how they interact with technology and services. This technology offers the potential for highly customized solutions, from AI-driven vehicle fleet management to tailored manufacturing processes. Automakers can benefit from AI's ability to analyze vast amounts of data, providing valuable insights for optimization and cost savings. Additionally, Gen AI can enhance communication and support, automating responses to common queries, and offering more efficient, personalized customer experiences. However, automakers must also consider the implications of data privacy and security when integrating GenAI into their operations, ensuring compliance with regulations and ethical practices. 

New models for AI-enabled organizations

As the automotive industry grapples with the profound implications of GenAI, forward-thinking companies are reimagining their organizational structures to remain agile and responsive to these technological shifts. In 2022, DXC Luxoft started a transformation process to develop a new organizational model called “Guilds and Chapters.” Guilds and Chapters are communities of professionals linked by deep experience and a genuine interest in specific technological topics; groups of individuals with shared skills or expertise who foster knowledge sharing and enhance the quality and consistency of decision-making and technological evolution within the organization. 

These groups are communities that transcend traditional team boundaries, encouraging cross-pollination of ideas and skills. This collaborative approach spurs innovation and ensures that the organization can adapt swiftly to technological advancements — such as GenAI. By fostering these collaborative networks of expertise, the various teams are key players in the decision-making process and allow us to continue to position ourselves as a leader of choice whenever a new technology wave comes along. In this article you can read more about the benefits that an agile-based company structure can bring to companies. 

Methodology

GenAI has revolutionized the landscape of technology, but while the potential for progress is immense, the challenges are equally formidable. To address the uncertainties and hesitations surrounding AI-based tools, our Vehicle Data Insights and Artificial Intelligence Chapter has spearheaded the development of a robust benchmarking framework, tailored to the needs of the automotive industry. This benchmarking framework quickly became a solid tool in the decision-making process for GenAI and software development. This benchmarking system is designed to rigorously evaluate and assess the efficacy and flexibility of AI-powered solutions. Whether you're a startup looking to make an informed choice or a tech giant seeking optimization of your SDLC for the automotive industry, our process provides invaluable insights tailored to the implementation use cases and technologies that compose your development ecosystem. It's not just a one-size-fits-all solution; it's a flexible, adaptable framework that can be tailored to companies’ specific needs.  

To ensure high quality standards for this process, we tested it internally, aiming to identify the most suitable AI-based code generator for our needs — i.e., to cover the main topics that car makers and automotive suppliers have to deal with in their product development. First, we conducted a deep market analysis to identify several off-the-shelf candidates to benchmark. In the meantime, thanks to the extensive know-how of our tech leads, we identified a wide range of KPIs and use cases to measure the performance of these products. An indicative selection of KPIs and use cases follows: 

Two independent groups of DXC Luxoft engineers created a secure, unbiased, automotive-specific benchmarking environment with an exhaustive set of code challenges and used four AI-based code generators to solve them. The tools used for the assessment were Amazon CodeWhisperer, GitHub Copilot, StarCoder and Tabnine. In parallel, our legal department delved into terms and conditions and defined the policies for using these tools in order to answer questions related to compliance, privacy, security and IP protection. 

The work we performed to design and engineer the assets described above allowed us to develop the capabilities needed for successfully accompanying clients and partners in defining and implementing a sustainable GenAI transition roadmap — one which considers the specific requirements and constraints of the automotive sector. 

These are just some of the features of our benchmarking framework. It has been meticulously devised to rigorously appraise the efficiency and versatility of AI-driven solutions for software development. Within the automotive sector, our framework provides key indicators which align with specific use cases of the development landscape and support the decision-making process at strategic and tactical levels. 

GenAI presents multifaceted challenges for the automotive industry. First, the adoption of GenAI in automotive software development raises concerns about intellectual property, as AI-generated content may include code or other assets that constitute intellectual property of third parties. A second point is guaranteeing compliance with ASPICE standards, which define best practices for software development in the automotive sector: It becomes crucial to ensure the reliability, safety and quality of AI-driven systems. In addition, establishing accountability and standards for AI-generated content is paramount, as companies must ensure that AI aligns with copyright laws and ethical guidelines. 

Lastly, the integration of GenAI into the processes related to autonomous vehicles poses challenges related to liability, particularly in the event of accidents. As AI takes on a more prominent role in decision-making, the legal responsibility for AI's actions and decisions becomes a complex matter, raising questions about liability and culpability in case of accidents or errors in autonomous vehicles. Regulators and legal experts are working hard to adapt legal frameworks to address these novel challenges and ensure that the integration of GenAI in the automotive sector occurs within a comprehensive and well-defined legal framework. 

Any AI deployment must follow ethical considerations to at least be compliant with laws such as the EU AI act and at best be inspiring to users. At DXC Luxoft, we follow our own internal guidelines for Responsible AI. The first two items which we line out in our Responsible AI guidelines are that “all AI use must be lawful” and “AI must respect data privacy”. As we enter the era of Generative AI, we must apply these guidelines to new technology. 

Let’s look at the two most contentious discussed legal issues: IP infringement and data leakage. The issue of IP infringement is hotly debated (a major court case which remains unresolved). The potential infringement arises if a large language model was trained on proprietary code and as code companion emits that proprietary code to the developer. A major cost risk emerges as unlawful integration of proprietary code into a vehicle might result in the need to stop and recall sold vehicles, rework the code base, and validate and deploy the changes. Data leakage through Generative AI tools has been widely reported, such as the high-profile case of Samsung employees leaking internal source code and meeting notes. 

Aware of the above challenges and aiming at navigating and anticipating them, DXC Luxoft Automotive created a GenAI Automotive Task Force that looks at this matter from a holistic perspective, combining the know-how brought by AI engineers with the complementary perspective of business leaders and a dedicated legal team. This allowed us to create — among the set of assets that will be revealed in the next story in this series — the Generative AI Benchmarking framework introduced in this article. As described, this framework is tailored to the needs of the automotive industry and has quickly become a solid partner in decision-making regarding Generative AI and software development. This framework is being adopted by our clients and partners that want to define and implement a sustainable Generative AI transition roadmap, which considers the specific requirements and constraints of the automotive sector. C

Looking forward

Identifying the right AI tool for accelerating software development in the automotive industry is a highly individualized choice for different actors in the value chain. Various factors influence this decision, such as budget constraints and automotive-specific technical requirements (Functional Safety, ASPICE, Software Quality, to mention some), or — coming to AI-specific aspects — the dynamic, exponentially changing and volatile technology landscape. As the paradigm of software re-usability and hardware abstraction grows within the automotive field, and with it, the lifetime of a portion of code, it is paramount to design, operate and continuously update an AI-based software development toolchain that delivers performance, robustness and scalability, and ensures compliance in terms of IP protection. Ultimately, the choice of a code generator is a strategic decision that should align with an organization's unique priorities, ensuring that it meets their current needs and positions them for future growth and innovation. To find out which code generator would benefit you most, or to have an open discussion on Generative AI for the automotive industry, contact one of our experts.