Assessing Driver Engagement in Assisted Driving

Applus+ IDIADA has been conducting research on driver engagement and mental workload in Assisted Driving systems through a study called ENGAGE with the primary aim to establish a robust methodology for evaluating driver engagement, in accordance with future regulations, using measurable metrics and a replicable process.  More broadly, the study seeks to help design Assisted Driving systems that align with human tendencies and enhance driver focus relative to the level of assistance provided.

The methodology for this study involves a three-stage iterative process:

1. Proving Ground Testing (2022 – 2023)

Participants drove on a test track in a continuous scenario, combining subjective metrics like mental workload and trust with objective measures such as Time to Collision (TTC). Results showed differences in driver engagement between medium and advanced L2 systems, with higher trust and lower mental workload in advanced L2 systems.

2. Focus Group (November-December 2023)

Held at the Polytechnic University of Catalonia (UPC), in Barcelona, these sessions highlighted generational differences in ADAS perceptions, with younger participants showing higher trust and acceptance. Situational awareness was identified as a crucial factor for proper engagement.

3. Dynamic Driving Simulator Tests (Planned for late 2024)

Using the DiM250 VI Grade dynamic simulator, this future phase aims to validate previous findings in a controlled environment, incorporating physiological measures and eye-tracking.

Proving Ground Testing: Combining Subjective and Objective Metrics

In this first phase, data collection focused on assessing driver engagement with two different L2 vehicles using a combination of subjective and objective metrics. The aim was to evaluate naïve driver performance and responsiveness to emergency scenarios.

The sample consisted of two groups: one of 20 participants driving a Golf 8 L2 medium vehicle, and another group of 19 participants driving a Tesla L2 Advanced vehicle, based on their Euro NCAP Assisted Driving grading. Participants were instructed to maintain a steady, low speed and follow a lead vehicle on a simple, monotonous track for 40 minutes, accompanied by a professional safety co-driver. During the penultimate lap, a dummy obstacle was unexpectedly placed in the middle of the lane to simulate a real-world emergency situation. The lead vehicle executed a cut-out manoeuvre 15 meters before the obstacle, leaving the participant to react based on their level of attention and engagement with the system (figure 1)

To comprehensively assess the driver's engagement, both subjective and objective data were collected. Subjective variables included mental workload and trust levels, while the objective metric analyzed was the time to collision.

Figure 1. Cut-out manoeuvre

Focus Group: Assessing ADAS Systems’ Perception

Focus groups were organized to review the findings from the first phase, shape the upcoming third phase, and gather qualitative insights on ADAS and driver engagement. Three sessions were conducted with 8 participants each, ensuring a balanced gender distribution and an age range from 20 to 60 years. Half of the participants had prior knowledge of ADAS systems, and the other half had been involved in the first proving ground phase. Each session began with ice-breaking activities and the division of participants into “novices” and “experts”.

Following a training session to establish a common knowledge base and compare the different perspectives, group discussions covered topics such as ADAS system perceptions, usage frequency, trust levels, factors influencing trust, mobility patterns, lifestyle impacts, safety risks, progress towards autonomous driving, ethical considerations, and the advantages and disadvantages of ADAS. Additionally, the sessions provided an opportunity to gather feedback on the first phase and identify areas for improving the methodology in the upcoming phases.

Driving Simulator Testing: Putting Past Research to the Test in Lab Conditions

This third phase will take place at IDIADA China Driving Simulator towards the end of the year and is currently in the preparation stage. The methodology will be based on the outcomes of the first phase and feedback gathered from the focus group sessions. It will consist of a simulated replication of the activities conducted in the initial phase, enhancing the complexity and richness of data collection due to the flexibility offered by the simulated environment, with the implementation of more driving and non-driving tasks.

The test procedure will remain consistent: participants will be instructed to follow another vehicle, initially without and then with ADAS systems engaged, involving a final obstacle scenario. This phase will provide a more comprehensive data collection process using driver's electroencephalographic (EEG) signals, electrocardiogram (ECG) signals, and eye-tracking systems.

Results and Discussion: Working Towards the Development of a Global Assessment Methodology

This study aims to develop a methodology to evaluate the level of engagement of the driver in Assisted Driving in different contexts of real and simulated environment.

The initial phase testing was carried out successfully. The collected data was compared between the Golf 8 and the Tesla Model 3, to highlight any variations in the level of engagement with L2 medium and advanced systems.

The analysis of subjective and objective metrics related to the mental workload perceived by drivers and the level of confidence revealed differences between the two types of L2 systems. With regard to subjective data, the perceived mental workload is low and the level of confidence is high in both systems. However, the vehicle equipped with a medium L2 system has a higher mental workload value than the vehicle equipped with an advanced L2 system As for the objective data, the results suggest that the type of L2 system used (advanced or medium) may influence a driver's reaction time and ability to take control of the vehicle to redirect the manoeuvre.

According to the results, participants perceived a higher level of confidence in the vehicle equipped with the advanced L2 system.

Regarding the results of the second phase, the overall perceived trust in the three groups suggests a significant generational gap. Younger individuals show high in trust levels in ADAS systems and in vehicles popularly considered more innovative from a naïve user perspective (e.g., Tesla), and agree on improved comfort and safety, assistance to less skilled drivers or drivers with specific physical limitations, perceived reduction of human error in driving, perceived comfort and reduced workload. Older participants show significant reluctance towards Assisted Driving systems, considering them distracting, excessively annoying and dangerous. All participants have in common, according to their own experience, the perceived risk of losing concentration due to overconfidence

An adaptation for driving simulator testing is being prepared for implementation in China in 2024, with the aim of assessing driver engagement in a different context, including potential cultural differences in driving habits and reactions to critical events. This phase will contribute to the enhancement of the study conducted thus far, providing valuable new insights and findings.

Conclusions & Assessments

The methodology here described has proven to be effective for evaluating driver engagement and gathering insights directly from individual points of view. Future steps include replicating the test track environment on proving grounds with different vehicle types, expanding the study and methodology to other cultures, and transferring the project to the United States for a comparative analysis across Europe, China, and the United States. This will enhance our understanding of driver behaviour in an increasingly automated vehicle landscape and inform safety measures for the application and implementation of autonomous vehicles.