Throughout human civilization, people have collaborated with machines to accomplish remarkable feats, from utilizing basic tools for construction to deploying sophisticated spacecraft for exploration. Today's artificial intelligence revolution is transforming this relationship, enabling unprecedented collaboration between humans and machines to tackle complex challenges through true human-machine partnerships.
While most research in human-machine teaming has concentrated on advancing the technological capabilities of AI systems, researchers at MIT Lincoln Laboratory identified a critical gap: insufficient attention to the human component of these teams. What happens when the AI functions flawlessly, but the human operator experiences cognitive fatigue or overwhelming stress?
"The field of human-machine teaming typically focuses on technology—how to monitor it, understand its functionality, and ensure optimal performance. However, effective teamwork requires mutual consideration," explains Michael Pietrucha, a tactical systems specialist at the laboratory. "Our research explores the reciprocal relationship where AI monitors and enhances human performance, creating a truly symbiotic partnership."
Pietrucha is part of a pioneering research team developing AI systems capable of detecting when cognitive fatigue impairs human performance. These intelligent systems can recommend appropriate interventions or, in critical situations, take autonomous action to help individuals recover or prevent potentially harmful outcomes.
Historical evidence demonstrates how human error has led to missed opportunities, accidents, and sometimes catastrophic consequences," notes Megan Blackwell, former deputy lead of internally funded biological science and technology research at the laboratory. "With neuromonitoring technologies becoming increasingly precise and portable, we can now envision systems that detect fatigue or cognitive overload in real-time. By monitoring these states, we can intervene before performance deteriorates or incidents occur."
This innovative approach builds upon decades of research at the laboratory focused on using technology to interpret human cognitive and emotional states. By collecting biometric data—including vocal patterns, facial expressions, and other physiological indicators—and processing these inputs with advanced AI algorithms, researchers have identified biomarkers for various psychological and neurobehavioral conditions. These biomarkers have enabled the creation of models that can accurately assess conditions such as depression with remarkable precision.
The research team is applying this biomarker expertise to develop AI systems that analyze an individual's cognitive state, evaluating levels of fatigue, stress, and cognitive load. By examining physiological data including voice patterns, facial expressions, heart rate variability, brain activity measurements, and eye movements, these systems gain comprehensive insights into human cognitive functioning.
The initial phase involves constructing a personalized cognitive model for each individual. "This cognitive model integrates various physiological inputs and monitors how these patterns change during fatiguing tasks," explains Thomas Quatieri, who leads several neurobehavioral biomarker research initiatives at the laboratory. "Through this process, the system establishes activity patterns and learns an individual's baseline cognitive state, including essential functions like auditory and visual attention and response times that are critical for maintaining safety and preventing undesirable outcomes."
Once this personalized baseline is established, the system can recognize deviations from normal patterns and predict whether these changes might lead to errors or diminished performance.
"Developing an accurate model presents significant challenges. Success is measured by the model's ability to predict performance accurately," states William Streilein, principal staff in the Lincoln Lab's Homeland Protection and Air Traffic Control Division. "Our goal is to create a system that not only identifies deviations but also predicts when these changes will interfere with task performance. Humans naturally compensate for stress or fatigue, so the critical capability is predicting when compensation will fail, necessitating intervention."
The range of potential interventions spans from simple adjustments to autonomous actions. Minor interventions might include recommendations for caffeine intake, lighting adjustments, or taking breaks for fresh air. More significant interventions could involve suggesting shift changes, transferring tasks to machines or other team members, or employing advanced techniques like transcranial direct current stimulation—a performance-restoring method that uses electrodes to stimulate specific brain regions and has demonstrated superior effectiveness to caffeine in combating fatigue with fewer side effects.
In extreme scenarios, the AI system might take life-preserving actions when humans are incapacitated. For instance, an AI teammate could execute an "ejection decision" for a fighter pilot who loses consciousness or the physical ability to eject themselves. Pietrucha, a retired U.S. Air Force colonel with extensive experience as a fighter/attack aviator, recognizes the value of such systems that "extend beyond analyzing flight parameters to include cognitive state assessment of aircrew, intervening only when necessary."
Determining optimal interventions requires consideration of numerous factors, including task requirements, intervention intensity, and user demographics. "Significant research remains to understand intervention effects and validate their safety," Streilein acknowledges. "Our ultimate objective is to implement personalized cognitive interventions and evaluate their impact on mission performance."
Beyond combat aviation, this technology promises benefits for other demanding or hazardous professions, including air traffic control, combat operations, disaster response, and emergency medicine. "Combat medics often work in overwhelming conditions with limited resources, experiencing exhaustion comparable to their colleagues," Blackwell observes. "Having an intelligent system to monitor their mental status and fatigue levels could prevent medical errors and alert others when intervention becomes necessary."
Currently, the research team is seeking sponsorship to advance their technology development. The upcoming year will focus on data collection to train their algorithms, beginning with intelligence analysts equipped with sensors while engaging in simulation games that replicate their job demands. "Intelligence analysts frequently face information overload and would benefit significantly from such systems," Streilein notes. "Their typical work environment—seated at computers in standard settings—facilitates easy instrumentation for physiological data collection and algorithm training."
"We're developing foundational capabilities in the near term," Quatieri explains, "but our long-term vision involves creating a more turnkey solution that maintains individualization while enabling widespread deployment, similar to how Siri functions universally but adapts quickly to individual users." The team ultimately envisions a universal background model capable of representing anyone while adapting to specific applications.
Such capabilities may prove essential for advancing future human-machine teams. As AI evolves to demonstrate increasingly human-like capabilities while remaining immune to mental stress, humans may emerge as the primary vulnerability in mission success. An AI teammate might possess precisely the right capabilities to support and enhance human performance when it matters most.
