As the aviation industry trends toward more autonomous flight technologies with the help of artificial intelligence (AI), one tech start-up is looking into a different use for AI in aircraft: to monitor pilots using facial sensing software.

Blueskeye AI, a U.K.-based software development company that specializes in facial analysis using AI, recently received a £20,000 ($24,600) award from the Aerospace Unlocking Potential (UP) program, a joint effort between the University of Nottingham and the Midlands Aerospace Alliance, to investigate how facial sensing technology could glean information about human behavior in aircraft cockpits.

The company uses facial recognition and voice analysis software “to look at medically and biophysically relevant behavior, so we can use it to help assess, diagnose, monitor, and treat medical conditions that actually change your expressive behavior,” Blueskeye AI founder and CEO Michel Valstar told FutureFlight.

Valstar explained that the AI software can be used to identify ailments such as fatigue, pain, depression, and anxiety, for example, by analyzing how facial muscles move and contract. “In aerospace we're looking primarily at the moment at the fatigue elements, but we have interest in some of the other medically relevant behaviors as well,” he said. While the company is currently focusing its efforts on identifying fatigue and other signs of psychological distress, the technology could someday be used to detect early signs of major health events like heart failure or cardiac arrest.

“The key thing is that we actually measure mental states, so we're inferring from your sequence of actions and your behavior over time that the mind is becoming fatigued way before you start actually showing it by nodding off and closing your eyes, at which point it can be too late,” Valstar explained. “The point is to provide information back to the pilot, that they are getting fatigued and might need to have a break or change with someone else.”

Blueskeye AI has been developing its face and voice analysis technology for 18 years, and the company is exploring a variety of applications for the software. For example, the company has developed an app for pregnant women that it claims can detect early signs of depression, and that app is currently undergoing clinical trials.

To implement this technology into an aircraft cockpit, the only equipment that needs to be installed is a small near-infrared camera, with a built-in microcomputer, pointed at the pilot or pilots. “From the view of their face, we can then detect the facial muscle actions in particular and where they're looking, and with the microphone we can try and pick up the voice if it's possible over the [sound of the] engine,” Valstar said.

Using its award from the Aerospace UP program, Blueskeye AI has installed a prototype in the cockpit of a small, two-seat airplane and used it to collect data on both the pilot and the passenger or co-pilot. “We’re basically collecting pilot data on pilots,” Valstar said. “We’re using that to see how it will operate in there and collecting data of the people in the plane to then run algorithms on and come up with the functional requirements for the next phase of this work.”

Valstar hopes that Blueskeye AI’s software will not only help make flying safer by identifying when pilots might be unfit to fly, but that it might also help the aviation industry as a whole come up with better policies for managing pilot fatigue. “There are already lots of rules in place from the industry, but the problem is that those rules are very fixed based on what is expected to be normal,” he said. Those rules, he added, are based on “a fixed number of hours that you can operate [an aircraft] before you need to switch, and they don’t actually take into account the actual state of the pilot.”

Rather than follow a “one-size-fits-all” approach to setting pilots’ schedules, airlines could use Blueskeye AI’s data analysis to determine when a pilot is too fatigued to keep working. On the flipside, the technology could also allow pilots to operate an aircraft for longer periods of time than what is allowed under existing rules, provided they are alert and well enough to keep working. “By introducing this measurement, you actually increase safety as well as flexibility,” Valstar said

AIN Products

SUBSCRIPTIONS

RESOURCES

ABOUT AIN