A multi-modal method is developed to continuously detect aviation safety risks in real-time across air traffic flows and pilot-controller communication messages. The method combines three models: (1) Deep Speech Pattern (DSP) analysis, (2) Speech-to-Text (STT) analysis, and (3) Intent Inference Analysis (IIA). DSP focuses on the audio aspects of the pilot-controller conversation; STT focuses on the words spoken in the conversation; and IIA focuses on inferring the pilot’s and controller’s intent. The three models independently cross-check system safety from different perspectives, identifying when the system is in a nominal, off-nominal, or anomalous condition. This multi-modal method is demonstrated through ATM applications in the New York Center (ZNY) within the U.S. National Airspace System (NAS).
Jasenka Rakas, Ph.D., is a faculty member in the Civil and Environmental Engineering Department and in the Mechanical Engineering Department’s Aerospace Engineering program, and is affiliated faculty with the Jacobs Institute for Design Innovation at the University of California (UC) Berkeley. She is Deputy Director of the UC Berkeley National Center of Excellence for Aviation Operations Research (NEXTOR III), founder of the UC Berkeley Airport Design Studio and Aviation Futures Research Lab, and Co-chair of the Sustainable Aviation Symposium. Her research interests include aviation systems safety, reliability, and resilience; aviation infrastructure performance; and their interactions with the environment. She has received numerous awards, including seven national competition awards from the FAA/ACRP.
Attendees are invited to join the session in person at NASA Ames Research Center, Building N210, Room 115 (a NASA badge is required) or online:
Microsoft Teams Need help?
Join the meeting now
Meeting ID: 219 053 318 025 74
Passcode: sM6Qo6to
About Air Time by NAMS-2
Air Time is a series of seminars on advanced aviation hosted by Crown Innovations, Inc., in collaboration with the University of California’s CITRIS and the Banatao Institute. The seminars feature leading experts on cutting-edge research who share interesting ideas on pertinent topics and innovative methodologies. Air Time speakers include subject matter experts from UC Berkeley, Merced, Davis, and Santa Cruz. The seminars take place weekly.
Crown Innovations, Inc. is the prime contractor for the NASA Academic Mission Services 2 (NAMS-2) contract. Contact the program management at nams2pmo@crownci.com for more information or to arrange a collaboration in your field.