Efficient VoIP Communications through LLM-based Real-Time Speech Reconstruction and Call Prioritization for Emergency Services

Danush Venkateshperumal, Rahman Abdul Rafi, Shakil Ahmed, Ashfaq Khokhar, Corresponding Author: Shakil Ahmed (email: [email protected]) · 2024

Read Efficient VoIP Communications through LLM-based Real-Time Speech Reconstruction and Call Prioritization for Emergency Services by Danush Venkateshperumal, Rahman Abdul Rafi, Shakil Ahmed, Ashfaq Khokhar, Corresponding Author: Shakil Ahmed (email: [email protected]) (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-16176.txt.

Introduction

Emergency calls often break up just when clear speech matters most. Danush Venkateshperumal, Rahman Abdul Rafi, Shakil Ahmed, and Ashfaq Khokhar wrote a 2024 paper on fixing weak internet voice calls used by emergency services. Packet loss, low bandwidth, poor signal, delay, and jitter all hurt call quality. Panic, speech problems, and loud noise make callers harder to understand, and short staffing adds more delay.

The authors propose large language models to rebuild broken speech and fill gaps in what was said. Their plan pairs live transcription with search-based generation and uses Twilio and AssemblyAI tools. Tests gave high precision and good text match scores. The work points to faster sorting of the most urgent calls first.

Catalog record

CREATORDanush Venkateshperumal, Rahman Abdul Rafi, Shakil Ahmed, Ashfaq Khokhar, Corresponding Author: Shakil Ahmed (email: [email protected])
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-16176

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Efficient VoIP Communications through LLM-based Real-Time Speech Reconstruction and Call Prioritization for Emergency Services Danush Venkateshperumal, Rahman Abdul Rafi, Shakil Ahmed, Ashfaq Khokhar, Corresponding Author: Shakil Ahmed (email: [email protected]) Abstract Emergency communications systems often face significant disruptions between victims and dispatchers due to packet loss, bandwidth constraints, poor signal quality, delay, and jitter in VoIP systems, leading to degraded quality of service in real-time and eventually making it challenging for dispatchers to interpret the urgency and details of a situation accurately. Additionally, victims in distress may struggle to convey critical information due to speech disorders, panic, and background noise, further exacerbating these challenges. This results in incomplete or ambiguous communication, hindering dispatchers’ ability to assess situations accurately and, when combined with staffing shortages in many emergency response centers, causing delays in coordination and the timely delivery of emergency assistance. This paper proposes solving the abovementioned issues by leveraging Large Language Models (LLMs) to improve emergency response efficiency. This is done by reconstructing incomplete speech, filling contextual gaps to provide meaningful outputs, and prioritizing the emergency calls accordingly through a multi-step process involving real-time data processing and advanced severity, rule-based and keyword classification techniques.. The system integrates real-time audio transcription and uses a Retrieval-Augmented Generation (RAG) model to generate relevant responses. This output is further used to classify the severity of the emergency to ensure that critical cases are prioritized using Twilio and AssmbleAI Application Programming Interfaces (APIs). Evaluation of the proposed model demonstrates high Conceptual Precision (100%) and favorable Bilingual Evaluation Understudy (BLEU) and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) scores across diverse emergency scenarios, indicating robust alignment with real-world needs 1 1 1 The code of paper is available at [ 1 ] . These results highlight the system’s potential to improve emergency communication efficiency, optimize response workflows, and prioritize critical cases effectively. I Introduction More than 240 million emergency calls are placed in the U.S. annually, and over half of the emergency centers within the U.S. face a staffing crisis, particularly during peak times within major cities [ 2 ] . The problem becomes severe in poor networking conditions, where connectivity issues compound the risk of miscommunication [ 3 ] . Such conditions degrade audio quality, resulting in fragmented and unintelligible communications that hinder dispatchers’ ability to assess and respond to emergencies accurately. Consequently, connectivity and staffing issues can lead to significant delays in emergency response, resource deployment, and the ability to address each call with the same level of urgency, potentially causing life-threatening situations [ 4 , 5 , 6 ] . Studies show the critical impact of response times on emergency mortality rates. For instance, each minute of delay in response time increases the mortality rate by 1% in critical trauma cases [ 7 ] . Additionally, delays of five minutes or more in notifying an ambulance can significantly increase fatality rates, up to 30% [ 8 ] . These statistics underscore the importance of efficient and timely communication in emergency services. Emergency communication systems rely heavily on voice communication via Internet protocols (VoIP). When victims dial the emergency number, dispatchers at a local emergency center manually assess the emergency details to respond. Nonetheless, these wireless calls may route to incorrect dispatch centers, leading to the caller’s location or number being lost, which can result in dangerous delays, especially if communication fails or the call drops [ 9 ] . VoIP 911 calls may not connect to the 911 call center serving the caller’s current location. In some cases, they may be routed to the administrative line of a 911 call center, which may not be staffed after hours or handled by trained 911 operators. Federal Communications Commission (FCC) [ 10 ] warns that even when VoIP 911 calls connect correctly, they may fail to automatically transmit the caller’s phone number and location information, making it difficult for dispatchers to locate the victim. Moreover, VoIP users are often required to manually provide their location information to their VoIP service providers and update it whenever they change locations to ensure proper functionality of the 911 service. VoIP services may become inoperable during power outages or internet failures, including when the network is congested, or bandwidth is constrained, further exacerbating delays in emergency response. Packet loss occurs when data packets fail to reach their destination, significantly impacting VoIP call quality. While small amounts of packet loss might not noticeably affect communication, higher rates can severely degrade audio quality. Maintaining packet loss below 1% is crucial to achieve voice quality comparable to traditional Public Switched Telephone Network (PSTN) services [ 11 ] . Loss rates between 5% and 10% can significantly affect call quality [ 12 ] . Moreover, burst packet losses where several consecutive packets are lost, can lead to noticeable gaps in audio, exacerbating the the degradation of call quality [ 13 , 14 ] . Bandwidth limitations, especially during periods of high network usage, can restrict the data capacity available for real-time audio transmission, causing delays or dropped audio packets. These constraints directly impact voice quality and can result in degraded call clarity due to reduced packet flow [ 15 , 11 ] . Unreliable or weak network connections contribute to degraded audio quality, leading to interruptions and call dropouts [ 14 ] . Environmental noise, such as background sounds or callers speaking softly to avoid detection, poses challenges to communication clarity. Background noise significantly affects voice quality, especially when combined with packet losses [ 15 ] . Additionally, using silence suppression in noisy conditions can cause “noise-pumping” effects, further degrading voice quality [ 11 ] . Delay refers to the time it takes for packets to reach their destination, while jitter is the variation in packet arrival times. Without effective buffering, these issues can lead to choppy or out-of-order audio, severely disrupting the flow of conversation [ 13 ] . Network failures can increase delay and jitter, negatively impacting call quality [ 14 ] . When responding to multiple emergency calls simultaneously, dispatchers often struggle to prioritize one case over another with appropriate urgency. This challenge is exacerbated by bandwidth constraints, VoIP limitations, and random and bursty packet loss, which degrade audio quality and make it difficult to assess each situation’s severity accurately. Additionally, poor signal quality and lack of communications interoperability, particularly involving critical communications equipment, can lead to severe miscommunications or delays. Dispatchers frequently transfer calls between various departments multiple times to ensure calls reach the appropriate emergency responders. This repeated transferring further increases the risk of miscommunications and delays, potentially causing life-threatening consequences [ 16 ] . The mortality rates in medical emergencies rise sharply when response times exceed five minutes [ 17 ] . Hence, it is essential to prevent scenarios involving degraded communication quality and ensure that cases are quickly and accurately classified based on urgency, all while providing clear context without causing any additional delay. Many local call centers operate independently, affecting not only call centers but also emergency medical services (EMS), particularly in ambulance allocation to emergency departments (EDs) [ 18 ] . This makes it hard to aggregate data and coordinate responses to the region’s victims [ 6 ] . There are also significant challenges in ensuring

Read the complete text