False alarms thwart remote patient monitoring early warnings
According to a recent clinical trial, wearable remote patient monitoring devices can provide early warnings of the deterioration of patients but also cause a high rate of false alarms.
The trial, details of which were published in JMIR Formative Research Journal, enrolled 410 patients at high risk of deterioration at Sheba Medical Center in Israel. Participants wore Biobeat Technologies’ wireless, noninvasive reflective photoplethysmography-based sensor, allowing the investigators to record variables such as heart rate and blood oxygen saturation every five minutes.
Using the data, the researchers calculated early warning scores every five minutes using three methods: the National Early Warning Score (NEWS), the Airway, Breathing, Circulation, Neurology, and Other (ABCNO) score, and deterioration criteria defined by the clinical team as a “wish list” score.
Among the 217 patients included in the final analysis, there were 24 cases of clinical deterioration. NEWS indicated a high alert for 16 of the patients. On average, the system provided the alert 29 hours before the clinical deterioration of the patient.
The ABCNO score indicated high alert in 18 of the patients, on average 38 hours before deterioration. The earliest, most comprehensive detection was achieved by the wish list scoring criteria, which detected all 24 cases of clinical deterioration 40 hours before medical staff, on average.
However, false positives were common. Across the analysis, 193 of the participants did not suffer clinical deterioration. NEWS provided early warning alerts for 150 of the patients. ABCNO delivered alerts for 162 of the patients. The wish list criteria delivered early warning alerts for all 193 of the patients.
The early warning systems, therefore, achieved high sensitivity but low specificity. In the case of the wish list criteria, the most extreme example, the sensitivity was 100% and the specificity was 0%, leading the authors of the paper to caution that the systems could lead to “provider fatigue” in real-world settings.
“This clearly shows that the ‘wish list’ criteria cannot be used for early warning,” wrote the authors of the paper. “However, it seems that by further improving these EWS systems, sensitivity could be kept high, while specificity would be higher. This was not achieved yet, but preliminary data from various studies implementing big data analysis of multiple physiologic parameters collected automatically and frequently already show promise in early detection of clinically significant changes.”
Potential areas for improvement include the adjustment of the early warning systems to provide tailored scores for different medical conditions.
