Pulse oximeters risk inaccuracy in patients with dark skin, leading to potential missed sleep apnea diagnoses. Some pulse ox manufacturers explore more inclusive testing.

By Alyx Arnett

During the COVID-19 pandemic, pulse oximeters—which are commonly used to detect oxygen saturation during overnight polysomnography, monitor patients during and after surgeries, and assess people with heart conditions, among other clinical applications—were increasingly utilized at home and in hospitals to evaluate and make treatment decisions for people with the virus.

But concerns quickly surfaced that the devices might give false negatives for hypoxemia in people with dark skin tones—a potentially life-threatening oversight.

The problem, while not new, gained renewed attention after an editorial published in the New England Journal of Medicine reported that Black patients had nearly three times the frequency of unrecognized hypoxemia, which was not detected by pulse oximetry, compared to White patients (17% versus 6.2%).

The US Food and Drug Administration (FDA) then issued a safety communication stating health care providers should be aware that multiple factors can affect the accuracy of pulse oximeter readings, including skin pigmentation.2 

And more reports surfaced. A retrospective analysis of over 7,000 patients found pulse oximeters overestimated arterial oxygen saturation in non-White patients, subsequently delaying these patients’ eligibility for COVID-19 medications.3 Letters from United States Senators and attorneys general urged the FDA to address the concerns. 

Key Takeaway: The COVID-19 pandemic elevated concerns over inaccuracies in pulse oximeter readings for dark-skinned individuals.

What’s the Problem With Pulse Oximeters?

A couple wears BodiMetrics’ circul pro Ring pulse oximeter, which received FDA 510(k) clearance last year and has been clinically validated to accurately measure oxygen levels in diverse skin tones. Photo credit: BodiMetrics

It has been recognized for decades that skin pigmentation can influence the accuracy of pulse oximeters. A study from 2005 found three pulse oximeters overestimated oxygen saturation during hypoxia in dark-skinned individuals.4

Pulse oximeters, which typically clip onto a fingertip, send beams of red and infrared light through tissues to evaluate blood oxygen saturation based on the light absorption levels.5 However, melanin can absorb light and affect the measurements.

Pulse oximeters are least accurate when blood oxygen saturation is below 80%.2 

FDA-cleared prescription pulse oximeters meet a minimum average accuracy, as validated through desaturation studies on healthy patients. These studies compare pulse oximeter readings against arterial blood gas levels for saturations ranging from 70%-100%. The standard accuracy, denoted as Accuracy Root Mean Square (ARMS), for recently approved pulse oximeters falls within 2% to 3% of arterial blood gas values.

However, the testing protocols may not fully address accuracy discrepancies across skin tones. The FDA’s latest 2013 requirements for 510(k) clearance necessitate that the study population include “two darkly pigmented subjects” or 15% of the total participants, whichever is greater. This criterion raises concerns about the representativeness of these studies, according to sleep specialist Meir Kryger, MD, professor emeritus of medicine at Yale School of Medicine. “Two 18-year-old guys could be representing 3 to 4 billion people,” Kryger says. 

Another issue is dark-skinned patients are classified based on ethnic self-reporting. “Someone can identify as African American, but they can have very light skin. Or they can be dark-skinned and identify as a different ethnicity,” says John Hastings, CEO of pulse oximeter manufacturer Nonin Medical. 

Also, over-the-counter pulse oximeters for “wellness” and not medical use are not subject to FDA scrutiny.

Key Takeaway: The melanin in dark skin can absorb the light needed for accurate pulse oximeter measurements. The FDA’s testing protocol may not fully address this skin-tone discrepancy. 

Implications for Sleep Medicine

Almost all sleep diagnostic systems utilize pulse oximetry as a key component. “If the pulse oximeter that’s being used as part of those systems isn’t accurate for darker skinned patients, it compromises the quality of the overall sleep diagnostic outcome for those very same patients, and some patients who may be in need of treatment may end up not receiving it,” Hastings says.

Indeed, research presented at the ATS 2023 International Conference suggests that pulse oximeters may be putting Black patients at a disadvantage by underestimating the severity of obstructive sleep apnea. The study of nearly 2,000 patients found that Black patients have smaller measured decreases in oxygen saturation during respiratory events compared to White participants.6   

Sleep specialists agree that results such as these represent a concern for sleep medicine. “We know that blood oxygen level is variable, and the worst time for people who have hypoxemia is during sleep,” Kryger says.

Heidi Riney, MD, chief medical officer at sleep diagnostics company Nox Health, says, “There exists a disconcerting tendency for oxygen saturations to be overestimated in this population, leading to an underrecognition of hypoxemia, which can have dire consequences. This oversight is especially concerning, as untreated obstructive sleep apnea, a condition prevalent in these populations, is known to detrimentally impact chronic health conditions. The underestimation of hypoxemia not only poses a risk of downplaying the severity of obstructive sleep apnea but also perpetuates health inequity, setting the stage for adverse health outcomes.”

The research presented at ATS concluded that further studies were needed to investigate how factors related to pulse oximeters and individual physiology affect disparities in oxygen saturation measurements during sleep among various ethnic groups.

Key Takeaway: Pulse oximetry inaccuracies in dark-skinned patients may lead to missed or underestimated sleep apnea diagnoses.

What’s Being Done to Address Racial Disparities?

In the meantime, in response to ongoing concerns about the performance disparities of pulse oximeters in individuals with darker skin, the FDA convened the Anesthesiology and Respiratory Therapy Devices Panel of the Medical Devices Advisory Committee on Nov 1, 2022. 

This meeting aimed to delve into the factors influencing pulse oximeter accuracy and performance, scrutinize evidence on their precision, and forge recommendations. It also sought to define the data that manufacturers must furnish to evaluate these devices.

The panel identified that variations in performance could also be attributed to factors beyond pigmentation, including perfusion and physical characteristics like obesity and finger width and breadth. 

The panel advocated for the standardization of skin pigmentation assessment and proposed enhancing the accuracy of pulse oximeters by suggesting a reduction in the ARMS threshold.

More recently, on Feb 2, the committee reconvened for an open public hearing to discuss the premarket evaluation of pulse oximeters. This meeting, aimed at refining the methodology for assessing performance with a focus on skin pigmentation, race, and ethnicity, reviewed approaches proposed in its November 2023 discussion paper.7

The panel deliberated on the types and volumes of data manufacturers should provide, encompassing both prescription and over-the-counter devices, alongside labeling considerations. With input from patients, clinicians, industry representatives, and other stakeholders, there was consensus on the need to update the 2013 pulse oximetry 510(k) submission guidance.

The panel supported using the Monk Skin Tone Scale—validated to capture race and ethnicity diversity in pigmentations within the United States—for subjective assessment and the individual typology angle—a method to measure skin pigmentation at the sensor site—for objective measurement. The panel advocated for the inclusion of participants representing a broad spectrum of skin tones (scores 2 through 9 on the Monk Skin Tone Scale) in clinical trials.

“It is a high priority for the FDA’s Center for Devices and Radiological Health to help ensure availability of safe and effective pulse oximeter devices intended for medical use. This is a complex issue, and the FDA continues to gather input from ongoing clinical research to help inform our decisions,” says an FDA spokesperson. 

Additionally, the University of California, San Francisco launched the Open Oximetry Project in late 2022 to address disparities in pulse oximeter performance. The multi-year initiative, uniting oximetry specialists, researchers, clinicians, manufacturers, regulatory agencies, and others, seeks to identify the extent and underlying reasons for oximeter inaccuracies in dark-skinned individuals, enhance performance standards, share performance data to encourage manufacturer responsibility, ensure regulatory compliance, and raise consumer awareness.

Key Takeaway: Efforts to address health equity disparities include FDA meetings to discuss accuracy and performance issues, proposals for improved testing standards, and initiatives like the Open Oximetry Project.

Nearly Half of Pulse Oximeters Fail in Diverse Populations

Nonin Medical’s Onyx Vantage (pictured) demonstrated the best performance of 11 pulse oximeters tested, with 1.9% ARMS. Photo credit: Nonin Medical

A study conducted as part of the Open Oximetry Project evaluated the performance of 11 fingertip pulse oximeters—four of which were FDA-cleared (Contec, Biolight, Masimo, and Nonin)—using the FDA’s 2013 guidance and International Organization for Standardization standards (2017).8

The study included 34 healthy individuals with diverse skin pigmentation and varying saturation levels. Skin pigmentation was assessed subjectively using a perceived Fitzpatrick Scale and objectively using the individual typology angle via spectrophotometry.

The study found that five devices had ARMS >3%, falling outside the FDA’s acceptable performance range. Of the four pulse oximeters that had received FDA 510(k) clearance, only two demonstrated ARMS ≤ 3%, Nonin and Masimo. The Nonin Onyx Vantage and Masimo MightySat had the best ARMS at 1.9% and 2.1%, respectively.

Researchers say the study underscores variable pulse oximeter performance resulting from vague regulatory guidelines on diversity inclusion and call for an end to using subjective, non-standardized skin color scales.

Key Takeaway: A study under the Open Oximetry Project found that nearly half of tested pulse oximeters failed to meet FDA standards in diverse populations.

Nonin and Masimo’s Pulse Oximetry Lead the Pack

Masimo’s MightySat fingertip pulse oximeter was granted FDA clearance for over-the-counter use in February. Photo credit: Masimo

Hastings at Nonin—the company that manufactured the first fingertip pulse oximeter in 1995—says the performance of its pulse ox, demonstrated in the study, reflects the advancements of the company’s technology over the years. “But with that being said, we’ve never been a company to be satisfied with the status quo, so we are continuing to make advancements,” he says. 

Nonin has included a “much larger subset” of dark-skinned patients in its studies dating years back, according to Hastings. Hastings would like to see FDA requirements for sample sizes increase and move toward an objective scale for measuring skin tone. 

Nox Medical, a manufacturer of diagnostic sleep tests, uses Nonin’s pulse oximeters in its systems. “We believe their pulse oximeters provide some of the most accurate blood oxygen saturation readings for diverse populations,” says Valgeir Tómasson, senior director of product management – medical systems at Nox Medical.

Meanwhile, Masimo’s MightySat fingertip pulse oximeter gained FDA 510(k) clearance for over-the-counter use last month, making it the first FDA-cleared medical fingertip pulse oximeter available to consumers without a prescription. The device uses Masimo SET pulse oximetry, which has shown no clinically significant difference in accuracy or bias between light- and dark-skinned individuals.9

Additionally, Masimo published a retrospective study showing its Masimo SET pulse oximetry was accurate across various skin pigmentations in low perfusion conditions—a known confounder.10 Researchers used data from 3,201 Black and 3,982 White individuals, and ARMS was 1.37% in all subjects with normal perfusion and 1.64% with low perfusion.

“The results of the study demonstrated that Masimo SET pulse oximeters with RD SET sensors are accurate for individuals of both Black and White skin pigmentation during both normal and low perfusion index conditions,” says William C. Wilson, MD, chief medical officer and senior vice president of clinical research and medical affairs as Masimo.

Key Takeaway: Pulse oximeters by Nonin and Masimo have studies supporting their accuracy in a diverse range of skin tones.

FDA-Cleared Smart Ring Accurate Across Skin Tones

The design of the circul pro Ring minimizes twisting and slipping, stabilizing the sensor on the finger’s “belly,” an area where there is generally less pigment. Photo credit: BodiMetrics

The FDA recently granted 510(k) clearance to BodiMetrics for its prescription-only circul pro Ring as a pulse oximeter that was proven clinically accurate in diverse skin tones.

The device features a square-like shape to prevent it from twisting around the wearer’s finger, stabilizing the sensor on the palmar side, where there is generally less pigment. The ring continuously measures oxygen saturation, taking a reading approximately every second, using reflectance photoplethysmography.

Clinical data submitted to the FDA tested the ring in 12 subjects, four of whom had dark pigmentation (33.3%), and found the ARMS was 1.85%, within the FDA’s allowed range of ≤3%. 

The device’s ability to accurately measure oxygen levels in individuals with dark skin pigmentation was reinforced by a July 2023 study involving 24 healthy participants—eight Black and 16 non-Black.11 The study compared the ring’s readings with simultaneous readings from arterial blood gas testing, finding “excellent” correlation between readings. 

“The most important thing—yes, I’m in the business, and yes, I want to sell my ring—is that I’d like to alert people about the problem,” says Neil Friedman, chief operating officer at BodiMetrics. “There’s racism within our healthcare system, and we have to provide equal health for all.” 

Kryger, a co-author of the July 2023 study, became aware of the circul pro Ring at the start of the pandemic when he was working in clinical practice, treating patients on CPAP machines. “I was told, ‘Go home. You’re not to come back until this pandemic is over,’” Kryger says. But he had no direct way to measure his patients’ blood oxygen, and companies that normally would have delivered these items to patients’ homes stopped, he says. 

He began to hunt for a device and ultimately came across the circul pro Ring. He was skeptical about whether it would work across various skin tones but became a believer after discussing the technology with the ring’s engineers and subsequently studying the device for himself.

He sent the rings to patients, facilitating remote settings adjustments on their CPAP machines. 

Kryger also co-authored a study testing the ring in 207 Chinese adults against polysomnography to detect sleep apnea.12 He found that the circul ring could detect sleep apnea with reasonable reliability. “So it can actually be useful in looking at desaturations in patients who have sleep apnea,” he says. 

Key Takeaway: The FDA has approved BodiMetrics’ circul pro Ring, a pulse oximeter shown to accurately measure oxygen saturation across various skin tones.

Change on the Horizon? 

BodiMetrics’ Friedman has noticed a change in the attitude in the healthcare industry in the last couple of months—after the attorneys general issued their letter to the FDA. “I’m seeing a willingness of people to say, ‘Yes, it’s wrong, and we better do something about it,’” he says.

Until significant change occurs, Kryger cautions clinicians to thoroughly understand the pulse oximeter they’re using. “Make sure you’re using a device that actually measures what you think it is measuring, and make sure you understand how the device works, whether it will work in everyone,” Kryger says. “Understand what happens when the light goes on.”

Key Takeaway: Recent advocacy prompts a noticeable shift in healthcare attitudes toward addressing biases in pulse oximetry.

Photo 209780133 © Cebas1 | Dreamstime.com