Devices like Jawbone (shown) and Fitbit are marketed as helping consumers track sleep. But how accurately are they measuring it?
Sleep and activity trackers like Jawbone, Fitbit, and Nike Fuel are now readily available on online shopping sites and at malls around the country. One actigraphy manufacturer argues that this surge in popularity in nonvalidated devices poses several dangers to the medical field.
By Thomas Kazlausky
As president of an actigraphy company, I’m often approached by individuals or companies asking me to include their new actigraph or smartphone sleep app in my product line. My response is always the same: “Show me the validation!” Not every combination of accelerometer and memory can be considered an “actigraph” from the perspective of a medical device provider or healthcare professionals who use actigraphy to make medical assessments. Specifically, devices like Fitbit, Jawbone, and Nike Fuel are not considered by their manufacturers as medical devices and don’t come with evidence of medical validity. They also are not FDA cleared to market as medical devices.
Unfortunately, many of these consumer device manufacturers make statements that the devices “measure sleep quality,” or are “great for insomnia or other sleep disorders,” which sounds suspiciously like medical claims to me. In my opinion, these consumer device manufacturers cannot continue to represent sleep as simply an element of lifestyle while continuing to make undocumented claims about their ability to measure it.
The National Sleep Foundation (NSF) has further confounded the issue by selecting a nonvalidated consumer sleep tracker called Lark as an Official Product of the NSF. When I complained about this lack of validation to the NSF, noting that selling Lark in the NSF’s Sleep Shop implied a validity that is not supported by evidence, I was told accuracy is a secondary concern and the purpose of selling the Lark is to promote an awareness of sleep in America.
The sleep medicine field has started to perform its own validation studies on sleep monitors marketed toward consumers. The results are starting to come in, and they are not good. For example, at the 2013 Conference on Pediatric Sleep Medicine meeting in Amelia Island, Fla, Lisa J. Meltzer, PhD, et al presented a poster entitled “Fitbit Is Not a Reliable Device for Estimating Pediatric Sleep-Wake Patterns” in which a polysomnogram (PSG) comparison with 49 children demonstrated an accuracy percent as low as 71%.
Agreement rates with PSG, not correlations, are essential and should include separate measures of sensitivity (ability to correctly identify sleep) and specificity (ability to correctly identify wake) during the intended sleep period. While being able to pick up napping during the day is indeed important, that should not be confused with some providers’ insistence that agreement should be measured over a 24-hour period. While this is a nifty new possibility available with the advent of 24-hour portable PSG, the resulting 24-hour actigraphy-PSG agreement rates are artificially inflated with a predominance of easily identified daytime wakefulness.
A study by Roger Cole et al (“Automatic Sleep/Wake Identification from Wrist Activity,” Sleep, Vol. 15, No. 5, 1992) found an Ambulatory Monitoring actigraph distinguished sleep from wakefulness with over 88% accuracy. If consumer-geared actigraphs don’t provide validation studies, are the devices worthwhile?
What about the ramifications of consumer-targeted devices on reimbursement? Unfortunately, third-party providers have published position papers citing any study they can find on actigraphs that demonstrates poor accuracy, and have ignored those with higher accuracy, then they conclude actigraphy is “experimental.” This situation can only get worse if healthcare professionals attempt to get reimbursement when employing lifestyle sleep “trackers” without any validation.
It doesn’t serve the patient to make judgment calls based on nonvalidated actigraphy. Doing so may even be worse than relying on a diary because people confuse objectivity with accuracy. Choose an actigraph with documented validity in the population you are examining. And if the payor rejects your claim, have studies on hand that show the device you are using is valid. It’s best to live by the mantra: “Show me the validation.”
Thomas Kazlausky is president of Ambulatory Monitoring Inc.
The opinions expressed here are by Thomas Kazlausky and not necessarily those of Sleep Review.
The NSF’s Response Regarding Lark
“Lark is an Official Product of the National Sleep Foundation, and, like our other official products, is engaged in our sleep health information program. We looked at all of the consumer sleep tracking devices available at the time and admired Lark’s ease of use and sales channels. We believe there is space for both clinical devices and consumer devices and that they serve two distinct purposes. We remain very excited about a future where the public uses sleep tracking devices.”
Share your opinion on consumer actigraphy products on Sleep Review’s LinkedIn group page: www.linkedin.com/groups/Sleep-Review-4041400
Ask Kazlausky your questions about actigraphy via our online Expert Insight department: