England’s publicly funded health care system, the National Health Service (NHS), has endorsed more than a dozen depression treatment apps, but there’s no proof that most of them actually work, according to a report published this week in the journal Evidence Based Mental Health. The authors of the report examined each of the fourteen depression apps the NHS lists in its app library and found that only two of them had been clinically validated using standard metrics.
Depression apps are computer-based programs that help people monitor their symptoms, provide education, and sometimes offer coping skills and therapy. But many of the NHS-endorsed apps “seemed a bit sketchy” or made strange recommendations, says Simon Leigh, an author of the paper and a senior health economist at Lifecode Solutions in Liverpool, UK. “I think it was rather dangerous of the NHS to endorse them without having the information.”
Leigh’s study is the latest in a slew of reports finding that mental health apps generally lack rigorous testing. Recent research papers on apps that treat bipolar disorder, eating disorders and post-traumatic stress disorder (PTSD) found a similar dearth of evidence validating app developers’ claims.
“Mobile health apps are increasing at an amazing rate, and most don’t have any empirical validation,” says Stephen Schueller, a clinical psychologist at Northwestern University who specializes in internet and mobile interventions for depression, and was not involved in the NHS study. “The last time I looked at the literature, there were five or six randomized controlled trials of an app for depression, and none of those apps were available on a public app store, and none ran on an iPhone or Android operating system. So anything that I would want to touch as a consumer has not been validated in a randomized controlled trial”—the gold standard of clinical research, he says.
The gap between consumer health technology and science to support it is nothing new, of course. Computerized systems that help physicians make clinical decisions fail two-thirds of the time, according to a paper published in June. And a report published last month by IMS Institute for Healthcare Informatics found that more than 165,000 mobile health apps are now available to consumers—twice as many as two years ago—severely outpacing the mechanisms by which physicians can assess them. Fifty percent of them have “limited functionality,” according to the report.
The fact that thousands of mental health apps may be bunk may not come as a surprise to many consumers. The trouble comes when an influential national health system like NHS puts its stamp of approval on an app without requiring standard mental health tests to be applied to it, says Leigh. Metrics the NHS commonly uses to accredit other mental health treatments include the Generalized Anxiety Disorder 7 (GAD-7) and the Patient Health Questionnaire-9 (PHQ-9). Only two of the apps listed on the NHS library—Big White Wall and Moodscope—used such metrics, Leigh’s study found.
The potential harm of a depression app that doesn’t work is that it could compound feelings of anxiety and lack of motivation. “I’m untreatable. I’m a failure. These are common symptoms of depression,” says Schueller. “Lack of motivation is another big one, so if a person gets motivated enough to download an app, and then it doesn’t work, we might have missed that window and they might not get motivated again,” he says. “But I’m not terribly worried that these apps might actually cause problems for a person,” he says.
NHS does not claim to have officially accredited the apps in its library, but “their badge is plastered all over them,” says Leigh. “And the NHS badge connotes an implicit level of quality.” The NHS says it chooses apps based on three criteria: that the apps are relevant to people in England, that they use information from a trusted source, and that they comply with legislation on appropriate use of data.
The NHS did not return Spectrum’s request for comment by press time. The agency has noted on its website that its health app library, which began in 2013, was a pilot project, and is scheduled to close this week. A new list of online mental health services has appeared on the NHS Choices website.
Other reports have criticized the NHS apps library as well. Imperial College London last month published a studying finding that many of the agency’s endorsed apps sent unencrypted personal and medical details over the internet.
The US Food and Drug Administration (FDA) so far hasn’t taken much of an active stance on validating mental health apps. The agency published guidance [pdf] in February this year, noting that apps that are intended to help with coping skills for people with depression and other psychiatric conditions may be subject to FDA oversight. One developer of cognitive health software, Akili Brain Interactive, plans to approach the FDA for approval of its therapeutic video games. The FDA in 2010 approved the first prescription-only diabetes app, called BlueStar.
Emily Waltz is a features editor at Spectrum covering power and energy. Prior to joining the staff in January 2024, Emily spent 18 years as a freelance journalist covering biotechnology, primarily for the Nature research journals and Spectrum. Her work has also appeared in Scientific American, Discover, Outside, and the New York Times. Emily has a master's degree from Columbia University Graduate School of Journalism and an undergraduate degree from Vanderbilt University. With every word she writes, Emily strives to say something true and useful. She posts on Twitter/X @EmWaltz and her portfolio can be found on her website.
