People who suffer with anxiety, depression or stress are often asked to record their mood changes throughout the day, helping psychologists fine-tune their treatment. But they often forget, recording only sparse information at best.
An emotion-sensing smartphone app that automatically generates a “mood diary” could give psychologists all the data they need.
The app is the brainchild of Matt Dobson and Duncan Barclay, founders of speech recognition firm EI Technologies, based in Saffron Walden, England. Instead of relying on people writing diaries, the app, called Xpression, listens for telltale changes in a person’s voice that indicate whether they’re in one of five emotional states: calm, happy, sad, angry or anxious/frightened. It then lists a person’s moods against the times they change, and automatically emails the list to their psychologist at the end of the day.
To work, the app has to be on always, listening out for the user’s voice once every second, whether they’re talking to family, friends, colleagues or even pets. It also listens in on phone calls. If the user is silent, the app does nothing. Crucially for the users’ privacy, it doesn’t record their words, instead seeking out telltale acoustic features, like pitch, that are indicative of emotional state.
This kind of emotion recognition via voice pattern already works well and is a “hot area” of research, says Stephen Cox, head of the speech processing lab at the University of East Anglia in Norwich, England, and a scientific adviser for the maker.
Initially, Xpression will send 200-millisecond-long acoustic snapshots to a remote server where a machine-learning system will work out a person’s emotional state before sending it back to the app for storage. Factors like voice loudness, intensity, changes in pitch and speaking pace allow the system to accurately estimate somebody’s emotional state.
“We extract acoustic features and let the machine-learning system work it out,” says Cox. This ability will be built into the app itself eventually, says Dobson.
There’s a strong need for this kind of technology, says Adrian Skinner, a clinical psychologist with the U.K.’s National Health Service in Harrogate, North Yorkshire, England.
“With conditions like depression, people tend to stop doing things like filling in mood diaries. If this app gives us more complete diaries it could help us better find the day-to-day triggers that raise or lower a patient’s mood,” he says.
An insurance company has already expressed an interest in using the app to ensure the workplace stress therapy it pays for is effective. Clinical trials are due to take place later this year.