Bluetooth bugs are making contact tracing apps spit out tons of errors

At the heart of all Bluetooth-based contact tracing apps is one fundamental issue: whether they actually help in the struggle to slow the spread of Covid-19. Whether they’re effective or not can be judged on two factors: epidemiologically and technically. But, months into the pandemic, there are still no clear answers.

“They're quite unreliable,” says Douglas Leith, the chair of computer systems at Trinity College Dublin, who has been analysing how well devices can use Bluetooth to measure distance. Leith says the reliability of apps to detect other devices around them accurately and calculate their difference can vary widely. Bluetooth wasn’t designed for contact tracing, but in the pandemic era it’s being held up as a potential solution to the crisis.

Contact tracing apps using Bluetooth Low Energy – which is the majority of apps around the world, including those using the Apple and Google protocols – detect other devices around them using Received Signal Strength Indication (RSSI). In short this is the signal strength. The loss in signal strength between a transmitter in one phone and the receiver in another device is used as a way to determine distance between phones.

The distance between phones is calculated every two and five minutes, this can vary in different countries, and is used to work out if someone being close to another person increases the chances of Covid-19 transmission. Apps are based on the understanding that the virus is more likely to pass from one person to another if they’re within two metres of that person for more than 15 minutes. (Although, as the variety of social distancing measurements around the world show, the science behind distances where transmission risk changes is tricky).

“There are at least four things that can have a big effect on the performance,” Leith says. None of these come as a surprise to engineers, he adds, as Bluetooth works on the same frequency band as microwave ovens and the frequencies are well studied. The type of phone, including the antennas, chipsets and way they’re combined, can all vary Bluetooth performance. How the app was tested before being released to the public can also have an impact, as can the materials of the environment where the app is being used – think thick concrete walls versus thin glass windows. All of these alter the registered RSSI.

Phone manufacturers have been trying to iron out some of the differences in how Bluetooth performs on their devices. Google is hard at work too and has created a method where device makers can configure Bluetooth settings more easily. This involves sticking phones on a tripod, rotating them and collecting lots of data.

A spreadsheet listing more than 12,000 models of phone – including manufacturers from all the big names right down to Coolpad and Tecno – shows how much variance there is. RSSI corrections range from 19 to -29 and the power transmitted ranges by up to 45 different points.

If that weren’t complicated enough, one of the most influential factors on signal strength is simply where a device is placed. “You see fluctuations due to small changes in the relative orientation of the phone,” Leith says. If your phone is in your back pocket, the signal transmitted to someone sitting or standing in front of you will be very different to if it is in your front pocket.

An experiment conducted on a tram by Leith and colleague Stephen Farrell found signal strength didn’t effectively calculate distances in that scenario. They used the app detection rules of the Italian, Swiss and German apps for their experiment. They found that similar ranges of signal strength were observed both between handsets which are less than two metres apart and handsets which are greater than two metres apart – all the way up to five metres. Such differences could have a big impact on Covid-19 transmission – but Bluetooth isn’t sensitive enough to spot them.

In the tram scenario, the lack of accuracy is caused by the metal walls, floor and ceiling, all of which reflect radio signals. “There are some places where we’re fairly sure it just won’t work. And that’s in environments where there’s a lot of metal,” Leith adds.

The inability for Bluetooth to always precisely measure how far away people are from each other is an open secret. And incorrect measurements can result in false notifications being sent. Telling people to self-isolate when there’s lesser risk of them having coronavirus can be economically damaging and erode trust in the overall track and trace system. On the flip side, not identifying people who have a greater risk of contracting the virus means it could spread more easily.

In August, ahead of the launch of the NHS Covid-19 app in England and Wales, the National Cybersecurity Centre discussed “the Bluetooth problem”. It said the issue of false positives – phones thinking their owners had been close to others when they hadn’t been within two metres – was something that was being worked on.

“If you were between 2m and 4m from an infected person, there’s currently a false positive rate of about 45 per cent,” the NCSC’s technical director Ian Levy wrote in a blog post. This means some people are sent notifications of potential infection despite being more than two metres away from someone who has tested positive for Covid-19.

“By way of a simple illustration, during the recent Leicester outbreak, the app would have generated ~50 false positives a day in a population of 330,000,” Mark Briers, from the Alan Turing Institute wrote in a blog post in August about the app’s modelling. That modelling has not yet been published.

Researchers are scrambling to improve the false positive rate. A spokesperson for the Department of Health and Social Care, the government department responsible for the app, says that since the NCSC analysis was published the false positive rate for people between two and four metres apart has been reduced by around 25 per cent. People who the app judges are at a high-risk of transmission but are actually low-risk have been reduced from 4.5 out of ten to three in ten, a DHSC spokesperson says.

They add that notifications from the app are only triggered once someone has been confirmed to have tested positive. “We are very clear, everyone who is contacted will have been in close contact with someone who has a confirmed case of coronavirus,” the spokesperson explains.

Key to the operation of the NHS Covid-19 app is a risk-scoring algorithm that combines Bluetooth distance and time data with when the person who tested positive developed symptoms. The DHSC and NHS can change the levels of risk that trigger notifications as more scientific advice becomes available. Briers at the Turing Institute and the NCSC are working to further refine the algorithm and detection methods used.

“The only way that you could minimise or decrease false positives is to decrease the sensitivity,” says Gary Hatke, an engineer at MIT and part of the PACT team working on contact tracing apps and the underlying technology. “Or you'd have to get Apple and Google to change the way the API is working.” Google says it has worked to make Android battery impact as low as possible and, along with Apple, is continually improving the notification system.

Other Bluetooth devices could tackle distance measurement better than phones. Singapore, which was first to develop a contact tracing app for Covid-19, is now trialling a wearable device. “It is possible that wearable devices could, in general, perform better than smartphones in helping to estimate likely exposure events,” says Ken Kolderup, the chief marketing office at Bluetooth SIG, the group that develops the standard. “In many cases, wearables such as wristbands are worn on the body in locations that will result in less interference, and thereby lead to more accurate estimations of the true distance between people.”

But, for the time being, Bluetooth contact tracing is will be done by millions of phones, not a smattering of wearables. Hatke says that the MIT team has shown that Bluetooth reliability can be improved on devices. He says apps can make more frequent RSSI measurements to help determine how far away other devices are, although this could require more battery power to be effective.

The other way, Hatke says, is “to do some self awareness on the phone. Every time I take a measurement, can I tell whether or not I’m in a pocket or not”. This could be through proximity sensors that can tell whether a phone’s surface is covered or near a body, or light sensors. “You could encode that information in the signal that is being transmitted,” Hatke says.

But Hatke and his colleague Marc Zissman still question the exact purpose of development for contact tracing apps. And this makes them harder to optimise. Are they meant to help find people who are unknown to those who have been infected (for instance, people on a train) or to help notify people to self-isolate faster than human contact tracing efforts can contact them?

Seven months into the pandemic, there’s still no obvious answer. Distance and time are only two indicators of transmission risk. Airflow, whether people are indoors or outdoors and the volume at which people are speaking are some of the other variables that impact the Covid-19’s ability to spread. But none of these can be measured easily by an app that collects very little data.

“We really need the data from the jurisdictions that are running this,” Zissman says. As a result, there’s little incentive for researchers to work out how contact tracing apps can be most effective. “There is no published data where we can take the country or state and tell you what these values are.”