A flurry of COVID-19 risk assessment tools have rolled out over the past months aiming to help people make decisions about what to do, and what not to do. But how useful are these tools as we enter what many experts are calling the most dangerous time of the pandemic thus far?
Experts have cautioned against relying on any one tool too heavily, but the urge to do so may be especially strong right now as people re-evaluate even basic outings, like grocery shopping, as cases surge and hospital ICUs near capacity. At the same time, many people also will be tempted to gather with family and friends over the December holidays and New Year, despite California’s stay-at-home order restricting such gatherings.
So I decided to try some of the most popular risk assessment tools to see what they could tell me about my risk profile in San Francisco right now, and then got experts to weigh in on my findings. The bigger question: How would these tools influence or alter my overall perception of risk, if at all?
The calculator UCSF’s Bob Wachter used
Created by a group of friends and housemates in San Francisco, the microCOVID Project calculator generates a numerical unit of risk, measured in “microCOVIDs,” and allows users to get incredibly specific in simulating the risk of an encounter: You can see the risk of flying on a plane wearing an N95 mask, and you can see the risk of kissing a stranger for 10 minutes in San Francisco — indoors or outdoors.
The tool also factors in prevalence, the portion of the local population that has the virus (which they calculate based on data consolidated from John Hopkins, Covid Act Now, and Our World in Data).
Using this app, I can simulate meeting up with five friends in San Francisco at a park — something that is forbidden under the current lockdown order, since the city allows meeting up with a maximum of only one other person who lives outside of your own household. I specify that we’re all wearing surgical masks, engaging in conversation, and sitting at least 6 feet apart for two hours.
This activity, according to the calculator, would add up to about 20 microCOVIDs, where each microCOVID is considered a 1-in-1 million chance of getting COVID-19. Moving the same gathering indoors — also not permitted — bumps my risk meter up to the app’s red “Very High Risk” territory, and gives me a total of 500 microCOVIDs.
A 500-in-1 million chance is hard to process, but when I plug in grocery shopping (assuming my normal 30-minute trip), I get a green “Low Risk” score of just nine microCOVIDs. And when I calculate the risk of a two-hour indoor gathering with five people in Providence, R.I., where the prevalence of the virus is much higher than it is in San Francisco right now, I get 1,000 microCOVIDs. Remove the masks from the picture and that jumps to 10,000.
“I think this is by far better than anything I’ve seen,” said Bob Wachter, chair of the Department of Medicine at UCSF, who used the app to generate an estimated risk of getting COVID when he chose to fly to Florida to visit his parents earlier this year, before the recent surge and the California lockdown that has barred nonessential travel. At the time, the calculator gave him a 40-in-1-million chance of getting COVID.
“Though not zero, it seemed worth it to see my folks,” he said on Twitter in September.
A basic tool to rate risk levels
The MyCOVIDRisk app, like the microCOVID calculator, asks for your location and information about the activity you’re considering, and then spits out a risk score from “Very Low” to “Very High,” based on the prevalence of the virus in your area.
“This can help, ideally, with both informing smart closures and with helping people to choose things that are safer,” Megan Ranney, emergency physician and co-creator of the tool, said.
Right now, the MyCOVIDRisk app tells me, meeting up with five friends (outdoors, for two hours, all wearing masks) is a “Very Low Risk” activity. Moving the gathering indoors changes the score to “High Risk.”
What about going to the grocery store for 30 minutes, assuming five masked strangers are also there? The app tells me this is “Medium Risk,” somewhere between seeing friends outside with masks on and seeing them indoors, or a 0.01%-0.1% chance of getting infected. The app offers ways that I can lower my risk at the store: by putting on an N95 mask, staying 9 feet apart from others, or wearing a face shield.
The app succeeds in making it clear that any indoor activities are much higher risk than socially distanced outdoor activities, taking a harm reduction-style approach, which acknowledges there are certain activities we can’t avoid, but can make safer.
But I didn’t find the app’s general risk ratings (”Very Low,” “Low,” etc.) as useful as the quantitative framework of the microCOVID calculator.
“Humans have a very difficult time, myself included, in appreciating what risk is,” said infectious disease expert John Swartzberg, “so I like that there are tools out there for people.” But he’s not sure how many people will actually modify their outcomes based on the results of any such tool. They can’t address “people’s own unconscious biases in how they interpret risk,” he said.
Tools to estimate the risk of gatherings
Japanese epidemiologist Yuki Furuse’s calculator lets users estimate the chance that there will be at least one infectious person at a gathering, based on his research for a scientific paper published in November. But it doesn’t let you specify particular types of outings, like meeting up with friends or going shopping for groceries — and it does require you to enter information about the overall population size and the number of daily new cases.
It shows that in San Francisco (with a population of roughly 870,000 and 223 daily new cases), the probability that one person is infected at a party of five people is 5.4%, assuming no one there has symptoms. Increasing the number of people at the event to 10 raises the probability to 10.6%.
“The probability does not describe how likely transmission is to occur,” Furuse said, noting that if people “follow the 3Ws” of washing their hands, wearing a mask, and watching their distance, the risk of getting an infection can be reduced.
“I’d like to make people scared, but not have them overreact,” Furuse said.
Georgia Tech’s COVID-19 Event Risk Assessment Planning Tool also estimates the chance of running into an infected person depending on the size and location of a gathering. For a 10-person event in San Francisco right now, that probability is about 11%, according to the map.
Increase the size of the gathering to 50 people and the likelihood increases to 43%. In Santa Clara County, the risk is 68%. And in Providence, R.I., it’s 95%.
Like Furuse’s calculator, the map doesn’t let users drill into different, specific types of gatherings, but it can be useful in visualizing how the level of spread in your community impacts your risk, or comparing the risk in one county to the risk in another.
Ellie Graeden, an adjunct professor for Global Health Science and Security at Georgetown University, said the Georgia Tech map is a good way to demonstrate how the level of spread in your community affects your risk.
“It’s a really useful way to communicate, for example, to your in-laws, why you can’t do Thanksgiving together,” Graeden said.
What I learned
Using these risk tools, especially the ones that allowed me to adjust variables to get as close as possible to the real encounters I’m faced with, was a useful reminder of how one small factor can change the entire risk profile of an activity.
A quick trip to Safeway could turn into an hour-long journey. If you must fly on an airplane, it makes a big difference whether the seats in your rows are empty or full. The risk level of a seemingly safe activity can increase dramatically depending on the prevalence of the virus where you are, which is constantly changing. And all the little risks of daily life can add up.
But it’s hard to say that any of the tools really helped me to understand risk in a new way. And experts say we simply don’t know yet if they will have a marked impact on our actual safety.
“Anything we can do to prevent transmission is terrific,” said George Rutherford, professor of epidemiology and biostatistics at UCSF, “and I’d be all for these sorts of things, but at some point in time we have to really figure out if they work or not.”
Jenna Fowler is a San Francisco Chronicle staff writer. Email: firstname.lastname@example.org