By now it should be apparent just how little we know about the coronavirus pandemic, from how to treat it to basic facts about what the “number of COVID cases” means. Even “deaths due to COVID” turns out to be difficult: both New York and UK have revised their numbers up to accommodate likely cases that hadn’t been counted, and there’s a brewing political battle over how to count them. This ignorance also affects pandemic modeling; the aspect I want to look at here is over what “social distancing” means.
COVID Models are for guiding action: we want to look at them and know when we can leave the house again (or, maybe more importantly, when the kids can leave the house again!). In that sense, models are inherently political, in several ways. First, the model doesn’t tell you what to do; the decision about what to do can be informed by the model, but it requires an entire apparatus of priorities, intuitions, and whatever else goes into a decision about what to do. Part of this is a second-order decision strategy about how to process uncertainty. I’ve suggested before that we’ve been operating on maximin; whether I’m right isn’t important here except to underline that the model is insufficient. Second, the construction of the model is going to embody a number of social judgments. I’m not being Latour here and arguing that whatever comes out of laboratories or complex instruments is broadly political (though he’s right). What I do want to say is that to use a model, you have to answer some antecedent questions what your use of the model is trying to achieve, and how society operates.
The reason to bring all this up is the simmering controversy about the adequacy of the IHME model. It’s apparently the basis of a lot of federal decision-making; it’s also been explicitly rejected by a lot of local and state agencies. Mecklenburg County (where Charlotte is), for example, has specified that it’s using the CHIME model out of Penn (more on that in a minute). Lots of epidemiologists hate the IHME model; they think its method (which involves projecting what happened in Wuhan and Lombardy onto other locations, a process that involves constant revision and correction; traditional epidemiology starts with variables like R0) is crap.
But anybody who’s looked for very long at different models will notice something else about the IHME: it says we can leave the house sooner! Knowing this fact already makes it hard to use models with confidence. If you are inclined to gloom and doom about COVID, you will avoid the IHME model; if you’re feeling optimistic, you’ll tend toward it (if you have kids at home, IHME is the ONLY model). Either could be examples of whatever the equivalent of confirmation bias is when applied to practical reason.
Mecklenburg County has been explicit: they are not using IHME because they don’t want to use such an optimistic model (to their immense credit, they also released the data and assumptions they input into CHIME). When planning for hospital capacity, better to be pessimistic and wrong than optimistic and wrong, basically. The County is also concerned that state-level results may not translate well down to counties, and Mecklenburg is by a substantial margin the most affected county in NC. The county is also using CHIME because it’s designed to predict hospital resource use, and they’re deliberately making that the key variable to manage.
Thus for some of the antecedent decisions about what model to use. Another antecedent decision seems to be built into the models, and I’ve seen almost zero discussion of it, but it makes a big difference. How much social distancing actually happens, and what does it mean to construct an indicator to measure it? I’ve alluded to this problem before, but it’s very clear in the choice between IHME and CHIME. CHIME lets you set different levels of social distancing; Mecklenburg is choosing the conservative 45% figure.
But what is social distancing? Per CHIME, it’s “the estimate of how much social contact is reduced in your region compared to no social distancing at all.” As for the effects of social distancing measures, the model refers you to the original Imperial College, UK model. That model outlines several forms of social distancing. For example, social distancing of the entire population is defined as:
“All households reduce contact outside household, school or workplace by 75%. School contact rates unchanged, workplace contact rates reduced by 25%. Household contact rates assumed to increase by 25%” (6).
CHIME then has to translate this into the percentage reduction used by its model. To do so, they follow a paper that makes the following assumptions: for adults (there is also a set for children), assume that there are 12 interactions a day, of which 40% are family, 30% friends, and 30% other. Then break down “other” into 30% essential and 70% essential. Finally, “we assume (guess) that the guidance for general social distancing (fewer in-person social gatherings, maintaining safe distance when meeting in person, etc.) will result in a 50% reduction of contact with friends and “Other” interactions (as defined above).” The model then gets to a 31% reduction in total contacts by closing schools, closing non-essential businesses, and the social distancing guidelines.
(Notice that to get to a 45% reduction, you need to do more, and that ramping up social-distancing is the only real wiggle room in the model. Assuming reasonable compliance with stay-at-home orders will virtually eliminate “friends” from the contacts, which radically changes the calculation. A lot also depends on what an “essential” other contact is).
If I’m reading all that correctly, the version of CHIME used by Mecklenburg County is assuming something less than closing everything and issuing a stay-at-home order for all but “essential” businesses, which are then operating with social distancing rules. But it assumes something more than general social distancing exhortations. But the model doesn’t support a precise rendering of this middle space.
What about IHME? The original version of that paper is also pretty pessimistic – forecasting a peak ICU demand in excess of 25% nationally (6), although its primary unit analysis is deaths, not hospital use. Comparisons beyond that are difficult, however, because the IHME model is based on the empirical trajectory of the epidemic in Wuhan (and to some extent Italy). The model looks at four social distancing measures: school closures, closing non-essential services, shelter-in-place, and major travel restrictions. How to weigh those?
“At this point in the epidemic, we have had to make arbitrary assumptions in our model on the equivalency between implementing 1, 2,or 3 measures–and we have implicitly assumed that implementing 3of 4 measures will be enough to follow a trajectory similar to Wuhan–but it is plausible that it requires all 4 measures. As more data accumulate, especially on the timing of deceleration of daily deaths, we may be able to empirically test which of these measures is more correlated with slowing the epidemic curve and reducing the ultimate death toll. Perhaps as important will be the question of adherence to social distancing mandates; it will take time to evaluate whether social distancing adherence is fundamentally different in the US compared to Wuhan.” (8)
So there you go. The models are neither measuring the same thing, nor measuring in the same way. And neither model is sufficiently granular about the meaning of ‘social distancing’ to make a policy-maker particularly confident about decisions as important as the ones about how to “open up” again.
I’m not saying all of this can’t be modeled. But there is a massive amount of priority-setting that’s going to have to go into any sensible decision to relax restrictions, and the modeling data we have really underdetermines that space. It’s not clear to me that we have any idea what percentage reduction existing measures have achieved, or what different levels of relaxing them would result in. Furthermore, it’s also unclear how the indicator “social distance” can be constructed; some of the cool stuff like cellphone geolocation is pretty rough for this purpose, and using Bluetooth devices is beset with problems of false positives. But for these models to be useful, you need to have an antecedent theory about the efficacy of social distancing policies and how to turn that into data that can be input into a model.
All of this may be completely wrongheaded of course. LA County has released some results of a serological study, and it suggests that about 4% of the County may have been infected. That’s 55x [sic!] higher than the official count. That’s an astonishing result, but it’s not out of line with similar early serological testing results in Santa Clara and Germany. Good for the case fatality rate, but terrible for our understanding of how social distancing works.
Meanwhile, Georgia is reopening a lot of stuff really fast, with no evidence that Gov. Kemp has though much about epidemiology (or really anything at all). Next time you find yourself calming down about Republicans cheating to win elections, remember his blatant voter suppression to get elected. I bet Stacey Abrams would be doing better.
Recent Comments