This week, I return to uncertainty. in this post I make two points; i) that even the modellers are confused about the distinction between risk and uncertainty; ii) that photographic models of reality trade-off realism for predictive-value. (In a seperate post, I will call attention to previously overlooked historical sources for the equation between uncertainty & randomness. It's a 'modest proposal'...)
First, on september, 15 2008 Lehman Brothers files for chapter 11; the day after (the 16th) the official Netherlands Bureau for Economic Policy Analysis (CPB) [the 'house that Tinbergen -- the first Nobel laureate -- built'] predicts economic growth of 1.25% for 2009. In reality the Dutch economy declines by about 4%; uncertainty in action! (In a remarkable self-study in Dutch this is analyzed as follows:."The example of Lehman describes a fundamental problem in preparing short-term forecasts, there are always risks, the estimate is by definition uncertain. It is important that users of estimates are aware of this and keep it in mind when making policy. Even if the collapse of Lehman Brothers had been accurately predicted, then the consequences of this unique event would probably have been seriously underestimated. For economics is not an exact science.“ (9; translate.google.com with minor changes)" As regular readers of this blog know, I am fascinated by uncertainty.
Unfortunately, the authors of the study don't make a crisp (Knightian) distinction between (measurable) risk and (unmeasurable) uncertainty (as regular readers of this blog know (see also, here, here), I am fascinated by it.) Instead they recogize only one kind (risk) and it comes in two epistemic categories: "by ‘risk’ we mean an event or circumstance that can impact the economy, but that more likely than not will not occur during the estimation/forecast period. Risks can be foreseen or unforeseen. An example of a foreseeable risk to economic growth was, since the outbreak of the credit crisis in 2007, for example, the collapse of a bank. An example of an unforeseeable risk is a terrorist attack. We cannot meaningfully take the second risk into account." (19-20; again translation by way of google.translate with minor modifications)
Note three things about their conceptual analysis: i) in their hands, "risk" is a very amorphous concept--an event that might happen, but probably not. Nowwhere in the report are we told how these events are systematically identified and how the low likelihood is established. (I am avoiding the language of probability.) ii) foreseeable risks can apparently be taken into account by the modellers, but apparently are treated as low likelihood and thus not placed inside the main model (or the numbers that are offered to their 'clients'--Dutch government policy-makers). We are not told on what grounds these decisions are made. iii) we are not told who establishes that and on what grounds what risks are unforeseeable--nor are we told on what grounds they cannot be meaningfully taken into account. (In countries that experience terrorism regularly -- Israel, Irak, the UK during 70s and 80s, etc -- quite a bit of planning against it takes place.) One general worry is that everything that did not happen last year is treated as a risk.
Second, one fascinating aspect about the self-study, is that CPB is confident that its main model of the Dutch economy is basically accurate (if I understood the authors of the report correctly, it can capture 90% of the variance). The model has 40 fundamental equations that capture the main movements of the economy; although the whole model has around 2600 equations. The main problem for the modelers is that useful data for one of the main exogenous values -- world trade, which is crucial to an 'open' economy like Holland-- lags reality about four months. Somewhat surprisingly, they don't seem to think this can be remedied, but that's because they rely on publicly available data (rather than spending real money collecting highly time sensitive data on trade). In his famous 1953 methodological article, Milton Friedman (who know the first generation modelers of the so-called Cowles commision -- the intellectual heirs of Tinbergen -- up close), warned that this kind of 'photographic' (and realistic) model of an economy might fair badly in predictions. Above the Dutch modelers respond by claiming that economics is not "exact" science, by which they mean it is not 'predictive'. (The model is VERY exact.) In a part of the report that I have not quoted, they also point out few other institutions do a better job at predicting, although they recognize a few exceptions. In fact, burried in the report the authors of the report make it clear that the aim of their model is not accurate predictions, but facilitating government planning and democratic bargaining. (I return to this some other time when I discuss their own understanding of the incentives of predictions.)
Recent Comments