How did Treasury policy-makers approach the economic response to Covid-19?

The UK Covid-19 Inquiry provides an opportunity to evaluate how expertise, models and informal analysis contributed to the overall outcome for mortality and the economy. Key questions surround the Treasury’s decision not to use integrated ‘epi-macro modelling’ to inform its policy-making.

With the worst of the pandemic now behind us, it is time to take stock of how the government dealt with the crisis and to learn important lessons for the future. The UK Covid-19 Inquiry is a good opportunity to review formally how the use of expertise in understanding and modelling the pandemic contributed to the overall outcome for mortality and the economy. The recently released terms of reference for the inquiry set out to do so.

Of particular importance are the economic and public health policies formulated by HM Treasury and SAGE (the Scientific Advisory Group for Emergencies), respectively. The Treasury’s policy-making and underlying analysis has been considerably more opaque than that of SAGE, which has periodically released minutes of meetings and supporting scientific evidence, such as modelling output.

A speech provides some insight

In a recent speech, the Treasury’s chief economic advisor Clare Lombardelli gave some insight into the analysis and modelling that took place behind closed doors at the Treasury and elsewhere in government. Little is known about how the Treasury analysed the crisis and how officials saw their role in guiding the economy through the pandemic, so this is a welcome, if rare, view into how economic policy was formulated.

In stark contrast, through most of the pandemic, SAGE members have gone out of their way to communicate strengths and weaknesses of their analysis. They have also acknowledged the basic uncertainties associated with their modelling and policy recommendations.

SAGE was relying on models of the epidemic and the Office for Budget Responsibility (OBR), the Bank of England and other modelling outfits were using models of the economy judgmentally adjusted to represent how the economy was affected by the pandemic. But there was no one in government making use of models that combine epidemiology with economics, modelling both how the pandemic affects the economy and vice versa.

In her speech, Clare Lombardelli justifies this by pointing to the sensitivity of these models to key assumptions, which made them poor tools for policy in her view. She boldly stated that: ‘… we could have constructed and estimated economic models all day long, and they would have been wrong. What we did do was think hard and look very carefully at all the data and evidence available and we used this to form our understanding and design the policy response’.

Not carefully thinking through epidemic-economy interactions is likely to have contributed to failures in UK government policy

As we will argue, forgoing the use of integrated epidemic-macroeconomic (‘epi-macro’) models with the argument that they are too sensitive to key assumptions is a non sequitur. This is especially true in the context of continuing to use both epidemiological models and macro models separately.

Macroeconomic models ignore the fact that economic behaviour – in particular ‘social consumption’ such as going to restaurants – affects how disease spreads. Epidemiological models ignore the fact that the epidemic then also affects economic behaviour: in particular, people may respond to increased virus prevalence through a ‘fear effect’ or ‘voluntary social distancing’ by lowering their social consumption.

We will also argue that by not carefully thinking through such interactions and the resulting policy trade-offs, the Treasury is likely to have contributed to failures in UK government policy: notably a very late first lockdown that was therefore longer than it needed to be; the ‘Eat Out to Help Out’ policy experiment, which aimed to boost the hospitality industry but ended up subsidising disease spread; and the very late and interrupted lockdown in the winter of 2020.

Informal thought processes are also models

Parameter sensitivity is just a brute fact of life, not one that can be avoided by some other mode of thinking. There is no policy strategy that does not involve the use of models, at least implicitly. The government may not have been conducting computer simulations of fully fleshed-out integrated models of the economy and epidemic. But it was using thought processes that speculated on how the economy and epidemic were going to evolve and interact, and how they would be affected by policies such as lockdowns.

These thought processes are also models. It is just that they are less transparent than formal models, and more susceptible to hidden biases or inconsistencies. The world is complex, with many different interrelated variables affecting each other. Governments have to represent this world somehow, with a view to affecting its future path through the use of different policy levers. There are just good and bad ways to represent it. In other words, setting aside formal models means relying instead on informal models.

The lockdown delays seemed to be motivated by wanting to save upfront economic costs. But this motivation did not appreciate that at least in a certain region of optimal policy, locking down achieves virus suppression, which ultimately helps the economy and health (see, for example, Giannitsarou et al, 2021).

This property of epi-macro systems is also evident in the empirical observation that once one controls for virus prevalence, the imposition of lockdowns reduces economic activity by very little or not at all (see, for example, Vlieghe, 2020; Andersen et al, 2020; and written evidence submitted by Mulheirn et al, 2020 to the Treasury Committee).

The Eat Out to Help Out policy seemed not to appreciate that the policy was encouraging and aggravating an externality – that is, it was generating more infections than would otherwise happen, by subsidising contacts through hospitality. This ultimately led to more mortality and also probably to less hospitality output (see, for example, Fetzer, 2021).

A better policy would have been to continue payments to the hospitality sector to encourage them to stay closed or to insist that meals must be consumed at home – ‘Stay Home to Help Out’. Alternatively, targeted financial transfers to workers and firms in the hospitality sector would have also avoided subsidising infection risk. To be fair to the Treasury, it was not alone in this error; for example, the Resolution Foundation suggested a similar policy.

Parameter sensitivity is not a licence to disregard modelling

The fact that integrated epidemic-economic modelling appeared to be sensitive to parameter choices would come as no surprise to anyone with expertise in the field, and is not a licence to disregard formal modelling. Biological models like those of epidemic dynamics often exhibit so-called ‘threshold’ phenomena, where small variations in parameters can cause radical changes in dynamics.

The most well-known of these is the now famous basic reproduction rate, which helps to determine whether infections will increase or decrease though its impact on the effective reproduction rate. Much of the government’s lockdown policy has been tied to this one number, so it is no stranger to tying important policy decisions to unknown parameters and quantities that have outsized influence on potential outcomes. Yet the Treasury instead shied away from confronting this delicate dependence on details and instead opted for broad-brush non-model analysis. This is a pity, for the fact that the models are sensitive to parameter choices actually teaches us something about the problems we are facing: namely, that it is extremely important to get the policy right and that more, not less, effort should go into getting the model and the parameters as right as possible.

Use of models in other policy institutions

In confronting other policy and analytical challenges, the OBR, the Bank of England and many other thinktanks and private sector bodies providing economic commentary on their own or others’ decisions routinely use macroeconomic models. These models are very sensitive to a lot of assumptions – about the interest sensitivity of demand, the fiscal space, how expectations are formed, how rational consumers and firms are, the degree of competition in product and labour markets, how labour supply responds to the real wage, and much else besides.

Such organisations take a view about these assumptions based on the evidence that they have, and take action or form their advice and commentary accordingly. There is no good reason why the Treasury should have chosen to do any differently in its modelling of Covid-19 and the economy.

In understanding how timid have been the Treasury’s attempts to get to grips with modelling the epidemic, it is instructive to consider the approach taken by SPI-M-O (the Scientific Pandemic Influenza Group on Modelling, Operational sub), which was responsible for SAGE’s epidemic modelling. This sub-group stated that:

‘HM Government requires estimates of the future epidemic that allow for short, medium, and longer-term planning for a range of operational considerations, including NHS capacity. It is challenging to model this with any degree of certainty as trajectories will be highly dependent on the timing and nature of policy decisions that are taken and the behaviour of individuals over the time range considered. It will also be affected by random fluctuations, which will become more significant when incidence is low. To reflect these fundamental uncertainties, it is important to consider a range of scenarios covering a reasonable set of assumptions.’

On a practical level, SPI-M-O produced several consensus statements, meant to aggregate and consolidate the analysis and research produced by a number of independent research groups: ‘Each SPI-M-O modelling group produce their own set of projections. These individual projections are combined to form a consensus and then reviewed by SPI-M-O and agreed by the Scientific Advisory Group for Emergencies (SAGE)’ (Imperial, Warwick, Faculty/NHS).

Unfortunately, no parallel approach was found on the economic side, and there appears to have been no attempt to procure any integrated modelling from outside experts, although the Treasury would have been in an excellent position to do so, as noted by members of SAGE.

In our view, the lockdown and hospitality subsidy errors could have been avoided by appreciating even relatively crude epi-macro models.

No one in UK government provided an integrated analysis of health-economy policy trade-offs

In a revealing recent interview, Rishi Sunak, now the prime minister and then the chancellor responsible for the government’s economic response to the pandemic, lamented the lack of consideration given to trade-offs in the analysis and policy recommendations emanating from SAGE. Yet SAGE was not tasked with, and indeed not equipped to, conduct such integrated economic-epidemic analysis, as emphasised recently by key member John Edmunds.

We know that SPI-M-O members explicitly disregarded any impact of lockdowns on the economy and that they were assured that such impact analysis would be carried out by the Treasury’s chief economic advisor. According to the SAGE minutes, ‘Policy makers will need to consider analysis of economic impacts and the associated harms alongside this epidemiological assessment. This work is underway under the auspices of the Chief Economist.’

Yet giving evidence to the Treasury Committee in November 2020, Clare Lombardelli stated that no such analysis was carried out, arguing that it was hard to do so and that the Treasury did not have the expertise.

As Rishi Sunak points out, there were indeed many trade-offs to consider, as was clear from the outset of the pandemic. Yet instead of systematically modelling and analysing these trade-offs, Treasury economists and the chancellor provided exclusively economic analysis, leaving the health side of the equation to the epidemic modellers in SAGE.

It was only at the prime minister’s desk that the different analyses were brought together and the trade-offs considered. Yet by that time, it was too late, for the analysis itself did nothing to consider those trade-offs in the first place.

On 15 March 2021, Matt Hancock, then secretary of state for health and social care, claimed that all evidence had been taken into account, but then went on to say that: ‘You have to balance all the different considerations. It's only at the prime minister's desk that all these different considerations come together... I'm responsible for the health aspects and then the huge economic response the chancellor is responsible for.’

All models are wrong, but that is not an excuse for discarding them

In her speech, Clare Lombardelli states that the epi-macro models would have been ‘wrong’. We agree with her on this point. In some sense, all models are wrong. They are necessarily abstractions that simplify reality, thereby missing some aspects of it. Some simplicity and inaccuracy (‘wrongness’) is necessary to make progress with gaining insight into the behaviour of the system in some ways that are salient to policy (for example, how costly lockdowns are for the economy).

The job of the analyst is to take a view of the plausible candidate set of models that best describe the economy-pandemic system, the distribution of the parameters within those models, and the distribution of the shocks that will hit them – and then to make recommendations accordingly. The evolution of the economy and pandemic will produce data that may invalidate all of these views.

But it is not logically coherent to discard the models as a consequence. As we have argued, there is no escape from models. The task is to set policy in light of what we know about how wrong they are, or are likely to be.

The speech by Clare Lombardelli was meant as an attempt at transparency, providing information to help us to evaluate the Treasury’s policies. But it actually avoids many of the most important questions: Which epi-macro models were used and to what end? Which assumptions were observed to be problematic? What evidence was used to try to narrow down the likely values for those assumptions?

These are questions that regular policy-making informed by models has to address and so we would expect them to be addressed here. What led to the abandonment of epi-macro models? With what exactly were they replaced? And how did this alternative reasoning (or in other words, these alternative models) inform the decisions actually taken? The material provided gives the impression of allowing us to take a look behind the scenes, but actually avoids all these important questions.

Why were there no objections from inside the Treasury?

The Treasury staff’s reasoning about the epidemic and economy was not necessarily to blame. It is possible that Eat Out to Help Out and the late lockdowns were undertaken in spite of, and not following, advice from teams using integrated epidemic-economic thinking.

But if this were the case, it would be surprising if those in charge of those efforts left their objections private forever. Given the stakes for the health and wealth of the rest of us, one may have in fact expected those objecting to resign; or at least to voice criticism, either at the time or subsequently. Given that Clare Lombardelli seems not to have voiced such criticism, it is reasonable to interpret her speech as defending government policy and explaining its successful underpinnings.

In the coming months, the UK Covid-19 Inquiry will look into a number of aspects of the crisis from a host of different perspectives, evaluating government policy and preparedness to learn lessons for the future. As we have seen, economic policy plays an integral role and it is important to get it right. We encourage the inquiry to take a detailed look at how the Treasury and the government more broadly formulated such policy and how economy-epidemic interactions were accounted for in decision-making.