How should we measure the impact of Covid-19 policies on our wellbeing?

Wellbeing is improved when people live longer, healthier lives. Measuring changes in life experience as well as life expectancy will enable richer evaluation of policies.

Policy responses to Covid-19 have been remarkably similar around the world – from lockdowns to mask-wearing mandates. In one way or another, non-medical policies designed to control the spread of the virus – known as non-pharmaceutical interventions (NPIs) – have been deployed to save as many lives as possible, almost irrespective of the costs.

During the January 2021 peak of the pandemic, over 1,000 people a day were dying from Covid-19. The lockdown measures imposed were therefore designed to save as many lives as possible – or limit the number of deaths. But these policies can also have unintended consequences: from delaying other crucial medial procedures to increasing financial strains or loneliness among individuals.

How long the lives that are saved will be and how well they will be lived have not featured prominently in policy discussion or media coverage.

It is impossible for anyone to determine what the outcome of different policy responses would have been. But governments can at least consider the impact on life expectancy and life experience for those directly affected by policies such as social distancing, mask-wearing and work from home directives.

These mandated NPIs, particularly lockdowns, have led to substantial decreases in wellbeing and mental health, as well as increases in loneliness (Office for National Statistics, ONS, 2021). Young people have been hit particularly hard, and those with home-schooling duties have also reported especially low emotional wellbeing (Banks and Xu, 2020; Lades et al, 2020).

One possible reason that these effects have not been considered to the same extent as physical health outcomes is that they cannot be quantified in the same way that mortality can. While it is relatively easy to count cases, hospitalisations, deaths and, recently, vaccinations, the toll on mental health, loneliness, and emotional wellbeing is much harder to quantify and it may also develop over time. As a result, there have been only limited attempts to use changes in wellbeing in the decision-making process.

Although it is challenging to do, changes in wellbeing can be captured. In fact, even if we look only at life expectancies and ignore life experiences completely, we might reach different conclusions about the effectiveness of the most restrictive rules, such as lockdowns. Alternatively, we may reach the same conclusions but with greater confidence in the decisions made.

From lives to life years

One way to analyse the effect of a policy would be to consider changes to the number of lives or, in other words, how many lives are saved by the intervention. Looking at life years, a measure of extending an individual’s life by a year or more, enables policy-makers to consider the lives of the rest of the population – including quality of life – over a longer timeframe.

In the case of Covid-19, taking a life years perspective would capture not just how many deaths were prevented by lockdown and other NPIs, but would also measure their effect on the life expectancies of others in the population.

For example, delays to operations or other health procedures during the pandemic are likely to have negatively affected people with existing conditions, such as cancer. It is estimated that delayed treatment of specific cancers will result in between 3,300 and 3,600 excess deaths in England within the next five years (Maringe et al, 2020). There were nearly ten million people waiting for surgical procedures in the UK in February 2021 during the height of the third lockdown, up from about four million prior to the pandemic (The Lancet Rheumatology, 2021).

Making calculations that take account of indirect or knock-on effects of a policy is routine in other areas of the public sector. For example, Public Health England’s ‘return on investment tool’ helps calculate the relationships between changes in employment on the one hand and resulting changes in healthcare and crime costs on the other (Public Health England, 2021).

Looking at life years, as opposed to lives only, would represent a significant step forward. It does not require large, revolutionary change to using a completely different concept of wellbeing, but it would help measure the longer-term ‘lifetime effects’ of particular interventions.

From life years to quality-adjusted life years

The quality-adjusted life year (or QALY) is an established measure that has been around for decades in medical decision-making to express changes in length of life and quality of life resulting from a particular intervention in a single index number.

By combining length of life and quality of life in a measure, we can account for how many life years a person lives as well as how these years are actually experienced by that person. An intervention that increases life years but decreases quality of life may, therefore, take on the same value as an intervention that decreases life years but increases quality of life. This allows policy-makers to make meaningful choices that consider both the quantity and quality of life.

The key question is how to calculate the ‘Q’ in the QALY. Quality-of-life values are obtained by asking individuals to consider hypothetically how they would value living longer if this meant living with particular quality-of-life-limiting health problems, such as reduced mobility and self-care, pain and discomfort, or poor mental health (Dolan, 1997). By making a choice between life years and improvements in each of these health states, the value that people attach to a particular health state can be located on a scale between 0 for death and 1 for full health.

It is also possible to use QALYs to calculate the ‘shadow price’ of health – in other words, improvements to health that could be made from an additional pound invested in the healthcare system. Sorting treatments in terms of how much QALY improvement per cost they generate, and implementing all treatments until, eventually, the health budget runs out, generates this shadow price of health, measured in terms of QALYs.

There are now established ‘price lists’ for different health states that can be readily used for policy appraisal. In the UK, the ‘threshold value’ (that is, the price that a health intervention should maximally cost to achieve a one-QALY increase in health) is around £30,000 (National Institute for Health and Care Excellence, NICE, 2013), rising to about £50,000 for end-of-life care (Collins and Latimer, 2013), in order to be deemed cost-effective.

Given the widespread use of QALYs in medical decision-making in the UK and elsewhere, it is surprising that they have not featured prominently in appraisals of pandemic response policies. This could be for various reasons, including that lives saved or lost are more salient, imminent, and a sensitive issue, or that perspectives from social science and health economics were missing in the pandemic decision-making process.

Using QALY values to look at the impact of lockdown in the UK, using the £30,000 per QALY threshold, one study did find that the benefits of easing lockdown outweigh the costs by a factor of about seven (Miles et al, 2020). This is probably even underestimating the effect, as their focus is mostly on lives lost and decreased output and not on mental health or other ‘hidden’ effects of lockdown.

Using QALYs as a measure of wellbeing allows policy-makers to account for lives as well as life experiences in evaluating policies. But we also know that people are not particularly good at predicting how certain health states will actually affect their quality of life.

They may overestimate the duration of the impact that a change in health may have on their lives, underestimate their capacity to adapt or focus too closely on specific details without seeing the broader picture (Dolan and Kahneman, 2008). For example, there is evidence that at the onset of physical disability, people overestimate the burden their disability will have on them in the future by underestimating their capacity to adapt to the condition in their daily lives (Odermatt and Stutzer, 2019).

What about using QALYs to measure policy effects beyond direct health outcomes? These could include being separated from a loved one for a long period due to travel restrictions or stress from concerns about jobs and income, which may have indirect effects on wellbeing. Although such scenarios may affect some of the health indicators underlying QALY values, they may end up being captured imperfectly, tilting policy appraisals in favour of policies that target health outcomes rather than wider wellbeing.

But being with loved ones and the ability to pay the bills are high priorities for many people. They could therefore be undervalued in the QALY approach, if not immediately health-related.

From quality-adjusted life years to wellbeing-adjusted life years

Consequently, including a measure of people’s wellbeing is important. This can be done by using people’s self-reports of how they are feeling – assessments of their own, subjective wellbeing – to adjust life years by their quality.

Wellbeing measures include people’s overall life evaluation on a day-to-day basis. Unlike QALYs, people are not asked to make hypothetical decisions, such as those mentioned above, but are surveyed as they go about their lives. This makes the measures less prone to bias resulting from the hypothetical trade-offs and allows researchers to capture important behavioural elements, such as people’s capacity to adapt to changing circumstances in their lives (Frederick and Loewenstein, 1999). Wellbeing measures allow people to express changes in their quality of life linked to health, education, loneliness and social relationships.

Researchers have estimated the wellbeing costs facing people in the UK in April 2020 using a large survey with nationally representative quotas (Fujiwara et al, 2020). Across a range of indicators, levels of wellbeing were substantially worse than they were a year earlier. The critical question here is whether it is possible to say how much of the effect is due to Covid-19 and how much is due to the policy response – that is, lockdowns and social distancing.

By separating out those who are unaffected by Covid-19 (in themselves, those close to them and psychologically) from those for whom Covid-19 had a direct effect, the study estimates that about one-third of the overall impact was a result of health-related costs and two-thirds because of the economic and social impacts on people from lockdowns.

Combining wellbeing with life years yields a wellbeing-adjusted life year measure, or ‘WELLBY’. (For an illustrative example of how WELLBYs can be used in a pandemic, see De Neve et al, 2020). This measure enables researchers and policy-makers to look at the impacts of a given policy on both life years and quality of life using a single index number.

While the wellbeing impacts of pandemic responses on various population groups have been well documented, none of these measures have featured directly in government appraisals of these responses. But they could be used to capture the effects of pandemic responses on mental health, loneliness or loss of education due to school closures, and put them on par with impacts on loss of life, income, unemployment and so on, to arrive at a balanced assessment of policies (as has been done, for example, in Layard et al, 2020).

Wellbeing is improved when people live longer and better lives. But during the pandemic, the evaluation framework of policies has not fully accounted for the ‘hidden’ effects that interventions have had on people’s day-to-day lives.

Adopting an approach that uses QALYs, or even WELLBYs, could help policy-makers make decisions that better capture their full effects. A full appraisal and balanced assessment of any intervention requires us to capture and quantify all of its possible short- and long-term ripple effects, not just the most immediate and obvious splash it creates.