Inflation past and present: how have we measured the rising cost of living?

Concerns about the cost of living and the need to measure it stretch back to ancient times. Two centuries of research on constructing price indices suggest that measuring the rate of inflation facing different households, sectors and regions is far from straightforward.

Inflation in the UK and elsewhere is rising. This increase in consumer prices is linked to soaring energy costs as a result of the war in Ukraine, as well as the considerable disruptions to global supply chains following the pandemic.

While the fundamental cause of the general increase in prices is debated (see King, 2022, and Capie, 2022, for a monetary interpretation), the rise in oil and gas prices is likely to squeeze real incomes in countries that are net importers of energy. This is resulting in what is now widely described as a cost of living crisis.

In the UK, real household disposable income per person – that is, income per person adjusted for the price of consumer goods – is predicted to fall by just under 2% in 2022, according to the Office for Budget Responsibility (OBR). This is a large fall in the context of the last 100 years and one that is typically only observed in large recessions and financial crises (see Figure 1).

Figure 1: Real household disposable income per person and inflation, 1920 to 2022

Panel A: Contributions of nominal income per person and inflation

Panel B: Real gross disposable income

Sources: Office for National Statistics (ONS); Sefton and Weale, 1995; Thomas and Dimsdale, 2017; OBR March 2022 Economic and Fiscal Outlook. The measure of inflation used here is based on the national accounts consumer expenditure deflator (CED). The 2022 observations are forecasts (shown in pale pink/blue).

Concerns about the cost of living and the need to measure it stretch back to ancient times. Two centuries of research on constructing price indices suggest that measuring the overall rate of inflation is far from straightforward.

So how did contemporaries attempt to measure inflation in the distant past? And how have economic historians attempted to reconstruct retrospective estimates of the cost of living using modern methods and techniques. This article looks at the issue in the context of UK economic history and recent developments in the construction of price indices.

Constructing a price index – what were the difficulties faced in the past?

There is a great deal of research on the appropriate construction of price indices (see Diewert, 1998; Johnson, 2015; Ralph et al, 2018). There are several key steps in constructing a modern-day price index and many difficult practical and conceptual issues arise for statistical agencies. So how did the commentators and statisticians of the past go about it and how successful were they?

Step 1: Which price index to measure?

For an open economy like the UK, there are several indices one might want to construct: for example, for consumer prices, producer prices, import and export prices, house prices and so on. No single measure of inflation is likely to serve all purposes.

A consumer price index – the focus of this article – typically attempts to measure the rate of inflation experienced by households. Such a measure is the basis of the UK’s inflation target that the government expects the Bank of England’s Monetary Policy Committee to achieve – currently set at 2%.

But a measure of consumer prices will include imported goods and when these increase relative to domestic firms’ prices, it means that consumer price inflation will move differently to that of, say, the GDP deflator, which is a common measure of the price of an economy’s output. (The GDP deflator is the price of a country’s output net of its imports, also known as the value-added deflator.)

Consumer prices also include taxes and distribution costs, which are passed on to households by producers and retailers. Before the 20^th century, data limitations largely restricted commentators to measures of producer or wholesale price inflation.

From the 18^th century onwards, these were increasingly available in newspapers and periodicals such as the London Gazette and The Economist. Indices derived from these prices often mixed domestic and imported goods, and the wholesale prices used could move very differently from retail prices.

Choices also arise in the coverage of a consumer price index. One could be constructed for the entire population or for subsets, such as for regions or different types of households (low-income and high-income, retired and non-retired, etc.).

Towards the end of the 19^th century there was a growing interest in the living standards of working households, in part due to the increasing industrial unrest in the lead-up to the First World War. By the early 20^th century, the UK Board of Trade had begun to attempt to measure the key elements of the cost of living faced by the ‘working classes’.

The imperative of the war saw the Board begin publishing the first ‘cost of living’ index, which combined prices from 5,500 retailers across 620 locations with weights from the Household Expenditure Survey of 1904.

After the Second World War, this developed into a more comprehensive index known as the retail price index or RPI, which, for many decades, was the reference measure of retail price inflation in the UK. The RPI was extended to all wage earners, but it excluded the top 4% of households by income and some of the poorest pensioner households given their very different spending patterns.

Today, the Office for National Statistics (ONS) provides a variety of measures of household inflation for the whole population and for different groups of consumers based on the consumer price index (CPI).

Step 2: Defining the concept – cost of goods versus cost of living?

The second challenge of any price index is to pin down the conceptual target – what exactly are we trying to measure?

Today, many official consumer price indices produced by statistical agencies are defined very precisely as cost of goods indices (COGIs). These attempt to measure the cost of a representative basket of goods typically purchased by households over time.

The composition of the basket and the weights applied to price changes are fixed for a period of time – typically a year for a monthly price index. But they are then updated periodically, reflecting updated spending patterns and new products, to provide what’s known as a chained index. A key challenge for statisticians, both now and in the past, is how to include new products in the index and how to incorporate increases in the quality of goods.

Many statistical agencies emphasise that the COGIs they construct are not necessarily the same thing as a theoretical cost of living index, which is an economic concept (derived from Konus, 1924). The latter has a precise definition: how has the cost of achieving a given living standard or utility changed over time, allowing for potential substitutability between goods.

A well-known result is that a fixed-base cost of goods index will, in general, overstate the rate of inflation relative to a true cost of living index. The cost of living concept, taken at face value, requires additional knowledge about consumers’ preferences and how they would respond to changes in prices.

This information is difficult or impossible for many statistical agencies to collect in a timely manner even today. Nevertheless, what are called ‘superlative’ price indices do exist – such as the Fisher ideal index (Diewert, 1976). These are practical to construct and consistent with the true cost of living index, provided that some basic assumptions about consumer behaviour hold.

But in the past, many contemporaries defined the cost of living as the cost to households of purchasing a set of essential goods and services deemed necessary for a subsistence or respectable standard of living. So when they referred to estimating the cost of living they, like modern statistical agencies, were in practice measuring a cost of goods index based on a basket of essentials.

Step 3: Sourcing the prices

The third challenge is sourcing the relevant price quotes for the goods and services in the basket. One of the first recorded attempts in the UK was at the start of the 18^th century by Bishop William Fleetwood, who was concerned with a fellowship at an Oxford college where the nominal terms had been unchanged since the mid-15^th century.

To explore how prices had changed in the long interval, he collected prices for a small set of goods and services, concluding that many had risen significantly, although he did not try to aggregate them to form an index.

Figure 2: Retrospective annual index of cost of living, monthly wheat prices and the monthly exchange rate on Hamburg

Panel A: Cost of living index

Panel B: Monthly wheat prices

Panel C: Hamburg shillings per £

Source: Allen, 2007; Gayer, Rostow and Schwartz, 1953; London Gazette

Before the mid-19^th century, statisticians interested in measuring the rate of inflation faced a trade-off. There was high frequency information on individual prices of goods such as wheat from the Corn Returns published weekly in the London Gazette (see Brunt and Cannon, 2013).

These data were timely and might vary alongside aggregate prices (see Figure 2, Panel A for a comparison with a retrospective estimate of workers’ cost of living between 1790 and 1821). But they could also be subject to idiosyncrasies such as poor harvests and global price pressures.

Exchange rates also provided a timely indicator. Indeed, the rates of exchange on Hamburg and Amsterdam were the focus of contemporaries involved in the Bullionist controversy, over whether paper notes should return to being convertible into gold on demand, during the French and Napoleonic Wars.

At that time, Britain was on a paper standard and the inflationary consequences were being debated. A falling rate of exchange was indicative of increasing prices but also subject to idiosyncratic influences (see Figure 2, Panel C).

In contrast, there was more comprehensive but infrequent information on a broader set of prices (such as from Evelyn, 1798, and Young, 1812). These could give a clearer signal of overall inflation but might also be long out of date.

It was not until the second half of the 19^th century that statisticians had a better understanding of inflation in real time. The Economist, The Statist and The Times also regularly reported (unweighted or crudely weighted) wholesale or producer price indices (based on the work of Jevons, 1865, and Sauerbeck, 1886).

Better and more timely information on retail prices developed throughout the 20^th century, especially after the Second World War. Today, a sample of hundreds of thousands of goods and services prices is collected each month.

Step 4: Aggregating the prices into an index

The fourth and final step is the method of bringing together and averaging price quotes into an overall index. In principle, this seems simple: you just weight the prices of each good by their importance in expenditure.

But when collecting price quotes for a single item or at very low levels of aggregation – known as the elementary level – there are typically no estimates of quantities or expenditure shares, and statisticians need to use unweighted averages. Even when quantities or expenditures at higher levels of aggregation are available, there are a number of methods of weighted averaging that in principle could be used. These issues were increasingly confronted by commentators and statisticians from the 18^th century onwards.

On the use of unweighted averages, major breakthroughs were made by Dutot (1738), Carli (1764) and Jevons (1863). In modern indices, Dutot and Jevons have generally been preferred to Carli at the elementary level of aggregation, based on a number of axiomatic, economic and statistical considerations.

Such considerations led to the discontinuation of RPI as a national statistic in the UK in 2013. Now, CPI and its variant CPIH (which includes the costs of owner occupied housing) are more widely used as they are based exclusively on Jevons and Dutot averaging.

Further conceptual and practical questions developed during the 19^th century on how to source and use the information on expenditure shares to deliver best on the concept being measured. For example, should one use expenditures shares at the beginning of the period, the end of the period or some optimal average of the two?

The important theoretical developments were made by Young (1812), Lowe (1823), Laspeyres (1871), Paasche (1874), Fisher (1921), Divisia (1926) and Tornqvist (1936). It was not until relatively recently that many of the axiomatic, economic and statistical concepts were developed and refined for use by statistical agencies in the construction of modern indices – see Diewert (1988).

On the practical side, statisticians in the 19^th century typically only had limited information on household budgets. What they had came from investigations by social commentators and parliamentary inquiries, which were limited in what they could feasibly collect and process.

Indeed, in the aftermath of the Second World War, the Board of Trade cost of living index was still based on the 1904 survey of households, albeit with minor updates to 1914. As a result, an interim index of retail prices began in 1947, where the weights were taken from a survey in 1938. RPI, which was launched in 1956, used a set of post-war weights that were regularly updated from more comprehensive surveys. The descendants of these are still in use today, for example, the Living Costs and Food Survey (LCF).

The improved coverage of surveys led to additional questions (Prais, 1998). For example, should one use aggregate expenditure shares for each type of good, which implicitly places more weight on the expenditure of richer households (a plutocratic weighting)? Or should one make use of survey-level data and measure the expenditure shares of each household individually and then weight them equally (a democratic index)? This idea has been revisited recently (Aitken and Weale, 2020) and the ONS has begun releasing experimental democratic indices of prices.

How have economic historians measured inflation retrospectively?

Many economic historians have gone back and revisited the prices and budgets collected by contemporaries, including tapping new and previously underused sources to calculate inflation retrospectively. Thanks to their efforts, we now have a relatively continuous record of inflation back to the early 13^th century (see Figure 3).

Figure 3: UK consumer price level, 1086 to 2021

Figure 4: Annual rate of UK consumer price inflation (including housing costs), 1209 to 2021

Sources: From 1209, the measure of consumer prices used in both charts includes housing costs (actual and owner occupiers’ rent) and is based on the aggregate expenditure weights of all households. From 1209 to 1830, it is based on the domestic expenditure deflator of Clark, 2015; from 1830 to 1949, it uses the consumers’ expenditure deflator based on Deane, 1968; Feinstein, 1972; and Sefton and Weale, 1995; and from 1949, it uses the long-run CPIH index recently produced by the Office for National Statistics. Between 1086 and 1209, the price index is a trend measure based on a more limited set of commodities based on Barratt, 1996, 2001, and Mayhew, 2013. Alternative measures of consumer prices such as the CPI and those based on the expenditure weights of the ‘working classes’ or wage earners can be found in Thomas and Dimsdale, 2017.

In many ways, a retrospective analysis is as challenging as contemporary estimation given the broader range of sources to choose from and modern techniques to apply. For example, a key issue was the use of prices collected from institutional sources such as the records of manors, Oxford and Cambridge colleges, hospitals and local government, which might have been subject to stickiness arising from fixed price contracts (Ashton, 1949).

The use of raw material and wholesale prices as a proxy for retail prices and the London-centric nature of many prices also raised concerns (Ashton, 1949). Other research was able to demonstrate that these problems were not as large as feared (Feinstein, 1995, 1998). But the price of items such as clothing proved just as difficult to measure as they do today, given the range and quality of goods available.

An important result of this retrospective analysis is that workers’ real wages grew relatively slowly during the Industrial Revolution – what’s referred to as ‘Engel’s Pause’ – and that it was almost a century before the majority of the working class obtained any economic benefits (Feinstein, 1995, 1998; Allen, 2007); though more optimistic estimates have been constructed (Clark, 2005).

Another good example of the use of modern methods is a recalculation of the early 20^th century cost of living index in interwar Britain (Gazeley, 1994). This work updated the expenditure weights from 1914 to 1938 to capture changes in consumption more accurately. It indicated lower inflation during the First World War and lower deflation during the slump of the 1930s than previously thought.

Conclusions

The gradual progress on measurement over the centuries by contemporaries and economic historians has improved our understanding of inflation over the long run. Figure 3 presents almost a millennium of price data in the UK and Figure 4 shows annual inflation from the beginning of the 13^th century, using the most consistent available price measures that cover all consumers and include the cost of housing.

Three things stand out. There have been several phases where there has been no trend increase in prices, interspersed by periods of large secular increases such as the early 13^th century, the 16^th century, the end of the 18^th century and since the start of the 20^th century.

Second, large bursts of inflation have often been associated with internal and external wars, which are then often followed by periods of deflation (with the notable exception of the Second World War).

Third, the positive and persistent rate of inflation that we have observed since 1940 contrasts with the volatility of prices exhibited for centuries before that, where for many periods, deflation was equally as likely as inflation.

That persistence could reflect a fundamental shift in the willingness of society to accept a non-zero and persistent rate of inflation as the optimal state of affairs, given the potential costs of deflation (Groth and Westaway, 2009). But equally, it could reflect difficult measurement issues, such as the improvement in the quality of goods over time, which means that true price stability is associated with positive measured inflation.

The anticipated increase in the cost of living over the next few years, with inflation rates expected to differ across sectors and household groups, is likely to bring many of these deeper questions back into sharp relief.