In new work, the Economics Observatory data team demonstrates how AI tools can be used to collect, clean and summarise price data. The automated food CPI generated is available faster, with broader coverage and at lower cost than human-collected data. It could be a complement to official statistics.
The UK is emerging from a two-year bout of inflation that hit the economy suddenly and unexpectedly. Over the same period, the use of artificial intelligence (AI), in the form of large language models (LLMs), has exploded. Could AI be used to build early warning systems that alert policy-makers to emerging price pressures?
The answer, in short, is yes. In a working paper released today, and presented at the Economic Statistics Centre of Excellence (ESCoE) conference on economic measurement at the University of Manchester, we introduce a new and growing dataset of daily grocery prices to help answer these questions.
We collect 25 million prices between July 2023 and April 2024 from seven supermarkets. The retailers represent 80% of UK grocery sales. We use the prices to examine the sources of food inflation, to study price rigidity at the firm level, and to build a new automatic consumer prices index or 'autoCPI', which is available daily.
Our paper shows how fast and efficient automated and distributed programmes can be. Our 100,000 daily prices are also rich, including loyalty discounts, regional availability and nutritional information. Automated systems are also fast: our prices are collected, cleaned and categorised with a lag of around six hours.
Using AI is like having an army of research assistants on our team. (The image on this page is also made by AI, in this case, OpenAI’s image generator, Dall-E).
The sheer scale of our data presents us with a daunting task: classification. Due to the huge ranges of products that UK supermarkets stock, we have almost 150,000 shop-product pairs to parse and categorise so that other useful economic data – for example, expenditure weights or trade exposures – may be merged in. To do this, we adopt a three-stage filter using the GPT-4 model produced by OpenAI.
Price data have several important potential uses. We establish a range of new daily facts on the frequency, size and timing of price changes. These questions – on 'price rigidity' – are core to the operation of macroeconomic models.
By further parsing our data for the origins of goods, we may be better able to establish the impacts of trade negotiations. The map below, from a forthcoming paper by Josh Hellings, shows the origins of UK supermarket items. Italian food, it seems, is what Britons like to eat most.
Figure 1: Products by LLM-identified origin
Source: Authors
Today’s main contribution is the food autoCPI. We track the official UK CPI basket by adopting tight matching criteria: this measure may be useful for policy-makers in periods when prices are moving quickly, or when it is unclear the extent to which price shocks are being passed through.
We also create a broader measure which allows for substitutes. Our measures are complements, rather than substitutes, to the official CPI; we hope that they will help strengthen the work being done to transform and improve UK statistics.
