Wednesday, August 27, 2008

Where there should be outrage, there is silence

This chart should produce howls of outrage. It compares the retail price index (RPI) to the consumer price index (CPI) since 1997, when the Bank of England gained its independence.

The chart illustrates how the RPI measure of inflation has outstripped the CPI counterpart. Today, the RPI is 13 percent higher than the CPI compared to 1997. Why should this be? After all, both measures are supposed to track the same phenomenon - inflation.

There are differences between the two measures which, in a statistical sense, explain this 13 percent divergence. However, a closer examination of these differences reveal that the RPI is a far superior measure of price changes. So what are these differences?

The basket used to calculate the CPI is a much more restricted measure of consumer expenditure. The CPI excludes council tax and mortgage interest payments. Both of these exclusions are highly dubious. The council tax covers essential services such as waste disposal, libraries, and education. If we have to pay more for them because council tax payments have gone up, then it is obvious that we are worse off.

The exclusion of mortgage interest payments is also nonsense. These interest payments represent the cost of home ownership, and if interest payments go up, it is obvious that households are becoming worse off.

The CPI also includes certain financial service charges. In this area, the CPI does capture something that is missing in the RPI, but the loss of information isn't that great since these charges are typically only a small component of overall household expenditure.

The RPI also use a more reliable data source when constructing the actual consumption basket. The RPI weights comes from the expenditure and food survey. In contrast, the CPI weights come from the household expenditure survey, which is primarily used for estimating the National accounts.

The two indices also use different mathematical formulas for combining the price is collected for each item in the basket. This is quite a complex issue, but the bottom line is fairly straightforward. In practice, the CPI formula ensures that the average price for each item is always lower than or equal to the average price for the same item within the RPI. In other words, CPI prices are always lower than or equal to RPI prices. If only this were true in real life.

The RPI tries to track the expenditure of the average household. It therefore sensibly excludes households in the top 4% of the income bracket. It also excludes some pensioners who tend to have highly atypical expenditure patterns. The CPI includes these groups.

As an aside, both measures of inflation are subject to outrageous Hedonic price adjustments. Actual inflation rates are reduced on the basis of arbitrary adjustments that supposedly capture the improved quality of goods. These changes are extremely widespread and have greatly undermined the validity of recent price data.

These differences may seem like an arcane issue that should only concern professional statisticians. Unfortunately, this issue really matters to all of us. The government chose CPI as the inflation target for the Bank of England. Since it produces consistently lower inflation rates, this target is a fix. It is therefore all the more shocking and worrying that the central bank have been unable to meet this relaxed and distorted target.

Measures of inflation based on the CPI are totally unreliable. We all know this to be true. The proof is hammered home every time we reach the supermarket tills and open up our purses. Prices are rising far faster than the headline rate suggests. This should be no surprise, the CPI has been designed to understate the true extent of inflation.


  1. No outrage?

    Maybe you should go to the pub!

  2. Nu Lab's biggest lie.

    Low inflation.

    House inflation meant our children and young workers getting poorer - the hidden story of Brown's 'boom' times.

  3. Maybe you are allowing your protest to be defined by others. Neither the CPI or the RPI are measures of inflation. They are measures of prices.
    Inflation is an increase in money supply.
    Look at credit expansion/contraction. You seem to state that CPI is totally unreliable yet continue to refer to CPI in your blog. It is -only- price measurement. Nothing to do with the inflation or deflation in Sterling.
    Great blog as usual.

  4. Well for more fun and interesting graphs, plot the Price Index (not the inflation rate) against M0. Do it as 2 separate graphs the beginning til 1992 and then 1992 onwards. (When you do the graph you'll see why).

    Put linear interpolations in both sections - look at the correlation.

  5. RPIY; RPIX; CPI...

    The latest lie is 'core CPI'...

    "The core CPI index excludes goods with high price volatility, such as food and energy..." (Wikipedia)

    Every time something gets pricey, they move to a new measure which doesn't include it...

    And if you can't exclude it, then try "Hedonic Substitution"

    If they can't afford beef, they'll eat chicken; if they can't afford chicken, they'll eat fish; if they can't afford...etc. anything... they will eat grass... paper...

    B. in C.

  6. B. in C.

    I know the Americans mess around with the weights to capture the substitution effects that you highlighted, I need to see whether the ONS do the same thing. However, the ONS do use hedonic quality adjustments, which are, in my view deeply dishonest.

    Core inflation is just a joke. Recently, I have noticed that the BoE have started to de-emphasize it. For example they don't actually publish a series. At least, I have never found a public series.


  7. Anonymous,

    It sounds like a treasure hunt, I might try it. I am going to guess that I will find no relationship post-1992.


  8. The biggest con was no longer linking the pound / dollar to any real commodity - such as gold.

    This means they have the gold in the central vaults, and the citizens get the paper promise... which tends to diminish in value, depending on how many promises are in circulation.

    Doesn't seem a fair exchange, somehow!

    I see the promise is still there: I promise to pay the bearer on demand twenty pounds...

    Twenty pounds of what? Twenty pounds of promises, I suppose.

    Hold on - that doesn't mean they promise me anything at all!!!!!

    B. in C.

  9. Using gold as a currency is barbaric. What we need is something like Keynes' bancor.

  10. "It sounds like a treasure hunt, I might try it. I am going to guess that I will find no relationship post-1992."

    Oh there is. I'll have to dig out my old spreadsheets, but from memory, it's a pretty good one too. The most revealing part is the equation of the interpolated line in the form of y=mx+c - especially when you figure out what m and c actually mean.

    I'll just leave the poser: If there is no money, can there be any prices?

  11. barbaric how?

    Because any time you save you know that your value is stored? Any time you borrow you do so at a low rate and with no inflation premium? Because governments cannot print money to "balance" a budget? Because you can take your money out of a country anytime the government gets punitive? Because goods and services get cheaper as productivity increases? Because you have an incentive to save and thus increase the pool of capital available to business?


  12. Alice,

    You suggest that “CPI has been designed to understate the true extent of inflation.” By whom was it designed with this aim in mind?

    CPI was designed by Eurostat in consultation with the national statistical offices of the EU member states. Moreover it was designed in the mid-1990s with the intention of being applicable on a European wide basis. Are you really suggesting that 15 years ago Eurostat deliberately designed CPI so that a future Labour Government could “fiddle” the inflation figures?

    Housing costs, particularly mortgage costs are excluded for two reasons:

    1. CPI is intended primarily as an inflation targeting measure. Such measures usually do exclude interest payments – otherwise the target measure is raised when the central bank raises interest rates. This is why the current Government and its Conservative predecessor both used RPIX rather than RPI as their IT measure.

    2. The housing market, particularly the rental market in many European states is far from being a free market. It is distorted by varying degrees of subsidy and regulation. Eurostat excluded housing costs from CPI because they were unable to come up with a methodology which could be applied Europe-wide. Since the introduction of the CPI measure, they have continued working on this problem and had hoped to include some measure of housing costs by 2010. My understanding is that this date has since slipped.

    You also seem highly dubious about hedonistic adjustments. Again, I do not see why. My first car cost 14K in 1990 (although I didn’t pay the new price). My current car cost 15.6k last year. It has a full leather interior, electrically adjusted seats, electrically heated seats, parking censors, air conditioning, climate control, ABS, cruise control, 12 air bags, a six CD player with 12 speakers, and a ten year anti-perforation warranty. My first car had none of these and started to rust after 2 years (it was a Rover). If I’d wanted to bring it up to the specification of my current car I would have had to spend at least 28K and it would still have been a pile of Longbridge-built rubbish! So, if you don’t take quality improvements into account it’s impossible to make sensible price comparisons over time, as your not comparing like with like.

    Young Mark

  13. yeah, yeah, yeah

  14. Alice,

    I’m afraid you’re wrong about the difference between HCPI and CPI. They are the same thing. It’s simply that the UK version of HCPI has been referred to as CPI since 2003. Obviously the weights will vary across the various member states of the EU, but the methodology doesn’t. In fact, Eurostat monitors members states’ statistical services to ensure they are complying with the HICP regulations.

    You say that RPI and CPI are measuring the same thing. This is only partly true, as they are doing so for different purposes. CPI is used primarily for inflation targeting, whilst RPI is used for measuring the cost of living. So when the Government indexes benefits, it does so using RPI, not CPI. Similarly index-linked savings certificates use RPI, not CPI. For wage bargaining purposes, both sides of industry use RPI, not CPI. Your accusation of fiddling might stand up if the Government had replaced RPI with CPI, or was downgrading its importance in some way, or attempting to hide the fact that mortgage interest payments are excluded from CPI. The fact is that it hasn’t done any of these things. Even when the Government started using CPI for IT purposes, it was simply a switch between two measures, both of which excluded mortgage interest payments.

    You say “Hedonistic adjustments only go in one direction” and that you’ve only ever seen two “one for digital cameras, the other for computers”. That’s because the ONS uses other techniques for making the quality adjustments for the great majority of goods. For the most part they use producer costs and option costs. Let’s say a car gets air-conditioning as standard and the price rises by £100. The ONS ask the manufacturer how much the new feature costs to make. If the manufacturer can’t supply an accurate figure, the ONS looks at how much the new feature costs as an option. One final word on quality adjustments, before we slide into a quagmire of stodgy white bread and rusting Austin Montego cars: within CPI, there are internationally agreed procedures for adjusting prices to reflect improved quality. RPI also makes quality adjustments. It, on the other hand, is a wholly British affair.

    Incidentally, if Eurostat do ever devise a method of including housing costs, my guess is that we will end up with two versions of CPI: one measure excluding housing costs, to be used for inflation targeting purposes, the other with such costs included, to be used as a measure of the rising cost of living.

    Young Mark

  15. Young Mark

    I understand that they comprise of the same subindices. I was referring to the fact that the weights are different. So the HCPI and the CPI give different index values. In the sense that they have different weights, they are not the same thing.

    The key point here is whether housing costs should be in the index. The answer is, of course, yes. In what sense is an increase in housing costs not inflation?

    As for the hedonic adjustments being "internationally agreed", so what? I don't agree with them, and I think I have made strong valid arguments why this adjustment is flawed. As for the alternative methods, hedonic adjustments are becoming very popular. My mind was made up on this issue when I went through two of two of the ONS notes on these issues in EMLR. Once I understood what the ONS were doing, I was appalled and horrified.

    Besides, you would agree that life doesn't always improve and quality adjustments should go both ways. We can agree that this makes sense. I am sure you can accept that particular weakness in the process.

    Mark, you haven't addressed my key point. Why have the two indices systematically diverged. They are both supposed to measure the same thing. A 13 percent divergence is huge. It simply doesn't make sense.

    You say that the two indices are used for different purposes and that the CPI is used for inflation targeting purposes. Why would you use an inflation measure that systematically understates the effect on the cost of living? That is bound to bring the whole inflation targetting exercise into disrepute. In short, I don't recognise this distinction between cost of living adjustment purposes and inflation targetting purposes.

    Again, thank you for your comments. I enjoyed reading them.


  16. "Let’s say a car gets air-conditioning as standard and the price rises by £100. The ONS ask the manufacturer how much the new feature costs to make."

    And with a Government looking ever vigilantly for "wind-fall" tax opportunities, the manufacturers all fall over each other to say "The cost is really small and we make shed-loads of money on this..."

    And the inquiring mind will wonder if that cost is in itself inflation adjusted.

    Mind you, since wheat costs more and more to make every year, it must be so much better than wheat of even five years ago.

  17. The more you find out about measuring inflation, the more it looks like a conceptual morass.

    I have a better idea. Forget about the CPI, target the money supply.

  18. "I have a better idea. Forget about the CPI, target the money supply."

    I've been doing a lot of "deflate by M0 rather than RPI" and it does correspond a lot better to what happens on the ground than by using RPI.

    Strangely, it seems "more accurate" than even M4, and that I can't explain.

  19. Alice,

    You say I haven’t addressed the reason for CPI/RPI divergence. The divergence arises for two reasons (as I’m sure you know): coverage effects and formula effects.

    The main coverage effect arises from the fact that various housing-related costs (depreciation, estate agent’s fees, insurance etc) are excluded from CPI. Because house prices have been rising rapidly over the last few years, the coverage effect has been exaggerated. When the switch was made to CPI in 2003, the ONS estimated that the coverage effect was responsible for about 0.5% per annum of the disparity between CPI and RPI (it has since risen). Over the next few years house prices will drop and the disparity will diminish. I don’t deny that the coverage effect exists. I do object however when people suggest that the Government or the ONS is somehow “fiddling” the CPI figure, when no evidence exists to support such a charge.

    The formula effect arises from the fact that individual prices are aggregated using the geometric mean in CPI, but the arithmetic mean in RPI. Using the geometric mean implies a substitution effect, whereas the arithmetic mean implies that people continue buying the same quantities of goods when their relative price changes. I would suggest that the former more closely represents consumer behaviour. In other words, RPI actually overstates the true rate of inflation. The ONS once again estimates the effect to give rise to a fairly stable 0.5% divergence between the two indices.

    You ask why mortgage interest payments should be excluded from CPI, simply because it is an IT measure. I would suggest it is because the BoE’s only weapon in the fight against inflation is its control of interest rates. It would be perverse in the extreme if inflation rose as a direct response of the Bank’s attempt to control it. This is why the Conservatives used RPIX, not RPI as their inflation target and why the present Government carried on using RPIX until its replacement with CPI in 2003.

    So long as people are aware of the differences in both composition and purpose between CPI and RPI, I see no reason why the IT regime should be brought into disrepute. Indeed, compared with previous attempts to control inflation, such as prices and incomes policies, monetary targeting and ERM membership, the current IT regime has a positively glowing reputation and unparalleled longevity.

    Young Mark

    P.S. Keep up the good work with your blog. It’s vastly superior and more entertaining than the drivel one reads in the mainstream press.

  20. Young Mark,

    My real concern is how the CPI fits into the monetary framework, which is inflation targetting. I think that the CPI, because it expressly excludes housing, and makes those hedonic changes, allows the central bank to generate huge amounts of liquidty. This generates asset bubbles, while the BoE hides behind these really low CPI inflation rates. We have ended up with a huge inflationary bubble, whatever the CPI might tell us. Now, that it is bursting, we will have to deal with these policy errors that the CPI obscured.

    I might do another post on hedonic price adjustments. I really do think that these adjustments are far to obscure and arbitrary.

    Thanks for the kind words about the blog. I've enjoyed responding to your comments, and I have learnt something.


  21. Alice,

    A fascinating debate, which I think we’ve probably exhausted for the moment.

    I agree that there has been a bubble in asset prices, most particularly house prices. The interesting question for me is “why did some asset prices inflate so much, whilst others, notably shares, didn’t”. Maybe this could be a topic for the future.

    Have a good weekend.

    Young Mark.