trainingreality helping people work better with people
return on training
Please press (at least!) one of these.
It costs you nothing, and (possibly) helps us spread the word!

Spurious certainties

Monday, 30 November 2009

Black or white. Yes or no. Specific. Measurable. Defined. All very useful things, in their place. All worth considering, at times. But they shouldn’t be allowed to define our approach in its entirety, because life simply isn’t as simple as that. We need to be comfortable with shades of grey in management.

On most scales, I’d consider myself to be a pretty measurement-based person. My degree was in economics (and, frightening though it is to admit, I rather enjoyed the statistics module!), and a significant chunk of my career involved work in the finance function, and included training with CIMA. So I understand the simple pleasures of accurate, measurable things, but I do get slightly frustrated when people aim to measure the immeasurable.

ROI (return on investment) is one of those things that is often used in a glib fashion to reassure people that they are spending their money wisely, and that the benefits of any expenditure significantly outweigh the actual expenditure. Of course, as a trainer, I’d love to be able to confirm to all of my clients that this is the case, but I’m honest enough to admit that (generally) I can’t - and others (generally) can’t either.

Below, I’m going to offer two training scenarios - broadly hypothetical, but based in reality, and show how it is (generally) reasonable to measure ROI in one case, and (generally) unreasonable to measure it in the other case.

The production line team

In most decent-sized manufacturing facilities these days, significant amounts of real-time data is available on team performance. In one part of my career, some of the most vital measures of performance were as follows:

Different production lines and different teams performed differently, and data was available on a continuous basis, cut pretty much whichever way you wanted.

I would argue that, for a shift team running a production line, it would be relatively easy (although not perfect) to measure the impact of training. There would be enough control data (un-trained teams, history, other lines and so on), and enough rapidly measurable variables to say with reasonable certainty what the effect of training has been.

The sales team

Initial assumptions about sales teams tend to focus on how easily their results can be measured, and, having worked on sales teams myself, I wouldn’t begin to deny this. However, simply because something is easy to measure doesn’t mean that it is easy to understand what affects it. It is easy to measure rainfall in my garden, but pretty tricky to perfectly understand the causes of it.

Sales teams, depending on the sophistication of the organisation, tend to be measured on (a combination of) sales volume, sales value, gross profit, or margin. These, as with the production measures, can be collected frequently, and assigned to specific teams of people. However, there are problems.

To say that a training course followed by an increase in volume/turnover/profitability means that the training course has worked, and that you can work out the ROI of the investment in training by comparing it to the additional profits generated is, at best, misleading. In all but the simplest of cases, there are too many other factors that impact sales performance (outside the skills, abilities, knowledge and attitudes of the team members), and too ill-defined a period of time over which an effect is caused, for any measure of ROI to be truly supportable by anyone who really understands how the world works.


The fundamental principle behind measuring the results of training is to what extent “ceteris paribus” applies - to what extent does everything else remain the same, so that training is the only input change and therefore can be the only factor causing a change in results.

It’s rare, in the real world, for ceteris paribus to apply. We have to live with, cope with, and thrive with, the shades of grey that exist, and be comfortable with a lack of certainty.


Be very careful when people offer you certainties - are they really, factually certain, or are they being offered as a comfort blanket, or, worse, hiding the reality. Ask yourself why someone might offer you a spurious certainty.

Please press (at least!) one of these.
It costs you nothing, and (possibly) helps us spread the word!
Please leave your thoughts and comments here

There are 12 comments
gravatar Aveek00
October 16, 2016 - 18:18

Good designs of Spurious certainties contain this article. Detail of these certainties and some other related detail mentioned this article which we should follow before using it. I actually looking a site which have reviews about but this website have good piece of information related to different topics.

gravatar ryan
October 14, 2016 - 13:07
Subject: free robux

Generate the roblox hack which will allow you to acquire the whole world in roblox as you will be fully entertained.

gravatar sanju
September 27, 2016 - 12:27
gravatar jame kurt
August 26, 2016 - 16:22

Yes it is time for action and have to taken strong decisions now days and i found Help With Best Assignment Writing Service | Assignment Point recently and they have to deliver me perfectly idea of assignment.

gravatar zorsha – hyderabad
May 17, 2016 - 13:15

Its now good info and good article.You need to be finished with the primary written work of the bestessays weeks before it is expected, so you have a lot of time to sort it up. Try not to abandon it so late that you're hurrying.

gravatar michael Jones
December 15, 2015 - 09:32

Really appreciate this wonderful post that you have provided for us.Great site and a great topic as well i really get amazed to read this. Its really good.
Assignment Help

gravatar Assignment help – USA
December 15, 2015 - 09:30
Subject: Nice

This is really great work. Thank you for sharing such a useful information here in the blog. https://www.allassignmenthelp.com

gravatar Adam maddy
June 18, 2015 - 19:23

This is a very amazing blog that you posted, I really appreciate your effort... pay for coursework-courseworkgeek

gravatar Maryleona – United States
May 05, 2015 - 14:35
Subject: Nice

I like this history You may record me in history Brilliant Assignment With your biting, You will find that your history educators mind an incredible arrangement about your written work.

gravatar Linser
April 04, 2015 - 10:56
Subject: education

We should always concerns about studies and some student looking custom essay writing online at comfortable rate but professional team always sort out their students issues.

gravatar Joan Henshaw – London
March 09, 2010 - 18:51

An excellent post Simon. As a provider of management training, I really couldn't agree more. It does bring some challenges in the marketing of training though e.g. Client: 'how will your training impact our profitability?' Me: 'Well it depends......'
Maybe next time I'll direct them to this blog!
Cheers, Joan

Reply to Joan Henshaw
gravatar Simon Roskrow – Yorkshire
March 09, 2010 - 21:21
Subject: Measuring training results

Please feel free to direct them this way! Having the confidence to be able to say that, with a very small number of exceptions, a genuine measure of ROI in training is impossible.

I hope that, when I have this conversation with clients, they appreciate the honesty. The training can work; it can deliver massive, organisation-shifting changes in performance...but to say that it's equivalent to a financial ROI calculation is simply misunderstanding the fundamentals of ROI.

We need smart ways of measuring training performance. Eric Pickles today derided work done by the Audit Commission to train their staff. He might be right, it might have been rubbish and a waste of taxpayers money - but, on the basis he attacked it from a method/process point of view rather than a results one undermines his approach. However, without smart, genuine ways of measuring performance changes ourselves, lets not be too surprised if others do it (badly) for us.

Thanks for hopefully kicking a debate off Joan.

Leave a Comment

? ?

Powered by TalkBack