This is currently a holding page to take you to sections on Analysing your results
and Basic descriptive statistics including an explanation of what data are and some of the most common tests used to analyse data.
The key function of this page is to outline the main statistical tests which you'll use for comparing data.
This section is divided into two sections, which is also kind-of how you need to approach any new data.
When you get some data, either by collecting it yourself or by having it supplied, it is well worth getting to know your data (cue music - "Getting to know you").
That way, when you run other tests, you have already spotted the outliers (ages over 120 or under 3 for written responses, for example, or heights of 15m instead of 1.5m) and you know the general shape of things (if cancer is expected to be 1% of the population, and your sample has cancer 90% of the population).
In the memorable words of Davis Balistracci, you plot the dots. For more information follow the link.
It’s worth explaining some simple things about valuation:
Terminology: Analysis is all about testing hypotheses (what you think might be happening). The null hypothesis (H0) is that any variation is due to chance. You can have any number of Hypotheses (H1 . . Hn) of specific things happening, though usually we just look for situations where deviation from the null hypothesis is unlikely to be due to chance.
If you know that the national average for a baby’s weight at 6 weeks is xx kgs, and 20 babies recorded by this one midwife during this year give an average 0.2kg less, then does this mean there is a problem or could this be due to chance?
Depending on your level of risk, you might say “if there’s a one in 10 chance that it’s not due to chance but something is going on (the mothers are all not feeding their babies properly, the midwife isn’t giving the right advice) then I want to investigate”, then you would look at the probability that this is due to chance and act accordingly.
Most statisticians use 1 in 20 (5%, <0.05) and 1 in 100 (1%, <0.01) or even 1 in 1000 (0.1% or <0.001) chance. I believe there are a great many situations in ealth where 1 in 10 (10%, 0.1) is a reasonable point at which to take action.
Real numbers (numbers of people, numbers of minutes, heights, numbers of activities) are what are called parametric. That is they are real numbers. This is crucial for using statistics – many are designed for using only with parametric data.
In the natural world, numbers tend to follow a normal distribution, as someone described it “a small amount at one end, a lot in the middle, and a small amount at the other end”; or in the case of height, most adults are roughly around average height, with only very few substantially shorter or taller than the average.
To look at whether the numbers in your population are normal (also often a requirement for statistics), plot the data as columns in categories or groups of data. It should form a bell-shape.
Times tend to plot as a skewed curve. Hardly anyone gets to see the clinician in no time at all (I’ve come across this occasionally, where the way the data are recorded, it’s the clinician who makes the decision whether a person should see the clinician, and of course sees them immediately. Watch for this as it’s not evidence of short waiting times, only of poor recording). So hardly anyone sees the clinician in no time at all, most people see them within a short period (whether that’s minutes or days), and perhaps half of the population don’t see the clinician quickly and sort-of tail off into the distance.
Numerically, the analyses to use are Kurtosis and Skewness.
Start with the Mean (average – MS Excel function =Average()) and Standard Deviation (amount of variation from the Mean – MS Excel function =Stdev()).
Categories are not numbers. The categories are mutually exclusive, such as Male or Female, Success or Failure, Sheffield or Barnsley or Doncaster.
You might have data for attendance at the Urgent Care Centre (UCC), and capture where people have come from. You want to know if referrals on to A&E are what could be expected.In other words, if three times as many people come from Sheffield as from either Barnsley or Doncaster, are the same proportion of people being referred to A&E or do they have different habits?
Use Chi squared (χ2) for this. This works out what you would expect from the totals, and compares this with what you actually have
Categories such as heights, as shown on the above normal curve, should not be used in this way as the categories are connected in a numeric way – they are on the same scale.
For example if you worked out how far people travelled to get to the UCC that would be a numeric score.
Combinations of categories and numeric data allow you to run useful tests such as Student’s t and ANOVA.
Where you have information which isn’t numeric but which can be ranked, for example satisfaction, then you should not treat it as a numeric score but as a non-parametric ranking.
You can also convert numeric scores to ranks, eg “closest, next closest” or “first ten, next ten” which sort-of “normalises” skewed data.
This is an excellent way to convert essentially qualitative data into quantitative.
Although the data are non-parametric not parametric, you can still perform a number of numeric statistical tests such as correlations.
Note percentages are NOT parametric or numeric scores.
Because they are proportions not absolute numbers, they don’t always fit the proper definitions for populations used in statistics, so be careful interpreting them.
Run the usual tests but make a note that there could be a problem.
The key function of this page is to outline the main statistical tests which you'll use for comparing data.
Where you are using actual numbers, the numbers of people coming, number of minutes wait, etc and you can reasonably expect a 'normal' distribution (you've no evidence that there is some bias), then use Parametric tests
compares two averages and determines if there's a real difference between them or if the difference is just due to natural variation (no statistics give absolute answers, they just give a level of confidence. One chance in 10 (10%), one chance in 20 (5%) and one chance in 100 (1%) are usually taken as the border lines for confidence that the difference, or similarity, is reliable.
Student's t-test was designed by William Gossett to cope with smaller numbers (usually taken to mean samples of fewer than 30 points) because the z-score starts to get inaccurate with smaller numbers. The t-test works for larger numbers too
Technically the F-test, this compares any number of averages essentially by comparing them pairwise and reporting if any are statistically different. Same as the t-test, this assumes that the distributions of the data are similar
Anovar is short for "Analysis of Variance" and the F-test is named after R.A. Fisher. Anovar is usually applied to one-way data but applies equally to more ways (see below for note)
requires that each measure has at least two bits of data attached; for example each week of data could have: number of users; average wait; number of staff hours; average daily temperature. You're looking for any association between any of these bits of data for example do waiting times go up when numbers attending go up, or down when staff hours go up. The easiest test to use is r2 (r-squared) or Product Moment Correlation Coefficient (Pearsons).
R2 values are between 0 and 1. The higher the value, the stronger the correlation. You'll always get a straight line between two points (i.e. for two points r2 will always be 1.0) so be sensible - with numbers of points below 30 you should be careful interpreting the results
Derek Rowntree lists how you might describe correlations for r, so I've updated it here for r2 as follows:
Note correlation doesn't indicate causation. There's a correlation between a person's weight and height, but extra weight does not CAUSE extra height. It's probably that extra height causes extra weight but you can't tell that from finding there's a correlation between the two.
The first tests listed are so-called Parametric tests, that is they work on absolute values - how high, how long, how heavy.
Non-parametric tests work on scores, such as % (the proportion, how many <b>of</b> how many, limited to a range usually 0 - 100), the rank (first, second, third - with no indication "by how much") and so on
Ranking can be useful if you can't assign a value, e.g. if people put their preferences in order.
Non-parametric tests should also be used if you've created artificial scores, such as ranking patient satisfaction
applied to two-way or more data (and one of the most popular tests in social sciences) - for one-way samples either use absolute values or compare Standard Errors of Proportions. Chi-Squared compares the observed frequencies (proportion * total) with the expected frequencies (average proportion * total, calculated from both totals row and column)
Spearman's Rank Coefficient
Essentially the same as the coefficient of correlation (and shown as a p2 value) but using ranks instead. Interpret the same way.
All of the above will be covered in more detail on accompanying pages
Footnote 1 - one-way,two-way and more. One way for a comparison of averages means that you're only comparing one difference between the groups, or populations. Men vs women. Trees vs Bushes vs Grasses. Red vs Blue vs Green vs Yellow vs Black.
Two way means you are comparing in two different groups, for example
and more ways just makes this more complicated
It's worth getting a simple book on statistics e.g. Derek Rowntree's "Statistics without tears"which will give you a general background - this would take too long on a web page when there are so many excellent resources out there. Other books on statistics can be found on this Statistics Resources
The most common ways to analyse data you've gathered from measuring results
So you've got your numbers, how many people, how much was done, using what resource; now you have to report it in a meaningful manner.
This is where Statistics fits in
If you compare two numbers, e.g. the number of users before the new service was introduced and the number after, then you can get one of three results:
MoreThe second number is higher than the first
The SameNo Change
This doesn't tell you whether you could expect the same change if you took measures a second time; for example you might have more people one week completely at random.
To counter this, you can use statistics, which estimate the 'likelihood' that one number is different from another, or the likelihood (H0 or Null Hypothesis) that they are the same.
Of course you need to go back to What constitutes success, or the benefits outlined in the Benefits Approach. To make the reports meaningful, you need to decide what they will demonstrate - 40 graphs and charts pinned to a notice board are pretty meaningless.
For example, if you want to show the service has improved and you've collected:
numbers of people using the service each day
average wait from arrival to being seen
average wait from appointment time to being seen
period from referral to appointment
follow-up or repeat appointments as a proportion of total day's appointments
number of that day's appointments that result in a follow- up
'see and treat' resolution of problem that day
which of these would show an improved service?
None! Comparing the figures with the week before, the month before, the year before would show if the service was improving or not (though it's important to remember to use run charts or averages - figures comparing one day to another are counter-productive).
Of course you also need enough clinical knowledge to know what direction of change represents an improvement - follow- up appointments may be good, and they may represent failure.
How do you analyse the data
Plot the dots
The first thing to do is to understand your data. What does it look like?
Is it "normal" or not?
parametric or non-parametric?
Does the data tell you what you need to know, or do you need to process it in some way before it will?
We'll explore Basic descriptive statistics and simple statistical tests in the next page
My life purpose is to inspire people to take life with both hands - to bring their heart and soul to their employment and put their enthusiasm into their work and professional life as well as their personal life.
By doing this, they not only bring their employer much greater outcomes and productivity for lower cost, but they also have much more fulfilling work.
I use benefits and benefits realisation as the way I engage people - helping organisations and individuals align on "WHY" we are here and "WHAT" is each team's contribution to the strategic objective, and each individual's contribution to the team's strategic objectives. The "HOW" follows naturally, and often, as people align with purpose and function, there's no need for management (in the sense of something imposed from outside)
Perhaps it is because they like people so much – computers seem far too simple?!
The result is that people spend hours on administration, fighting with their spreadsheet to try to make it do the budget; or endlessly retyping from one application to another to complete the mandatory reporting that both the Health Service and Social Care think is compulsory. In the end, the very thing they hate the most comes to dominate their working day.
Lots of people struggle to understand the mountains of data that, let's face it, you have to understand if you're going to plan ahead or to make things better.
I met a “data expert” the other day, who'd struggled for half a day putting a large data file into MS Access (the database), filtering for the bit of data he absolutely had to have, and then putting it into Excel to cross-reference (very slowly). With the right tool for the job (in this case, MS Access for much more of the job) it took about 10 minutes which allowed him to get on with his life (helping cancer patients). The next time he needed to do it, it took a couple of minutes.
In another Strategic Health Authority, they have been assessing their health commissioning and providing organisations against quality standards. This means that each service (eg Vascular Care) in each organisation (eg an Acute Trust/ Hospital, or an Out of Hours service) fills in a form saying whether they are doing 'best practice', the things likely to lead to the best outcomes for patients. Then the SHA has to collect the forms (by email, luckily), retype the data into a spreadsheet (because the form is in MS Word), cross reference some 500 or so specific local Quality Standards (which define exactly what needs to be done) against the 16 Care Quality Commission (CQC) essential standards (which are general so it's difficult to know what you need to do to improve), compile the reports by PCT cluster and send the results back out. After they'd taken 10 days to do 1 service in 1 PCT cluster, they realised it was a hopeless job (hmm, 14 services X 15 health areas - that's 10 years of work per year and we haven't even started on the Mental Health standards) and asked me to look at automating.
Now they have a self-creating spreadsheet, where they fill in the data and press a button, and the computer does the tedious work, and doesn't make mistakes. Managers and staff can get on with improving the quality of care for patients! Two things really stand out
Over the years I've taken on some pretty tough assignments (I know I don't go out there and save lives, but I try to make sure the right decisions are made about numbers of people to train, facilities to build, etc). For the Emergency Care Practitioner new ways of working project, I developed the economic models that said how many ECPs we'd need, and therefore how many paramedics, how many ambulance technicians, ambulances, cars; also how many A&E departments and appropriate staffing, and of course additional staff for the community services where people would be referred directly. I've been given questions like "I need to know all the academic literature on Head and Neck cancer, compare it with our own survey data, and present options. Oh by the way I need it in 4 weeks' time and I've only got money for 10 days' work" and the same for Domestic Violence and Obesity (and others!). I've looked at "Why do we need to ask User Experience? Is it just feel-good or is it worth doing?"
I couldn't do this without making my computers sing and dance. I automate all the time to get the best result, accurately, within the time and budget available. Now is the time to let you get access to all of this, without having to define your problem as a whole big project. As one friend (who still comes back for more) says: “what I like about you is that you can understand what I want even when I can’t explain it very well”.
Maybe it can help you. Do you want to spend your time on improving care, improving the quality of your services? Do you wish that your paperwork would magically sort itself out? Do you want to see results rather than struggle to get there?
I’d be delighted to talk about your options.
Marc Woods, Paralympic medallist and motivational speaker, came to speak to health service staff in Easington today.
He is superb, and in between explaining how he changed from moderately successful (a county level teenage swimmer who was sufficiently good that he didn’t have to try, so he didn’t try), to realising how much he had lost when his leg was amputated below the knee, to deciding to make something of his life, Marc made a number of very important points.
If you don’t really REALLY want to win, then you won’t win. If you are trying to swim a world record time, then it might not be fast enough to beat the others who are also trying to swim a world record time. You need to WIN, to beat everyone else – self leadership
If anyone in your team doesn’t REALLY want to win, then your team won’t win. You are as strong as your weakest link (see the look of disappointment on the faces of two of the silver medallists, Marc included) – clarity on your goals
Everyone who has anything to do with your success is on your team, from the person who makes sure that the water is clean and the ropes laid out at the training park, to the coach and those who swim. Everyone needs to feel involved, and everyone has to be COMMITTED to winning gold. It’s up to the leader to get them there
Personal relationships count. When the chips are down, that’s what pulls you through. When you need to make changes, then do it on the basis of the strengths and weaknesses of each player, and with everyone’s agreement. Communication is all ways, communication doesn’t just “come out of the tip of my finger” as Marc said about his early days as British Swimming Team Leader. Marc said at this point “do you know someone who leads by talking AT you instead of with you? Don’t all look at him/her, that would give the game away”.
It’s a clear reminder of how easy it is to get complacent. We’re good enough, so we don’t try to be the very best. We sometimes can’t be bothered to get up and go to training, so arrange enough different reasons why you should that at least one will mean something to you when you are tired, aching all over, and hungry.
What will I do differently?
Give myself five reasons to be the best at benefits realisation:
It inspires people to enjoy their work, because they can see what a difference they will make
I enjoy understanding what a difference people are trying to make, and helping them make it
In health and care, the difference that frontline teams make is real and makes a real difference, improving the quality of people’s lives
I respect the people I work with and who mentor me
I love it when people say “thank-you” and “you did a good job”
Every day, live on purpose to be the best I can at whatever I’m doing. If it’s a relaxing day, then I’m going to be the best at relaxing, and so on
Spread the message
The problem with the IS/IT investment is that people often don’t think about the benefits - "of course we need it". Ward and Daniel illustrate this from a survey reported in 2003 (Lambert and Edwards – also at Cranfield University) as follows:
· 55% of respondents think that an appraisal of the IS investment is important
· Only 22% have a process to perform this appraisal
· Only 10% consider the implications of business change from their IS investment
· And ultimately – only 27% of projects appear to justify the investment made
This is surprising, given how important investment appraisal is in the preparation of the business case. It begs the question: are honest business cases prepared for IS investment?
The business case – "benefits cost analysis"
The business case is designed to explain the financial justification for any investment of resources. Examples of approaches include:
· ROI - Return On Investment
· NPV – Net Present Value
· ROA/ROCE – Return On Assets/Return On Capital Employed
· And many others, that may illustrate a non-financial return to a resource investment
In principle of course, the business case will illustrate what the reward is compared to the risk, and what the return is for the resources engaged. Typically a business case will make a case to: do things better, do new things, and/or stop doing things.
Working with stakeholders
Very little gets done unless you involve people. Ward and Daniel into some detail on how to engage stakeholders; in particular took about a stakeholder perspective, planning the business change, and valuing the benefits.
One point make repeatedly is that the value from IT investment comes from the business changes – "IT enabled change" not just the technology. Therefore it is critical to involve the business managers, and to identify, and satisfy, their business needs. Technology can help to integrate processes and coordinate resources (Value Linking), to increase the speed at which the business performs (Value Acceleration), and to realign job roles and organisational structures with valued processes (Value Restructuring).
Therefore it's important to identify the stakeholders for the project (who's relevant).
Stakeholders can be categorised in a number of different ways, including their influence, apart make decisions, and their attitude towards the project, and many of these methods are illustrated. Having identified the stakeholders, identify what the benefits/dis-benefits ("what's in it for me?") are for the most important, and what change activities will be necessary, and from this identify where the resistance may come.
It isn't always the explicit changes that create resistance. Often it is a secondary or perceived change such as:
· Increased accountability, less discretion and autonomy
· Individual performances more visible
· Having to rely on others to achieve performance
· Collective rather than individual recognition/reward
· A learning curve
Stakeholders can be approached in a number of different ways, with varying results:
In most organisations, substantial investment and change decisions are made by senior management. Typically decisions are justified with a rational presentation (eg a Business Case), which clearly fits into the top left-hand box of the diagram above.
· The solid arrow in the illustration above shows management approach following the organisational behaviour. The decision to proceed with the investment is made at a senior level. The project then works with (‘coalition’) the business to build trust. It is implemented by finding quick wins and self-interest, and minimising dis-benefits to bring about the business change.
· The blue dotted (top) arrow follows a rational approach. By not building trust that the top management has the interests of the whole business at heart, divisions and teams will seek to avoid blame, by removing business change that represents a risk. Inevitably this means that the final solution involves less change and therefore less benefit (note that not all change is a benefit, although all benefits involve change)
· The yellow lower arrow illustrates solution imposed without discussion. Once this imposed solution reaches the teams who need to make the changes, each team will create their own objections, and the delays will result in individual negotiations resulting uncoordinated and inconsistent solutions, which will miss many of the overall benefits.
Implementing a benefits-led approach
Most decisions are already made on the basis of “benefits”, but often the only person who understands this is the person making the decision, and it isn’t written down.
Why would you change the way your business works to use a Benefits-Led approach? It can be for either tactical reasons (precipitated by a crisis; recognising business is disenfranchised from decisions over investment; a way to reengage business), or strategic (decide to improve the process or to improve governance standards).
What does it actually mean? Decisions about investment in technology inevitably, and always, affect people – as Ward and Daniel put it: "techno-social". This means that they aren't simply a decision about money, they are a decision about changing the way of working. Changes to technology will affect the business, in one way or another (after all, otherwise why would you bother investing?), the business, and the people in the business, need to play a part in making the decision.
How? Implementing a benefits approach will depend on the context. Ward and Daniel give three example contexts where thought needs to be given before implementing a benefits led approach:
· Public sector – multitude of stakeholders (the agenda is not just financial viability, but also service to the population, and an ever-changing political agenda); and frequently a mandatory requirement that has to be met (but it won't be met by mandating compliance, it still requires a consensus to make the relevant business changes)
· Small business – small businesses don’t typically have specialist project managers and benefits managers. Everyone is involved, and everyone contributes. The benefits led approach needs to recognise the specific benefits for each individual, as tangible rather than nebulous (for the greater good) benefits
· Large organisation, multi-site – although the sites may appear to be clones of each other, the benefits identified for one site may not apply to every site. For example, each site starts from a different point, and they have different stakeholders. It is tempting to do the benefits planning once, and then roll it out on this basis, but some form of benefits planning needs to be done with each site in order to build consensus and agreement for the changes as they apply to that site, otherwise we will end up with the illustration of the yellow arrow – different sites create different objections requiring different work-arounds, which means systems that don’t work in harmony
The book is a good review of a vast amount of academic literature to date (2006), and used in Cranfield University to teach the most widely taught benefits approach in the UK.
To my mind, it may miss the point in some areas, especially the use of the highly regulated and formalised Benefits Dependency Network which can be very difficult for line with "common-sense". Bradley's version of the benefits dependency network takes a very different, and far less formal approach, and I prefer it.
It isn’t often that I write a piece which blatantly advertises a product. But a friend had to get his Masters dissertation in on time, and was fighting with the University-issued stats package, so I turned to my favourite Statistical Analysis software.
It’s my favourite because:
It does all of the statistical functions I need, in a clear and sensible way (yes you can do them in MS Excel but you have to jump through hoops)
It does Run Charts and Quality Management out of the box (you have to buy additional modules for SAS, SPSS, etc)
It’s low cost/ low maintenance (It is designed for working people rather than academics – I’m a working person with academic pretensions)
It has a really good help system
I’ve been using Minitab (www.minitab.com) for years, but I was still stunned to see this new feature – Assistant.
Plot the Dots
(to quote Davis Balastracci)
When I click the Graphical Analysis button, it opens up a flow chart. This looks like a useful help screen, but it turns out that it’s live – it looks at your data for you! Of course you can plot graphs in Excel, but do you know which graphs to plot?
Test your hypothesis
This is where the rubber meets the road. What am I trying to test, and am I right? I mean, look at the choices:
Compare one sample with a target
Compare two samples with each other
Compare more than two samples
ANOVA (Analysis of Variance) for continuous variables, and Chi-Square for discreet groups (male/female, ethnic group, etc) are my favourites.
It does reveal its working roots – I found that I had to use Chi-Square % Defective to test whether the balance of experienced/new staff differed between the company subdivisions (so if you are experienced, you are defective – actually that could be right?).
Looking at the results
But the way it presents the results had me excited! As an infrequent user, I get to stare at the Probability, F ratios, T ratios and everything else and think “is bigger better, or is it smaller that is better?”
Here’s what look like a set of dots where the human eye would say “OK I can see the trend, but it isn’t very close”
Minitab tells you (from the Assistant) not just the Probability that there is a relationship (nice bar chart with “degrees of blueness”), but also that in spite of the high probability of a relationship it only represents 26% of the variation (degrees of greenness) and shows the correlation too. All on a single sheet.
Go and download your own trial copy: www.mintab.com (yes, Quality Companion is about service improvement and lean – that’s worth downloading too)