Evaluating the effectiveness of the training delivered is part of the role of Learning & Development professionals and critical to demonstrating adding value to the business. This has been done in the past by using test scores and upward/downward trend analysis e.g. beginning with a pre-test and ending with a post-test, and maybe an aggregation of a volume of participants over time. Then there is the survey or ‘smiley sheet’ to evaluate the learning experiences across programs to monitor a program or an aggregated trend.
Some might say that we are already making ‘data-driven decisions in training functions. However, this holds true only if you analyse the data, draw conclusions from it and take data-based actions. Data is being collected, but how often is it really being used?
Data leads to insights; business owners and managers can turn those insights into decisions and actions that improve the business. This is the power of data. [Bernard Marr, Forbes.com, 2016]
Training or any particular learning solution is often hard to isolate as having business impact. However, data can now be collected during the learning experiences. But an organisation trying to stay relevant and ahead of the skills gap curve in today’s environment will need to be making data-driven decisions to maximise resources and opportunities. Careful planning and design, based on actual data, can help training be a better-targeted experience and relevant results achieved.
Steps To Making Data-Driven Decisions
Inspired by the10 step process outlined by business leader, Bernard Marr, [Forbes, 2016], here is how it might work with learning:
- Start with your strategy – what are you trying to achieve with your go-to-market strategy?
- Identify the knowledge and skills required to achieve the desired results
- Prioritise the topics you want to work on – what knowledge and skills will make the most impact on achieving your strategy?
- Look at what data you already have access to, and any data that needs to be collected. Do you have assessment results that indicate gaps in knowledge and skills? Have you prioritised the gaps according to impact and urgency? You need to focus to avoid being overwhelmed by low-value data.
- Identify the questions that are still unanswered – for example, do you know why any haps exist?
- Identify the data needed to answer the most pressing questions which will help you create a plan to close the gaps.
- Assess the cost of obtaining the data – is it justifiable? This is why you need to identify the data you really need as we can get carried away obtaining unnecessary data and invest too heavily with no ROI.
- Collect the data – buy it, collect it yourself or collect it with an external partner. This could be an assessment initiative, a survey, and/or data already collected by First Line Sales managers during rep rides.
- Analyse the data – what is it telling you? Do you need help interpreting it? Can it be presented in a visual format and easily understood?
- Present the insights to relevant stakeholders.
- Apply the learning. Create your learning program based on identified gaps and a sound understanding of why they exist, according to potential business impact. Do something based on the insights!
Numbers That matter
You are probably already collecting some data from your learning activities, such as participant feedback; however, if the data is sitting on different platforms and therefore in different databases, it may not be easily accessed or used. The utopia is to get the data into one place in a reusable format and use a visualisation tool to interpret it. This is the same ambition the commercial functions have to conduct cross channel analytics. ‘Good data’ needs to be quick and easy to access. ‘Numbers that matter’ are based on picking relevant, actionable metrics.
Both qualitative and quantitative data can be useful in the pursuit of insights into learning needs and priorities.
Basic data which is often available include:
- Profile Data – who are your learners and what are their preferences?
- Financial Data – are we achieving our financial objectives using budgeted resources?
- Activity Data – how do our planned and realised activities match our objectives?
- Market Data – what is happening in the marketplace that may require new knowledge and skills?
- Training Data – what training has taken place, by whom, when, and what were the results.
There are several ways Big Data analytics, which often combines different sources and AI or machine learning, can provide new insights:
- Diagnostic Data – what happened in the past and why;
- Predictive Data – what might happen next based on patterns of the past e.g. performance on the job;
- Prescriptive Data – what should happen next based on the data received;
- Descriptive Data – what is happening right now in real-time; and
- Outcome Data – what made that happen [customer behaviour and preferences].
Shift to Data-Driven Learning
Data-driven decisions are proven to drive increased productivity and effectiveness. Data-driven decision-making is an approach to making decisions using data to back them up. The quality of the data is a critical success factor of course - bad data informs bad decisions! The ability to analyse data and draw valuable insights may in itself be a required area for skills building in your organisation.
Implementing data-driven decisions in a learning environment means collecting and using data to identify learning solutions needed, and how to best design and deliver those solutions. Training or any particular learning solution is often hard to isolate as having a business impact, but the collection of new types of data, combined with improved analytical and decision-making skills, offers new opportunities to really understand the learners, customise learning solutions and also to demonstrate real business impact.
If you are interested in our mobile learning solutions on digital channels, contact Actando.
The Actando Consulting Team