How to Measure the Efficiency of Microlearning

Higher Level Learning – a learning company in Manchester

Evaluating Microlearning

The field of evaluation is well established and as a result, there are a number of different approaches and theories, but evaluate microlearning as you would any other learning. In order to develop a practical, non-labour intensive and therefore cost-effective tool for evaluating microlearning, opt for a goal-based approach (see 2. below) whilst making certain assumptions regarding environmental variables.

Evaluation Theories

Robinson (2002) lists the five main approaches developed by different evaluation theorists, below. ‘While the different approaches are all attempting to answer similar questions (about the worth of programmes and elements of them), the emphasis on various aspects (purpose, focus and evaluator’s role) varies, as does the underpinning theory. However, all

share at least one feature: a concern for rigor in the conduct of the evaluation and a concern for reliable and systematic evidence to support any conclusions.’

Approach Emphasis Focusing issues Evaluator’s role Specific information needs
1. Experimental Research design What effects
result from
programme
activities and
can they be
generalized?
Expert/ scientist Outcome measures.
Learner characteristics.
Variation in treatment.
Other influences on learners.
Availability of a control group.
2. Goal oriented Goals and
objectives
What are the
programme’s
goals and
objectives, and
how can they be measured?
Measurement
specialist
Specific programme
objectives.
Criterion-referenced
outcome measures.
3. Decision focused
[CIPP]
Decision
making
Which decisions
need to be
made and what
information will
be relevant?
Decision
support person,
provider of
information.
Stage of programme
development.
Cycle of decision making.
Data gathering and reporting.
4. User-oriented Information
users or clients
Who are the
intended
information
users and what
information will
be most useful?
Collaborator Personal and
organizational
dynamics.
Group information
needs.
Programme history.
Intended uses of information.
5. Responsive Personal
understanding
Which people
have a stake in
the programme
and what are
their points of
view?
Counsellor/
facilitator/
collaborator
Variation in individual and group perspectives.
Stakeholder concerns and participation in
determining and framing the data.
Programme history.
Variation in measures and sites.

How to measure the efficacy of elearning / learning Part 1

Higher Level Learning – a learning company in Manchester

How to measure the efficacy of microlearning

Whether you’re an elearning designer or microlearning developer or anyone working in the field of learning, measuring the efficacy of microlearning or learning in general is sometimes regarded as a difficult or almost impossible task especially measuring the transfer of skills in the workplace.  Consequently it is rarely attempted or only ‘surface scratched’. We’ve done some work on this at Higher Level Learning and can hopefully provide one method that works and has been tried and tested in the work environment. We call it SeET – Snapshot elearning Effectiveness Tool.

Our criteria for success was:

  • A practical solution that can be used by staff with little experience of evaluation
  • A tool that steps beyond Kirkpatrick level one and measures behavioural changes in the workplace based on completed elearning (or just learning)
  • A solution that can be quickly deployed and easily measured in the workplace
  • A cost-effective solution
  • the tool should not be limited to evaluating elearning and could be adapted for use in other areas

One question before we go any further:

Why measure microlearning or learning at all?

In the current financial climate, it’s more important than ever for elearning and learning programmes to demonstrate their value and the benefits they provide to an organisation. This means creating programmes and initiatives clearly linked to business objectives and having effective tools in place to measure their success (hence the creation of SeET).