What does ‘impact’ look like for digital HCP engagement and medical education?

Within the pharmaceutical industry, many continue to rely on ‘volume’ metrics; basic headline statistics like total reach, email open rates or page views. But truthfully, these offer limited insight into the real impact of an activity, and giving them undue focus may result in programmes that are poorly optimised to deliver on core objectives.

Meanwhile, it has proven far more difficult for Industry to demonstrate quality of engagement and measure outcomes that more closely signal progress against the overarching ambition to improve patient care and, ultimately, health.

Designed in the context of traditional, in-person learning, this is what Donald Moore et al sought to address in 2009 with their framework for outcomes measurement in continuing medical education (CME).

With a focus on measuring impact on healthcare professionals’ (HCP) knowledge, confidence and clinical decision-making, it responded to the problem that those participating in CME previously only needed to demonstrate attendance to receive certification, with no meaningful evidence of learning required.

Providers of digital medical education have subsequently adopted ‘Moore’s Level’ assessments, employing pre- and post-learning surveys to demonstrate impact for online programmes. However, with significant shifts in HCP engagement behaviour observed over the last 15 years, it has become clear that new approaches are required in the digital space.

More than ever, HCPs are consuming educational content and gaining CME points through digital channels that offer greater flexibility of engagement than traditional CME.

For example, over 90% of HCPs using EPG Health’s independent medical education platform Medthority (www.medthority.com) follow a unique educational journey that, on average, spans more than four sessions.

Preferring to consume bite-sized educational material in 5-10 minute chunks, HCPs are engaging in an incremental style of learning online that Moore’s framework was never designed to address.

Currently, Moore’s Level assessments as applied to non-linear digital programmes lack something that was taken for granted in the traditional CME setting: transparency of individual HCP engagement and content consumption.

Instead, many providers of online medical education simply assume that active engagement has taken place. However, it has been demonstrably possible for HCPs to complete post-learning assessments and gain certification without any evidence of meaningful learning time with which to confidently validate the outcomes measured.

Recognising this challenge, EPG Health sought to adapt Moore’s framework to enable more robust impact measurement for digital medical education, while continuing to support HCPs with the flexibility of online learning they have become accustomed to.  This culminated in the development of a unique digital Impact Outcomes Framework.

The framework – which is applied to educational programmes hosted on the Medthority platform – considers all four core areas covered by Moore et al (Stage of learning, Educational design, Outcomes and Assessment), and takes additional steps to fit these concepts into a digital setting.

Underpinned by deep, technology-driven insight into HCP behaviour, the approach sees HCPs tracked through five stages of learning as they engage with the content, usually over the course of multiple sessions. Once the HCP has consumed sufficient content to indicate engaged learning, key ‘Impact Outcomes’ for resulting changes in knowledge, confidence, competence and clinical behaviour can be reliably assessed.

The Impact Outcomes Framework is enabled by the vast wealth of data that EPG Health can now collect and utilise to measure all aspects of HCP behaviour and engagement within a Medthority educational programme.

By tapping into this data – including correlating the specific content HCPs have consumed to their learning assessment results – it is possible to build a full picture of an HCP’s learning journey. Aggregated together, this provides powerful insights. The data doesn’t just inform Impact Outcomes reporting – revealing how engagement has influenced HCP knowledge and competence – but also has an instrumental role to play in how learning objectives are addressed through programme design. It contributes to ever-evolving understanding of how HCPs learn best, and how engagement is driven through communication outreach to them.

With this level of data available at our fingertips, it is possible to target HCPs based on their recent learning activity, develop audience segmentation based on digital behaviour (not just demographics) to adapt communication, and react promptly to changing HCP needs and preferences.

All of this means we can create relevant and increasingly personalised ‘teachable moments’ that will resonate with HCPs, building their awareness of potential gaps in their knowledge and helping them to address these.

Combining an effective communication strategy and hyper-relevant content makes delivering the right educational message to the right HCP easier, resulting in higher levels of engagement and reducing the need to rely on vanity metrics. Instead, the focus can be put onto building meaningful Impact Outcomes.

EPG Health is the publisher of Medthority, an independent medical education website for healthcare professionals worldwide. To find out more about the Impact Outcomes Framework, visit bit.ly/Impact_Outcomes

Matthew Hellyer is Head of Insight at EPG Health