State of play of CME in Europe in 2014: proceedings from the Seventh Annual Meeting of the European CME Forum


State of play of CME in Europe in 2014: proceedings from the Seventh Annual Meeting of the European CME Forum

Eugene Pozniak1 and Rick Flemming2

1Siyemi Learning, University of Manchester Innovation Centre, Manchester, UK;
 2Aspire Scientific Ltd, Bollington, UK


European CME Forum is a not-for-profit organisation that was established in 2007 in order to bring together all stakeholder groups with an interest in European CME and promote multi-channel discussion in an independent and neutral environment. This report summarises the presentations and discussions that took place at the 7th Annual Meeting of the European CME Forum in London on 13–14 November 2014. The meeting was held at a time of great uncertainty in European CME and gave attendees opportunity to consider many unanswered questions regarding how CME in Europe will be funded, accredited and regulated in the future. The programme for the forum was developed based on a needs assessment conducted among a variety of CME stakeholders in Europe and beyond. This exercise identified a number of issues that are rarely covered at similar gatherings and which were therefore given prominence during the meeting. Chief among these “hot topics” were how to ensure effective measurement of outcomes in CME programmes and how to encourage and manage the transparency of relationships between industry and healthcare professionals. Other subjects covered in depth during the forum included the future funding of CME, e-learning innovations and potential, and the value, or otherwise, of CME accreditation. The forum made use of a number of interactive meeting formats which ensured the days’ proceedings were characterised by a series of lively discussions and stimulating debates.

Keywords: CME, accreditation, Europe, providers, best practice, funding, regulation

Received: 20 February 2015 Accepted: 2 March 2015 Published online: 27 March 2015

Journal of European CME (JECME) 2015. © 2015 Eugene Pozniak and Rick Flemming. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (, allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.

Citation: Journal of European CME 2015, 4: 27680 -


European CME Forum is a not-for-profit organisation that was established in 2007 in order to bring together all stakeholder groups with an interest in European CME and promote multi-channel discussion in an independent and neutral environment. The 7th Annual Meeting of the European CME Forum was held in London on 13–14 November 2014. The meeting was opened by Eugene Pozniak (Programme Director, European CME Forum; UK), who began by posing the provocative question “Is European CME in crisis?” With many uncertainties in the way that European CME will be funded, accredited and regulated in the future, the seventh gathering of the forum could not have occurred at a more opportune time.

Meeting attendees were then provided with an overview of significant milestones in European CME over the previous 12 months (Table 1). Firstly, an update on developments with respect to the Journal of European CME was provided. This online-only, open-access journal accepts manuscripts that report on any aspect of CME–CPD practice (see The Editor-in-Chief is Professor Robin Stevenson (UK) and the journal is published by Co-Action Publishing. Target time to publication from manuscript submission is now 6–8 weeks. An update on the activities of the Good CME Practice Group (gCMEp; see was also provided. Membership of gCMEp now extends to 18 European CME providers. gCMEp is currently developing a CME toolkit that will provide guidance on securing funding for CME-accredited activities and detailed instructions on the effective delivery of these activities.

Table 1.  Milestones in the 12 months prior to the 7th European CME Forum.
Time period Milestones
Q4 2013–Q1 2014
  • December 2013: European Directive 2013/55 on the recognition of professional qualifications released by the European Commission, focuses on national CME in accreditation in Europe
  • December 2013: GSK announces it will move away from direct sponsorship of individual healthcare professionals to arm’s length funding and will stop making direct payments for speaker engagements
  • February 2014: 2nd UEMS-EACCME provider meeting, Brussels
Q2 2014
  • May 2014: gCMEp Spring Meeting, London
  • June 2014: GAME Meeting, Miami
  • June 2014: EBAC Provider Meeting, Frankfurt
Q3 2014
  • August 2014: AMEE: Essential Course in CME (University of Toronto) + specific sessions on CME led by GAME
  • August 2014: ACCME letter to CMS to state that CME should not be part of the US Physician Payments Sunshine Act
  • August 2014: ACCME announces that accreditation is now open to non-US providers
  • August 2014: US Physician Payments Sunshine Act – reporting goes live
  • September 2014: EFPIA announces it is considering an approval system for independent education
  • September 2014: 3rd Cologne Consensus Conference
Q4 2014
  • October 2014: GAME regional meeting takes place in Mumbai, India
  • October 2014: EUCOMED statement – direct industry sponsorship of healthcare professionals to third-party organised conferences to be phased out by 2018
  • October 2014: President Putin gets involved in CME – announces that Russia must have a CME system in place by end of 2016
  • November 2014: Two critical ACCME publications on the effectiveness of CME, and the relationship between commercial support and bias in CME activities are published
  • November 2014: European CME Forum “Day 0” – involved meetings of gCMEp, iPACME, scientific societies and accreditors
ACCME, Accreditation Council for Continuing Medical Education; AMEE, Association for Medical Education in Europe; CMS, Centers for Medicare & Medicaid Services; EBAC, European Board for Accreditation in Cardiology; EFPIA, European Federation of Pharmaceutical Industries and Associations; GAME, Global Alliance for Medical Education; gCMEp, Good CME Practice Group; iPACME, International Pharmaceutical Alliance for CME; UEMS-EACCME, European Union of Medical Specialists-European Accreditation Council for Continuing Medical Education.

To plan the programme for the 7th Annual Meeting of the European CME Forum, a formal online needs assessment survey was conducted among CME stakeholders. The survey was promoted by email and social media (LinkedIn, Twitter) and responses were gathered from 81 individuals. A number of different scenarios were identified as those that currently cause the most concern for CME stakeholders (Figure 1). When asked to rate potential topics for discussion at the forum in order of importance, two issues not often discussed at similar meetings were the highest ranked: firstly, “measuring outcomes of CME programmes” and secondly, “transparency of relationships between industry and healthcare professionals.” Other high-ranking topics were, in order, “how CME can affect clinical practice,” “how to guarantee the quality of CME programmes,” and “how to carry out a good needs assessment.” Accordingly, the programme for the 7th Annual Meeting was developed to ensure it was as relevant as possible to CME stakeholders’ current concerns and interests.

Fig 1
Figure 1.  Responses to the pre-forum survey question, “What scenarios cause you the most concern?” (n=81).

Session 1: What is the need for assessing needs?

The first session of the meeting was led by Kiki Lombarts, Professor of Professional Performance at the University of Amsterdam (Netherlands), and was focused on the “need for needs assessments in CME,” a topic that is undoubtedly becoming an increasingly important component of any CME programme delivered in Europe. Professor Lombarts began, however, by highlighting key conclusions on the effectiveness of CME contained in a report commissioned by the US Accreditation Council for Continuing Medical Education (ACCME) and published in 2014. This synthesis of systematic reviews concluded that CME positively impacts both physician performance and patient outcomes (although the former is more reliably impacted than the latter). Notably, the report also acknowledges that many commentators consider the CME system to be deeply flawed, citing, for example, the focus on participation rather than improvement, as well as the lack of emphasis on helping healthcare professionals, enhances their competence and performance in daily practice. However, the idea that the value of CME should rest on its capacity directly to influence practice is challenged by other experts who consider this to be an “impoverished view of how change in clinical practice actually occurs.”1

As acknowledged in the ACCME report, a proven way to increase the effectiveness of a CME activity is to ensure it is based on an assessment of need. Professor Lombarts asked forum attendees to answer the question, “Before you design/organise/attend/accredit a CME event, do you perform a needs assessment?” The answer “Eehhhh … I really should” was selected by 24% of attendees; “Yes, but I probably could be doing it in a more robust way” by 46%; and “Duh … I am a professional!” by 31%. The purpose of a needs assessment is to identify the gap between what is needed and current levels of practice. It is important to distinguish between the educational needs of the collective (i.e. those that relate to the gap between current professional performance/patient health outcomes and the highest quality performance/best possible health outcomes) and the learning needs of the individual (which can be identified by an exploration of the issues that created the gap in each individual).

The three pillars of professional performance in doctors

In the second part of the session, Professor Lombarts shared part of her professorial acceptance speech entitled “Professional performance of doctors: between time and technology”. The importance of professional values is captured by a quote from Dr Donald Barwick: “When values are weak, rules are insufficient; when values are strong, rules are not necessary.” For doctors and other healthcare professionals, it is possible to identify three pillars of professional performance (Figure 2). The first pillar – continuously striving for excellence – is characterised by intrinsic motivation; a modest and humble nature that allows acknowledgement of one’s own limitations and a willingness to learn from others; a dedication to knowledge; and a drive for improvement that leads to the seeking-out of challenging activities further to improve performance. The second pillar – compassionate care – can be achieved by following the maxim “treat others how you want to be treated,” and by acknowledging the importance of the health care professional–patient relationship. Unfortunately, empathy among doctors is declining, a development driven by two main factors: confrontation with clinical reality and experience of distress (e.g. lack of time, lack of role models, cynicism and arrogance). The ideal healthcare professional has a cool head and a warm heart and Professor Lombarts asked forum attendees to consider whether there is a role for CME in facilitating compassionate care. Some attendees thought that there was no such role for CME because, for example, “CME has its borders,” while others disagreed and thought that CME could make a contribution, at the very least in raising awareness of the existence of the three pillars of professional performance.

Fig 2
Figure 2.  The three pillars of professional performance in doctors (Lombarts).

Accountability is the third and final pillar of professional performance and, if present, ensures that trust from patients and society in individual professionals and in collective professions is gained or maintained. Professor Lombarts asked each member of the audience to consider the main focus of the last CME event that he/she attended, organised and/or accredited. Overall, 74% of attendees stated that the main focus was on contributing to excellent professional performance, while only 6 and 21% stated that the main focus was on contributing to more humane professional performance or more accountable healthcare delivery, respectively.

Needs assessment in multiple sclerosis

To illustrate some of the concepts discussed in Session 1 with practical examples, Suzanne Murray (AXDEV Group) and Veronique Moy (Merck Serono) presented details of three complementary and iterative international needs assessments performed in the area of multiple sclerosis. The way in which these assessments contributed to the design of needs-based educational activities was strongly emphasised. The presentation also demonstrated the importance of moving needs assessment findings beyond CME, including the potential to disseminate findings through multiple channels, thereby increasing awareness of clinical practice challenges among key stakeholders. Furthermore, the potential to leverage needs assessment findings for strategic and operational planning, with the ultimate goal of health system improvement, was also discussed.

Session 2: How will CME be funded?

This discussion session explored how sources of funding for CME are changing and how external factors such as the US Sunshine Act may affect future funding models. Session chair Jacqui Thornton (Independent, UK) began by asking Professor Reinhard Griebenow (European Cardiology Section Foundation and University of Cologne, Germany) to provide an overview of how CME is traditionally funded in Europe. Professor Griebenow reminded attendees that the European Union comprises 28 member states and that CME funding models are not uniform across EU countries; therefore it is difficult to talk in general terms about the “situation in Europe.” Nevertheless, there are certain similarities across countries, perhaps most notable of which is that the great majority of education at regional and local levels is actually “informal” learning (i.e. unaccredited activities generally organised by physicians or other healthcare professionals). Of those activities that are accredited, a decreasing proportion is funded by industry. Michel Ballieu (European CanCer Organisation [ECCO], Belgium) stated that around 6 or 7 years ago, approximately 70% of CME activities accredited by the Accreditation Council for Oncology Education (ACOE) – the European Specialty Accreditation Board (ESAB) run by ECCO – were supported by industry; now this figure is below 50%. Perhaps surprisingly, this trend has had no adverse impact on participation rates; in fact, participation in ACOE-accredited events is growing, presumably reflecting increased use of alternative funding models (e.g. institutional support; self-funding). Notably, while many observers appear to assume that CME funded by industry is inherently biased, this view is not supported by the available evidence. In fact, a recent literature review commissioned by the ACCME did not identify any data-based research to support or refute the hypothesis that commercial support leads to bias in accredited CME.2 The same review noted that, in post-programme evaluations, physicians perceive levels of commercial bias in industry-funded programmes to be very low and at a similar level to those that occur in programmes that are free from commercial support.

As in Europe, only a relatively small percentage of CME activities in the US are funded by industry, according to Kate Regnier (ACCME, USA). Of the approximately 140,000 activities offered by ACCME-accredited providers in 2013, around 17% received commercial support. Undoubtedly the biggest difference in CME funding practices between Europe and the US is that, in the latter, participants in accredited CME activities cannot receive funding from industry directly. Instead, provision of commercial support grants to accredited providers is the only method by which industry can fund US CME. The session panellists discussed whether a move towards this indirect method of industry support is happening in Europe. Although there are undoubtedly some examples of this – notably in the announcement by GSK in December 2013 that it would stop direct funding of healthcare professionals participating in CME activities – this is not a universal trend. Dr Eva Thalmann (Janssen Cilag) noted that some pharmaceutical companies are actually increasing CME funding levels. The distinction between direct and indirect industry funding of CME was considered by some forum attendees to be a matter of semantics; instead, it was argued, the important issue is whether there is clarity of process and transparency. In response, Michel Ballieu noted that the UK in particular is very advanced in terms of anti-bribery law and thus this distinction is not a semantic one and can, in fact, have a serious impact on a medical society organising a CME-accredited meeting, for example. In the experience of one UK healthcare professional attending the forum, direct industry sponsorship of CME is now less common than in the past and there is now more focus on local meetings and e-learning, the latter being particularly easy to fit around work and home-life commitments.

Another model of CME funding observed in the US and perhaps likely to emerge in Europe is that of pooled funding. In this model, multiple pharmaceutical companies and/or other stakeholders contribute to a single funding pot for a particular CME activity. Although this model might seem attractive at first glance, it could in some experts’ opinions lead to a marked reduction in industry support due to the fact that companies may be reluctant to pay into a pool because of concerns regarding potential off-label promotion or the need to show to tax authorities that they are funding areas of relevance. Such fears could be assuaged, however, if pooled funding is sufficiently targeted to a specific, relevant CME activity. In the US, for example, the Food and Drug Administration has obliged all pharmaceutical companies producing extended-release and long-acting opioids to participate in a pooled fund for the commercial support of CME activities on safety education and training. Major initiatives in diabetes education are underway in Spain and France in which various stakeholders are making different types of contribution (e.g. funding from government; time from academics/healthcare professionals). Insurance companies are another potential stakeholder in CME. Kate Regnier confirmed that the ACCME does accredit professional liability insurance companies, as well as permitting them to fund CME activities. In Europe, insurance companies currently do not provide education themselves, but there have been some cases in which they have provided funding.

Discussions then refocused on the funding situation with respect to informal, non-accredited educational activities. Although these activities are considered by some to be the most important contributor to learning, they remain severely underfunded. In the UK at least, medical schools have effectively turned their back on CME. Thus, relevant stakeholders such as the UK General Medical Council, or the medical schools themselves, should be encouraged to fund and provide CME. There could even be an argument for industry support of CME in medical schools although this would be controversial. Alternatively, it could be argued that government should contribute to the costs of CME activities, in particular those that aim to maintain competence. Such spending would likely be cost-effective if the amount of money reserved for medical negligence claims is taken into account. Currently, the lack of financial support for informal, non-accredited CME is concerning, particularly as performance levels of healthcare professionals are decreasing as the general population ages. The US Physicians Payments Sunshine Act is now active and obliges pharmaceutical companies to report direct and indirect payments and other transfers of values to US physicians and teaching hospitals. Commercial support grants provided by pharmaceutical companies to ACCME-accredited providers are currently exempt from reporting, as long as the grants meet established criteria, including that the funding company has no control over the content of the activity or the selection of the speakers/authors of the activity.

Session 3: e-Learning innovations and potential

Session 3 reviewed and examined the latest technology in medical e-learning. Session chair Professor Peter Henning (Karlsruhe University of Applied Sciences, Germany) emphasised that the focus would be on providing practical examples of what is happening now in e-learning, rather than on the underlying theory.

e-Learning innovations and potential

Carsten Germer (CompuGroup Medical, Germany) described how different e-learning approaches can be used in CME. A particularly exciting development has been the creation of virtual patients such as those who “attend” the 28 specialty clinics that form part of the INMEDEA Simulator.3 Using such systems, patients and clinics can be created and modified as required and medical pathways for different diseases can be adapted. Other advantages of these virtual simulations include their compatibility with interactive learning methods and the ease with which they can be accredited for CME credits. Adapted portals of virtual clinics have been generated for pharmaceutical companies and for non-profit organisations such as the German Bundeswehr. One project delivered in 2012/2013 for a local physician association in South West Germany featured 15 virtual post-mortem examinations and was successfully completed by approximately 2000 physicians, each of whom received up to 45 CME credits. Current challenges in the field of e-learning include: a lack of defined standards/quality control measurements; the fact that, in Germany, e-learning, has become a distinct category (“Fortbildungsordnung”) but is still not widely accepted by German institutions; and the fact that users need to be guided (because if users encounter technical difficulties with e-learning, most will stop immediately). Compared with individual e-learning at home or in an office, blended learning events involving a speaker and/or an e-learning guide seem the best option to ensure effective e-learning.

The potential of virtual patients in medical education and CME

Professor Martin Haag (Heilbronn University of Applied Sciences, Germany) talked further about the use of virtual patients in CME and medical education in general. Virtual patients can be defined as a “specific type of computer programme that simulates real-life clinical scenarios”; using this system, “learners emulate the roles of health care providers to obtain a medical history, conduct a physical exam, and make diagnostic and therapeutic decisions.”4 First developed in the early 1970s, virtual patients have grown considerably in their complexity and capabilities. However, use of virtual patients in CME is not yet widespread, particularly in comparison to its use in medical education. In addition to the challenges described by Carsen Germer, barriers to adoption of virtual patients in CME include their significantly higher costs compared with traditional CME activities, difficulties with finding sponsors, and the large amount of effort involved in development of virtual patient platforms. The processes involved in the creation of a virtual patient are summarised in Figure 3. Unsurprisingly, it can take many hundreds of hours to develop a virtual patient, depending on the complexity of the patient, the interactivity employed, and multimedia platforms utilised. Despite such challenges, virtual patients are likely to become ever more important in CME. This trend is likely to be driven by a lack of appropriate “real-life” patients and also the general desire to decrease the amount of time and money that is spent supporting live CME activities. Interestingly, acceptance of virtual patients among medical students and their teachers is high and this approach is considered an effective learning method by these groups. As virtual patients are expensive to develop, different strategies have been employed to lower costs. One such strategy is to share and repurpose existing cases with other partners. For example, the electronic Virtual Patients (eViP) project, co-funded by the European Commission, has created a repository of 320 repurposed and enriched virtual patients, all of which are available under a Creative Commons License (see

Fig 3
Figure 3.  Processes involved in authoring a virtual patient (Haag).

Interdisciplinary e-learning for diabetes specialists – Is multi-accreditation the future?

At present, most CME activities are only targeted at a single profession. Thomas Kleinoeder (KWHC GmbH, Germany) reported the first results of a project aimed at enabling “multi-accreditation” of an e-learning CME activity by different physician and nurse organisations. The project was aimed at a range of German diabetes specialists including general physicians, diabetes consultants, diabetes assistants, nurses and medical assistants. The approach adopted involved the identification of topics with common learning needs, development of various e-learning materials, and accreditation by a relevant medical association and diabetes nurses organisation. E-learning materials included animated slides, video interviews, checklists, downloads and participant tests. Over the first few weeks of the project, over 700 individuals had participated, 250 certificates of completion had been issued, and no significant technical issues were reported. The project was particularly attractive to diabetes nurses who participated in higher numbers than physicians. These initial findings suggest that depending on the topic to be covered, interdisciplinary e-learning can make sense, providing that learning needs for the different professions can be matched. For this approach to gain widespread acceptance, common criteria for accreditation will need to be developed.

Future trends

Peter Henning closed the session by predicting some of the future trends in e-learning. He noted that it is unarguable that the world is witnessing an explosion in the use of smartphones and that “24/7 connectivity” has become a new lifestyle for many people, especially the young. People have become used to seeking-out any kind of information desired on the internet and thus there have been rapid changes in informal methods of learning. However, with a few exceptions, learning organisations are not changing in response to these new demands. In the future, information will need to be increasingly personalised, a trend that should offer new opportunities for accreditation providers. Learning will also become increasingly independent of space and time – after all, who wants to learn in a classroom? – and will develop into a much more personal experience in which the learner is in control of the learning process. Of note, Europe can be considered to be a leader in terms of the development of these adaptive learning environments. Overall, there is no escaping the fact that we are about to witness a dramatic increase in the use of e-learning in CME.

Session 4: CME on trial

This panel session was moderated by Jacqui Thornton and addressed the deliberately controversial questions, “What is the point of CME accreditation?” and “Why do healthcare professionals want it or need it and what precisely are the roles of the provider and accreditor?” Providing a physician’s point of view, Mark Westwood (St Bartholomew’s Hospital, UK) stated that CME forms an essential part of revalidation and, in that respect, is very important; beyond that, however, the value of accreditation is perhaps less easy to define. Jennifer Gordon (Royal College of Physicians and Surgeons of Canada) suggested that accreditation can act as a marker of quality, offering reassurance that a certain level of adherence to ethical and educational standards has been achieved. She also emphasised, however, that there are many valuable education activities that for various different reasons are not accredited.

In many people’s experience, there is a wide variation in the quality of accredited events within Europe and thus it was generally agreed there needs to be marked change in the way that European accreditation of CME is managed and regulated. The main difference between Europe and the US with respect to accreditation procedures is that, in the US, it is generally the CME providers that are accredited, while in Europe, accreditation is almost always activity based. Much of the discussion in the session focused on whether the European or US approach to CME accreditation was the most effective, although most contributors were reluctant to make such a judgement. An important distinction is that in the US, there is a predominant national system (ACCME), while in Europe the situation is much more complex due to the fact that accreditation procedures differ by country. Caroline Hager (Directorate General for Health and Consumers, European Commission) provided an update on some of the work performed by the European Commission to understand variations in accreditation procedures across Europe.5 This work has confirmed that there are huge differences in procedures, regulations, and in the type and understanding of terminology employed with respect to CME. One of the main future aims of the European Commission is to overcome some of this complexity by enhancing transparency and promoting the exchange of good practice.

Regulation of CME is an on-going issue and there is some concern that a “regulatory vacuum” is appearing in Europe that may necessitate a shift from the current model of self-regulation to one involving some degree of governmental control. Reinhard Griebenow stated, however, that CME stakeholders should be very wary of encouraging government regulation, as the evidence from other sectors suggest that it does not work and can lead to a “regulatory jungle.” From a physician’s perspective, Mark Westwood suggested that, while most people agree there is a real need for change, he is not too concerned about exactly which method, or methods, of accreditation are chosen. What is clearly needed, however, is a more rigorous assessment of what is happening in individual CME activities; currently, quality of learning varies far too much.

Another key question is whether CME has a real effect on patient outcomes and/or safety. Professor André Tichelli (European Hematology Association & University Hospital Basel, Switzerland) and Mark Westwood agreed that, while it is easy to see a potential link between CME and outcomes, there is currently little direct evidence to prove this relationship, perhaps largely because of the technical difficulties involved in studying it. Caroline Hager stated that the European Commission had attempted to find published evidence of a link between CME and patient safety but that very little information was available. However, delegate discussion identified that there are in fact many projects that have shown a link between CME and outcomes; sadly, they are very often not being published. Reasons for non-publication include a lack of resource (e.g. hospitals involved in the projects do not have the means to take the evidence into the public domain), as well as a desire to withhold intellectual property. Providing a different perspective on accreditation, Robin Stevenson stated that he is fairly relaxed about the importance of this process, as the really important educational activities are those that go on informally in the hospitals. Indeed, it is vital that everyone appreciates the value of this “old-fashioned” learning. Professor Don Moore (Vanderbilt University, USA) noted that this fact has been recognised in the US where it is possible to designate CME credit for informal activities including journal clubs and other regularly scheduled series.

Session 5: In conversation with …

In this session, leading experts in CME explored the issues of the day in a series of head-to-head conversations. Robin Stevenson and Don Moore began by contrasting the current situation in Europe and the US with respect to CME/CPD. Professor Stevenson noted that, when browsing the websites of big US institutions, he nearly always sees a “Department of CME.” In contrast, this is rarely observed in Europe; a situation that is consistent with a general lack of investment in CME at the hospital level. It is saddening that in Europe, CME specialists do not have input into hospital-based learning in the same way as in the US. Europe is faced with worsening levels of clinical performance and, unfortunately, the response of governments to this problem has been somewhat aggressive; for example, implementation of mandatory revalidation for doctors. It could be said that a “stick” approach for managing clinical performance is branded in Europe, whereas in the US a “carrot” is used. Turning towards possible future trends in CME, Professor Moore stated that he believed CME will be more focused on performance than on simply “sitting in a lecture theatre.” Such a change, however, will require a different approach to learning and will necessitate that all involved in CME understand the underlying educational issues. It is vital to consider the design of an educational activity to ensure effective knowledge transfer and changes in behaviour. Another important trend will be inter-professional education. The challenge here lies in how to bring the whole team together so that they all benefit from a CME activity. The idea of the “adaptive workplace learner” is also likely to take hold. In medicine, change is rapid and constant. For a physician to work in this environment, it is necessary to develop adaptive expertise, as routine expertise only works some of the time.

The role of industry in CME was discussed by Dr Eva Thalmann and Dr Edwin Borman (European Union of Medical Specialists-European Accreditation Council for Continuing Medical Education [UEMS-EACCME] & Shrewsbury and Telford Hospital NHS Trust, UK). Dr Thalmann acknowledged that while the provision of financial support is undoubtedly a key function of industry, companies such as Janssen Cilag hope to be seen as a valued partner in CME, not just as a financial contributor. Industry has a lot of medical, scientific and other knowledge to share and, like all CME stakeholders, wants to contribute to the basic goal of improving patient care. With respect to regulation of CME, Dr Borman described how there has been a lot of debate in recent times on whether self-regulation can be successful or whether some degree of governmental control may be required. Dr Thalmann believed that both the medical profession and industry are capable of self-regulation, highlighting the massive changes that have occurred over the past 6 or 7 years in terms of transparency. Nevertheless, further travel in this direction is required and it is likely that the next 6 or 7 years in CME will also be focused on disclosure, transparency and the general need for greater openness. If in this time all partners in CME can prove that self-regulation is effective, it will then be possible for everyone to focus on what really matters; namely, healthcare professional education and improving patient care.

Lawrence Sherman (Prova Education, USA) began his conversation with Edwin Borman by posing the question, “What do you, as an accreditor, think are the problems we as providers have with the current CME system?” Dr Borman recognised many issues for providers, not least the question of simple survival in a highly competitive environment. There are an increasing number of new players in the system, and while many of these are responsible providers, others are less reputable. In terms of processes, it is clearly not easy to organise a conference or other CME activity. The conversation also covered the topic of how reviewers of CME proposals are selected by accreditors. Dr Borman stated that while a certain amount of self-selection occurs (i.e. the system relies on volunteers), it is important that reviewers can provide examples of relevant expertise in CME. To ensure quality and consistency of the accreditation process, EACCME does cross-check some reviews; for example, office staff will flag to the Secretary General if there is considerable discrepancy between two reviewers. One source of frustration identified by some providers is the sometimes inappropriate feedback received from accreditors on approaches to learning, particularly as this can sometimes seem to originate from an individual lacking specific learning knowledge. Dr Borman described how, at the EACCME, some reviewers are very experienced in learning; furthermore, there are steps in place to ensure feedback on learning approaches, or on any other aspect of a proposal, is factually correct. In terms of innovation in CME, Dr Borman stated that he believes CME providers should stretch accreditors in terms of the type of activities they submit. For example, “bite-size” learning activities (e.g. 15-minute podcasts) are likely to become increasingly important. At present, there is no method for accrediting such short periods of learning, although bundling of these activities may prove to be the best approach.

Session 6: The measure of outcomes

In this role-playing session, Don Moore – Professor of Medical Education and Administration at Vanderbilt University School of Medicine and developer of what has become known as “Moore’s Pyramid” or the “Seven Levels of Outcomes”6 – acted as a CME/CPD planner who was holding a planning meeting with a CME course director (an endocrinologist and director of a diabetes centre played by Brittany Grohs (PCM Scientific, UK)). The learning objectives of the session were to enable delegates to describe and discuss: (1) an outcomes framework for planning and assessing CME activities; (2) using backward planning with the outcomes framework; and (3) one approach to planning and assessing a CME activity using backward planning and the outcomes framework.

The goal of the imagined meeting was to discuss planning for a 3-hour session for general practitioners (GPs) on diabetes during a 2-day conference. At the start of the meeting, the CME/CPD planner asked the course director what she hoped to achieve in the course. Her answer (“There has been some important research in the last year that GPs need to know about so they can manage their patients with diabetes”) led the planner to state that it would be necessary to gather more detailed information in order to accomplish what the course director wanted. Turning to a diagram of the outcomes framework (Figure 4), the planner stated it would help them make decisions about how to plan an educational activity that will achieve the desired results. To understand the framework, it is important to realise that an outcome is defined as the result of an action or activity, while an educational outcome is the result of an educational activity. The outcomes framework includes seven levels of outcomes, the first of which, “participation,” may be quantified by the number of physicians who participated in a CME activity, or as the number who are engaged in such an activity. Progression through the different levels of outcomes captures the degree to which: participants have had their expectations met (“satisfaction”); participants have learnt about the topic in question (“learning”); participants can show in an educational setting how to do what they have been taught (“competence”); participants can do what they have been taught in their practices (“performance”); the health status of patients improves due to changes in the practice behaviour of participants (“patient health”); and, finally, the health status of a community of patients improves due to changes in the practice behaviour of participants (“community health”).

Fig 4
Figure 4.  Outcomes framework for planning and assessing CME (Moore).

For most CME activities, the most realistic outcomes are changes in competence and/or performance. If improvements in patient or community health are desired, a research project is usually needed to support identification of desired outcomes, as well as the level of evidence that would be required to show that the outcomes had been achieved. In the meeting played out during this session, it became clear that there were insufficient available data to accurately define a target outcome in terms of community health (e.g. “to reduce the percentage of patients in the community with metabolic syndrome from 30 to 15%”) or patient health (e.g. “to reduce glycosylated haemoglobin from 11.5 to 7.0”). Instead, the course director believed that focusing on GP competence would be the most effective approach. Competence would be defined as the level of adherence to a key diabetes guideline achieved by the GPs during the learning activity itself. Of note, the course director believed that currently these guidelines were followed completely only 50% of the time. It was not considered possible to measure performance (i.e. adherence to the guideline in practice) as there were insufficient resources to obtain performance data; furthermore, it is unclear if all the GPs would cooperate with data collection.

During planning, it is important to ask for each level of the framework, “What is?” and “What should be?” (i.e. “What is the gap?”). By starting with the end in mind, the desired result can be identified, as can the level of evidence required to prove the result was achieved. This backward planning process then moves on to planning the learning activities necessary to achieve the desired outcome (the “plan for learning”). In the current example, for competence, “what is?” was identified as “guideline adherence to the diabetes guideline = 66%” and “what should be?” was defined as “guideline adherence to the diabetes guideline = 100%.” According to the planner, most patients with diabetes referred from the community to specialist care are out of control in terms of diabetes care, are obese, and have laboratory findings that suggest metabolic syndrome. Furthermore, evidence from various studies suggests that using insulin has helped patients who have metabolic syndrome and difficulty controlling their blood glucose. Therefore, the planner believed it would be beneficial for the GPs taking part in the CME activity to learn how to use insulin with these patients. The planner therefore recommended that three scenarios be developed and used at the beginning of the activity to determine the participant’s level of understanding of the use of insulin in managing patients with difficult-to-control diabetes. The help of faculty should be solicited to develop these scenarios and other content of the activity, making sure that it is relevant to accomplishing the desired results. In general, too many people think the desired result of an educational activity occurs by a “miracle”; in fact, educational outcomes do not just happen, they have to be planned for deliberately. A final critical step is to ensure the outcomes are measured. In this case, a “commitment to change” statement was developed which participants were requested to sign immediately after the activity. This statement confirmed participants’ belief that they had developed or improved their ability to use insulin therapy to manage difficult-to-control diabetes and that they would incorporate this knowledge into their practice. It also confirmed participants’ willingness to respond to a request for information about the results of this commitment 6 months later.

Clinician’s International Treatment Quality Manifesto project

At the end of Session 6, Celeste Kolanko and Alisa Pearlstone (PCM Scientific) presented a case study that illustrated a methodology for developing a commitment to change programme during a series of live CME events. Performed in the area of addiction, the project began in 2010 with a needs assessment of 300 clinicians treating opioid-dependent patients in four countries. This exercise identified a number of issues, of which the most critical were those associated with suboptimal management of safety issues (specifically: too much prescribing of benzodiazepines; little cardiac screening and poor knowledge of cardiac issues; poor knowledge of “red-flag” drug interaction combinations) and poor standards of care (suboptimal use of psychosocial interventions; limited multidisciplinary collaboration and communication). The response to these issues was to develop an annual 3-day CME-accredited conference that was attended by 250–500 GPs, additional psychiatrists, and pain specialists from up to 30 countries and supported by multiple sources of funding from industry (Schering Plough, MSD, and RB Pharmaceuticals). At each conference, delegates were encouraged to be “active learners” by submitting text-based messages that provided their ideas for how the presented data and expert opinion could translate into changes in their clinical practice. These ideas were then amalgamated into thematic segments and shared with an expert faculty panel as proposals for change. Through the structured process illustrated in Figure 5, each idea for change in practice was debated and considered and, if accepted by >75% of delegates, adopted and entered in the international treatment quality manifesto and follow-up study. Around 12 new items were included in the manifesto each year. Delegates were provided with a copy of the manifesto and then participated in a comprehensive 12-month survey aimed at measuring the implementation and maintenance of each adopted item. Each year, data from the project have shown a high rate of practice change across all areas. For example, in 2011, 75% of responders reported a reduction in benzodiazepine prescribing while, in 2012, there was a 19% increase in the detection of cases of QTc prolongation, driven by a marked rise in cardiac screening.

Fig 5
Figure 5.  Process for creating a commitment to change programme during a series of live CME events: the Clinician’s International Treatment Quality Manifesto project (Kolanko & Pearlstone).

Lunch with the learners

The second lunchtime of the meeting featured the now regular forum in which a panel of learners provided their opinions on, and personal experiences of CME and medical education. The session was chaired by Lawrence Sherman and the panel comprised three cardiologists from the UK, Dr Debashish Das (Whipps Cross Hospital), Dr Abhishek Joshi (London Chest Hospital), and Dr Dan Sado (King’s College Hospital). To start the session, the panel were asked to describe what they liked and what they did not like about the learning experience in medicine. The ability to perform a task that previously eluded them, the deconstruction of complex information, and the use of experiential learning were all identified as possible indicators of a good learning experience. In contrast, a poor experience might be characterised by making information unnecessarily complicated, by an inappropriate focus on a sponsor’s product, or by the education method used being too obvious (“seeing the theory poking the content”). Elaborating on the latter point, Dr Joshi stated that if it becomes clear what education structure has been used for a particular event, it can lead to a stilted, non-responsive approach to learning. Many healthcare professionals have a basic – and some a more advanced – understanding of learning theory. CME providers should be aware of this and ensure that the tools used to deliver information do not drown out the information itself.

The panel had mixed experiences with e-learning. One problem, at least within the UK National Health Service, is that e-learning is often used for the most mundane of tasks; for example, the completion of health and safety training upon initiation of a new job. This can lead to a negative perception of this method of learning. If e-learning is didactic and lacking interactivity, it can be considered no better than a textbook. Incorporation of new technologies, especially gamification, would be a welcome advance to the panel’s e-learning experiences. The panel were asked to describe what the impact of a pharmaceutical company providing funding for a CME event would be on their perception of that event. In response, it was stated that the content and format of the event itself would be the main influencers. The worse experiences are those in which the event is focused on a company’s product; for example, one of the panel had recently attended a training session on presentation skills that turned out to be a lesson on how to sell the sponsor’s drug. This kind of experience leaves a poor impression. In contrast, however, some industry-sponsored learning can be very effective. The potential value of multidisciplinary training was also discussed. All of the panel could see the value of this approach: as a doctor progresses through his career and becomes more and more specialised, it becomes increasingly necessary for him or her to rely on team members with different types of expertise. Although the need for multidisciplinary training was clear, to date the panel’s experiences with such learning experiences was very limited.

Session 7: CME quality and compliance challenge

In this innovative session, two delegates were invited to participate in a mock pitch for CME approval. The delegates acted as CME providers pitting their wit against a panel of “CME investors” who provided the perspective of a CME accreditor (Reinhard Griebenow), an education expert (Dr Jonas Nordquist, Karolinska Institutet, Sweden), a course director (Mark Westwood), and a learner (Dan Sado). Delegates were also able to ask questions of the speakers.

Toby Borger (Springer Healthcare) presented details of a theoretical CME-accredited programme on the management of gout to be aimed at UK GPs and funded by the pharmaceutical industry. The second pitch was made by Götz-Johannes Peiseler (Omnia-Med), who represented a CME provider that interacts with approximately 5000 GPs a year, continually asking them about their interests and needs. The activity described by Dr Peiseler was an educational programme on novel oral anticoagulants (NOACs). After each presentation the “CME investors” and delegates questioned the speaker, elucidating further details with probing questions and points for discussion.

Session 8: The CME unsession

The final session of the forum was an unstructured session designed to ensure delegates had the opportunity to discuss any remaining issues and pose remaining unanswered questions. Led by Lawrence Sherman, the session covered a variety of topics including the difficulties in, and solutions for, attracting GPs to participate in CME activities; the need to bring together existing e-learning content into a single “on-demand” electronic portfolio; and the on-going challenge in ensuring the appropriate disclosure of potential conflict of interests. A detailed discussion was held on whether disclosures should focus solely on financial conflicts of interest or whether non-financial ties such as services-in-kind or personal relationships should also be covered. In Europe, although the focus is generally on financial support by industry, there is a general expectation that a faculty member will disclose anything that might be considered relevant. Of course, the challenge faced by other CME stakeholders is that this process is reliant on the honesty of the individual making the disclosure. The provision of industry sponsorship for attendance at medical meetings was also discussed. In general, this practice is now declining, although there are some situations in which this may still be considered valid; for example, the provision of travel fellowships for junior doctors.


The Seventh Annual Meeting of the European CME Forum was held at a time of increasing uncertainty about the future direction of CME in Europe. Around 100 delegates gathered from Europe and beyond to participate in a meeting that featured lively discussions and stimulating debates on a variety of important topics including the need for reform of accreditation, on-going uncertainty about funding, and the potential future direction of travel with respect to the regulation of CME in Europe. The meeting also featured presentations that showcased new and evidence-based approaches to e-learning and the measurement of educational outcomes. Looking forward, there are many unanswered questions regarding the future of CME in Europe. There is no doubt that there will be much new to discuss at the 8th Annual Meeting of the European CME Form which will take place in Manchester, UK on 11–13 November 2015.


  1. Cervero RM, Gaines JK. Effectiveness of continuing medical education: updated synthesis of systematic reviews. A report commissioned and funded by the Accreditation Council for Continuing Medical Education; 2014. Available at:, accessed December 9, 2014.
  2. Cervero RM, Gaines JK. Is there a relationship between commercial support and bias in continuing medical education activities? An updated literature review. A report commissioned and funded by the Accreditation Council for Continuing Medical Education; 2014. Available at:, accessed January 28, 2015.
  3. Braun J, van den Berg R, Baraliakos X, Boehm H, Burgos-Vargas R, Collantes-Estevez E, et al. 2010 update of the ASAS/EULAR recommendations for the management of ankylosing spondylitis. Ann Rheum Dis 2011;70:896–904. PubMed Abstract | PubMed Central Full Text | Publisher Full Text
  4. Cook DA, Triola MM. Virtual patients: a critical literature review and proposed next steps. Med Educ 2009;43:303–11. PubMed Abstract | Publisher Full Text
  5. Putrik P, Ramiro S, Kvien TK, Sokka T, Pavlova M, Uhlig T, et al. Inequities in access to biologic and synthetic DMARDs across 46 European countries. Ann Rheum Dis 2014;73:198–206. PubMed Abstract | Publisher Full Text
  6. Moore DE, Jr., Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof 2009;29:1–15. PubMed Abstract | Publisher Full Text

Correspondence: Eugene Pozniak, Siyemi Learning, University of Manchester Innovation Centre, Arch 28, North Campus Incubator, Sackville Street, Manchester M60 1QD, UK. E-mail:

About The Authors

Eugene Pozniak
Siyemi Learning, Manchester
United Kingdom

Rick Flemming
Aspire Scientific Ltd, Bollington, Cheshire
United Kingdom

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Related Content