The Results Are In: Pilot Testers Give MOCA-Peds "Thumbs Up"
By Trevor Fleck, MPA
The American Board of Pediatrics (ABP) MOCA-Peds 2017 pilot is off to a strong start. All pediatricians whose MOC examination was due to be taken in 2017 were eligible to participate in the pilot. Of those, more than 5,000 are participating. Nearly all pilot participants (98%) completed the 20 questions available in Quarter 1 (Q1). Responding to a follow-up survey, most reported that the questions are relevant, adequate support is available, and the technical delivery system functions effectively. These results were gratifying for ABP staff members and the dozens of pediatrician volunteers who’ve been working more than 16 months to develop and launch the pilot.
To recap: MOCA-Peds is a new method for assessing the knowledge of pediatricians as part of Maintenance of Certification (MOC). An alternative to the examination given at a secure testing facility every 10 years, MOCA-Peds delivers 20 multiple-choice questions to a participating pediatrician’s portfolio quarterly. These questions can be answered any time during that quarter at the diplomate’s convenience.
Once a question is opened, the pediatrician has 5 minutes to answer. Feedback--including the correct answer, supporting references, and a rationale to foster enhanced learning--is immediate. Once the 20 questions for a quarter are completed, participants are asked to fill out a survey to help assess and improve the MOCA-Peds material and process.
In the Q1 survey, pilot participants were asked about the system, use of resources, technical issues, and customer support. More than 83% of the diplomates responded with valuable feedback on their experience. Some of the highlights include:
- 92% Agreed/Strongly Agreed that the overall MOCA-Peds IT system was satisfactory
- 90% Agreed/Strongly Agreed that the number of notifications received was appropriate
- 80% Agreed/Strongly Agreed that questions were relevant to general pediatrics
- 72% Agreed/Strongly Agreed that they had enough time to answer questions
- 79% Agreed/Strongly Agreed that the question difficulty was appropriate
- 81% Agreed/Strongly Agreed that the overall ABP customer support experience was satisfactory
- 54% Said they did not study in advance
Survey respondents were also asked for feedback regarding technical issues. Of those who responded, 16% indicated that they experienced some sort of technical difficulty, ranging from internet connectivity to frozen screens. This feedback has been instrumental and several enhancements have been released to address the issues. Now, halfway through Quarter 2 (Q2), the ABP has received only a fraction of the technical issue-related messages received in Q1.
Along with survey feedback, the ABP is conducting focus groups after each quarter to dive into the nuances of the survey feedback received. Already, we have received constructive feedback on various aspects of the MOCA-Peds experience and will continue using this feedback to make improvements in 2018 and beyond.
At the start of Q2 on April 1, the ABP released a mobile app version of MOCA-Peds, available on Android and Apple mobile devices. Diplomates can check out a tutorial of the mobile app by visiting the MOCA-Peds page on ABP.org.
About the Author
Trevor Fleck, MPA, is the MOCA-Peds Program Manager at the American Board of Pediatrics (ABP). With a background in project management and process improvement, he oversees the design, planning, implementation, and ongoing management of the MOCA-Peds program. Prior to joining the ABP, Trevor has worked for the Department of Veterans Affairs, USAID, and the Robert Wood Johnson Clinical Scholars Program.