Blog

By Linda A. Althouse, PhD

During the design of MOCA-Peds we spent a considerable amount of time and effort discussing security. After all, the plan is for MOCA-Peds to replace the current MOC Part 3 examination, which currently takes place at a secure testing center. So what does exam security mean for MOCA-Peds, when it can be taken anytime, anywhere?

We know the vast majority of pediatricians are honest, but, unfortunately, we have had a few incidents on past exams that require us to take this issue seriously. As a credentialing organization, we cannot ignore security of our assessment program, because security measures help ensure the validity of the assessment scores (and the decisions made based on those scores).

So, let us review a few security-related items for MOCA-Peds:

• ABP Honor Code and MOCA-Peds Participation Agreement – You will need to attest to these the first time you sign on to MOCA-Peds. The ABP Honor Code applies to everything you do with the ABP. The MOCA-Peds Participation Agreement is specific to MOCA-Peds.     
• Copyrighting – The ABP copyrights all questions and related material (e.g., rationales). That means you cannot copy them or distribute them without the permission of the ABP.
• Communication – You cannot discuss the questions or specifics about any questions with your colleagues. Remember, one of the benefits of MOCA-Peds is that you can do it when it best fits your schedule. As a result, you may see a question before your colleague sees it.
• Collaboration – Collaboration is not permitted when answering the questions. MOCA-Peds is not intended to be a team exercise. We realize collaboration could help with learning, but the primary purpose of MOCA-Peds is still to provide an assessment of knowledge. If collaboration were permitted, we could not be assured that we are assessing a specific individual’s knowledge.
• Use of books or online resources – Consulting reference material is permitted within the time allotted for the question. However, as noted in earlier communications, questions are developed with the intent that reference material is not needed.

This is uncharted territory for the assessment community. High-stakes exams are typically proctored in some way.  While the external security environment of MOCA-Peds is not equal to that of a proctored exam, the ABP places the same value on exam security for this assessment as it does for its other exams.  It is important for all diplomates to recognize that any security breach is not only unprofessional, but also a violation of the ABP Honor Code, requiring the ABP to respond accordingly.   

To ensure the security of MOCA-Peds, several checks have been put in place. First, the assessment can only be accessed through your personal physician portfolio.  We will also be conducting statistical analyses to determine if questions are getting easier over time, which may indicate whether questions are being improperly shared. We also have web patrolling software that searches for leaked test content. Depending on what we see after MOCA-Peds is launched, we may also consider adding additional security measures, such as remote proctoring or biometric authentication.

This is your certification and for it to remain meaningful there needs to be integrity in the process. In addition, for MOCA-Peds to be successful, we need to be assured that it is being used as intended. As such, we need your help. Should you see or suspect any potential security breaches with MOCA-Peds, please let us know. You can email us directly at mocapeds@abpeds.org. If you want to learn more about exam security in general, you can visit our exam security page. 

About the Author

Linda A. Althouse, PhD is the Vice President of Psychometrics & Assessment Services at the American Board of Pediatrics (ABP). With expertise in research and psychometrics (the development of examination and measurement instruments), Dr. Althouse is responsible for ensuring that the Board produces valid and reliable examinations that accurately assess a physician’s knowledge base. She is responsible for producing the ABP's annual workforce data book, and works closely with the ABP Research Advisory Committee, which develops new research initiatives and efforts related to board certification. Her doctorate is in educational psychology with a concentrated focus on measurement and assessment.