By Andy Dwyer, PhD
The American Board of Pediatrics (ABP) wants all of its exams to assess pediatricians’ grasp of knowledge critical to current pediatric practice.
To help ensure this was happening, in 2016, the ABP conducted a large-scale study that analyzed the clinical practice of general pediatricians. The results of this study were used to make significant revisions to the general pediatrics content outline. The content outline, sometimes referred to as the “test blueprint,” specifies the content areas that are being assessed by the ABP’s general pediatrics exams, including the in-training, initial certification, and maintenance of certification exams. The purpose of this study was to ensure that the ABP’s exams are measuring the knowledge required currently for safe and effective clinical practice.
The practice analysis study consisted of three major phases. In the first phase, a diverse panel of 13 practicing general pediatricians met in person for three days to delineate the critical tasks performed by general pediatricians in clinical practice. They also identified and categorized the knowledge required to safely and effectively perform those tasks. The panel’s work produced an initial draft of the general pediatrics content outline.
In the second phase, all board certified general pediatricians (N = 69,117) were invited to review the draft content outline and provide feedback through an online survey. Survey respondents rated each of the content areas listed within the survey using two rating scales:
Survey respondents also had the opportunity to identify any important knowledge areas that they felt were missing from the outline through open-ended comment boxes.
In the third phase, the survey results were used to make final revisions to the outline and to establish the percentage of exam questions associated with each content area. Knowledge areas rated as “highly critical” and “frequently required in practice” were weighted more heavily than knowledge areas rated as “less critical” and/or “less frequently required.” In this way, an empirical link between the knowledge required for practice and the knowledge areas being assessed by the ABP’s general pediatrics examinations was established.
Beginning September 1, 2017, all new versions of the ABP’s general pediatrics examinations will adhere to the specifications provided within the new general pediatrics content outline.
Andy Dwyer is the Director of Psychometrics for the American Board of Pediatrics (ABP) and is involved in work designed to ensure and improve the validity of the ABP’s examination programs. Dr. Dwyer holds a PhD in Educational Psychology and an MS in Statistics from the University of Nebraska-Lincoln.