The American Board of Internal Medicine (ABIM) has found a concordance of almost 70% between questions asked on its Internal Medicine Maintenance of Certification (IM-MOC) examination and conditions treated by US general internists in offices and hospitals. Its findings appear in an article published online June 13 in JAMA.
Recently, the IM-MOC examination has drawn fire from physicians, who say it is burdensome, expensive, too frequent, unfair, and most important, irrelevant to patients they see in daily practice and the cognitive skills required to treat them. In some jurisdictions, however, MOC-based board certification is a requirement for physician hospital privileges and reimbursement from insurers.
Bradley Gray, PhD, a health services researcher at ABIM headquarters in Philadelphia, Pennsylvania, and colleagues compared the percentages across 186 categories of medical conditions seen by general internists with the corresponding percentages in 3461 questions on IM-MOC examinations taken during 2010 to 2013. They considered both office visits and hospitalizations.
The nationally representative estimates of conditions seen in practice were calculated from primary diagnoses in the 2010 to 2013 National Ambulatory Medical Care Surveys and the 2010 National Hospital Discharge Survey.
Using strict concordance criteria, the researchers found 69.0% of examination questions were concordant. "This finding indicates that the IM-MOC examination has generally been consistent with the conditions seen in practice," the researchers write. "However, with 31.0% of examination questions categorized as discordant, the study also identified potential opportunities for improvement."
Dr Gray was somewhat surprised at the high level of concordance. "We used such stringent criteria that even a movement of two questions out of 180 categories could move you from concordant to discordant in a category. So in some sense I'm surprised we didn't find more things discordant," he told Medscape Medical News.
The study's partial validation may help counter criticisms of irrelevance leveled by some critics of the certification exam, which traditionally has been taken every 10 years at a test center, but will be offered in 2018 as a 2-year "knowledge check-in" option completed at a place of a physician's choosing.
With the objective of assessing whether question percentages differed from corresponding percentages for conditions seen in 13,832 office visits and 108,472 hospital stays, the study identified a number of areas of discordance. Although concordance with clinical practice emerged for 2389 exam questions, 1072 questions showed discordance, being overrepresented or underrepresented when compared with prevalence in clinical practice.
The study observed an excess of questions on liver disease and hematological malignancies and less-than-prevalent percentages for lower respiratory tract infections and localized joint syndromes.
However, the sheer frequency of a condition is not the only measure of its importance in medical care. "Unlike the common cold, Ebola is not frequently seen, but you really want your doctor to know how to handle it," Dr Gray said.
The study also found more concordance with conditions seen in office settings than hospital settings. "This may reflect that the examination is for general internists who practice in an outpatient setting or both outpatient and inpatient settings," the authors write, noting that a separate examination is available for general internists practicing only in inpatient settings.
In a related editorial, internists Adam B. Schwartz, MD, from New York University School of Medicine, New York City, and J. Sanford Schwartz, MD, from the Perelman School of Medicine in Philadelphia, Pennsylvania, agree that board certification cannot be the full measure of a clinician, especially for assessing patient interactions, clinical judgment, and patient management. "Nevertheless, board certification influences patient expectations, physician assessment and hiring, and insurer inclusion and payment," they write, adding that the certification process has attracted increasing criticism as costs in time and money have grown.
The editorialists note a dilemma of balance. Although physicians are trained to focus on common diagnoses, they must still be able to identify uncommon but serious diseases. "Finding a balance is critical, and using standardized, well-defined clinical coding data, as Gray and colleagues did in their study, is a reasonable place to start," the editoralists write. But coding cannot always capture the greater complexity of some visits.
As to the two practice settings studied, the commentators point out that primary care practitioners and hospitalists work in increasingly separate arenas of care. Although the former may treat more chronic hypertension, the latter may see more acute hypotension. "What is relevant to one internist may not be relevant to another, making the design of the examination more difficult and the usefulness of such an examination less clear," Dr Schwartz and Dr Schwartz write.
They call for future studies by independent investigators to confirm the ABIM analyses and "evaluate other measures of clinical validity, such as physician judgment and decision making, including understanding and application of epidemiology, statistical reasoning, and disease prevention."
This study was supported by the ABIM. Dr Gray and two coauthors are ABIM employees. Another coauthor is a member of the ABIM's Council and board of directors. Dr Adam B. Schwartz is board-certified by the ABIM and is enrolled in the ABIM's Maintenance of Certification program. Dr J. Sanford Schwartz has time-unlimited ABIM board certification and has served as an ABIM consultant. He also reports consultancy or scientific advisory committee membership with Bayer, Blue Cross and Blue Shield Associations, Pfizer, and Takeda.
JAMA. 2017;317:2288-2289, 2317-2324. Article abstract, Editorial extract
Medscape Medical News © 2017 WebMD, LLC
Send comments and news tips to news@medscape.net.
Cite this: ABIM Validates Certification Exam's Relevance to Practice - Medscape - Jun 14, 2017.
Comments