Many electronic health record vendors have not met the usability testing requirements of the Office of the National Coordinator for Health IT (ONC), although their products received certification from ONC-authorized certification bodies, according to a study published in the September 8 issue of JAMA.
As part of ONC's certification rules, EHR vendors are required to attest to incorporating user-centered design (UCD). This is "a process that places the cognitive and information needs at the forefront of software development," write Raj M. Ratwani, PhD, from the National Center for Human Factors in Healthcare, MedStar Health, Washington, DC, and colleagues. In addition, they note, ONC requires the vendors to conduct formal usability testing of eight different EHR capabilities.
Specifically, ONC mandates that EHR vendors:
provide a written statement naming the UCD process they used, and describing it if it is not an industry standard;
provide the results of usability tests, including the number, clinical background, and demographics of the participants; and
make the usability reports public once the product is certified.
To measure compliance with these requirements, Dr Ratwani and colleagues examined certification reports for the 50 EHR vendors whose products were used by the highest number of providers attesting to meaningful use between April 1, 2013, and November 30, 2014. In addition to extracting the stated UCD process from each report, the researchers focused on the usability tests for computerized physician order entry (CPOE), because it is primarily used by clinicians and can compromise patient safety if poorly designed.
Of the 50 certified vendor reports, nine were not publicly available. Of the remaining 41 vendors, 14 (34%) had not stated their UCD process, as required by ONC; 19 (46%) used an industry standard for UCD; and six (15%) used an internally developed UCD process.
For the usability tests, ONC has endorsed guidelines from the National Institute of Standards and Technology. Those guidelines state that usability testing should include at least 15 representative end-user participants.
However, the authors found that there was considerable variability in the number of participants enrolled in the usability tests (mean, 14; standard deviation, 10; range, 4 - 51). Twenty-six vendors (63%) used fewer than 15 participants, seven (17%) used no physician participants, and two (5%) used their own employees. Only nine (22%) used at least 15 participants with clinical backgrounds, as required in the guidelines.
Of the 41 available vendor reports, five (12%) lacked sufficient detail to determine whether physicians participated and 21 (51%) failed to provide the required demographic details.
The researchers said their findings revealed a lack of adherence to ONC certification requirements and usability testing standards among several widely used, certified EHRs. They added, "The lack of adherence to usability testing may be a major factor contributing to the poor usability experienced by clinicians."
Richelle Koopman, MD, an associate professor of family medicine at the University of Missouri and an expert on EHR usability, said that the poor compliance of EHR vendors with the usability testing criteria showed that vendors had assigned usability a low priority because of their need to meet other certification requirements tied to meaningful use.
"Usability is part of the EHR vendors' mission," she told Medscape Medical News. "However, the other meaningful use requirements have been driving the agenda of EHRs for a long time."
Despite the failure of ONC to enforce its own usability testing requirements, she said, "I think [ONC's] priorities are moving toward usability in a bigger way. This was their first stab at it, and it was probably a minimal standard that people were able to meet. The next standards will be higher."
A recent study by Dr Koopman and colleagues found that physicians valued the assessment and plan sections of EHRs most highly and regarded the review of systems feature as useless in patient care. The authors recommended that less valuable parts of the EHR be deemphasized or hidden when physicians review the progress note and that the note be redesigned "to match the workflow and information needs of its primary consumer."
"Poor usability is a giant overhead on physician time, and that's really expensive," Dr Koopman noted. "If you're not using physician time well, you're paying for a lot of waste."
The authors have disclosed no relevant financial relationships. Dr Koopman reports receiving grants from California Healthcare Foundation and ONC.
JAMA. 2015;214:1070-1071. Abstract
Medscape Medical News © 2015 WebMD, LLC
Send comments and news tips to news@medscape.net.
Cite this: Usability Does Not Appear to Be High Priority for EHR Vendors - Medscape - Sep 09, 2015.
Comments