It was a pleasure to work with you, as you and your team made this process a more pleasant experience for our team and the participants.
Karen Perry
The Usability People conduct and report summative usability evaluations using the NISTIR 7742 Customized Common Industry Format Template for EHR Usability Testing—suitable for satisfying the Safety-enhanced Design criteria portion of your 2015 Edition certification.
The Office of the National Coordinator for Health Information Technology has made several enhancements to the Safety-enhanced (aka usability) testing portion of the certification criteria.
The Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) established a Merit-based Incentive Payment System (MIPS) that, in part, is based upon compliance with certain rules. The following describes what is required to obtain the ONC Certification for Safety-enhanced Design (§170.315.g.3) for an EHR.
Meaningful use, which CMS changed to "advancing care information" and rolled out under the Medicare Merit-based Incentive Payment System, is now being called "promoting interoperability."
Summative Usability testing for safety enhanced design (§170.315.g.3) involves recruiting targeted users as test participants (Doctors, Nurses, and other medical practitioners) and asking these users to complete a set of pre-defined tasks. An expert test facilitator conducts the testing via an established test protocol while the test sessions are recorded and later analyzed.
Usability testing cannot happen overnight, and it is important that you reach out early so that the required testing can be completed in time for your appointment with an ONC-ACB.
We recommend that you allow at least 4-6 weeks in your submission timeline for the preparation, conducting and reporting of a summative usability test for the ONC 2015 Safety-enhanced Design Certification. In a public comment on the certification process, Drummond (one of the ONC ATCB's) stated that
"...the usability test effort is not a small undertaking for EHR vendors. In fact, it (is) essentially one of the most difficult criteria to complete."
ONC states that a User-centered design processes must be applied to each capability an EHR technology includes that is specified in the following certification criteria: §170.315(a)(1); § 170.315(a)(2); § 170.315(a)(6); § 170.315(a)(7); § 170.315(a)(8); § 170.315(a)(16); §170.315(b)(3); and § 170.315(b)(4). Health IT developers must select an industry standard User-Centered design process because compliance with this certification criterion requires submission of the name, description, and citation (URL and/or publication citation) of the process that was selected. Examples of method(s) that could be employed for UCD, including ISO 9241-11, ISO 13407, ISO 16982, ISO/IEC 62366, ISO 9241-210 and NISTIR 7741.
We recommend that EHR vendors follow the NISTIR 7741 and ISO 9241-11 standard. ISO-9241 Part 11: (1998) pertains to the extent to which a product can be used by specified users to achieve specified goals with:
Effectiveness (Task completion by users),
Efficiency (Task on time) and
Satisfaction (responded by user in term of experience) in a specified context of use (users, tasks, equipment & environments).
Summative usability testing is an industry standard usability methodology (see http://bit.ly/13WYqTU ) During the testing a trained user experience professional works with the EHR vendor to create a series of representative tasks and then measures the amount of time it takes to complete the task (time on task), how many and what type of errors occur (error rates), and satisfaction with the task interaction (user satisfaction). The results —as measured by things like task times, assisted and un-assisted completion rates and standardized satisfaction scores --can be benchmarked against current industry standards.
The application of user-centered design (UCD) during development and summative testing is limited to only those nine certification criteria specified in this certification criterion and only for which certification is sought, namely [80 FR 62670]:
§ 170.315 (a)(1) Computerized provider order entry (CPOE) – medications
§ 170.315 (a)(2) Computerized provider order entry (CPOE) – laboratory
§ 170.315 (a)(3) Computerized provider order entry (CPOE) – diagnostic imaging
§ 170.315 (a)(4) Drug-drug, drug-allergy interaction checks for CPOE
§ 170.315 (a)(5) Demographics
§ 170.315 (a)(9) Clinical decision support (expires on January 1, 2025)
§ 170.315 (a)(14) Implantable device list
§ 170.315 (b)(2) Clinical information reconciliation and incorporation
§ 170.315 (b)(3) Electronic prescribing
§ 170.315 (b)(11) Decision support interventions
In accordance with NISTIR 7804, ONC requires that the test scenarios be based upon an analysis of critical use risks for patient safety, which can be mitigated or eliminated by improvements to the user interface design.
Results from the summative study should be created and presented to the ONC using the Customized Common Industry Format (CCIF) Template for EHR Usability Testing (NISTIR 7742 see http://1.usa.gov/gAZFDt ) These reports typically include an executive summary, an introduction, a method section, discussion of the results and any appendices .
ONC also requires that Health IT developers include the specific information from the NISTIR 7742 “Customized Common Industry Format Template for Electronic Health Record Usability Testing. All of the data elements described in NISTIR 7742 and every section must be completed, including “major findings” and “areas for improvement.”
ONC identified the requirements for the submission of the demographic characteristics of test participants. These test participant characteristics must reflect the audience of current and future users of the EHR.
The following are required for submission to the ONC:
The Usability People conduct and report summative usability evaluations using the NISTIR 7742 Customized Common Industry Format Template for EHR Usability Testing—suitable for satisfying the Safety-enhanced Design criteria portion of your ONC certification.