Wednesday, July 14, 2010 2:15 PM

Human Factors Engineering (HFE), also known as usability engineering or ergonomics, is the study of how humans interact with machines and complex systems.  HFE entails overt consideration for all aspects of user interaction with a device and is assessed continuously during the product development process using usability testing.  As medical devices are becoming more complex – due, in part, to greater availability of technology and the sophisticated tasks asked of them; human factors considerations are increasingly critical.  This is particularly true for medical devices intended to be used directly by patients.

FDA’s Center for Devices and Radiological Health (CDRH) receives over 100,000 Medical Device Reports (MDRs) per year, and approximately one-third of these reference user or use error.  FDA has initiated several regulatory mechanisms to ensure adequate consideration of HFE in device design.  Some of these mechanisms include site inspections of manufacturers, review and approval of medical devices, and review of medical device incident reports (MDRs, complaints).

 

During an inspection of a manufacturer, FDA may review design control procedures and activities to ensure that human factors were adequately considered in the development process.  FDA conducts inspections to ensure compliance with applicable regulations including the regulations pertaining to human factors cited below:

 

• 21CFR§820.30(c) Design Inputs:  Each manufacturer shall establish and maintain procedures to ensure that the design requirements relating to a device are appropriate and address the intended use of the device, including the needs of the user and patient.

 

• 21CFR§820.30(g):  Design Validation:  Each manufacturer shall establish and maintain procedures for validating the device design.  Design validation shall be performed under defined operating conditions on initial production units, lots, or batches, or their equivalents.  Design validation shall ensure that devices conform to defined user needs and intended uses and shall include testing of production units under actual or simulated use conditions.

 

• 21CFR§820.100(a)(1) Corrective and Preventive Action:   Analyzing processes, work operations, concessions, quality audit reports, quality records, service records, complaints, returned product, and other sources of quality data to identify existing and potential causes of nonconforming product, or other quality problems.

 

An investment in human factors engineering and usability assessments by manufacturers is not only important to remain in compliance with FDA regulations but it may also:

 

• lower the number of use and user errors,
• reduce risks associated with use of the device,
• lower training costs, and,
• reduce costly and unnecessary service and support.

 

Device firms can contribute to the health care cost-containment aims of the current healthcare debate by making devices less prone to human error according to medical device industry representatives.  The medical device trade association, AdvaMed, was among six national organizations that, in May of 2009, collectively pledged to help slow the growth in health care costs over the next 10 years by 1.5%, for a savings of $2 trillion or more.  The device industry's focus on product design issues to help reduce medical errors is one of the two commitments from AdvaMed outlined in a letter to the President from the six trade groups following the meeting, and human factors was identified as a major target for member companies.

 

When errors involving medical devices recur repeatedly, people typically blame the users instead of the real culprit, which is often a poorly designed interface between the medical device and user.  It is common in medical device development for all engineering resources to be placed on the internal workings of the device, i.e. bringing the concept to a functional, manufacturable device, with minimal effort spent regarding the design relating to the logistics of how people will be interacting with the device and potential stressors to that interaction.  Medical device designers and manufacturers need to consider the user (patient, family, prescriber of the device, etc.) characteristics, including the person’s abilities and training and their expectations of the device.  They also need to focus on the device-user interface including instructions for use, and the environment in which the device is used (noise, lighting, workload, movement/vibration, competing attentions, etc.).

 

Human Factors and usability are integral to the overall Risk Management efforts in device design.  There are two methods by which these can be assessed: 

 

• Analytic approach:   Classic risk management techniques may be used including operational analysis, analysis of similar systems, Failure Modes Effects Analysis (FMEA), Fault Tree Analysis (FTA), and Critical Incident Technique and Hazard and Operability Studies (HAZOP) that include use and user error as inputs.

 

• Empirical approach:   Actual use studies with the device including “walk throughs” and usability studies can be conducted to focus on detecting and measuring use and user error risks.

 

Manufacturers are required not only to use human factors principles to repeatedly test the product in all phases of design, but also to validate the ultimate device design.  Validation entails testing the device, either in an actual clinical situation or a simulation, and documenting that the device conforms to the individual user’s needs.  This requires manufacturers to have in place a process that ensures adequate consideration of human factors in the design and development of medical devices.  Some examples of human factors considerations include:

 

• Use of labels and displays that can easily be read and interpreted (consider environment).
• Use of colors and contrasts that minimize ambiguity and add information redundancy when possible, e.g., red alarm lights to further convey a dangerous condition.
• Recognize the implications for visually-impaired or hearing-impaired users.
• Evaluate, in advance, to determine if viewing angle limitations, such as those associated with LCD or LED displays, are likely to be problematic under expected use conditions.
• Manage the tones, intensities and types of audible alarms; avoid the same combination of tones and intensities for differing alarm conditions; ensure that device alarms are distinct from the many other devices that are likely to be in use in the immediate working environment;  and never provide the user the ability to permanently silence audible alarms or turn alarm volumes to less than ambient noise levels, i.e., 55–65 dB without a thorough risk assessment.
• Seek to make the alarm compatible with the level of the threat.  Humans tend to equate both the volume and character of an alarm with its severity.  In other words, do not assign an ear–splitting klaxon for a relatively minor alarm condition.
• Provide the user with tactile feedback whenever possible and appropriate.  Humans possess touch receptors that are sensitive to both displacement and viscoelastic resistance.  Positive detent push buttons and keypads communicate to users that their actions were sufficient. However, today there are new technologies, such as proximity capacitive sensing, that do not require positive detent push buttons.  Users may lose the tactile response of pushing a button, but audio or visual (lights or LEDs) feedback can be programmed into the system to return a satisfactory response  (think iPhone!).  These interfaces are also easier to clean and maintain, since users don´t have direct contact with the circuits inside the device.

 

Usability Evaluation

 

Evaluation of usability entails a systematic progression of testing during and at the end of the device development process.  Early in the development process, i.e. during product definition, input to the design concept based on prototypes or graphics can be obtained from representative users.  This type of evaluation is often referred to as “formative testing”.  These sessions can be interactive with the user to obtain subjective input. 

 

As the product design matures, further user input on the product or product components should be gathered on the higher-level design.  This type of evaluation is often referred to as “summative testing”.  Ideally summative testing is performed while changes to the design can still be implemented without significant delays to the product development schedule.  These sessions should be less interactive with the users, providing only enough instruction to make the leap from the prototype provided for testing, to the final product. 

 

Finally, at the end of the development process, usability of the product should be evaluated against a performance standard.  This type of evaluation provides validation of the usability of the device.  If a device trial will be conducted to evaluate safety and efficacy, usability objectives can be included to validate usability of the device within the overall design of the larger study.

 

As in the development of any clinical trial, whether usability is being evaluated in the context of the larger trial or as a stand-alone study, an intimate understanding of the device indications, user population, device function, intended use of the device, and associated risks is needed in order to design the study.  The essential elements of clinical trial design apply also to stand-alone usability studies:  definition of study objectives and pass/fail criteria, study design, test environment considerations, sample size, user eligibility criteria, procedures, device components to be included in the testing, and study assessments.

 

• Objectives:  A basic premise of usability testing is that users will demonstrate that the device can be used as intended.  Of paramount importance in developing objectives for usability testing is consideration of scenarios where patients or clinicians are likely to use or misuse the device in such a way that could increase safety risk.  In developing the objectives, the mitigations identified in the risk assessment provide ripe fodder for test scenarios and pass/fail criteria.  Additional objectives may include evaluation of labeling, e.g. assessing whether or not users can find the information needed to perform the required tasks.  Marketing claims may also be considered during development of the objectives for usability testing, e.g. ease of use ratings, assessment of how quickly the device can be used to perform specified tasks, perceptions of device use, etc.

 

• Study design:  The study design can be simple or more complex, depending on the objectives, whether randomization is required, and duration of user participation.  Some usability studies can be conducted within one session, others may require users to take the device home and return later to provide use information (collected via a diary or preferably stored electronically in the device).

 

• Sample size:  The number of subjects, of course, depends on the objectives and study design.  A biostatistician should be consulted when considering the sample size requirements for the study.  The goal in any study is to be sufficiently confident that the results are true and reliable for making business decisions regarding the product.  Good decisions are based on good data!

 

• Subject eligibility criteria:  Subject eligibility criteria generally exclude anyone associated with the development team due to inherent bias as well as anyone with prior knowledge of how the device works.  Inclusion of employees of the manufacturer requires special consideration and protection of the volunteers as vulnerable research subjects.  Subjects for the study should be selected from the same population as will be targeted for use of the device.  So if your company employees are not representative of the intended user population, it’s best to solicit subjects from a different pool who are likely to use the device.

 

• Study procedures:  Study procedures should be explicit and carried out in the same manner for each subject.  Consider scripting the introductory explanations, and follow the procedure in the same order for each subject.  It is also important to conduct a “dry-run” of the procedures prior to finalization of the protocol in order to ensure the steps are logistically feasible and can be completed in the time expected.  If someone from the design team will be present or conducting the testing, they must refrain from providing assistance to the user.  Development engineers seem to be innately compelled to help the subjects when they are struggling with a use task, and this is ultimately not helpful in gathering accurate usability data.  Having a design engineer present to observe during the study is key for developers having a full understanding of any use issues that arise and, thus, facilitates insightful design enhancements.  It can be hard for developers to believe that the users had difficulty and that changes might be necessary, but seeing is believing! 

 

• Testing environment:  The setting of the study also requires consideration for mimicking the real use scenario, i.e. lighting, noise, temperature, etc.  Commonly, an office or clinic is acceptable.  Sometimes it is necessary for the subjects to use the device at home for a period of time, or to use the device in another location that cannot be simulated within an office setting.  Consideration should be given for devices intended to be used during patient transport situations such as ambulances or other emergency vehicles and how such an environment may impact the user interaction with the device.

 

• Device system or components:  The components to be tested should reflect the goals of the study.  If the study is for formative purposes or even summative of a particular function, then the components involved can be studied separately.  If the study is for validation of the design, then the full system should be included. 

 

• Study assessments:  The study assessments should be as objective as possible.  For example, a checklist can be developed for whether or not the subject correctly performed discrete tasks that are required for use of the device.  Consider whether a second attempt following training is appropriate for failed tasks.  The data collection forms should be designed to capture all pertinent information, including any subject demographic information that is pertinent to device use, results of each step, and use difficulties encountered.  A subject questionnaire using Likert scales is often used to collect subject perceptions of ease of use.  For example, subjects might be asked to rate a task such as removal of the device on a scale of 1 to 5, with 1 = very difficult, 2 = somewhat difficult, 3 = neutral, 4 = somewhat easy, and 5 = very easy to remove.

 

For usability studies, as in any clinical trial, the safety and welfare of the subjects must be protected.  Thus, informed consent and Institutional Review Board (IRB) approval are generally required.   If the study is  “…being conducted to collect data to support either a clinical investigation or a marketing application, then the study must comply with 21 CFR Part 50 [protection of human subjects], and should comply with 21 CFR Part 56 [Institutional Review Boards] and  21 CFR Parts 50.1(a), 50.20, 56.101(a) and 56.103.” (Cited from:  Information Sheet Guidance for IRBs, Clinical Investigators, and Sponsors, Frequently Asked Questions About Medical Devices, January 2006, available on the FDA website)  The FDA’s perspective is that all human subject research, even sociology and educational studies, require IRB approval.

 

While the reality is that the subjects’ ability to perform tasks using the device will affect the results of the study, the mindset for both the moderator(s) as well as the subjects should be that it is the device being tested, not the subjects.  It is important to preface the testing with this assurance to each subject.  Naturally, subjects usually want to appear smart by quickly figuring out how to complete the assigned tasks and have the device work well to please the tester.  So it is important to emphasize that any problems encountered provide really important information and will help the developers make the product better before it goes to market.  If no problems are encountered during the progression of usability testing, either the design is exceptional, or possibly, the testing is inadequate and unidentified problems may be encountered later.  From a manufacturer’s perspective, the motto for medical device usability should be to test often and early to avoid surprises during validation testing, or worse, after market introduction.  

 

Human factors engineering is a “hot button issue” at the FDA particularly for devices intended to be used by patients.  Risk management is already integral to both the premarket and postmarket responsibilities of medical device manufacturers and a significant amount of risk can be mitigated through proper HFE methods.  Furthermore, cost containment is a requirement of the health care debate and all stakeholders will be asked to contribute.  AdvaMed has committed to reducing errors though promoting HFE with its membership as part of the healthcare cost containment efforts.

 

 

 

Guidance Documents and Papers:

 

Do It By Design:  An Introduction to Human Factors in Medical Devices.   Sawyer, C., FDA; 1997.  This is FDA’s “primer” on Human Factors and can be found among the CDRH guidance documents on the FDA website.  It provides an overview of HF issues and checklists to help manufacturers consider HF issues in medical device design.

 

Medical Device Use-Safety:  Incorporating Human Factors Engineering into Risk Management, July 18, 2000.  This is the latest formal guidance from FDA on Human Factors and presents an excellent discussion of how HFE fits into the device Risk Management strategy.

 

Human Factors Principles for Medical Device Labeling.  Callan, J.R., & Gwynne, J.W. (1993).   San Diego, CA: Pacific Sciences & Engineering Group.  This report was commissioned by FDA and can be found among the CDRH guidance documents online.

 

Write it Right: Recommendations for Developing User Instruction Manuals for Medical Devices Used in Home Health Care, Backinger, C., & Kingsley, P. (1993). (HHS Pub. FDA 93-4258). Rockville, MD.: U.S. Department of Health and Human Services, Food and Drug Administration.  This document is available among CDRH guidance documents online.

 

Books:

 

Designing Usability into Medical Products; Michael E. Wiklund and Stephen B. Wilcox; foreword by Matthew B. Weinger.  CRC Press, 2005.

 

Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests; Jeffrey Rubin and Dana Chisnell.  Wiley Publishing, Inc., 2008.

 

Standards:

 

ANSI/AAMI HE75:2009, Human factors engineering – Design of medical devices.    Just Published!

 

This is an encyclopedic document containing more than a thousand human factors guidelines for the design of safe, effective, and user-friendly medical devices.  This document tells you how to make a patient monitor’s digital readouts legible; what features enhance a portable CT scanner’s mobility; how to design alarms that draw attention in noisy environments, and a myriad of other ways to improve medical device safety, effectiveness, and usability.

 

ANSI/AAMI HE74:2001, “Human Factors Design Process for Medical Devices” (Arlington, VA: AAMI, 2001).

 

The AAMI Human Factors Engineering Committee developed this process–oriented standard to provide manufacturers with a structured approach to user interface design; helping them develop safe and usable medical devices.  It also helps them respond to the increasing number of national and international human factors standards in the medical field and the promulgation of new governmental regulations (based on ISO 9001) pertaining to medical device user interface design. This standard includes an overview of the human factors engineering (HFE) discipline; a discussion on the benefits of HFE; a review of the HFE process and associated analysis and design techniques and a discussion on implementation issues; and relevant national and international standards and regulations.

 

ANSI/AAMI HE48:1993, Human factors engineering guidelines and preferred practices for the design of medical devices.

 

This standard is based on the military specification MIL-STD-1472D, a standard for the US Department of Defense for the human factors of weapons and military control systems and is hardware-oriented.  It provides ergonomic information and human factors engineering guidance so that optimum user and patient safety, system safety and performance, and operator effectiveness will be reflected in medical device design.  Specifically, the recommended practice deals with the controls, displays, consoles, size, weight, and general user interface design of medical devices and is extensively illustrated.  User instructions, manuals, software, and algorithms associated with medical devices are also discussed briefly.

 

IEC 62366:2007 -- Medical devices - Application of usability engineering to medical devices.

 

This standard was developed to help manufacturers improve the usability and safety of medical devices.  The standard recognizes that the use of all medical devices has associated risks and provides an engineering process for identifying, assessing and mitigating those risks.  It describes a process that addresses medical device use errors and divides those errors into categories to guide their analysis.  This process can be used to assess and mitigate risks caused by the usability problems associated with the normal and abnormal use of a medical device.