A Summary of Risk Management Tools for Human Factors
Based on the book written by Edmond W. Israelski and William H. Muto: Risk Management: How to Assess and Control Risk Related to Human Error in Products and Systems
Risk management is one of the most critical steps of the Human Factors Engineering process. It is invaluable particularly in the healthcare and medical device industries. After all, if engineers can't identify and then mitigate the elements of a product or system that could lead to harm, serious negative consequences for users, patients, and the company can occur.
However, risk management can be challenging to understand! There are so many strategies, tools and Do Not’s, a beginner can feel overwhelmed.
Unlike other books on risk management, Israelski and Muto offer step by step instructions on how to apply popular tools such as FMEA (failure modes and effects analysis) and FTA (fault tree analysis) through a human centered perspective.
I highly recommend their book to anyone wishing to learn more about the topic as it relates to use error and human interactions with products and systems. With two detailed case studies - an AED defibrillator and a blood glucose meter - this book is an excellent introduction to an often complicated, but essential step of the human factors engineering process.
While I encourage you all to buy your own copy and derive your own insights, here is a summary of some chapters that resonated with the Loring Human Factors (LHF) Team.
To read, expand each section below:
Importance of Risk Management
Since people’s perception of risk is often incorrect (ex: people have a greater fear of airplane crashes than car crashes, even though airplane crashes occur less frequently), it is important to establish a robust and thorough risk management process to avoid biases and assumptions during device development. Israelski and Muto recommend gathering stakeholders from different functions (clinical, regulatory, quality, engineering, etc.) to form a diverse risk management team. Different perspectives will ensure the team is able to capture most foreseeable risks, assign appropriate severity ratings, and develop comprehensive mitigations. Although not specifically mentioned by the authors, bringing in external consultants with diverse industry experience can be helpful in drafting the risk documentation, since their expertise may bring in a fresh perspective on device usability. Our team has helped many of our clients draft the URRA according to FDA expectations and most importantly, conducted numerous studies that validate mitigations and identify unexpected risks. Of course, as the authors rightfully point out, risk management does not end after you have identified, prioritized, and mitigated risks through an HF validation study. After a product/system is released to the public, the risk management team must then conduct post-market surveillance in case a new risk is found. Just like many other deliverables involved in a product life cycle, risk assessment and management reports are “living documents.”
Risk Terminology and Tools
Before we dive in further, here are some essential terms to keep in mind:
Hazard - potential source of harm
Hazardous Situation - circumstances in which people, property, or the environment are exposed to one or more hazards
Harm - physical injury or damage to health of people or damage to property or the environment
Use-Related Risk (or Risk) - combination of the probability of occurrence of harm and the severity of that harm. In other words: Risk = Likelihood x Severity
Use Error (NOT user error or human error 😉) – when the outcome of using a product was different than intended, but not due to a technical malfunction. Here are some types of use errors:
Errors of Omission – User fails to perform an action that should have been done. For example, a lapse means a user does not perform an intended action due to failure of memory.
Errors of Commission – User performs an action that should not have been done.
Slip – Intending to perform an action, but the action is not executed as intended
Mistake – Performing an incorrect action because incorrect knowledge or misinterpreted information
Violation – intentionally performing an inappropriate or prohibited action (manufacturers do not need to consider abnormal uses, only those that are foreseeable)
Two of the most common risk management tools are:
FMEA – Failure Modes and Effects Analysis (such as a design FMEA - dFMEA or a use FMEA - uFMEA, etc.)
FTA – Fault Tree Analysis
There are many other tools (spoilers: HAZOPs, Delphi Technique, etc.) the authors mention, but we’ll be focusing on these two in this article.
Note: You may have also heard the term URRA (use-related risk analysis)! A URRA is a simplified version that focuses mostly on the uFMEA columns that FDA cares about (see Table 1 in the next section).
Failure Modes and Effects Analysis: FMEA
A uFMEA is a straightforward, “bottom-up” approach to identifying use errors and risks. Check out the graphic below, which outlines the steps described by Israelski and Muto. Note: In a uFMEA, use error = failure mode.
Figure 1: Steps to create a uFMEA, according to Israelski and Muto. Diagram was created by LHF team to summarize the uFMEA chapter in their book. Regarding medical devices and systems, you may already be familiar with a uFMEA since the FDA requires risks to be documented in a format similar to a uFMEA. If this is new information to you, contact the Loring HF team for some regulatory support! Here’s a glimpse of the FDA’s risk table with an example entry:
Table 1: Based on the FDA’s Content of Human Factors Information in Medical Device Marketing Submissions, which was published in December 2022.
Notice how this format excludes a column for likelihood? The FDA asks device manufacturers to focus only on severity, which is a stance Israelski and Muto challenge. They agree that the challenges associated with estimating probabilities for novel technologies and various medical conditions make it difficult to develop accurate likelihood values, but as you may remember, there are two parts to risk: severity and likelihood of occurrence. Ignoring one or the other impacts how realistic risk mitigations can be. The authors believe the FDA’s stance means manufacturers are evaluating residual harm, rather than residual risk, and “reducing harm is much more difficult to accomplish.” Our founder and principal consultant, Beth Loring, recently discussed this point during the April 2023 AAMI Human Factors Standards Committee meeting.
Fault Tree Analysis: FTA
Since a uFMEA allows risks to be dealt with sequentially and individually, it is not necessarily the best approach for identifying errors happening simultaneously. When there are multiple errors leading to a hazardous situation, a fault tree analysis (FTA) may be more suitable.
FTA is a top-down approach, where an undesired event or hazardous situation is first identified, followed by potential use errors that could have led to that situation. See below for a graphic illustrating the process in the book.
Figure 2: Steps to create an FTA, according to Israelski and Muto. Diagram was created by LHF team to summarize the FTA chapter in their book.
The FTA is certainly a more quantitative approach to risk analysis, so make sure to have a calculator ready and talk frequently with your clinical team! Israelski and Muto have some excellent advice on what to do if there is minimal data about probabilities. Most importantly, they provide an in-depth explanation of gates, which can be a tough concept to grasp. It took me a few re-reads before I caught on and I bet they’ve only scratched the surface.
Conclusion:
There are plenty more topics discussed in the book, so we’ll wrap up here to prevent additional spoilers. If you want to see a uFMEA and FTA conducted in their entirety, we (*once again*) encourage you to buy the book and take a look at Chapters 5 and 6, where Israelski and Muto apply these tools to an AED defibrillator and a blood glucose meter.
Once you do, we'd love to hear your thoughts. Do you agree or disagree with their stance about omitting likelihood in the FDA’s uFMEA format? Do you conduct a uFMEA or FTA differently that described? Comment below or reach out to Loring HF to start a conversation!
References:
Israelski, E.W., & Muto, W.H. (2022). Risk management: How to assess and control risk related to human error in products and systems. Human Factors and Ergonomics Society.