Human Factors

Human Factors proves your device can be used by your users. If your users cannot safely use your device, you don’t have a product.

BSC has led multiple products through successful Human Factors processes and gained FDA approval. BSC can do the same for your product.

EFFECTS.jpg

Formative Studies

Formative studies establish how the device will work. They are done early in design and development using mock ups and prototypes and establish whether users will be able to use the device as intended. They help identify issues early enough in the design process to still allow for design changes.

Formatives are more casual than a summative study and as a result are much less expensive to run. Formatives can be as simple as having a few people from your office try using your device to as formal as a summative study with the intended users on the final device. A formative typically consists of:

  • Formative Protocol

  • Formative Script/Testing Plan

  • Testing

  • Formative Report

The Formative Protocol is a document that outlines what will be tested in the formative. It describes different prototypes, the differences between them, and any other tools that will be used. Often this gets wrapped into the Formative Script/Testing Plan which defines the tasks the users will attempt.

The Formative Report summarizes the findings from the Formative Study, such as which workflows users found easy, any places they had difficulties, and when designs are compared which design was better and why.

Boston Systems Consulting has extensive experience conducting Formative Studies from simply bringing in nurses, running through a few different designs, and gathering feedback up to formal affairs involving a strict script, professional moderation, and observers behind two way mirrors. BSC can ensure you gather all the information you need quickly and efficiently.

IMG_4850.jpg

Summative Study

A Summative Study is a hugely important for your success. The FDA closely scrutinizes these studies, the studies take a lot of effort, and the difference between success and failure is small. A summative study validates that the device is safe and effective for use. Technically, the Summative Study validates that the Use FMEA is accurate and the Use FMEA claims that the device is safe for use.

The steps for are a Summative are:

  1. User Workflow Diagram

  2. Use FMEA

  3. Task List

  4. Summative Protocol

  5. Summative Script

  6. Testing

  7. Disposition of observations

  8. Summative Report

  9. Human Factors Report

The first step is to create a User Workflow Diagram. This documents all user actions and decisions from initial setup through powering down, cleaning, and alarm response. The Use FMEA examines each step in the workflow and asks ‘how could the user mess this up’ and ‘is that bad’? The Use FMEA documents each potential user error based on the workflow and any observed use errors in prior human factors testing. For more information on how to perform an FMEA see our blog.

The Use FMEA will dictate the task list that the user’s need to perform during the test. The FDA requires the task list to include all high severity failure modes in the Use FMEA. They insist that probability cannot be a factor in determining the tasks, however high severity can be defined by each project. You can include other tasks in the Summative Study, however I would recommend sticking to what is strictly required and placing any other tasks in a formative study.

The Summative Protocol is a document describing how the testing will be run. It includes a detailed breakdown of users, testers, testing locations, and test procedures. While the protocol is developed the Summative Script is also developed. This contains both a training script and the testing script. Once the summative starts it is very difficult to change these scripts. I strongly recommend performing a dry run with 1-3 users prior to starting testing to iron out training, set up, and script issues.

Testing involves a moderator reading the script to the users. BSC partners with skilled moderators for that role. For a summative, a skilled moderator makes a huge difference. They need to stick to the script while keeping the user relaxed and focused over the entire duration of the test. BSC observes each user in real time to provide feedback to the moderator. This allows us to quickly identify areas where the user’s actions could impact the Use FMEA and gather root cause information instantly. For instance, if a participant does not follow aseptic procedures during setup the root cause could be because the design of the device makes it impossible or because the testing is using a simulated patient and they thought it was unnecessary. Determining the difference between those two scenarios is the difference between a successful product and having to change the design. It is a lot easier to follow up while the participant is being tested rather than calling them weeks later. Having a systems engineer who knows the Use FMEA and risk profile providing real time feedback greatly simplifies the dispositioning of observations.

Both during and after testing is complete, all observations are dispositioned. Observations are classified based on whether they are use errors, test artifacts, or acceptable use. All use errors need to captured by the Use FMEA and the acceptability of the risk from those use errors is determined by the Use FMEA.

The Summative Report builds upon the protocol and includes the results of the testing. It outlines the conclusions and includes the disposition of all use errors and test artifacts. It also details any differences between the final device and the devices used in the summative including the rationale for why those differences do not impact the testing. All user facing changes after the summative should be avoided wherever possible.

The Human Factors Report combines the Summative Report with all of the formatives, showing how the formatives shaped the device through development and how that resulted in the Summative Study validating the accuracy of the Use FMEA.