Importance of test protocol for medical device projects

When planning any testing project, it is critical you develop a test protocol. Understanding the study objectives and methods will help ensure a successful testing project and mitigate risk for ongoing and future testing programs. The protocol should convey a clear understanding of why and how the test will be completed, what the necessary materials are, and, most importantly, how the data will be presented and analyzed. 

We often get asked: “Do I need a test protocol?” or “How do I develop one?” This document outlines the typical sections of a protocol. We highly recommend having test protocols in place before starting any study and will work with you to develop your testing strategy. 

Why is having a test plan so critical? 

Whether you are performing tests internally or outsourcing them to a trusted partner, the test protocol serves as the primary communication tool for capturing expectations, acceptance criteria, regulatory requirements, and applicable standards, and walks you through the steps to create a plan for the project. It’s almost like a map. A protocol also documents this information accordingly for repeatability in the future, and for keeping the team on the same page with revision control. Inevitably, as you move on from your current position, someone in the future will either try to replicate your study or use the information for future testing purposes. Having a clear protocol helps prevent uncertainty in the future regarding what was done, why it was done, and how it was performed. 

How do you start? 

There are several standard sections to test protocols, and depending on your regulatory strategy, you may want to include all of them or modify them as needed to align with your design history file. Some of these items may be outlined in other documents and may be redundant to have in both places. Typical protocol sections are outlined below:

1. Scope


2. Purpose


3. Reference Documents


4. Test Samples


5. Materials and Equipment


6. Methods


7. Acceptance Criteria


8. Exception Conditions


9. Data Analysis and Documentation Requirements


10. References

Revision history

Proper test protocols can take time to develop, and adequate resources should be allocated at the beginning of the projects. Writing protocols retroactively or mid-way through the project because you are trying to save time often results in delays and unclear expectations in test reports. Delays can be easily prevented if the correct steps are mapped out, particularly if you do not perform testing routinely. In our experience, the most successful customers are the ones that put in the forethought to identify the regulatory needs and risks at the beginning, then mitigate them through the protocols instead of doing it on the fly.

The FDA issued a guidance document on December 20 of 2019 on the Recommended Content and Format of Non-Clinical Bench Performance Testing Information in Premarket Submissions that may be helpful to review. 

Scope, background, and purpose – what is the goal and why?

Most protocols start with a scope or background for the project. It lays the groundwork for the device and why it is being evaluated. The objective or purpose defines the output. A clear objective sets the tone for the project and identifies what questions are being answered or what hypothesis is being made. Several examples are listed below in order of increasing impact and clarity:

1. To perform testing to evaluate the strength of the partial knee replacement.


2. To characterize the anteroposterior, mediolateral, and rotational constraint behavior of a unicondylar knee replacement system.


3. This testing covers how unicondylar knee replacement constraint may be quantified according to motion delineated by the inherent articular design as determined under specific loading conditions in an in vitro environment. This testing characterizes anteroposterior draw, mediolateral shear, and rotary laxity in reference to ASTM F1223. The results will be compared for substantial equivalence to a predicate.

Reference documents

The first question we ask a new customer is, “Do you have a test standard that you are looking to test to?”. Often the answer is yes. If not, then there may be a similar standard to reference or deviate from. This part of the protocol should list any relevant standards and comment on any expected deviations. The FDA guidance documents can also be references as they may outline the required testing.

Test Samples, materials, and equipment

The Materials section should outline all of the components that are needed for testing, quantities, and any other relevant information. Maintaining traceability to ensure a clear expectation of what is to be tested is also essential.

An example table is shown below:

Another item that is helpful in the test samples section is the worst-case justification. For the medical device industry, it is critical to identify the worst-case configurations for testing. It is helpful to include this in the protocol along with any finite element analysis results because this is often the only insight the user or future team members have into why this specific device size was selected. Being able to understand this information helps to quickly identify issues or additional risks and address them at the beginning of the projects.

The Materials section also contains the test equipment or apparatus that will be used. Identifying the load capacities and type of test frame can help ensure that the expected loads are within the calibrated ranges and minimize any equipment variability. General tuning parameters can also be included to reduce any risk. Fixtures and testing environments can be outlined, although they can also be listed in the Methods sections.

Methods

The Methods of Procedure sections outline the actual testing steps. They often align with the applicable FDA guidance documents and ASTM or ISO testing specifications. If you are not sure how to get started with this section, the Procedures section in the standards is often the best place to go. 

The Methods section should outline the test parameters for the testing project:

1. Specimen Preparation - customer-specific assembly instructions


2. Test Configurations


3. Testing Frequency


4. Target Cycles


5. Target Load Data or Method for Load Selection


6. Testing Environment


7. Testing Rates


8. Control Modes (force/displacement)


9. R Ratios


10. Waveforms

This section should also reference any previous studies or test reports to ensure repeatability. If the Methods section has several different test configurations, the parameters listed above should be outlined for each test type. A key area of the Methods section is an outline listing any specific assembly, disassembly, or handling information. Test block or loading orientations, tightening torques, and fluid collection information should also be outlined.

Another critical piece is documenting any custom fixturing or setup instructions. Often, a schematic of the test setup or drawing of the fixtures is included to make it clear how the device was tested. This is particularly important when developing custom test methods. 

Acceptance criteria and exception conditions

Defining the data, you are looking for and what you are comparing against needs to be documented. Determining acceptance criteria based on previous or predicate testing upfront helps everyone involved evaluate the data that is being generated, and quickly make adjustments if needed. It also drives the procedure section to make sure there is an “apples to apples” comparison to predicates. If there are no predicates available for your device, a clear definition of acceptance criteria is paramount. Any exceptions, changes, or modifications to the test protocol should also be noted and justified.

Data analysis and documentation requirements

The test protocol plan should also include the reporting requirements of your company’s internal reporting procedures. Similar to the methods section, the ASTM or ISO test standards outline the common deliverables for the testing. However, if you also require raw data, specific photographs, or inspections, these should be clearly outlined in this section. Statistical analysis information should also be outlined here for data evaluation.

The regulatory environment you plan on submitting to should also be considered. PMDA in Japan, for example, has different requirements than that of other regulatory bodies and requires additional information and flowcharts. Knowing this in advance will make it easier to capture the necessary data and take proactive measures to mitigate any submission risks. 

References

The References section should list any appropriate references, such as test standards, regulatory guidance, previous test reports, published literature, etc. This section helps identify where the information in the protocol came from and where to access that information for more details.

Revision history and signatures

Once the protocol is completed, we recommend having signature and revision control to ensure that all parties are aware of any changes and to get everyone on the same page. Study owners, local management, and testing laboratory sign-offs are common. Additional signatures can be added as required. This ensures multiple people have eyes on the program. A sample signature block is shown below. Depending on internal processes, some protocols have signature blocks on the front page, while others have them at the end.

Conclusion

Often customers have purchased or developed a set amount of parts and have one opportunity to get testing programs right. There is a substantial difference in testing programs that have testing protocols implemented and those that do not. Always invest in a protocol and test plan. A protocol and plan will mitigate your risk, prevent confusion, set clear expectations, and preserve the necessary information for future reference and use.

Element is ISO 17025 accredited and has one of the most expansive medical device testing scopes in the world ranging from orthopedics and cardiovascular implants testing to EMC/EMI/product safety testing, and biological and packaging evaluations. Contact us to discuss how we can help with your test project.