Since the concept of writing a weekly blog and actually having a following is somewhat foreign to me, I was not sure what to expect or the reception I would receive writing a weekly column. So far, the support and responses have been overwhelmingly positive; and for that, I would personally like to thank the readers.

Additionally, I will continue to write about and provide relevant information associated with supplier quality (four installments remaining) and will eventually begin addressing quality and regulatory concerns influencing the medical device industry as a whole.

Warning letter violation
The next time you have some time, I strongly suggest visiting the U.S. Food and Drug Administration website and spending some time traversing though the warning letters. It never ceases to amaze me the sheer volume and type of violations identified by the agency, on a regular basis. In fact, the information depicted within these warning letters can be easily employed by any organization wishing to remain out of the proverbial FDA doghouse.

In support of this installment, I have found additional pieces of stellar quality performances, resulting in egregious violations of the QSR. The FDA has graciously acknowledged these violations with warning letters. Warning Letter #1 was issued in January 2010; and Warning Letter # 2 was issued in November 2009.

Warning letter #1
Failure to establish and maintain procedures to ensure that all received product conforms to specified requirements, as required by 21 CFR 820.50. For example, your firm has not performed testing for 3 out of 4 incoming lots received by one of your suppliers of disposable delivery sets, (b)(4)
Your firm also receives disposable delivery sets from (b)(4). This firm is not on your approved supplier list and you did not have adequate documentation upon receipt to demonstrate that these incoming sets met required specifications.
Also, your firm contracted with an unapproved supplier to print out labels for the Sentinel pumps, which included serial numbers. Your firm did not perform any incoming inspection of the printed labels to demonstrate that they met required specifications. As a result, your firm received two complaints that the labels were smudging and chipping off.

Warning letter #2
Failure to establish and maintain adequate procedures for identifying valid statistical techniques required to establish, control, and verify the acceptability of process capability and product characteristics in production and process control systems, as required by 21 CFR 820.250.
For example, for inspecting absorbable suture material per (b)(4) the firm uses (b)(4) criteria for sampling that is not in accordance with GB-2828-1-2003/ISO 2859-1 1999. This Reference for Sampling by Attributes states that if the acceptance number is above the acceptance level, then reject the sampled lot without retesting. The firm’s procedure allows for retesting if the initial sampling test fails.

Regulations and requirements
The regulations provide a little more specificity when addressing measurement capabilities, a fundamental requirement for an effective Gage R & R Study. Remember – Devine Guidance Rule # 2 – “measuring and monitoring equipment shall be calibrated and maintained.” In fact, the expectation is that the approach to metrology pursued by medical device manufacturers be well documented with ability to trace back to a recognized national standard, e.g., National Institute of Standards and Technology (NIST).

So what is Devine Guidance Rule # 1? “Compliance to regulations is not optional.” That said, the key regulations providing oversight are:

1. QSR – Subpart G – Production and Process Controls

820.72 Inspection, Measuring, and Test Equipment

(a) Control of inspection, measuring, and test equipment. Each manufacturer shall ensure that all inspection, measuring, and test equipment, including mechanical, automated, or electronic inspection and test equipment, is suitable for its intended purposes and is capable of producing valid results. Each manufacturer shall establish and maintain procedures to ensure that equipment is routinely calibrated, inspected, checked, and maintained. The procedures shall include provisions for handling, preservation, and storage of equipment, so that its accuracy and fitness for use are maintained. These activities shall be documented.

(b) Calibration. Calibration procedures shall include specific directions and limits for accuracy and precision. When accuracy and precision limits are not met, there shall be provisions for remedial action to reestablish the limits and to evaluate whether there was any adverse effect on the device’s quality. These activities shall be documented.

(1) Calibration standards. Calibration standards used for inspection, measuring, and test equipment shall be traceable to national or international standards. If national or international standards are not practical or available, the manufacturer shall use an independent reproducible standard. If no applicable standard exists, the manufacturer shall establish and maintain an in-house standard.

(2) Calibration records. The equipment identification, calibration dates, the individual performing each calibration, and the next calibration date shall be documented. These records shall be displayed on or near each piece of equipment or shall be readily available to the personnel using such equipment and to the individuals responsible for calibrating the equipment.

2. QSR – Subpart H – Acceptance Activities


Section 820.8

(a) General. Each manufacturer shall establish and maintain procedures for acceptance activities. Acceptance activities include inspections, tests, or other verification activities.

(b) Receiving acceptance activities. Each manufacturer shall establish and maintain procedures for acceptance of incoming product. Incoming product shall be inspected, tested, or otherwise verified as conforming to specified requirements. Acceptance or rejection shall be documented.

3. QSR – Subpart 0 – Statistical Techniques

(a) Where appropriate, each manufacturer shall establish and maintain procedures for identifying valid statistical techniques required for establishing, controlling, and verifying the acceptability of process capability and product characteristics.

(b) Sampling plans, when used, shall be written and based on a valid statistical rationale. Each manufacturer shall establish and maintain procedures to ensure that sampling methods are adequate for their intended use and to ensure that when changes occur, the sampling plans are reviewed. These activities shall be documented.

4. MDD (Please note the MDD references EN ISO 13485, the Harmonized Standard for Medical Device Quality Management Systems)

Article 3 – Essential Requirements: The devices must meet the essential requirements set out in Annex I, which apply to them, taking account of the intended purpose of the devices concerned.

Where a relevant hazard exists, devices, which are also machinery within the meaning of Article 2(a) of Directive 2006/42/EC of the European Parliament and of the Council of 17 May 2006 on machinery (5) shall also meet the essential health and safety requirements set out in Annex I to that Directive to the extent to which those essential health and safety requirements are more specific than the essential requirements set out in Annex I to this Directive.

Annex II.3 EC Declaration of Conformity – Full Quality Assurance System: The manufacturer must ensure application of the quality system approved for the design, manufacture and final inspection of the products concerned, as specified in Section 3 and is subject to audit as laid down in Section 3.3 and 4 and to Conformity surveillance as specified in Section 5.

5. EN ISO 13485 – 7.6 Control of Monitoring & Measuring Devices (recommend reading ISO 10012)

The organization shall determine the monitoring and measurement to be undertaken and the monitoring and measuring devices needed to provide evidence of conformity of product to determined requirements.

Gage R & R

Once the features required for measurement have been identified, the execution of a Gage R & R study becomes the foundation for determining accuracy of the measurement system. According to research performed by Kappele and Raffaldi, “a gage repeatability and reproducibility study can determine if a measurement system is acceptable for the required measurement.”

Additionally, an increase in variability results in the instability of output quality. For example, if the measured variation is large, the potential for defects being introduced into products and resulting in overall poor product quality will increase substantially. To mitigate issues surrounding an unstable measurement system, gage performance curves provide an effective solution for determining the overall probability and the acceptance of a non-conforming component or the rejection of an acceptable component. According to Mehta and Kauffman, “normally, for the acceptable measuring system, no more than 9 percent of the tolerance should be consumed by the measurement variation or Gage R & R error.”

The performance of Gage R & R ensures that the measurement modalities selected are repeatable and accurate. Measurement accuracy is one of the salient fundamentals associated with establishing successful measurement correlation. In a paper written by Bangert, according to Rusty Eckstrom, “the decision that you make on data is only as good as the data provided.” An inherent part of any medical device manufacturer’s quality system is the ability to perform accurate measurements in the pursuit of determining the acceptability of finished medical devices.

Now true story time – several years ago, I was visiting a supplier and witnessed the performance of a Gage R & R study. The inspector was using calipers to measure a small cylinder. The gage was linked to a data acquisition system. Now instead of breaking contact and measuring 22 components, the inspector depressed the data acquisition button 22 times. Guess what, the data set was perfect, too bad there was no value in the exercise.

From my perspective, an effective Gage R & R has multiple elements. The first element (not necessarily in order) is adherence to Devine Guidance – Rule # 2. The second element is adherence to Devine Guidance – Rule #3; “Document the results of all events in writing (figuratively speaking).” Why – because (broken record time) if it is not documented in writing, the event did not occur.

The third element is to ensure inspectors performing the Gage R & R Study are adequately trained and the training documented within a training record and the record retained. The fourth element is to ensure a minimum of three distinct samples, premised on a validated sampling plan, is randomly selected for the study. The fifth element is to ensure multiple inspectors are employed within the study. The sixth element is to ensure the samples are serialized for measurement traceability. Why? Because you just might want to think about repeating the Gage R & R Study, in house, to ensure you (a medical device manufacturer) and your supplier, achieve the same results. I am a big proponent of Minitab™ for use in Gage R & R Studies (once again, I am not a paid spokesperson for this product). The seventh element is to ensure the measurement data is reviewed and the results make sense. If there were anomalies noted in the data set, these must be addressed through root-cause analysis.

Reduced inspection programs

For reduced inspection programs, the same concept of documenting results and the decision tree employed to reduce inspection is necessary. In short, not wanting to perform acceptance inspection is not a valid reduction in defensive-receiving inspection program. Go ahead an attempt to explain to the FDA or your notified body that your reduced-inspection program is based on inspecting when you feel like it or can find the time. Then remember to try and spell “Warning Letter” or “Major Deviation.”

Reduced inspection programs are a subset of receiving inspection. A reduction in inspection activities can result in significant cost savings for an organization. Skip-lot inspection plans are a derivative of the commonly accepted continuous sampling plans. H. F. Dodge, one of the founding members of the American Society for Quality, developed skip-lot inspection plans in 1955.

According to Vijayaraghavan, the creation of a skip-lot sampling plan requires the adherence to a 6-step model and the application of a reference plan. The premise of a skip-lot inspection model is when the quality is good, as evidenced by inspection results, a fraction of the lots will require inspection, after the establishment of initial lot acceptability. The six steps identified by Vijayaraghavan, for the skip-lot sampling model, e.g., SkSP-3, are as follows:

1. Perform normal inspection employing the reference plan;
2. When the inspection and acceptance of a predetermined number of consecutive lots has occurred, switch to a skip-lot approach;
3. When a lot has failed during the application of skip-lot inspection, begin inspecting every lot until a predetermined level of acceptance has occurred;
4. If an additional failure occurs, return to normal inspection;
5. When an acceptable trend has been re-established, during the performance of normal inspection, return to skip-lot-inspection; and
6. Address and/or correct all non-conformances.

Prior to commencement of a reduced inspection program, understanding process capability is germane. According to Fink and Margavio, determining the stability of a process through the employment of control charts is mandatory before pursuing any inspection modality, i.e., 100 percent inspection, sampling inspection, etc. An organization needs to determine the level of sustainable quality, when measured against the specification. Once process capability is determined, the level of ongoing inspection requirements and the linkage to an economic model can occur. The determination of the economic cost of quality and the profitability associated with different inspection policies can occur, once process capability is established.

A fundamental goal of any organization is to determine warranted levels of defensive-receiving inspection, if any. Ericson believed reduced inspection levels were warranted, premised on the results collected from normal sample inspection. Ericson was also a big fan of supplier-provided statistical data as a tool to reduce the defensive-receiving inspection burden. All inspection programs structured to remove or reduce costs associated with the reduction in the burden of defensive-receiving inspection will favorably influence the reduction in the total cost of quality. According to Freiesleben, inspection never actually solves a quality problem. In fact, Freiesleben believed an optimal level of quality could not effectively be linked or driven by cost factors. 

Conclusion
When properly employed, Gage R & R and reduced inspection programs can have a positive influence in reducing expenditures associated with escalating costs of quality. The real questions needing to be answered are: (1) how much inspection is necessary; (2) are the results of inspection activities accurate; (3) is the entire inspection program well documented; (4) how much is the defensive-receiving inspection program costing the organization; and (5) is the inspection program defendable in front of an investigator or auditor?

In the next edition of Devine Guidance, I will provide a list of all of the Devine Guidance Rules and dive into the Supplier Corrective Action Request (SCAR) process. In closing, thank you for joining me and I look forward to having read the next installment of Devine Guidance. 

References:

1. Bangert, M. (2008, September). The fundamentals of data collection. Quality 47(9).
2. Code of Federal Regulation. (2008, April). Title 21 Part 820: Quality system regulation. Washington, D.C.: U. S. Government Printing Office.
3. Devine. C. (2009, July). Exploring the effectiveness of defensive-receiving inspection for medical device manufacturers: a mixed method study. Published doctoral dissertation. Northcentral University. Prescott Valley, AZ. 
4. Dodge, H. (1955). Skip-lot sampling plan. Industrial Quality Control, 11(5).
5. Ericson, J. (2006, November). Lean inspection through supplier partnership. Quality Progress, 39(11), 36-41.
6. FDA – U.S. Food and Drug Administration Website. (2009). Warning letters. Retrieved February 3, 2010, from http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/ucm193670.htm
7. Fink, R., & Margavio, T. (1994). Economic models for single sample acceptance sampling plans, no inspection, and 100 percent inspection. Decision Sciences, 25(4).
8. Freiesleben, J. (2006). Costs and benefits of inspection systems and optimal allocation for uniform defect propensity. The International Journal of Quality & Reliability, 23(5). Retrieved January 24, 2007, from http://proquest.umi.com
9. Kappele, W., & Raffaldi, J. (2006, June). Gage R & R improves quality and profitability. Quality, 45(6). Retrieved October 31, 2008, from http://proquest.umi.com
10. Medical Device Directive. (1993). Council Directive 93/42/EEC. Medical Device Safety Service. Retrieved January 11, 2010, from http://directive93-42-eec.htm
11. Medical devices – quality management systems – requirements for regulatory purposes. (2007). EN ISO 13485:2003/AC:2007.
12. Mehta, M., & Kauffman, P. (2006, August). Improve Gage R & R results. Six Sigma Forum Magazine, 5(4). Retrieved October 31, 2008, from http://proquest.umi.com
13. Poor supplier control causing recalls, FDA says; contract is key to success. (2007, May). The Sheet – Medical Device Quality Control, 11(6). Danvers, MA.
14. Vijayaraghavan, R. (2000, September). Design and evaluation of skip-lot sampling plans of type SkSP-3. Journal of Applied Statistics, 27(7)