DFA admits to inconsistencies
The preliminary report of the Working Group is quite revealing

Last week, the FDA’s Centre for Devices and Radiological Health issued its plans for the next steps in the revamp of its services but the report reveals inconsistencies which need to be looked at more closely. Those in the know will remember that the CDRH report was released back in August 2010, when its 510(k) Working Group listed more than 60 recommendations grouped aimed at improving the Center's effectiveness.

Of the reports’s seven findings, the first five are self-evident. They are:
[1] There is insufficient clarity with respect to the definition of "substantial equivalence".
[2] CDRH's current practice allows for the use of some types of predicates that may not be appropriate.
[3] The de novo pathway is important and has not been optimally used across the Center.
[4] It is challenging for reviewers to obtain the information they need to make well-supported clearance decisions.
[5] CDRH's knowledge management infrastructure is limited.

Variations in expertise could cause inconsistencies
Finding 6 stands out because it exposes an issue of considerable importance but does so in typically obtuse language that may cause many to miss its significance. The actual wording of Finding 6 reads as follows: Variations in the expertise, experience and training of reviewers and managers may contribute to inconsistencies in the decision making.

Fact is, more than 30% of reviewers have less than two years experience under their belt. This means that one-third of the 510(k)s submitted to the FDA will be reviewed by people with little agency history in their heads. One former high-level manager from CDRH advises everyone to study the preambles to every regulation. This is because preambles have legal standing, but most CDRH reviewers have never read them. When asked how the CDRH could give a warning letter to a sponsor over a consent form that had been approved by an IDE reviewer, a mid-level manager from CDRH scoffed and said: "Reviewers don't have time to read all that stuff."

Reviewer memos
Reviewers prepare an internal "review memo" for every 510(k) reviewed. Each review memo documents the rationale for a given decision. The Working Group Report states that less experienced reviewers write lengthier review memos, which are presumably more efficient. Something unexplained seems to happen with reviewers with more than ten years’ experience; their highest percentage falls into an "unknown page count" column as shown in the report.

The Working Group did not make recommendations specific to review memos. The CDG recommends the very existence of review memos be more widely publicized. A careful search of the FDA website, Device Advice, and Google turned up very little information about review memos. In fact, the only information found concerned the 501(k) Quality Review Program. That item states that a random selection of 510(k)s will be selected for review each quarter. A phone call to CDRH's Office of Device Evaluation gleaned the information that an internal review memo is written by the reviewer for every 510(k) reviewed.

The memos are not generally offered to the submitter, and are available under the Freedom of Information Act. A call to FOI Services Inc. confirmed this information and clarified that copies of review memos for specific 510(k)s can be obtained through their office. The review memo is of vital importance in gaining insight into the reviewer's thinking. Because the memos are available under Freedom of Information, why not make them available in the 510(k) database so that everyone can see them without making an independent FOI request? It would contribute to better submissions and more consistent reviews. In addition, if a 510(k) submission was not cleared or received an NSE decision, the review memo should automatically be included in the FDA letter to the submitter.

Third-Party Review
510(k)  for Class I and Class II devices that do not require clinical data may go through a pre-review by an accredited third-party. While data show that submissions that have gone through pre-review tend to go through the final FDA review faster, there are no data to show if the total review process (pre-review plus FDA review) is longer or shorter. Concerns have been raised about the quality and consistency of third-party reviews.

The Working Group recommends regular evaluation of what devices are eligible for third-party review and that the agency enhance its third-party training programs. The third-party review program should be promptly expanded as planned in the Next Steps. Medical device companies would be well-advised to shop around for reviewers who have experience in their type of product, check their schedule for reviewing, and assess the cost of the review. Typically the reviewer will require less than a month to give tentative approval (or discuss deficiencies) for a submission.

Third-Party Review
The Working Group recommends that reviewer training, professional development, and knowledge-sharing be enhanced. This is a good idea. A lack of reviewer experience can have serious consequences for manufacturers. Submissions are not reviewed consistently, reviewers may not have the scientific training to adequately review the submission, decisions may be delayed, review decisions may be inadequately documented, unnecessary or even outdated or inappropriate testing may be requested. In one situation, the reviewer of an IVD submission utilizing de-identified data asked the submitter to obtain additional information from the clinical subjects—an illegality because it would violate privacy rules, as well as impossible because the subject's identity was not known. In the case of the IDE approved informed consent form, BiMo nevertheless issued a warning letter for alleged deficiences.

The CDRH cannot do regular assessments
Finding 7 of the report admits that the CDRH is no position to do regular assessments. But the 510(k) Quality Review Program is the CDRH's primary tool for assessing the quality of 510(k) reviews. Through this program, managers in ODE and OIVD assess a sample of review memoranda on a quarterly basis. A standardized checklist is used to evaluate the completeness of each review memorandum, but not the adequacy or appropriateness of the reviewer’s decision making rationale and explanation.

Further, although CDRH collects information on device performance in the postmarket setting, important limitations, including the inability to consistently link postmarket events to specific 510(k)s, make this information, in isolation, an unreliable measure of program effectiveness.

The Working Group recommended that CDRH develop metrics to continuously assess the quality, consistency, and effectiveness of the 510(k) program, to periodically audit 510(k) review decisions, and to measure the effect of any actions taken to improve the program. We hope that such information will be made publicly available in ODE's annual reports. Submitters can only increase the quality of their submissions if they have examples to work from.

This text was edited from an article written by Nancy J Stark, President, Clinical Device Group Inc. For more information, phone 773-489-5721 or email cdginc@clinicaldevice.com.