| Comment Record |
|
Commentor |
Mr. Kim Nitahara |
Date/Time |
2001-12-24 15:59:58 |
|
Organization |
META Solutions, Inc. |
|
Category |
Company |
| Comments for FDA General |
| Questions |
|
1. General Comments
|
To whom this may concern,
These comments are provided for the Docket 00D-1538 - Draft Guidance for Industry: Electronic Records; Electronic Signatures, Validation. We represent a consulting company that provides 21 CFR 11 services to pharmaceutical, biotechnology, medical device, software vendors, and contract research companies. We also provide software development services, clinical site monitoring, and clinical data management services that comply with 21 CFR 11 regulations.
Our comments follow:
5.1 System Requirements Specifications
Regarding paragraph 1, Regardless of whether the computer system is developed in-house, developed by a contractor, or purchased off-the-shelf, establishing documented end user requirements is extremely important for computer systems validation, it is suggested that the Guidance should note that the size and complexity of an electronic recordkeeping system will vary greatly because of the broad coverage of the regulations, but that the need to confirm that the system can consistently meet requirements is the basis of the validation. For example, a large electronic document management system has the same basic validation requirement as a SAS program that is written to perform a standard data edit verification in a clinical study. This comment reflects an industry problem with the definition of a system and the relative amount of work that is required for ad hoc programs and customized software that is used for regulatory purposes. In certain cases, such as the current practice of re-using SAS programs and changing certain variables for a new study, the use of procedural controls and simple checklists can meet the basic validation needs without the formal and extensive documents that typically accompany a very large system implementation. This clarification in the Guidance would be extremely beneficial because a one-size-fits-all approach to validation has resulted in unreasonably complicated (and expensive) efforts in the industry to date.
Regarding paragraph 2, Other factors not specifically addressed in part 11 may also impact on electronic record trustworthiness, integrity and system performance. and the following three bullets that refer to system performance. The term system performance means different things to different segments of the community. The term performance is typically related to network speed or software (speed) performance. Other than these network or database/application performance measures, it is difficult to establish suitable and non-subjective measures of system performance. The term is not used in 21 CFR 11 regulations, unless it refers to meeting requirements. The introduction of system performance in this Guidance requires clarification and explanation of the reason for this new issue.
5.1 System Requirements Specifications
Regarding paragraph 1, you should obtain evidence that the computer system implements those needs correctly and that they are traceable to system design requirements and specifications. The industry standard is to confirm that the user requirements are met to assure that a system has been validated. The documentation of traceability between design requirements and validation testing is an expectation for large systems projects. However, it is unusual to demonstrate traceability between the 1) requirement documents and 2) the programmer's specification documents as part of the validation effort. This is considered to be a part of the system design effort, and is a responsibility of the developers. Perhaps this should be rewritten traceable to system design requirement specifications if the intent is not to confirm traceability between the 'three' separate documents (requirements, specifications and validation plan).
5.2 Documentation of Validation Activity
Regarding 5.2.3 Validation Report, Whenever possible, test results should be expressed in quantified terms rather than stated as pass/fail. and 5.4.3 How test results should be expressed, Quantifiable test results should be recorded in quantified rather than qualified (pass/fail) terms. These statements have raised many industry questions and require further clarification and/or examples. Does this refer to the documentation of the test script results, or the description of the test results in the Validation Report? In general, the industry practice is to document whether a specific test script has passed or failed, and either include a printout as evidence, or include sufficient detail in the written test script to enable the test to be recreated. The term quantified suggests a statistically-based evaluation process, or the reporting of results in a measurable form. An example from the Agency of the use of quantified terms would be very useful and reduce the problems that would arise from a misinterpretation of the Guidance.
5.4 Dynamic Testing
Regarding 5.4.1 Key Testing Considerations and 5.4.2 Software testing should include, there is a need for additional qualification of the relative need for implementation of the guidance regarding test conditions, simulation tests, live, user-site tests, structural testing, functional testing' and program build testing. This is necessary because of the significant difference in the type, complexity, duration and purpose of computer systems and programs that must comply with 21 CFR 11 regulations. It is recommended that each of these sections first describe the principles for dynamic testing, but indicate that the presented level of detail and scrutiny is appropriate for large systems. It is important to assure that the same principles apply to all systems and programs that must meet part 11 requirements, although the specific methods might depend on the circumstances.
Regarding 5.6 Extent of Validation, it is suggested that this statement should be moved to the beginning of the document to describe the different expectations for systems that have different levels of regulatory risk and system complexity. It should include a further elaboration of the recommended adherence to validation principles. It would also be valuable to acknowledge the significant difference between in-house developed systems, custom-developed systems, purchased (off-the-shelf) systems, integrated systems (combinations), and end-user-configurable systems.
6. Special Considerations
6.1 Commercial, Off-The-Shelf Software
Regarding 6.1.2 Software Structural Integrity, which indicates that Where source code is not available for examination, end users should infer the adequacy of software structural integrity by doing all of the following: conducting research into the program's use history and evaluating the supplier's software development activities, it is recommended that the latter evaluation is much more useful than the former research activity. The Guidance notes that both should be done, but does not provide an assessment of the relative difference in the reliability and accuracy of the two approaches. In some cases, such as a custom-developed system, the research into the program's use history would not be useful, and research of others' experiences might be of limited value.
Thank you for the opportunity to provide our comments on this draft document.
Kim Nitahara for META Solutions, Inc.
December 24, 2001
|
|