Docket Management
Docket: 01N-0322 - Institutional Review Boards
Comment Number: EC -4

Accepted - Volume 1

Comment Record
Commentor Dr. Stacey Berg Date/Time 2002-05-23 10:54:37
Organization Baylor College of Medicine
Category Academic

Comments for FDA General
Questions
1. General Comments We feel that the proposed rule is unnecessary and will create an enormous new administrative burden on IRBs without improving the protecting of human research subjects. In summary: * We have no evidence that “IRB shopping” is a significant problem. The questions outlined above should be answered thoroughly before any change in regulations or guidance is made. * Because every IRB has the responsibility to conduct a complete review of every protocol (at least in the absence of some sort of consortium agreement), there is no value to an IRB to learn that another IRB has approved that protocol, either outright or with stipulations. * Disapprovals could be circulated, along with the reasons for disapproval if available. However, IRBs should not be burdened with having to justify disapproval in a way that would be persuasive to another IRB. * The rules must explicitly state that the disposition of the protocol by another IRB may NEVER be considered in any enforcement, compliance, or regulatory determination involving the local IRB. This is completely critical; otherwise, disapproval by a single IRB will effectively mean that no IRB will be able to approve the protocol. * There would be no effective way to assess compliance with the proposed rule, and thus no guarantee that the rule, even if passed, would be implemented. Detailed responses to each question in the RFC follow. 1. How significant is the problem of IRB shopping? The OIG report refers to ``a few situations'' where IRB shopping supposedly occurred, but does not offer any quantitative estimate. FDA seeks information on how frequently IRB shopping occurs, the circumstances in which it occurs, and the nature of the different conclusions reached by the IRBs. For example, what number or percentage of sponsors and investigators engage in IRB shopping? What issues lead to IRB shopping? Is IRB shopping more prevalent where certain FDA-regulated products are involved or more likely to occur in certain types of research or under certain other situations? What sorts of differences in IRB conclusions are observed? Are there particular areas of disagreement that suggest a wider issue, such as review of certain trial practices or standards? Is IRB shopping more prevalent when the protocol includes or excludes certain populations (such as women and minorities)? Information on specific occurrences of IRB shopping and disagreement would be useful to help determine the seriousness of the problem. We have no evidence that “IRB shopping” is a significant problem. The questions outlined above should be answered thoroughly before any change in regulations or guidance is made. 2. Who should make these disclosures? The OIG report recommended that sponsors and investigators inform IRBs about any prior reviews, but FDA's experience suggests that there is some variation as to the person who seeks IRB review. In some instances, a sponsor, rather than an investigator, will seek IRB review, especially in the case of devices. One way to deal with these variations could be to require the person who sought the prior review, whether he or she is a sponsor, investigator, or both a sponsor and investigator, to make the required disclosures. As FDA considered this issue further, questions arose as to whether sponsors and investigators should have a duty to [[Page 10116]] inform IRBs about any prior reviews, even if the sponsor or investigator had not sought the prior review, but somehow knew about it. For example, if investigator X and investigator Y were using the same protocol, and if investigator X knew that an IRB had disapproved investigator Y's protocol, should investigator X inform his or her IRB about that disapproval even though it involved a different investigator? If the sponsor knew that an IRB had disapproved investigator Y's protocol, should it notify investigator X so that he or she could inform his or her IRB? FDA invites comment on these issues. Sponsors know the outcome of all IRB reviews of their studies, whereas investigators are rarely in a position to know the outcome of any IRB review besides their own. This is quite analogous to the situation with adverse events. It would be highly impractical to require investigators at individual sites to obtain the records of IRB reviews at all the other sites of a multicenter study, just as it would be absurd to require investigators to distribute the results of their own IRB reviews to all the other sites. Furthermore, even if investigators did make such an attempt, what would happen if the number of sites involved expanded? Would investigators be required to monitor continually the sites involved in a study and to document that all sites were notified of all IRB actions, even those that predated a site’s inclusion by months or years? Such a requirement would be virtually impossible to fulfill. It is quite alarming that FDA would even consider a requirement for investigators to report a review that they “somehow knew about.” Even if FDA encouraged such a practice, how would it be enforced? One can envision FDA criticizing an investigator for not disclosing a review on the basis that the investigator must “somehow” have known about it. Similarly, would investigators feel that they needed to report to their IRBs (or to FDA) any kind of gossip that they might have overheard regarding the disposition of a protocol by another IRB, for fear that if not they would be accused of having not disclosed an IRB review about which they knew? 3. Who should receive the disclosures? The OIG report states that IRB's that are reviewing or are going to review a protocol should be informed about prior IRB reviews. This assumes that the prior IRB's decision is known at the time the second IRB is asked to review the protocol. But what happens if the new IRB has already approved the protocol at the time the prior IRB's decision becomes known? Would information about prior IRB reviews still be helpful? One could argue that sponsors and investigators should inform new IRBs about prior IRB reviews, even if the new IRB has already approved the protocol, because the prior reviews might be relevant to the new IRBs continuing review of a protocol. Disapprovals could be submitted to IRBs involved in review of the study. Even if the receiving IRB has already approved the study, the information about the disapproval could be useful in continuing review. 4. What information should be disclosed? The type of information to be disclosed depends on the purpose of the disclosure. If the purpose is solely to be certain that an IRB is aware of a prior adverse conclusion, perhaps only unfavorable prior reviews would need to be disclosed. If the purpose of the disclosure is to ensure that IRBs receive all relevant information about a study, it might be appropriate to disclose all prior IRB decisions, both positive and negative. Should all prior IRB reviews, including approvals, be disclosed? It would be counterproductive to contemplate sending all IRB review information to all IRBs involved in reviewing a study. First, and most importantly, no improvement in human subject protection would result (see #s 5 and 6 below) . Second, IRBs would be burdened with a huge volume of material from outside institutions which would require review. This in turn would dilute the time and attention given to the review of the protocol itself, which would almost certainly weaken the protections that subjects receive now. 5. If a proposal would not require disclosure of all prior IRB decisions, what information should be disclosed? Even if the purpose of disclosure is solely to be sure an IRB is aware of an unfavorable IRB review, there could be different degrees of disclosure. An unfavorable IRB decision could encompass complete disapproval of a protocol, a decision to approve a protocol with stipulations, and a request for significant changes to a protocol. Even a decision to require additional reviews by the IRB could be considered as an unfavorable decision. A requirement to disclose only prior unfavorable IRB reviews may presume that an unfavorable review is more likely to be correct than a favorable review. If one presumes that the earlier IRB correctly disapproved, or requested modifications of, a protocol, then a new IRB could, indeed, benefit from knowing about that decision. This could be the case, for example, if the earlier IRB disapproved a protocol because one of its scientific members recognized that the investigational product would present a greater risk of harm to research subjects than was acknowledged in the informed consent document, based on that member's knowledge of certain animal studies. This information would be helpful to a new IRB, particularly if its scientific members did not possess the same expertise as the earlier IRB. On the other hand, a favorable decision by a prior IRB with superior expertise in a particular case could also be of value to a subsequent IRB as well. Conversely, in cases where an initial review, either favorable or unfavorable, was not well-founded, information about the earlier IRB's review decision may offer little or no value to a new IRB and might lead to an ill-considered, ``defensive'' acceptance or rejection of a satisfactory proposal. For example, if an IRB was associated with an institution, and the institution was well-known or had a good reputation, a subsequent IRB might be inclined to follow the first IRB's decision even if the first IRB's decision was not well-founded. The question raises a very important point about the duties of individual IRBs. Because every IRB has the responsibility to conduct a complete review of every protocol (at least in the absence of some sort of consortium agreement), there is no value to an IRB to learn that another IRB has approved that protocol, either outright or with stipulations. The other important question raised here is whether a particular IRB, for one reason or another, might uncover a substantial risk to human subjects that a second IRB would miss, and whether the second IRB would therefore have done a better job protecting its local subjects if it had had the benefit of the first IRB’s judgment. This question should be answered with data before a regulation is implemented on the assumption that such cases are common. Hundreds of multicenter trials have been reviewed by a large number of IRBs in the past few years; the reviews could be studied for discordance in the risks uncovered. For research involving an investigational product, one could argue that the most efficient way to ensure that all relevant risks are described in the protocol would be for FDA, rather an individual IRBs, to be responsible for this part of the review. In the example provided above, a scientific member of an IRB has knowledge of preclinical data pertaining to an investigational product, and as a result of this member’s provision of the data, his IRB rejects the assessment of risk in the consent form and disapproves the study. Human subjects would be better protected if FDA conducted this level of scientific review before approval of the IND. If FDA did so, no IRB would be at risk of inadvertently approving a protocol that did not include all the relevant information about an investigational product’s risks. Regarding the disclosure of approval with stipulations, it should be noted that many questions and stipulations from IRBs to investigators reflect local conditions and would not be meaningful to any other IRB. Some examples include stipulations to provide consent forms in additional languages depending on the local population; stipulations that wording of contraception information follow local requirements; requirements for liability disclaimer language to conform with institutional standards; and so forth. None of this information would be helpful to another IRB reviewing the same protocol. Disapprovals could be circulated, along with the reasons for disapproval if available. However, IRBs should not be burdened with having to justify disapproval in a way that would be persuasive to another IRB. More importantly, for the reasons outlined in the next paragraph, the potential harm of circulating disapprovals outweighs the potential benefits of doing so. This question raises a critical concern regarding “defensive” disapprovals. Given the increasing legal liability faced by investigators and IRBs, it is easy to envision a scenario in which IRBs feel compelled to adhere to a standard in which the most unfavorable review of which it is aware sets the bar for further review of the protocol. It is equally essay to envision a scenario in which FDA, OHRP, or another regulatory body criticizes one IRB for failing to raise concerns identified by another IRB. For these reasons, any rules or guidance requiring disclosure of IRB reviews to another IRB should state explicitly that the decision for approval or disapproval of a protocol rests entirely with the local IRB. Furthermore, the rules must explicitly state that the disposition of the protocol by another IRB may NEVER be considered in any enforcement, compliance, or regulatory determination involving the local IRB. This is completely critical; otherwise, disapproval by a single IRB will effectively mean that no IRB will be able to approve the protocol. 6. To permit a subsequent IRB to assess the value of a prior IRB decision, should information about the basis for the prior decision be disclosed? Currently, IRBs are not generally required to document the reasons for approving a study, so if a proposed rule would require all IRB decisions to be disclosed, IRBs might have to explain their reasons for approving a study. Should the disclosed information include information about the composition and expertise of the prior IRB's members? What would be the additional burden on IRBs if FDA required the disclosure of the basis for all or even some IRB review decisions? How would this affect the time needed to conduct an IRB review? In effect, by asking this question, FDA admits that if an IRB were required to consider other IRBs’ actions in its review of a protocol, the IRB would have to assess the quality (“value”) of the other IRBs’ work. This is an extremely important point, for it admits that the rule-making discussed throughout this document would creates an entirely new category of research review, namely, the review by local IRBs of other IRB’s decision-making capability. It is difficult if not impossible for one IRB to assess the value of another IRB’s decision. By regulation, IRBs must consider the local research context. This context cannot be easily made known to an outside IRB. Therefore the outside IRB by definition will be unable to assess the research from the local IRB’s point of view. Similarly, the outside IRB will have considerable difficulty assessing the IRB’s review taken out of context. The basis of all IRB approvals is the determination after substantive review that the research meets federal and institutional guidelines for the protection of human research subjects. The basis for approval does not vary from protocol to protocol. Therefore there is nothing for IRBs to disclose regarding why a specific protocol is approved. The knowledge that an outside IRB has approved a protocol only tells a local IRB that the outside IRB felt that the protocol met all requirements for approval. Since the local IRB must make the same determination for itself from its own review of the protocol, knowledge of the outside IRB’s approval contributes nothing to the local IRB. The composition of IRBs is often if not always publicly available. However, it is quite difficult to summarize the expertise of members, which quite often extends into areas not obvious from such demographic information as departmental affiliation. Furthermore, IRBs understand the expertise of their own members through frequent interaction and substantive discussion of many protocols. Like the local research context, this level of understanding is not easily transmitted to an outside IRB; the outside IRB therefore has little to gain from knowing the composition of the local IRB. Any requirement for IRBs to provide additional information about their membership, expertise, review process, or basis for protocol approval would create a significant additional burden for IRBs that the OIG report itself notes are already overburdened. Furthermore, the disclosures discussed in this question would not provide any meaningful information to another IRB. Therefore a requirement for these sorts of disclosures would mean more meaningless work for all IRBs, with no advancement in the protections for human research subjects. 7. How should FDA enforce the requirement? The OIG report did not suggest any method for enforcing a requirement that these disclosures about prior IRB reviews occur. What would be an appropriate sanction to impose on an investigator or sponsor for failure to comply with a disclosure requirement? FDA must learn about a violation before it can consider what sanctions might be imposed. The OIG report did not recommend that sponsors and investigators inform FDA about any prior IRB reviews; it only recommended that sponsors and investigators inform IRBs. If FDA has no knowledge about the prior IRB review, the agency might find it difficult to detect noncompliance. FDA invites comment on how it might enforce the requirement efficiently. There should be no requirement for the FDA to enforce. If one steps back and attempts to envision a process in which actions of one IRB are reported to all other IRBs reviewing a study, the absurdity of such a requirement quickly becomes obvious. For a multicenter study involving hundreds of centers-- a common scenario-- the protocol be initially submitted to the individual IRBs over a span of many months. In fact, the protocol may be in annual or continuing review at some centers while it is in initial review at other centers. If the IRB at each center had to take into account all the IRB actions at every other center, an endless loop would result in which it would become impossible for any IRB to approve the study! 8. Are There Other Ways to Deal with IRB Shopping Other Than Disclosure of Prior IRB Reviews? Although the OIG report recommended requiring disclosure of prior IRB reviews, there may be other ways to deal with IRB shopping. Therefore, if the problem of IRB shopping is significant enough to warrant Federal regulatory action, are there other requirements that could be employed to address the problem besides mandating disclosure of prior IRB reviews? As discussed in #1, we strongly disagree with the proposed rule-making, since no compelling evidence has been presented supporting the need for the rule. However, even if “IRB shopping” is identified as a real problem, there are better ways to solve it than by requiring all IRBs to review all other IRBs’ reviews (which is a brute force approach to a problem that even in the OIG report is identified as “OIG had heard of a few situations”). Clearly, the vast majority of protocols that are submitted to more than one IRB are multicenter studies, rather than sequential submissions designed to find the “easiest” IRB. A simple way to alert IRBs to the possibility of IRB shopping would be to require the sponsor or investigator to answer the question “Has this protocol ever been withdrawn from consideration at or disapproved by another IRB?” A “yes” answer could lead to further investigation in these specific instances, without drowning IRBs in a sea of additional paperwork.. In summary, we strongly recommend against adoption of a rule that would require IRB reviews to be disclosed to other IRBs. * The proposed rule-making requiring that IRBs be informed of other IRBs reviews raises a large number of substantial concerns, with no evidence that there is a problem that needs to be fixed, and no evidence that the rule proposed would fix the problem if it does exist. * The proposed rule would create a vast new burden for IRBs (and potentially for investigators), because the rule-making discussed throughout this document would creates an entirely new category of research review, the review by local IRBs of other IRB’s decision-making capability. * It is likely, due to the unending circle of reviews-based-upon-reviews that would be created, that the entire review process would become grid-locked in the case of multicenter studies. * There would be no effective way to assess compliance, and thus no guarantee that the rule, even if passed, would be implemented. * Most importantly, there is nothing in the proposed rule that would enhance the protection of human research subjects, and much that would dilute the protections that are already in place. Stacey Berg, M.D. Chair, Institutional Review Boards for Human Subject Research for Baylor College of Medicine and Affiliated Hospitals Kathleen J. Motil, M.D. Ph.D. Chair, Institutional Review Boards for Human Subject Research for Baylor College of Medicine and Affiliated Hospitals Janet Allen, Director, Assurance and Compliance Services Baylor College of Medicine Addison Taylor, M.D., Ph.D. Associate Dean for Clinical Research Baylor College of Medicine James Patrick, Ph.D. Vice President and Dean for Research Baylor College of Medicine




EC -4