Department of Health and Human Services
Public Health Services
Food and Drug Administration
FDA Public Hearing
Thursday, October 19, 1995
3727 Colesville Road
Silver Spring, Maryland
- Mitchell E. Daniels, Jr., North American, Pharmaceutical Operations, Eli Lilly & Co.
- Stephen Stefano, Health Management Division,, Glaxo Wellcome, Inc.
- Bryan R. Luce, Ph.D., Panel on Cost-Effectiveness in Health and Medicine and MEDTAP International, Arlington, Virginia
- Alan L. Hillman, M.D., M.B.A., Task Force on Principles for Economic Analysis of Health Care Technology, and Leonard Davis Institute of Health Economics, University of Pennsylvania
- Jonathan Lax, The Marketing Audit, Philadelphia, Pennsylvania
- Elizabeth Moench, MediciGroup, Inc., King of Prussia, Pennsylvania
- Questions for Speakers
PANEL 1 PROCEEDINGS
I am Mitchell E. Daniels, Jr. The views I am expressing are presented on behalf of Eli Lilly and Company, who paid my travel and expenses today. We are involved in the manufacture of prescription drugs and also involved as the owner of a pharmacy benefit manager known as PCS Health Systems.
Is that adequate?
MS. PEDERSEN: Yes. Thank you.
MR. DANIELS: Madam Chairman, thanks for the chance to be here and to present views on a topic that I think is overdue for inspection. I think that Ms. Woodcock stated well, if anything, understated the dimension of the changes that have come to the health care marketplace and that bring us together here today. It will be my contention in these brief remarks that these changes have brought us really to the dawn of a new era of quality in American health care, an era of great promise for us as patients and collectively as payers for health care as it is delivered here and that this quality will be driven by the most powerful of human impulses, which is the universal human concern for the betterment of our personal health.
As we move into this era of more rational and more effective and more cost effective health care, it would be unnecessary and, moveover, counterproductive for FDA to move beyond its traditional and critical core mission and intrude upon this area just as it is becoming airborne, counterproductive in the sense that it would inevitably slow down the learning process about what works in health care, slow down the dissemination of what is learned and its constant refinement, and then secondarily, only distract and divert resources from the core mission, which we all, I am sure, in this room support and want to see FDA succeed in.
Let me then embellish those remarks very briefly. Managed care, as I experience it every day in the marketplace, is beginning to drive from its primitive origins now very much in the direction of higher and higher quality standards. As consumers or patients, we will demand this, and as those who try to serve that marketplace, we will be driven, ordered by our customers to play a role in enhancing that quality.
The American health care system, I think it's only candid to say--here's a personal view--is the best in the world but not nearly what it ought to be or soon will be. Variability, which throughout enterprise, throughout life, is invariably the indicator of poor quality, is, as we all know, rampant in American health care. We know this in the hospital context, whether we look at mortality rates or utilization rates for high-cost procedures, CABGs, C-section, so forth.
It is only honest to say that variability is just as widespread in use of prescription drugs. It is further, I think, only honest to say that the pharmaceutical industry has historically benefitted from that variability, which often led to over-utilization or imprecise utilization that benefitted individual companies. Speaking just for one such company, we believe that it is unacceptable to acquiesce, let alone collude, in further irrational care. In fact, we have a duty, and our customers certainly believe we have a duty, to help bring greater rationality. That's the process on which we are now embarked.
There is a very precise parallel, I think, familiar to many in this room between what is happening in health care and what has already happened in enterprises of all kinds and also in many agencies in the public sector and that is the application of what we have come to call the principles of total quality to the processes of caring for patients.
Put simply, this involves the identification of variability which almost always is an index of defects; the identification, then, of best practices, what is working best among this wide spread of varied practices; and then an attempt to standardize around what appears to be at a given point in time the best practice, the best practice for performing certain operations right on down to the best practice for administering pharmaceutical-based solutions.
Beyond that standardization, a process of constant measurement, collecting information on what appears to work, what produces the best outcomes, and then the never-ending continuous refinement of the process.
More information freely flowing is the indispensable enabler of this entire loop. That requires a maximum of freedom for data coming into the system about what works and freedom for data moving out of the analysis of the first data set so that best practices can be more widely understood and standardized around.
This is the direction managed care is going, the direction already arrived at by the best of these organizations, and they are placing new and quite appropriate demands on businesses like ours. They are demanding that we justify the value of the products we bring and do so in real world settings. No one here need fear any lack of sophistication or toughness or suspicion of bias on the part of the customers I deal with day in and day out. They apply not merely high but the highest of tests and thresholds to information that we might bring them where it does apply to our own products.
I noted with some irony the recent awards in the last month by the ASHP, which is a society of managed care pharmacists. They gave eight or nine awards for the highest quality health economic studies of the previous year, those deemed most useful in improving the quality and the cost effectiveness of various pharmaceutical solutions. Not one of those oscar winners would have passed the so-called substantial evidence test that some, at least, believe ought to be applied to economic research as it has been historically applied to clinical research.
There is a message in that, I think, that speaks powerfully when those most accountable, responsible, and whose business success depends on delivering higher quality solutions have, frankly, no interest in sterile, as they see it, antiseptic, and artificial research settings.
Managed care is further demanding that we as would-be suppliers of pharmaceuticals recognize that while the right prescription is critical, our duty does not end at the prescription pad. What happens outside the proper match of pharmaceutical and pharmaceutical dose to patient is sometimes much more important to what we're all supposed to be concerned about, which is the eventual result for that patient.
Contrary to our traditional industry line, the biggest risk to most patients is not that they won't get a premium drug or they won't get a premium price drug. The greatest risk is that they will get a redundant drug or an interactive drug or an excess dose of the right drug or that the patient will get the right drug but that he won't fill the prescription, or if he does, he won't take the whole prescription. We are being asked to assist managed care in trying to make sure that these activities external to the actual prescription come out right more often than they have in the past.
This includes compliance. I think we all know about the sad statistics in terms of both the high incidence of non-compliance and the costs clinically and economically that flow from that. I think we all know, and we certainly as a company active in diabetes care know very directly that appropriate outcome for a patient has at least as much to do with lifestyle choices of diet and exercise and so forth as it does with anything that can be found within the confines of a product label.
Prevention of redundant or conflicting therapy is often as important. Our sister company, PCS, has in scan after scan of data bases identified patients, for example, an elderly population in which 5,000 of less than 100,000 patients were receiving concombinant therapy in ten or more therapeutic classes at the same time. A like number was receiving treatment from more than five doctors at the same time. Every one of those doctors might have been prescribing within label and might have been doing an appropriate job clinically, but the outcome for many of those patients had to be less than optimal, given what has been historically inadequate information.
These are the roles that we are being asked to play and that we, as one company, are enthusiastically volunteering to play, to assist these emerging quality-oriented organizations in, first, the collection of data and its analysis and its dissemination, all under their instructions and all on the terms on which they rightly insist.
So my summation would simply be an appeal to the Panel and to those that will advise to walk very carefully in intruding into a process that has so much promise and yet would be so, I think, endangered by arbitrary or ham-handed regulation and restriction of the free flow of information when ample safeguards, I can certainly testify, are present in the marketplace.
The ultimate safeguard, I think, will be a better educated, more inquisitive customer who will have more, not less, choice in the health care system of the future, more informed choice in the sense that he or she will have access to customer satisfaction data and quality data on the basis of which to pick not individual physicians but systems of physicians and paramedical personnel who will have charge of delivering higher quality care not only after the fact but, increasingly, by preventing events before they happen.
I hope that the Panel will reflect carefully on the promise of the system now evolving in this country and will advise FDA to not only maintain but renew its commitment to the critically important core mission of establishing the gold standard for pharmaceutical use worldwide and ensuring that safe and efficacious medications and devices come to American patients as swiftly as possible. Thank you for your time and this opportunity.
MS. PEDERSEN: Thank you, Mr. Daniels.
MR. STEFANO: Correct. I have her testimony, which we will be submitting.
MS. PEDERSEN: Fine.
MR. STEFANO: Good afternoon. My name is Stephen Stefano and I am the Vice President and General Manager in charge of Glaxo Wellcome's Health Management Division.
As I previously mentioned, Dr. Jane Osterhaus, Director of Pharmaco-Economics, will not make her presentation today because of the limited time available. However, we have a written copy of Dr. Osterhaus's testimony for you.
MS. PEDERSEN: Thank you.
MR. STEFANO: Is that sufficient?
MS. PEDERSEN: Can you do the financial disclosures? Are you not making any further statement?
MR. STEFANO: No, I will be making a further statement.
MS. PEDERSEN: Okay. If you could speak to the financial disclosures.
MR. STEFANO: I work for Glaxo Wellcome in their Health Management Division and also came here via Glaxo Wellcome.
MS. PEDERSEN: And presenting on their behalf?
MR. STEFANO: And presenting on their behalf, as well.
MS. PEDERSEN: Thank you.
MR. STEFANO: The Glaxo Wellcome Health Management Division markets our products to managed care organizations such as health maintenance organizations and pharmacy benefit management companies. As the largest pharmaceutical manufacturer in the country, we have a major commitment to marketing in the managed care environment. Therefore, we have a great interest in this hearing and in FDA's follow-up to this hearing.
First, I would like to identify what we are and what we are not. Glaxo Wellcome's business is the research, development, and manufacture of prescription drug products. While we market our products in both the managed care and non-managed care environments, we do not own an HMO, PBM, mail order, or other companies in the distribution chain of prescription drugs.
In this respect, Glaxo Wellcome represents the purest prototype to the FDA. In general, everything we communicate about our products is regulated by the FDA and every time we communicate about our products, we identify ourselves as Glaxo Wellcome.
As an initial matter, I would like to state that Glaxo Wellcome supports the FDA reform legislation currently being circulated by our trade association, the Pharmaceutical Research and Manufacturers Association. Among other things, that legislation calls for less restrictive FDA regulation of the distribution of pharmaco-economic data. And let me emphasize that we support this FDA reform legislation as it would apply equally to both manufacturers that do and those that do not own a pharmacy management subsidiary.
My point this afternoon does not address the substance of any particular FDA regulation and we are not asking for more regulation. Instead, my point is that whatever the level of FDA regulation, it should be applied even-handedly both to manufacturers and manufacturer-owned pharmacy management businesses, such as PBMs and mail order subsidiaries.
Today, I would like to address whether a drug manufacturer should be responsible for drug claims communicated by the subsidiary business which it controls. We strongly believe that all drug manufacturers should be treated equally with regard to the claims they communicate about their products. This is true whether the claims are communicated directly by a drug manufacturer or indirectly through its subsidiary business, such as a PBM company or mail order pharmacy.
This equal treatment does not require a change in the law, an extension of FDA's enforcement policies, or issuance of new guidelines. It simply involves the application of agency principles which FDA and other governmental departments have traditionally invoked.
What is currently happening in the managed care marketplace? What we have seen is that FDA concentrates on regulating promotion issued by drug manufacturers about their products. FDA does not address the promotion issued about those same products by subsidiary businesses owned by drug manufacturers. As a result, FDA's uneven treatment has contributed to a two-tiered system of marketing in the health care environment.
Can companies with subsidiary businesses such as PBMs use this situation to their advantage? Presently, yes. I am personally aware of situations where drug manufacturers' subsidiary business will promote the manufacturer's products in concert with the manufacturer. The manufacturer's claims will stay within FDA rules. The subsidiary business will parrot the manufacturer's claims but will go beyond those claims in ways that the manufacturer cannot.
We have seen manufacturer-owned PBMs and mail order pharmacies make cost effectiveness claims and superior efficacy and safety claims about the manufacturer's products which the manufacturers themselves would not be allowed to make under current application of FDA regulation. To the best of our knowledge, those claims are not filed with FDA like drug manufacturers' claims must be, and therefore FDA is currently unable to monitor them for accuracy and fair balance.
This puts manufacturers who are committed to abide by FDA's rules, like Glaxo Wellcome, at a competitive disadvantage. For example, we are aware of manufacturer-owned PBM companies that issue letters to doctors stating that our product is less cost effective or less efficacious than the parent manufacturer's product. These claims are unsupportable and the manufacturer would not be able to make them directly. Once again, this puts a company like Glaxo Wellcome in a disadvantage position.
We do not believe that the creation of an independent formulary committee in a manufacturer-owned PBM or mail order pharmacy solves the problem. An independent formulary committee only creates a list of drugs. The rest of the PBM or mail order pharmacy promotes those products. It is a reasonable assumption that the PBM or mail order pharmacy will promote those drugs in ways that favor the parent company. Therefore, the parent company should be accountable to the FDA for the promotion. In fact, some of the examples I have highlighted have come from subsidiary businesses which maintain an independent formulary committee.
Are we saying that a drug manufacturer should always be held accountable for what its subsidiary businesses say about the manufacturer's products? It depends. There should be a presumption that a subsidiary owned by a drug manufacturer is controlled by the manufacturer. This control should make the manufacturer accountable for the subsidiary's communications about the manufacturer's products. This presumption may be overcome only by clear proof that the subsidiary is run independently from the manufacturer's interest in promoting its product.
Independence is a hard thing to demonstrate when the subsidiary is owned by the manufacturer. At the very least, there should be an operating or policy document that forbids the manufacturer from directing or controlling the communications issued by the subsidiary concerning the manufacturer's product or competing products. Any indication that the subsidiary's communications are part of or associated with the promotional campaign conducted by the parent or that subsidiary is issuing promotional claims which the parent has been directed by FDA to cease making should support the presumption that independence is lacking.
Finally, we realize that a manufacturer inevitably will impose controls over the bottom-line profitability of its subsidiary businesses. That standing alone does not negate the independence I am describing. However, where the financial performance objectives the manufacturer imposes on its subsidiary business relate to sales of the manufacturer's products rather than the performance of the subsidiary's core business, independence in terms of product promotion is lacking.
All that I have covered so far relates to the need for FDA to apply equal treatment to pharmacy management businesses which are owned by drug manufacturers. With regard to management businesses which are not owned by drug manufacturers, we see no need for FDA to change its current activities. However, there is one exception. Where a pharmacy management business distributes product-specific literature supplied by a drug manufacturer or contractually agrees to create and distribute comparative claims regarding the manufacturer's products, then the pharmacy management business becomes the agent of the manufacturer for purposes of promotion.
As such, this product-specific literature or those claims should fall within current FDA regulation of promotion. This is consistent with FDA's longstanding policy that it regulates promotion distributed by or on behalf of drug manufacturers.
In closing, I would like to reemphasize that we support PHARMA's FDA reform legislation and its application to all manufacturers. We are not asking for more regulation. Rather, we believe that the FDA regulations should be applied even-handedly to manufacturers and manufacturer-owned pharmacy management subsidiaries such as PBMs and mail order businesses.
We would like to thank FDA for this opportunity to state our views and to express our willingness to engage in further dialogue on these important issues. Thank you very much.
MS. PEDERSEN: Thank you, Mr. Stefano.
MR. LUCE: We considered it and we will do seven-and-a-half and seven-and-a-half.
MS. PEDERSEN: For convenience, perhaps you could do the financial disclosure, both of you, at the beginning.
MR. LUCE: Yes, I am going to do that.
MS. PEDERSEN: Thank you.
MR. LUCE: Thank you, Madam Chairperson. I am Bryan Luce, the Chief Executive Officer of MEDTAP International, a firm specializing in outcomes research which includes the field now known as pharmaco-economics.
With me today is Dr. Alan Hillman, Director of the Center for Health Policy of the Leonard Davis Institute of the University of Pennsylvania. Dr. Hillman was the organizer and I served as a member of a task force on principles of economic analysis of health care technologies.
Among many other responsibilities and activities, both of us consult with and perform outcome studies with pharmaceutical firms, both under personal and institutional contracts and grants. I am representing my firm and myself. Dr. Hillman is representing himself. Our organizations are paying for our time and expenses today.
We would like to begin, Madam Chairman, by thanking the FDA for this opportunity to present our views and commending it for holding these hearings. In our testimony today, we will make the three major following points.
MR. LUCE: The market for pharmaceuticals is both demanding and supplying the evidence of value for money. FDA should do nothing to hinder this information transfer, but it could play a positive role.
Second, the standard of evidence to substantiate a cost effectiveness claim is different than it is for safety and efficacy. FDA's regulation of pharmaco-economic claims should reflect this difference.
Three, legitimate concerns have been raised about the validity and potential for bias of pharmaco-economics research. The private sector is responding to these concerns. However, FDA may be able to play a positive role here, as well.
MR. LUCE: We would like to begin our testimony with an interesting example. In the late 1970s, Congress requested the Office of Technology Assessment to study whether Medicare should cover the pneumococcal vaccine which had just been approved by FDA. The data was very poor for a coverage decision. Using an epidemiologic model to estimate cost effectiveness, OTA gathered data from multiple sources, including clinical trials, claims data bases, and expert opinion.
MR. LUCE: The results were very positive, $1,000 per life saved. Uncertainty abounded, as can be seen in this slide. Estimates of efficacy, duration of immunity, and costs varied tremendously. Yet even when extreme values were used in the model, the conclusions held up. The vaccine seemed cost effective. Based on this study, the pneumococcal vaccine became the first Medicare preventive benefit.
Was this a good decision? At the time, it was hard to say. Was it an informed decision? Certainly more so than if a study had not been done. Did the analysis meet FDA's criteria of two well-controlled trials? Not even close. Could two well-controlled trials have been done? No, but if they could have been, they would have been very costly and taken years to conduct, thus not meeting Medicare's needs. But the study was useful, timely, in itself cost effective.
Medicare's coverage decision is quite different from FDA's decision to permit marketing of the vaccine. In turn, the methods used and the standards of evidence were different. More to the point, the consequences of making a mistake are different. Approving an unsafe vaccine would be disastrous. A safe but inefficacious vaccine would be a serious breach of faith. However, a safe, efficacious, but cost ineffective vaccine is simply a bad buy.
We now turn to our first major point. The key to understanding--the first major point is, the market is demanding and supplying evidence of value. The key to understanding the uses, usefulness, and required rigor of pharmaco-economic research and resulting claims is understanding the needs of the consumer, in particular, in this case especially, managed care. Health care decision makers need information to make good decisions. They demand evidence of value for money.
Industry is responding. For the first time, empirical cost effectiveness data is being developed within clinical trials and unique prospective cost effectiveness trials are being funded. In addition, methodological work is being funded in quality of life, utility assessment, and modeling techniques. Neither the device nor diagnostic industries, the medical specialty societies, nor government is coming close to matching this level of investment in outcomes research and cost effectiveness analysis.
Yet cost effectiveness research is desperately needed for all categories of medical technologies, certainly not just pharmaceuticals. Whatever policies FDA may establish, care should be taken not to dampen this market-generated investment and research.
We now turn to our second major point, which is that the standard of evidence is different for cost effectiveness. It would seem that society's tolerance of uncertainty depends upon the circumstance. For decisions about drugs, for instance, societies have very low tolerance of uncertainty for safety issues, especially catastrophic safety concerns.
MR. LUCE: As we move from safety to efficacy to effectiveness to cost effectiveness, the purpose for the information changes, as does the nature of the decision. In turn, society's threshold for uncertainty relaxes. Pricing and coverage decisions are just not as important as registering a dangerous or inefficacious drug.
For example, a decision maker may be comfortable being right 75 percent of the time for a reimbursement policy yet want near 100 percent assurance before subjecting patients to serious risk.
Thus, I am left with severe cognitive dissidence when on one day I participate in the Public Health Service sponsored panel on Cost Effectiveness in Health and Medicine, which will be releasing its report soon, whose mission is to develop methodological standards and guidelines for cost effectiveness analysis based primarily on modeling, then review the draft FDA pharmaco-economic guidelines where the underlying premise seems to be that validity of scientific findings can only come from double-blind randomized control trials as analyzed by classical statistical analysis and bounded by the 95 percent confidence interval.
So to what extent does managed care need protection from false and misleading claims? Today, these decision makers are increasingly large, highly organized, and sophisticated business operations, as exemplified by national managed care organizations and pharmaceutical benefit managers. Evidence indicates that they are skeptical, even dismissal, of claims of cost effectiveness. We do not think they are in much need of protection.
However, as we will note in our closing remarks, FDA could possibly play a positive role in helping decision makers assess the validity of claims without hindering the transfer of information inherent in the claims themselves.
DR. HILLMAN : Thank you, and if you could circulate the reprints, that would be helpful.
The third of the issues that we would like to stress concerns the validity of and the potential for systematic bias in outcomes research in pharmaco-economics. If this research were known and proven to be 100 percent valid, reliable, and unbiased, then the question with respect to promotions would only be, does this promotion accurately reflect the underlying research?
Unfortunately, this is not the case. Pharmaco-economics research is subject to systematic bias, much more so than traditional clinical research because it's unstandardized, there are few qualified peer reviewers, indeed, few people who understand it. There is often significant interpretation of these studies which is subjective in nature, and there are usually many assumptions which are made, as in the cases, as Dr. Luce said, when we base these things on models.
We also need to look at the stakeholder's reasons for possibly leaving the door open to bias. The sponsors are usually the pharmaceutical industry. Obviously, they stand to gain the most from positive results, and sometimes there is subtle pressure that is exerted, whether it is meant or not.
Researchers are also in a crunch for funding and they would like to find positive results because that often enamors them to the sponsors, who then call them back, indeed for more funding and more research and consulting time.
Editors are known to be biased toward publishing positive research, and by positive research we mean research that finds a difference, and it is a well-known bias among editors.
Are these concerns real, the concerns about bias? Unfortunately, yes. These things happen and they happen frequently and there are a lot of shoddy pharmaco-economic studies in the marketplace now. Fortunately, many groups are working on guidelines for acceptable methodology, including the governments of Australia and Canada, the American Public Health Service panel that Dr. Luce mentioned, and several other people and groups.
Also, the private sector has addressed the validity and bias issue directly, publishing the recommendations that we just distributed in the Annals of Internal Medicine. This was an effort of a group that I organized and that Dr. Luce was a primary player in called the Task Force on Principles for Economic Analysis. We had 30 members who met by teleconference and met in person over the course of two years, including members from professional societies and government, the AHCPR, the Institute of Medicine, the FDA, HCFA, the CDC, et cetera, as well as a variety of researchers from different venues and the pharmaceutical industry and a number of other people.
The goal was different than deciding on what was correct methodology. The goal was to decide how to best structure the relationship between industry and researchers.
DR. HILLMAN: The task force recommendations included, among other things, the requirement of guarantees for researcher independence and a requirement of guarantee of freedom to publish findings, regardless of how they made the sponsor's product look. In this way, the task force hopes that it has created a kind of Good Housekeeping seal of approval and we think that authors should state whether or not they have followed these guidelines.
The task force was funded by no-strings-attached grants from 13 pharmaceutical companies to the University of Pennsylvania. These guidelines plus the more methodologically-oriented guidelines compliment each other well.
So, in summary, what we have said is that there are concerns about potential bias in the research that provides the unique standards of evidence underlying a pharmaco-economic claim or promotion. However, the marketplace is attempting to deal with this problem. The FDA's responsibility is to help the marketplace get there.
So what do we specifically suggest in the context of these three important concerns? First, we believe that the draft FDA guidelines would inappropriately hinder the marketplace. Therefore, we suggest that FDA change strategies by establishing a panel of disinterested but knowledgeable parties, perhaps from within the FDA, from private sector research, academic research, and other places, and this panel should establish a series of hierarchial disclaimer statements which would be affixed to promotional material prominently. These statements would reflect the panel's judgment about the validity and reliability of the research that underlies the specific promotional material that it is evaluating. This process would be mandatory.
DR. HILLMAN: For example, in the best situation, the disclaimer would read something like, "This promotion accurately reflects and is based on research that meets generally accepted principles as set forth by the Task Force on Principles for Economic Analysis" and any other criteria that the FDA chooses to use. In a less desirable situation, the disclaimer might read, "This promotion is based on research that does not adhere to", or perhaps violates, "generally accepted principles", et cetera, et cetera.
In this way, the FDA would act like an editorial body by submitting a promotion to an expert "peer review", if you will, a kind of peer review. We believe the pharmaceutical companies, acting in accordance with the rules of the invisible hand in the marketplace, would rapidly and voluntarily withdraw promotional material that did not achieve an acceptable rating.
This effort would be inexpensive and would not be time consuming. It would not require FDA to make acceptable/not acceptable decisions or to order some promotions withdrawn. The FDA would be judging and reporting two things, the merit of the research underlying the promotional material and whether the promotion accurately reflects the research. The FDA would have an effective mechanism by which to regulate promotions and hence protect the public, a mechanism that would facilitate the American marketplace's search for good information by which to make decisions.
On behalf of Dr. Luce and myself, thank you for your time and consideration.
MS. PEDERSEN: Thank you, Dr. Hillman and Dr. Luce.
MS. MOENCH: Four-and-a-half and 11-and-a-half.
MS. PEDERSEN: Hold on one second while I get some technical assistance.
MS. PEDERSEN: The way it is set, it will be green for the longer period of time and yellow for four-and-a-half minutes, but if you would both do the financial disclosures first so that we can then get straight in.
MS. MOENCH : I am Elizabeth Moench, President of MediciGroup. We are based in Princeton, New Jersey, with offices in King of Prussia. The views are those of the market and also observations of my own. Our organization, my company is paying for my expenses here and we have our feet in a number of different camps, both managed care, pharmaceutical, clinical research organizations, and also health care management consulting companies.
I would like to say we applaud FDA's initiatives to understand the changing health care environment, and furthermore, we agree that FDA must also understand the needs of the end users. It is to this point that if we look at a three-legged stool with FDA, the pharmaceutical industry, and managed care, there is, in fact, one leg missing, and that is managed care. In fact, if we are going to be contemplating regulations that are charged with public policy, we must first consider the opinions of those for whom regulations are intended to serve and to protect.
I think from managed care's perspective in terms of what we hear, it is rather akin to, "Hi, we're from the FDA and we're here to help you."
What I'd like to do is I'd actually like to tell you that today we're going to be presenting some findings of some research which actually points out to the fact that as FDA needs to understand the managed care market, so, too, must managed care understand the motives of FDA. They also in this tenuous new and evolving field need to have closer relations and understanding of the intentions of the pharmaceutical industry, too.
What does all this mean? It means, in fact, that there are some natural checks and balances that we see in the marketplace, and what we are going to be doing is presenting, and I think we are the only two who will be presenting some actual research of opinions, an opinion survey of managed care executives.
What I'd like to just quickly go over is that, in fact, in terms of key findings, what was very interesting was that there was almost no awareness among those that we surveyed to today's hearings. There was also lack of awareness about recently published guidelines for managed care. That was also quite striking.
What we found was that managed care companies want guidelines and criteria; they don't want regulations. They trust very much their own forums, they trust their own data, but they want to have some tools with which to validate what they are doing on a local level and compare that to national norms.
What managed care companies are doing in terms of bias is, in fact, they have put together a series of home-grown rules, and we are going to be going through some of those with you. So there are ways in which managed care companies are detecting bias, such as are the studies applicable to my clinical setting? The publications of results, are they in peer review journals? Is there a disclosure of conflicts of interest? The understanding of reasons why a study was terminated, et cetera.
What we found, also, was that MCOs are cautious and guided about the pharmaceutical industry, academia, FDA, and others, and in some comments that we had about academia is, well, we are seeing some of the same faces and same names on different studies saying different things, and also that the field of pharmaco-economics is new and, in fact, in some cases, for large companies, they have been involved in pharmaco-economics for about 4.8 years compared to 2.5 for smaller companies.
In terms of key observations, I think we can say that MCOs have clearly consolidated the buying power in the changing environment that was alluded to right up front and that pharmaceutical companies really cannot afford to lose credibility in a marketplace where it is converging and the power is in the hands of so few players. But FDA and the industry must understand the needs and concerns and opinions of managed care, and also, let's realize that managed care decision makers bear the risk financially and clinically of the decisions that they make.
So with that, I would like to just turn it over to my colleague, Jonathan Lax.
MR. LAX : Thank you, Liz. My name is Jonathan Lax. I am President of the Marketing Audit in Philadelphia. I paid for my own railroad ticket. We work for a wide variety of companies in this field, pharmaceutical companies as well as managed care companies. The opinions, though, are those of our own and those of the marketplace, since we did this study. I would also like to mention that I own stock in various mutual funds which may or may not hold stock in pharmaceutical companies. I have found that whenever I buy a stock in a pharmaceutical company, it goes down.
MR. LAX: The first slide, please.
MR. LAX: Again, my company does competitive intelligence and high-level market research. I am also blessed with a part-time job. I teach medical history taking to second- and third-year medical students at Hahneman University and Medical College of Pennsylvania.
The next slide, please?
MR. LAX: This is a snapshot of where 43 different MCO organizations and some consultants believe their field is today. Our objectives were to survey a group of MCOs as to their current sophistication in performing outcome studies, determine types of strategic relationships, determine user titles and user groups internal to the MCOs, determine a mix of in-house versus partnered studies, explore the HMO and pharmaco-economist opinions concerning FDA regulations, guidelines, development of the discipline at large, and the need for checks and balances.
The next slide?
MR. LAX: A very, very interesting study for us. Normally when we call people, we have to keep calling them back to get an interview. Very, very different in this case because people wanted to call us back. Not only that, but they wanted to spend over an hour on average in talking about this subject with us on both an interview and open-ended basis.
The methodology was long in-depth telephone calls with directors of outcome research, medical directors, director of pharmacy, managers of health care economics, and research managers. Respondents were promised personal anonymity. MCOs were chosen to be diverse among size, geography, sophistication, and different styles of organization.
We identified specific names through secondary research known to be leaders. We tried to survey those and we filled in the rest of the group with semi-random choices.
The next slide, please?
MR. LAX: We interviewed--58 percent of the interviews were large MCOs, and here we're using the GHAA-level breakdowns. Twelve percent were medium ones. Nine percent were small. Of the executives within these organizations, 18 percent were presidents, senior vice presidents, or vice presidents, so we got very, very high-level views within these organizations. Twenty-one percent were consultants, and here there were two industry consultants and the rest were medical school and provider and academician-type consultants.
Interviews were started on September 12 and were completed on October 13. Interviews gathered both metrics and verbatim comments. The next slide shows a little bit of a map of the geographical diversity. The next slide, please.
MR. LAX: There are some faults in this kind of methodology which I want to point to you up front. One of our clients asked for a standard of error, which we calculated, but we don't normally calculate a standard of error in a small sort of anecdotal study like this.
Respondents were not completely random. We sought out some specific outcomes developers to get their specific opinions for our health care clients.
Most questions were open ended, requiring analysis of verbatims, which is tricky. I talked with Lou Morris beforehand and we were going to argue a little bit later about what is real and what is unreal data. Only six questions were forced choices. Averages were numeric averages of numbers given. They are unweighted. Ranges were also averaged.
MR. LAX: In terms of conclusions, there was a wide range of sophistication in the ability of MCOs to perform or use outcomes studies. The average MCO has only 3.9 years of experience in performing studies and the range for the whole group of 43 was four months to 20 years.
User groups and MCOs haven't been totally defined and the real end users are not always known. Regulations are not wanted, nor are they necessary. Guidelines or criteria would be helpful, but other groups are seen as also playing a role--GHAA, ECRA, NCQA, and possibly others.
The next slide?
MR. LAX: Managed care companies can and do detect bias in pharmaceutical company studies. The natural distrust of PHARMA and some academics provides a useful check and balance and sets a higher standard for the studies in which these MCOs participate. Pharmaco-economists know what they want from PHARMA companies in terms of control of data, financial support, design of studies, implementation, and dissemination and use.
The next slide?
MR. LAX: There is a lack of awareness among pharmaco-economists about this particular meeting and the FDA's overall effort. This disconnect between the regulators--we would rather call you guideline formulators than regulators--and the users may create both practical and political problems in the future, and here is where we get into our opinion at this moment.
We believe that this field should be nurtured carefully rather than shackled with new regulations. Guidelines can and should be developed, but first by the users, and these can evolve with the maturity of this new science.
The next slide, please?
MR. LAX: This is our best guess as to the sophistication of the various folks we interviewed, and remember, we picked out some who were very, very sophisticated because we specifically wanted their opinions. Fifty percent we counted as very sophisticated, 28 percent moderately sophisticated, and 22 percent as beginners, and this is plotted versus size of number of covered lives.
The next slide, please?
MR. LAX: There is some feeling that cooperating with pharmaceutical companies is really a good idea, and this quotation notes that. There is a quotation that notes that. They do have home-grown rules about what they will do in terms of cooperating with pharmaceutical companies and the home-grown rules are on the next slide.
MR. LAX: The top concerns of working with pharmaceutical companies to disseminate studies, the top mentions were bias, 29 percent of mentions; freedom to control the study, 13 percent; no concerns whatsoever, eight percent; validity of the study, eight percent; confidentiality, seven percent; appearance of collusion on our part, five percent; cost, five percent; drug companies' motives and integrity, five percent; accurate data, four percent; and potential value in using the study, three percent.
The next slide, please?
MR. LAX: There are a series of home-grown partnership rules in terms of working with pharmaceutical companies, and these are those that we found in the study.
The next slide, please?
MR. LAX: Why don't we go to comments?
MR. LAX: A series of comments which were the most important part of the study, some were very favorable in terms of the FDA's involvement. "The FDA could provide some basic standards with what is necessary." "I think the FDA could play a role if we were to partner with a pharmaceutical firm and the FDA is probably in the best position from a regulatory perspective to prepare and promulgate guidelines."
The next slide, please?
MR. LAX: Others had sort of medium-warm views. "I don't think the FDA should be involved if they are going to overwhelm us with bureaucracy." "I don't think the FDA needs to control the dissemination. Controlling content and accuracy may be a good role, but they shouldn't attempt to control dissemination." "I'm not one to encourage a lot of Federal regulation. I'd rather not see regulation, but I would rather see it required for drug approval, some kind of study of cost benefits."
The next slide?
MR. LAX: Some said there may be some role for the FDA to play. There are some needs for developing benchmarking criteria to judge the study. If they were developing an initial rating system, we would numerically decide how well the study met the overall criteria.
One person said, "I'm not really sure, but my gut feeling is that the industry should be regulating itself. FDA jurisdiction might impede some research. I'd rather see the industry being self-regulating."
Another person said, "I believe that the industry regulates itself through peer review and publication."
The next slide?
MR. LAX: Some highly negative comments, "The FDA needs to stay focused on what is safe and what is effective." "This will change the role of the FDA dramatically." "The needs of the marketplace change more rapidly than FDA. If it takes two years to come out with findings, the product may no longer have any value." And finally, "It's baloney. The government has no business dictating research methods to anybody. They are not going to get it. I'd get out of the business and fight them full time if they get jurisdiction. They have no business in this area."
The next slide, please?
MR. LAX: Is there a role for the FDA in regulating dissemination or do you believe that you have control? Forty-two percent said that there is a role for the FDA and 58 percent, roughly 58 percent, said that we have the situation under control.
The next slide?
MR. LAX: What are your thoughts on the appropriateness of the FDA seeking jurisdiction? Fifty-five percent said it was inappropriate, 18 percent said it was appropriate, and the mixed opinion was about 26 percent. We found out, by the way, pharmaco-economists frequently have mixed opinions.
MR. LAX: The next slide, please?
MR. LAX: What organization other than the FDA should prepare and promulgate guidelines for the preparation and use of pharmaco-economic data? A small number of responses, but of these responses, the NCQA came up with a 53 percent response rate.
The next slide, please?
MR. LAX: How many pharmaco-economists planned to attend this meeting? The answer was zero. None of the respondents planned to attend this meeting. Roughly half of the respondents professed surprise that this meeting is even being held.
The next slide, please?
MR. LAX: We then asked them at the end of the survey, what would you say to the FDA about outcomes measurement if you were to appear at the October 19 meeting? Thirty-three percent said, set guidelines, do not regulate. Twenty-two percent said, stay out of this are. Twenty-two percent said, use caution as this area is still developing. And 19 percent said, encourage this area as a facilitator, not a regulator.
MR. LAX: In terms of conclusions, we found out that the FDA needs to prove itself to managed care. The FDA will have to reach out and collaborate with both users and designers, including ECRI and the PHARMA companies to work out guidelines. And finally, this is a new field. In most companies, it's been in use less than four years. Users haven't been identified and the relationships with pharmaceutical companies haven't been finalized. There is a healthy skepticism in this field and that provides built-in checks and balances. Thank you.
MS. PEDERSEN: Thank you very much.
Our final panelist, who was in panel one, was not able to be with us today, so that completes the presentations for Panel 1.
I would ask the FDA Panel if they have any questions. Dr. Temple?
DR. TEMPLE: Let me just introduce my question by saying what I think our stated proposed thoughts are, which are that in a pharmaco-economic analysis, there are two components. One component is the difference between two therapies or the effect of a particular therapy on some clinical measurement. That could be death, it could be hospitalization, it conceivably could be time out from work or something like that.
Then, having asserted that, one tries to, in various ways, put a value on that, using modeling techniques or getting impressions of what people value and do a whole lot of things. It is the first part of that where we have said that we have a role. The second part, we have basically said, disclose your methods and that's enough for us. I just want to be sure that that distinction is made.
My questions go really to how much flexibility there should be in the first part of that. There is a suggestion that methods other than controlled trials are good and should be accepted, and I want to probe that a little bit.
I guess I also want to take note of a recent Science article from July in which large numbers of card-carrying epidemiology folks said that differences of less than three-fold coming from non-randomized trials were, on the whole, not credible. There were real experts in there. I was quoted saying that, too, but some people who really know also said that.
So imagine, just for the moment, a scenario in which someone goes to some data bases and discovers that--this is actually for the first speaker--someone goes to a data base and discovers a lot of suicides in people taking Prozac, finds that in that population, this rate is far worse than the rate in people on tricyclics or on some other SSRIs, and having established that there is a particular difference in outcome that is perceived as important, proceeds to make various value judgments based on that.
This is, in fact, not different from a situation that actually occurred. Lilly's rebuttal to that was to look at all their control trials, find that the suicide rates were the same for Prozac as for placebo and other therapies, and therefore they weren't particularly worried about that.
But in a world where claims can be made based on all kinds of data, not necessarily implausible data but not randomized data, presumably a competitor of Lilly could blanket the world with these kinds of claims and go to HMOs and say, you're just crazy not to use the other thing. The difference in value is this much, et cetera.
I would be interested in a reaction not just from Lilly representatives, who obviously have a stake in my example, but from other people about that possibility and what one should say about it.
MS. PEDERSEN: Mr. Daniels, do you want to start off?
MR. DANIELS: It's an interesting hypothetical.
MR. DANIELS: I guess my first reaction, Doctor, is I think it's an important subject but it misses the point of this discussion because it falls, from my hearing of it, into the province of purely clinical outcome, a very important matter for FDA to retain its historic role in.
This is not, as I understood it, the sort of study, let alone utilization of one, that I think we were invited here to address. In other words, this was not a matter in which the safety and efficacy of a product had been established through properly scientific means and then the debate ensued based on other kinds of data about efficacy per dollar spent, is usually the way I think of pharmaco-economic studies.
It is as the data or use of it that moves into that area that I think very legitimate questions have to be asked about what value FDA can add. I think you heard me but also others here take the point of view that there are many bias filters already in place, and if anything, they may be higher, at least in some quarters of managed care, than they are, let's say, in academia.
It's perfectly plausible to me that we could bring to the attention or a company could bring to the attention of managed care a study which met all of the guidelines that experts like we've heard from might suggest and still have it rejected, just based on its province.
If one is not confident about the bias builders, I think it still leaves the question, who is being protected. As long as the patients' interest in safety and efficacy is strictly protected, which we strongly support, I think there is a legitimate question why it should be the province of government in any of its agencies to protect managed care organizations from economic miscalculations. There is a very effective self-correcting mechanism there, namely, those who make enough such bad judgments will be penalized in the marketplace.
And finally, if there is at the end of the day a need for some baseline protection against purely fraudulent claims, there are other alternatives than FDA, which has such a critical and burdensome mission to carry out already.
DR. WOODCOCK: Could I extrapolate something, because I think a lot of issues were raised by this panel, many of which were the ones that I raised in the beginning, that we really want to investigate in this hearing issue about PBMs and so forth, but what we are discussing right now is pharmaco-economic studies.
Several points were raised by you all, and I think we ought to just lay the groundwork of what we regulate and what we're talking about. I think that would be helpful, because there may have been some confusion here. FDA is not proposing anywhere any regulation of pharmaco-economic studies conducted by managed care organizations for their own purposes. FDA regulates the dissemination of information about medical products by the manufacturers of those products.
One point which was raised earlier in this panel, who are those manufacturers, if they own other subsidiary companies, but another issue is that if a pharmaco-economic study is conducted or caused to be conducted by a manufacturer, then we regard that dissemination as sort of part of promotion or advertising. So just for the benefit of the audience, that point.
Now, the issue has been raised by Dr. Temple, we regulate safety and efficacy claims made by manufacturers about their products, and everyone, it sounds like, on the panel is at peace with that. But one of the issues with pharmaco-economic studies as they are often conducted is they involve extrapolations beyond the actual data that is discovered in randomized clinical trials or otherwise about it, and that was well illustrated in the pneumococcal vaccine example. Some of those extrapolations often, although not always, have to do with the safety and efficacy in the future, shall we say, or as extrapolated to broader populations or something like that.
So I think the question that Bob is asking is the slippery slope question, which is if those sorts of extrapolations and conclusions would be permitted in pharmaco-economic studies, what would limit a company from arriving at the same conclusion about a competitor's product in a limited area, such as suicide, and why is that different. Is that--
DR. TEMPLE: Yes. It can also be part of the pharmaco-economic claim itself.
DR. WOODCOCK: Well, that is a different--
DR. TEMPLE: In the most direct way. You have to make some assumptions, for example, for an anti-hypertensive about what--
DR. WOODCOCK: Well, that's the kind of extrapolation you're talking about.
DR. TEMPLE: Exactly, right.
DR. WOODCOCK: So you're saying, if that were part of a pharmaco-economic study which would be this anti-depressant has more suicides and therefore there are more consequences from overdose attempts or whatever to the managed care organization, that would be an extrapolation that you feel a competitor might take umbrage at because it was arrived at from a data base?
DR. TEMPLE: Right, and I don't want to be disingenuous. My larger question is, what kind of data go into these things? What kind of data should we rely on, because there were a number of people who said randomized trials aren't important anymore, and I just want to be sure we know what that's referring to.
There are the parts of a pharmaco-economic analysis that relate to what the benefits of the drug are. That's one part. Then there's the part that extrapolates those benefits, costs them out, makes a lot of judgments, and that's a different part which I would say we on the whole don't want to regulate because it's not really a clinical claim.
But what goes into those things may be a clinical claim and I was using an example that I thought would provoke interest from Lilly, because it was a drug that they were interested in, to illustrate the dangers of certain kinds of data that don't have clear standards.
Now, I understand Dr. Hillman and Dr. Luce think there are standards growing, but I would submit that you don't even really know how many studies were started, much less how many were interrupted for some reason. It's very hard to get a handle on that unless some new system for doing that comes along.
DR. WOODCOCK: But the question that you have on the table is, you are questioning how do people feel about the inclusion of those types of conclusions, which could be medical claims type of conclusions, pharmaco-economic studies--
DR. TEMPLE: Yes, yes.
DR. WOODCOCK: --when they ordinarily wouldn't be accepted as the basis for a label claim for the drug, is that right?
DR. TEMPLE: That's absolutely right, yes.
MR. LUCE: Could I address this? I think you have a problem. I agree with you. But you can't regulate half of a ratio. In point of fact, cost effectiveness studies do have a numerator and a denominator, and, in fact, the denominator does extrapolate from your efficacy and safety information which you tightly regulate and I think all of us would agree is a good thing to do and we're not challenging that. So it does call into question what your position should be relative to claims that do extrapolate from that within the cost effectiveness framework.
I think we have to be creative about this, to tell the truth. I don't think the answer is to say, well, okay. We'll only regulate the denominator. That doesn't make any sense at all. If nothing else, the numerator is derivative from that denominator. The reason you have differences in cost for the most part is the denominator changes. People are healthier or sicker or you are projecting that they will be healthier or sicker and that then affects what the numerator says. So it's the same issue anyway.
I toyed around with some ideas. One would be strictly a disclaimer that, in fact, any claim made within this cost effectiveness promotional claim in no way negates the labeling of the drug or implies that this product is any more efficacious than the label does or something of that nature. But I don't think there's any way around letting the free market work in developing the kind of value-for-money information it is demanding and willing to pay for without letting go some concerns about a back door inference that, in fact, products are more efficacious or more safe than the actual labeling would permit fully. I think somehow we have to deal with that through a disclaimer, is my best suggestion.
MS. PEDERSEN: Mr. Lax, did you want to comment?
MR. LAX: Yes. I would like to welcome you all to the wonderful world of the social sciences. We use things called case controlled studies. We use historical controls. And anybody who has ever studied economics starts forecasting on the second day at school. Again, the denominator side, you are going to have to get used to mushy data. I think the important thing is the assumptions that went into those data and those extrapolations and those forecasts are laid out for the reader to decide.
MS. PEDERSEN: Are there other members of the panel that wish to comment?
DR. HILLMAN: Yes.
MS. PEDERSEN: Dr. Hillman?
DR. HILLMAN: I also think that there is a major problem, and I think it is the underlying philosophy by which the FDA uses randomized controlled clinical trials as kind of a gold standard. I think what we need to do is open that up for question and perhaps be less parochial about what we use as evidence.
Now, you know that RCTs bring us--they're an extrapolation in themselves. They bring us into an artificial experimental world where we are hovering over patients. We make sure they take their pills. We choose them carefully. We bring them in every week rather than every month and things like that, and that's not the real world of medicine. We all know that doctors don't use drugs the way the labels say they do.
So RCTs are one method of accumulating data, but they are by no means the perfect way or even the correct way. I have yet to be convinced that they are the correct way to collect data by the FDA. I think if we take a step back and say that, okay, RCTs work well on the safety and efficacy challenge so let's leave them alone there, but now we have a new challenge and that new challenge has to be met by new standards of data and new standards of analysis. If we continue to hide behind RCT as the gold standard, we are going to miss the opportunity to create the information the marketplace wants and we are going to be sticking our head in the sand, because, quite frankly, the RCT is flawed.
MS. PEDERSEN: Ms. Moench?
MS. MOENCH: Yes, I would like to pick up on both those points because one of the things that our research did find was that there was an expression by several executives, managed care executives, that there was an overuse of randomized clinically controlled studies. And, in fact, if you look at the spectrum of analyses that they are currently using in managed care, it includes from case control to randomized to perspective to cohort studies and they are asking for that flexibility.
MS. PEDERSEN: Dr. Woodcock?
DR. WOODCOCK: Yes. I would like to ask you and Mr. Lax, I still got the flavor from your presentation--how much of your presentation related to the perception of managed care that FDA wanted to regulate the studies they themselves conducted or evaluated?
MR. LAX: In terms of the actual studies, probably about a tenth of the study. It was what we chose to tabulate in time for this hearing. But we gathered a lot of other data that is going to be important to our health care clients.
DR. WOODCOCK: My question was, it sounds like your respondents were under the impression that FDA wished to regulate the studies they themselves conducted.
MS. MOENCH: I think the question that was in their mind in terms of the slippery slope is what happens when they look at the partnership role with the pharmaceutical industry. Where does jurisdiction stop and where does it begin? So I think that that was an issue that came up, which is, okay, if there is going to be some regulation and some standards put in place from FDA, where does that stop? And if we choose to do a study in partnership with a pharmaceutical company, which regulations or which guidelines, or what are we actually bound by, and I think that that was a very big question.
DR. WOODCOCK: So clarity of jurisdiction and the intent of any guidelines or regulations and who is actually being regulated is an important point that we need to address? That's very helpful.
MS. PEDERSEN: Ms. Baylor-Henry?
MS. BAYLOR-HENRY: This is for Mr. Daniels, and then after Mr. Daniels, I would like for Mr. Stefano to answer, please. It's a question that was raised in the Federal Register notice, and so I'd like for you to address it.
The question is, how should FDA address methods employed by pharmaceutical manufacturers to switch patients from one drug therapy to another similar product, and just a tag-on question, do you believe that there should be some type of disclosure made to the patient that there is a relationship between the manufacturer and the PBM?
MR. DANIELS: I would answer the first question. In my experience, manufacturers aren't and won't be switching anybody. Managed care organizations or those providing the care will. They alone will decide whether and on what basis any such--doctors will be encouraged to make any such switch in the interests of greater effectiveness or cost effectiveness. Anyone who imagines differently hasn't talked to any of my customers.
I think disclosure is very important. Speaking now on the pharmacy benefit management of that service in the marketplace, I think disclosure to patients, disclosure to physicians is both appropriate and essential.
MS. PEDERSEN: Mr. Stefano?
MR. STEFANO: I agree with that answer. I agree with both parts, especially the part about disclosure of the manufacturer if it owns a subsidiary, a pharmacy benefit management program, is essential.
MS. PEDERSEN: Dr. Temple?
DR. TEMPLE: It would be helpful for us to see the examples of studies that the ASHP thought were excellent that you mentioned, Mr. Daniels. Can you put us in touch with either a bibliographic list or something so we can--
MR. DANIELS: Yes, sir, Doctor. I read their press release and had personal knowledge of one of the eight or nine studies, but if you would like, I would be glad to undertake the project.
DR. TEMPLE: Even the press release would get us started. We could then try to find them.
MR. DANIELS: I can get it for you.
MS. PEDERSEN: Thank you. Ms. Stifano?
MS. STIFANO: Dr. Hillman, you brought up a real interesting point about the use of randomized controlled clinical trials as being biased, as well, when you're looking at pharmaco-economics or cost effectiveness claims. We are seeing some companies come in that are doing clinical or generating pharmaco-economic data based on their randomized control clinical trials.
Have you looked and done a comparison to see if the retrospective analyses that you have done, say, on a cohort, are so different from the data that's being generated while a study is ongoing?
DR. HILLMAN: I have not done a systematic study of that specific question but I have had the opportunity to work over the course of several years on a project where we first used models and retrospective data and then we analyzed prospective data. Believe it or not, some of these other types of analytic methods bring us right to the very same conclusion that the randomized control clinical trial does, and when they didn't, it was a very tentative conclusion by the RCT with--it just barely made statistical significance, so it wasn't that robust altogether.
And I wanted to just clarify one thing. I don't think that RCTs are only biased with respect to the pharmaco-economics. I think they're always biased with respect to clinical findings, and how a drug works under the best possible circumstances does not reflect how it works in actual clinical practice. I think that's irrefutable.
MS. STIFANO: Dr. Morris?
DR. MORRIS: I have been struck, even before your last statement, the use of the term "bias", it has been used by several members of the panel but yet in very different ways. When I grew up, they taught us that bias meant some kind of systematic deviation from the truth, and in terms of advertising regulation that's an important issue. And yet what we've heard from different members of the panel is the term bias used in very different ways meaning very different things.
I was struck by Mr. Lax's use of the term, was it home-grown rules to avoid bias, and yet when I look at what those rules are, they don't seem to get at this basic issue of does the study systematically vary from the truth.
Your own evaluation of sophistication of the groups, if I understand what the panel is suggesting, is that overall, you have a very sophisticated audience who can detect bias, so therefore the FDA doesn't need to have the same degree of oversight to this audience because they can do it themselves, and yet I don't see from the rules applied the degree of sophistication that, say, an epidemiologist or someone of that ilk would apply to determining bias.
So maybe could you speak to more specifically what you mean by bias? This is, I guess, for several members of the panel. What does it mean and what is the sophistication in? I'm still--there's a lot of kind of uncertainty here.
MR. LAX: Engaging in sophistication was the level of experience, the number of studies they are doing, the number of pharmaco-economists on staff, and the number of users of those studies.
I agree with you that the word "bias" probably had a different meaning to everybody that we spoke to. I guess it's sort of like pornography. I know it when I see it.
MS. PEDERSEN: Are there other members of the panel who would like to respond?
DR. HILLMAN: Yes. I think, Dr. Morris, that it is important to define the word "bias", you are quite right. We are all using it differently. I would like to suggest that the word bias, and Brian and I were just talking about this last night, has to do with the systematic introduction of incorrect information or information that sees only one side of an issue and it is not kind of mistakes made that even though people are trying to do their best and--I don't think it is the kind of bias that was referred to in the study that was done.
I would also point out that there was another study done a couple of years ago by the Zitter Group that was the same kind of analysis only it was directed at medical directors of all HMOs and simply asked two or three questions. Do you believe the evidence in outcomes and pharmaco-economics that is derived from work done by pharmaceutical companies? The answer is no. And do you use it? The answer is no.
And although I think the study here may be showing a partial changing of what's going on in the marketplace, I still do think that there is not the large number of well-trained people out there who can easily differentiate a good study from a bad study for a managed care organization.
MS. PEDERSEN: Dr. Luce, did you want to comment?
MR. LUCE: Yes. I don't mean to leave the impression that the consumer that we're talking about, in this case, managed care are the major institutional consumers, can screen out all of the bias that's thrown at them. These studies use dirty data, in a sense. I mean, they use assumptions. They do extrapolate. There is systematic bias and non-systematic bias in these studies. They are not randomized controlled trials, which, obviously, are designed to eliminate as much bias as possible.
My point is that the consumer is a savvy consumer. They may not know much about how to screen out the bias and even when to recognize a good versus a bad study, but most of the analyses that I have seen and some that I have done indicate that the managed care consumer doesn't pay any attention to these studies anyway, to a great extent. They screen out a lot of stuff. They make decisions based on a lot of things.
My major point, or a major point here, is that they just don't need your protection, that they are smart, that if, in fact, they make mistakes, it's not that big of deal. They're going to make bad buys, and if they make too many bad buys, they're going to go out of business. So that's really a big part of it.
The other part of at least our testimony is that, in fact, the market is self-correcting. If you have been watching the field at all, the field is getting better and better and better, and, in fact, the particular people or the industry that you're trying to regulate or that you're mandated to regulate is the leaders of our country funding the research and improving the methods and actually producing empirical data in spite of what FDA's role is, more so than any other aspect of the country that is improving all of these techniques and providing better data for decision making.
So I just don't see the need there, and I think there is some danger, in fact, a significant danger, of slowing down very important information that can help this country make better decisions about health care resource allocation.
MS. PEDERSEN: Mr. Lax, a quick comment?
MR. LAX: I think Dr. Morris's point about the word "bias" and Dr. Hillman's points about sophistication reinforce our point about the fact that this is a very, very young field, and I wonder whether when you have a young field you regulate it or guideline it into sophistication.
MS. PEDERSEN: Thank you.
I think Dr. Temple has a question.
MR. DANIELS: I'm sorry, if I may just follow very briefly and knit together a couple of things we have talked about. Increasingly, the studies--and maybe one day the only studies--in which our customers are going to place confidence are those done in their own population or in some like organization elsewhere, which is to say that the same skepticism that they--not skepticism of validity but skepticism of utility with which they would view double-blind controlled, as they see them, somewhat artificial, they would probably extend to academic studies of the kind that would meet the highest standards this learned group might bring to bear.
At the end of the day, what they want to know is, will a given care plan, which probably involves not drug versus drug but a basket of drugs wrapped in management programs or disease management programs that take into account the other factors impinging on patient care and results, will that work in my demography, in my geography, with my set of physicians? That is why these partnerships which Ms. Moench mentioned will be more and more common and why I think it will be hard to separate that which is industry sponsored from that which is managed care sponsored.
MS. PEDERSEN: Dr. Temple?
DR. TEMPLE: There is nothing about properly done trials that prevents them from being done in HMO environments and I think most people would say that seems like a very good place to do them. There is also nothing that says they can't use real world qualities and things like that.
The problem with epidemiologic methods is not that they are theoretically wrong or fundamentally unsound. It's that it's hard to tell in a given case whether this is a good one or this is a bad one. It's hard to know. So it would be very helpful to us if we could get some of the details of Dr. Hillman's analysis of situations where an analysis either did or didn't predict the same result in a controlled trial. To the extent you can supply those things or have already published them, that would be very helpful.
There are other data in the same arena showing that the non-randomized group of people, say, subjected to a surgical procedure, had about the same outcome as the group that was randomized in the surgical procedure. So there's certainly no contention that the answer is always wrong, just because you didn't randomize it. It's that it's hard to know whether it's right or wrong because the standards aren't very well set up for evaluating them. So anything you can contribute to that would be helpful.
DR. HILLMAN: I would be happy to provide those to you, but I just want to reiterate and try one last time to get past this obstacle where I think we are all, or part of the room has, with respect to randomized controlled trials. I submit to you that RCTs are wrong just as often as any other form of analysis because in the end, what we want to know is what happens in the real world to real patients when they are seen by real doctors. So if we knew that, then we could all make better decisions. Is that fair? Do we agree on that?
DR. TEMPLE: You need to provide the evidence that supports that, but it could be true, certainly.
DR. HILLMAN: No, no, the evidence that supports the contention that if we knew how a drug acted in the real world on real patients--
DR. TEMPLE: No, you must think there is evidence that shows that drugs that appear to work in controlled trials don't actually work in other settings. There are not many studies to that effect. It certainly seems possible, but I don't think there's a lot of data that show that.
DR. HILLMAN: Well, there aren't many studies to that effect because they have been quashed by the people who need to have some output for their studies that they fund, i.e., the pharmaceutical industry.
I will give up, but I think it's common sense--I think it's a little bit of common sense to say that our goal should be to determine information for the marketplace that helps us understand how real patients react to real medicines in actual clinical practice as guided by real doctors, and I don't think that's a matter for a randomized controlled trial. I think that's a matter of common sense.
MS. PEDERSEN: Dr. Woodcock?
DR. WOODCOCK: I just wanted to say, I agree with the sentiment. I don't think the methodologic disputes are what we are talking about today. The issue of generalizability is what you are raising.
DR. HILLMAN: Yes, absolutely.
DR. WOODCOCK: I think we understand that. I think everyone here on the Panel and the people here understand the generalizability issues and they may be more or less severe depending on the trial design and they may be just as severe in an epidemiologic study done in a particular setting. So why don't we sort of smoke the peace pipe on that one and go on?
DR. HILLMAN: I'll smoke anything at this point.
MS. PEDERSEN: Are there any other questions? Other questions?
MS. PEDERSEN: I think on that note we are prepared to take a break. Why don't we resume at 3:30?
MS. PEDERSEN: We can proceed with the second panel for this afternoon. For your information, the last two speakers listed on the agenda will not be appearing. Loren Abdulezer and Irving Schwab will not be appearing this afternoon.
I also wanted to ask whether there is anyone here from the Pharmacists Planning Services of California? Is there a representative?
MS. PEDERSEN: If not, we do have a note from Frederick Mayer asking that his comments previously submitted to the agency be read into the record. Those comments are entitled "Impact of Pharmaceutical Company PBM Relationship and Managed Care Promotions on FDA Regulation", dated in his submission to us August 16, 1995. Apparently, there is no one here to present this information but it will be made part of the record.
Now we would like to proceed. The four members of the panel are, as I understand it, Allan Zimmerman, William Zellmer, Calvin Knowlton, and Juliet Goodfriend, and we will begin with Allan Zimmerman, if you could begin with the financial disclosure.