FTS-HHS FDA CDRH
Moderator: Margaret Gomez
May 26, 2010
6:00 am CT
Woman: ...they thought of overnight.
Woman: (Joy), we're getting ready to start. (Joy)?
Melissa Eackle: Good morning. My name is Melissa Eackle. I'm the network leader for Infection Control Dental General Hospital and Infusion Pump Network. I'm glad to see so many of you returned from yesterday.
It was quite a lively discussion after the session ended. A lot of you stayed around. We had some really good conversation.
Today we're going to be talking about the guidance which is just one piece of our infusion pump initiative that was announced recently.
And our first speaker is going to be Lieutenant Commander Alan Stevens. He's the infusion pump team leader for the FDA Center for Devices and Radiological Health in the Office of Device Valuation.
He is the team leader and responsible for insuring consistent review of infusion pumps and coordinating policy for pre-market reviews of infusion pumps.
Prior to joining the Office of Device Valuation he worked in the Office of Compliance where he managed regulatory compliance action for infusion pumps. He has a Bachelors of Science in Mechanical Engineering from the University of Maryland.
Please welcome Lieutenant Commander Alan Stevens.
Alan Stevens: Good morning. I want to thank everybody that I've spoken to already and I think I've already lost my voice in the past half an hour. So I, you know, I look forward to today. I think we're going to have an interesting discussion and I expect that we're going to be getting a lot of questions and comments.
So if we could have the next slide. We're going to be focusing on the guidance document. You just gave away my whole presentation. We're going to be focusing on the guidance documents today.
I'm going to start with an overview to present our kind of each section of the guidance and what our thinking was and including the information in the guidance document.
And then we're going to have a series of presentations focusing on specific sections of the guidance that we think are of interest to you. Next slide.
I just - I first want to start by recapping at a high level, the information that we presented yesterday about why we're here today, why we've made the recommendations in the guidance document.
So we're, you know, we're concerned about infusion pumps because of the risk to health presented by these devices when they fail. And just in the past five years, you know, I put some of the information on the screen - some of the adverse events and the recalls that we've seen.
Recalls, you know, infusion pumps are a leader in device recalls. And depending on how you parse the recall data out there are actually (unintelligible) for recalls.
For example, if you look at - just looking at the class one recalls which are the highest risk classification for recalls, the infusion pumps have the most class of recalls for the past five years. In addition to the recalls we have over 56,000 adverse events which include over 700 deaths.
And I just want to address quickly, one, you know, aspect that has been recurring with respect to the numbers of adverse events and the denominator data. You know, denominator I think is an important question.
However, you know, when we think about taking action and doing, you know, taking an action like we've done I think we have to look at the total picture. And, you know, we look primarily at the recalls as direct actual confirmed failures of devices where we felt like the issues were preventable.
That combined with the fact that we know that there's underreporting of adverse events. So while we don't have solid denominator data we don't have solid numerator data either.
So what we do have is a picture of a large number of recalls, a large number of adverse events which compared to other device classifications, the numbers of adverse events we see for infusion pumps are relatively high.
So this information to us presents a compelling case for why we should take an action. So, you know, after we decided that, you know, perhaps we should take a look at the information to find out if there is something that could be done, we first had to look at the data and the information that we had.
And when we're looking at conducting analysis of data we have to look at it from several different perspectives.
So one perspective that we look at was a very high level view if we could - we wanted to determine if we could kind of parse the information down to identify if there were particular areas of infusion pumps that were having problems.
They - it was just the insulin pumps or just syringe pumps that we could focus on to address most of the problems. And what we found were that we were observing recalls and failures with each pump type wherever the pumps were used. And it wasn't just one or two or three manufacturers in each pump area.
But the problems that we were observing were occurring throughout the industry. Next slide.
So we had to dig a little bit deeper and we looked at the underlying data primarily in the recalls because that's where we had the root cause analyses and the confirmed failures.
And what we found were that we could categorize the root causes into several high level categories. And I've put those on the screen - software, (unintelligible) factors, hardware and manufacturing quality.
And what we found after we conducted the analysis is that we're not really looking at anything unique or specific to infusion pumps. We could be talking about any software driven electromechanical device or even product. You know, I think my computer crashes on a weekly basis.
So we're not dealing with unique infusion pump problems. So we had to think about that a little bit. And what we, you know, for my opinion, you know, one of the contributing factors that's just the - the large number of use and the frequency of use.
You know, some of these devices are in use 24 hours a day. So when we think about levels and probability of failures for latent based software anomalies, the expectation is that these are going to occur if they do exist.
And the other, you know, as we have been saying is that we believe that these problems are largely preventable. Next slide.
So this analysis that I just summarized very quickly - foreign (unintelligible) for our infusion pump improvement initiative that we announced on April 23rd.
In addition to the guidance document we've also announced activities where we're proactively facilitating device improvements and increasing user awareness and this workshop being one of those activities.
And I would certainly recommend - I'm sure many of you have already looked at the Web site that we put up and all of the information we have. But for anybody that may have not, you know, I put the link on the screen.
And for the rest of the presentations today we're going to be focus on the guidance document. Next slide.
So I'm going to walk sequentially through the guidance document starting with the title, Total Product Lifecycle or TPLC.
We intentionally selected this title to indicate a new approach for how we're viewing infusion pumps and how we're going to be reviewing them, focusing on using known post market data to inform our premarket reviews; the guidance document being a prime example of this new approach.
You know, we heavily relied upon the past market information we have, what we were developing the guidance document. Other areas premarket - post market information rather, that we have available to us. I have identified some of those on the screen.
A TPLC sheet - this is a single report. And I believe in the transparency initiative the TPLC sheet or at least a sample of what that is, is publicly available for the public to be used and to look at what I'm talking about.
But this is a single report that we can as reviewers, use on individual physicians to pull up information in a single place about the sponsor or the submission, with respect to post market adverse events, recalls and other post market information.
We also have other more typical post market information available to us as reviewers and special reports, recalls, adverse reports through the MDRs. And we also have frequent collaboration with our post market colleagues to identify any new potential safety signals that we need to address in our premarket reviews. Next slide.
So the scope of the document is primarily on the infusion pumps that are classified under 21 CFR888-5725.
And I've identified the product codes on the screen. It's general infusion pump, PCA pump, (unintelligible) pump, insulin pump, insulin bolus pump, (unintelligible) pump, infusion pump accessories.
One thing I want to note, you know, we did exclude the class three infusion pumps such as the implantable pumps, from the scope of the guidance document.
However, you know, we have never historically had to - at least in the preclinical engineering side, we haven't had two separate review processes for infusion pumps. And, you know, the guidance document does represent our thinking on how we're going to be reviewing infusion pumps moving forward.
Next slide please. The device description - I think that this section is pretty self explanatory. You know, we are getting pretty detailed with respect to the information we're asking for. And the purpose for that is to provide the reviewer an understanding of the device under review.
We do view infusion pumps as part of a drug delivery system that we need to understand how the device we're reviewing fits within that system, any features and information that we need to be aware of and ask questions about. And it helps us to inform our review process. Next slide.
So now we come to the first new big part of the guidance document which I think, you know, based on the comments and the feedback I've been getting, I think there's a lot of interest in this section of the guidance document. We'll be having a presentation to get more in depth on the assurance case later on.
But, you know, what the assurance case is it's a formal method for presenting claims about the device and then providing the evidence and linking the evidence with a convincing set of arguments.
And we do expect that the assurance case is going to cover every aspect of the infusion pump system design.
And, you know, our prior review process prior to, you know, releasing the guidance document, you know, what we were looking at was a claim of substantial safety and effectiveness to the predicate. And the submission would include a series of evidence or performance data.
And it was up to the reviewer to make the link between the evidence and to demonstrate for themselves, to - to make the claims.
And what we're asking now - what we're expecting is that the sponsors themselves are going to be making the arguments and not just high level arguments but actually detailed formal structured arguments using the assurance case method to present it to us in a way that's reviewable.
And I do just want to mention that I mean this is the first time that we have - that agency recommended that this method be used for the premarket review of the device. Next slide.
So within the assurance case section we identified known hazards and this is where the post market - the TPLC approach comes into play. The hazards we identified were based on the average (unintelligible) that we have received.
We identified hazards as a starting point or a framework for sponsors to conduct their own analyses. So we do expect that sponsors are going to be identifying any additional hazards that may exist in their systems. We identified what we thought were common or well known hazards.
But that is not intended necessarily to be a complete set of hazards. We only identified 56. So - but we do expect that each hazard we identified within the guidance document is going to be a (trust) within your submission with respect to whether or not it applies to your submission. Next slide please.
This is just an example of how we have structured the hazards within the guidance document. We tried to break them down logically with respect to how they impacted the system.
And within each section we've identified specific pieces of evidence that we believe are going to be common - commonly needed for every submission. Next slide please. So this is just some examples of the evidence that we have identified in the guidance document.
For example, flow rate accuracy, you know, we see flow rate accuracy typically for most submissions. And that's to be expected.
You know, I think one of the differences now is that, you know, where now we see, you know, maybe one or two cases and it's up to us as reviewers to determine that the test conditions are appropriate for the intended use.
The assurance case is really going to rely on the manufacturer to make the case to us for why the range of test conditions and maybe environmental conditions and the range of flow rates that are tested, are actually - demonstrate that the device stays with respect to flow rate accuracy, for example.
Other examples of evidence that we expect to see for most submissions - (human) factors, clinical evaluation, software, drug device, interactions, compatibility and reliability. I have one - a couple of comments I want to make on the drug device compatibility.
This is something that we have been asking for, for the past few years. And what we're expecting - and this is based primarily on the indications for use for the device so if you're identifying specific drug types such as antibiotics, you know, we expect you to focus on those areas.
But if it's - it's driven primarily by the indications and the routes of administration for the device. And what we're looking for is one, is to make sure that there are approved medications for delivery through those routes of administration.
And once you've identified those that are approved for infusion drug pump to assess those that you've identified for compatibility with the drug contact and components of your system. And then those would be identified in the labeling.
And one last thing I want to say on this slide, with respect to reliability, you know, one of the things that struck me yesterday, just listening to the presentations is the users' perception that the reliability of these systems is not what they would expect.
And I think that, you know, we certainly didn't tell users what to say. And so, you know, when I heard that that was almost a confirmation of the fact that we do need to be addressing the reliability of these systems.
And so what we're expecting is not simply - we're expecting a formal reliability analysis which is different than just identifying certain reliability parameters such as meantime between failure for a set of a components.
But actually identifying a reliability specification and then formally documenting and analyzing how the system meets that reliability specification.
Again, we're looking at this from a system perspective so we're interested primarily in the system reliability as opposed to component level reliability. And where possible we expect that to be determinative. Next slide.
So clinical evaluation is the next big section of the document.
And the assurance case section actually ends with a discussion of the use hazards. And we flow right into the clinical evaluation.
And this is intentional because we viewed the discussion of use hazards and human factors under the use hazards and the clinical evaluation to be two complementary evaluations. And, you know, the human factors, you know, that we have been increasing our emphasis on this over the past few years.
And this is a really important and critical aspect of our premarket reviews. And they're risk based assessments. And we do expect that the clinical evaluation is not going to take the place by the simulated human factors that we have been conducting over the past few years.
We still expect that the simulated, you know, I'm using that term simulated - but the human factors, you know, in the kind of - I don't know what the technical term is - simulated use testing - thank you Mark, is still going to be conducted.
And, you know, we do have a heavy emphasis on that aspect. But we believe that the clinical evaluation is necessary as well to address the risks of the use of the device and performance of the device. Infusion pumps are significant risks.
So doing some studies is going to require an IDE. And our expectation is that the IDE submission is going to include the assurance case up to and including the simulated use testing.
And we will be looking primarily to simulated use testing in the IDE, to demonstrate that all of the hazards have been mitigated to the extent possible, prior to doing a human study. Next slide.
So in the guidance document we identify the criteria for when we expect clinical evaluations are going to be needed. New devices, major changes in modifications, the intended use and modifications to correct problems with the user interface. Next slide.
Risk management - I'm not going to spend too much time at this moment because I'm going to be giving a presentation on that a little bit later this morning. But I just want to identify the documents that we have identified in that section of the guidance.
The risk management report, system architecture, design requirements, much of these documents are probably already being produced and so this isn't necessarily new for some (mini) sponsors. It is new for us to be asking for it thought. So we did want to address it.
And I'll talk about that a little bit later on. Next slide please. Pre-clearance inspections - the guidance document outlines where we have this authority to ask for this and our findings.
The law actually requires FDA to document the findings for when we are going to request pre-clearance inspections for (unintelligible) devices.
While in draft we're - if we want to do a pre-clearance inspection, while the guidance is in draft we're going to have to document a similar finding for each submission that we want to conduct a pre-clearance inspection. And we are doing that on a case by case basis now.
And clearance - and I want to make a point of this in case it's not clear. Clearance is a (unintelligible). It's going to be withheld if quality system deviations are identified during the inspections until the deviations are corrected. Next slide.
So the guidance - again, similar to the clinical evaluation section, the guidance document - when we expect to be conducting pre-clearance inspections. And again, we're going to have a presentation later on this that's going to get into this in more detail.
So I'm not going to spend time on this now. Next slide.
Labeling - similar to, you know, I think - I think the recommendations we're making are pretty self explanatory. Some of the - I just want to call out a few issues.
We do expect that the claims and the labeling are going to be consistent with the indications per use with respect of level specificity. So if the indications are general - the claims about the use of the device in the labeling should be similarly general.
And we do review those to make sure that the claims are consistent. And where the labeling goes beyond or makes more specific claims, you know, we will be asking questions and we have been. I mean this isn't new.
We will be asking questions about the indications for use, are they updating the indications for use or modifying the labeling to be more consistent. Again, we do expect that the labeling is going to identify the medications that have been evaluated for compatibility with the system.
And where the pump is intended for home use or foreseeably intended for home use we do recommend patient labeling. Exactly. So the guidance ends with a section on post market surveillance. Again, we're, you know, we have an emphasis on the total product lifecycle.
We're reinforcing manufacturing reporting requirements. That information isn't new but we are reinforcing it. Again, you know, we have an emphasis on understanding and getting good data. And we had some presentations on the issues with that yesterday and we're going to have more today.
And we're identifying answers to common reportability questions that we received. And we're trying to provide those answers publicly so that there is a consistent expectation on that aspect. Next slide please.
So we have an appendix to the guidance that identifies risks to health and the corresponding medications.
And when we move to special controls as we have indicated we're going to, this section is going to be much more prominent and sponsors are going to be required to demonstrate that they've mitigated each risk to health. So I would recommend you pay attention to this section.
And please - please make comments on the risks to health that we've identified in the mitigation. We do appreciate those comments. Next slide please.
So just to summarize, you know, we have, you know, we've looked at the root causes of the recalls and we tried to develop the guidance document to directly address systematically the root causes and the problems that we have observed with infusion pumps.
You know, in the presentations yesterday we identified design and manufacturing as being issues. Engineering issues, not specific, you know, there is - we didn't identify, you know, if you do these three things then we can address 80% of the problems.
You know, we couldn't identify that. What we identified were engineering, design issues and manufacturing issues.
So we have structured the document to address directly those issues so that we can improve the premarket process and to improve the, you know, to have a level playing field with respect to our expectations for premarket physicians.
You know, one of the comments that, you know, just in talking yesterday, many of you is that, you know, we feel like we do many of these aspects. Maybe it's not presented through the assurance case but we do many of these, you know, risk management and design controls.
And that's to be expected. And, you know, when we went through this, you know, our expectation was that if you're following the quality system and the intent of the design control you should already be doing much of this. So, you know, I think that's to be applauded.
And we, you know, certainly look forward to working with many of you to - the comments on the guidance and hope we can address any of the questions that you might have today. Thank you.
Alan Stevens: Apparently I'm next.
Melissa Eackle: Hi. One of our speakers is coming in on the Red Eye so Alan got bumped up where he's going to be covering the risk management report. So he'll be with you in just a moment. Thank you.
Alan Stevens: I thought I was going to have a moment to look at my notes. That's okay. Here you go. Okay. Can we go to the next slide please? All right, so my presentation on the risk management section is going to be pretty succinct. My goal isn't to tell anybody how to do risk management.
There are plenty of standards and workshops about how to do risk management. However what, you know, my goal with this presentation is just to convey our expectations for - thank you - for what we expect to see in the submissions. Next slide please.
So there - FDA has recognized ISO 14971 for risk management of medical devices. And I recommend you use it. Many of you already do use it standard. And certainly this is a good approach. And we, you know, we recognize the standard so, next slide please.
So for the risk management report, you know, we are expecting that the report should summarize the final results of your risk management activities. And should demonstrate that the risk management plan has been implemented. Overall residual risks are acceptable.
And we're particularly interested in understanding the thinking of and how you have evaluated estimated risks and how you've concluded that the risks are acceptable. And the report should provide context and analysis for your conclusions.
So what we're looking for is really a good understanding of how you have evaluated the risks for your device and how you manage those risks.
You know, some of the reports that I've seen not just for infusion pump submissions I see every now and then a risk management report included in a submission. And I've seen anywhere from two or three pages to much longer reports.
And we're, you know, two or three pages is probably not adequate to document and to summarize the risk management activities.
You know, we are looking for a good analysis and to provide the reviewer an understanding for your risk management and how you have evaluated, estimated and concluded the risks are acceptable for your system. Next slide please.
So when you're explaining your risk management activities again, you know, we want you to explain how you've estimated risk, how you've evaluated the risks, how you've identified the risks and the control measures and provide a justification for risk control measures with respect to is it a design control, is it a protective measure, is it labeling, why is that acceptable?
And then once you've implemented the risk control how have you evaluated the risks associated with the controlled self? And at the end of the day, you know, there are going to be residual risks.
And so we're interesting in understanding the risk benefit and how you've analyzed them to be acceptable. You know, simple statements such as, you know, acceptable - the risks are acceptable. That doesn't help me as the reviewer understand that they are acceptable.
I believe that you believe that they're acceptable. But, you know, we're trying to make an independent evaluation about the residual risks with respect to the device and how it's going to impact the users. And so we really need to understand how you've come to that conclusion. Next slide please.
System architectures - again, I don't think this is, you know, anything relatively new or, you know, something that you perhaps don't already have. But we are looking for a document to indicate, you know, how the functional requirements are allocated in each system.
And the purpose is to provide us as a (reader) of this, you know, I talked about this (device) description section and we're getting into a lot more detail than we have in the past. And this is to provide us as a reviewer, a comprehensive understanding about the design on the device and how - I can hear myself.
Just a second. Give me a second. So take a - collect my thoughts. Okay, so device description. You know, we are getting a lot more detail with respect to the description of the device and getting down into how the device is designed, how you've allocated functional requirements.
And what we're really trying to understand is how you've designed the system and where do we need to ask questions. You know, and this all ties back into the assurance case. Have you evaluated all of the risks? Have the hazards been mitigated?
And we, you know, we need all of this information to provide us a comprehensive view of the system. Again, you know, the root causes that we identified from our perspective, are indicative of, you know, systematic design issues.
And so it's not - it's almost kind of like looking for a needle in a haystack per se because, you know, we can't just say, you know, if we identified every recall problem say, key balance and we're going to evaluate key balance for every single submission, well in the next submission down the road that might not be a problem. That might not be a problem with key balance.
We might be overlooking something else. And so we're trying to evaluate systematically the system and the device that we're reviewing. And so this kind of information helps us to understand comprehensively the submission itself.
And within context, you know, the assurance case and risk management and all of the information that you're providing to us is just another piece of information that provides context to us. Next slide please. Design requirements, documents - again, this is nothing new.
And it's already required under the quality system. You know, we're not looking at this from the quality system inspector's point of view, the investigation to understand that you have a process in place and, you know, from the quality system aspect we're actually looking at the decisions that are being made and do they make sense.
Are they consistent? Are there conflicting requirements? And again, do the - I mean do the design outputs meet the inputs? Again, you know, we're looking at the decision-making. Does it make sense from a risk management standpoint and does it tie back into your assurance case? Next slide please.
So this is to summarize, you know, the information that you provide particularly in the risk management report. You should document and provide the analysis.
We are looking for conclusions in your - to understand how you've designed your system and why you've made the decisions that you've made. That concludes my presentation.
Melissa Eackle: I want to thank Lieutenant Commander Stevens for stepping up there. All righty, our next speaker is Mr. Ron Kaye. He represents the Human Factors Program at FDA Center for Devices and Radiological Health. He leads the Human Factors and Device Use Safety Group.
The purpose of his program is to insure that new medical devices are safe and effective when used by the intended user population.
The effort primarily involves reviewing new device submissions, promoting effective and focused human factors evaluation, promoting good design practices for medical devices, developing FDA's Human Factors guidance and participating in national/international human factors standards.
The team also contributes to analysis of post market reports of use related area and to device recalls when use error is involved. Mr. Ron Kaye has worked in human factors for 27 years and has been with the FDA Center for Devices and Radiological Health for 14 years.
Prior to this, before joining the FDA, he worked in human factors and human performance testing, training analysis and research on safety critical systems such as nuclear power plant control rooms, military weapons and communication systems, aircraft cockpit systems, air traffic control instrumentation and medical devices.
He has a Bachelors in Psychology and Biology and a Masters in Applied Psychology. Please welcome Mr. Ron Kaye.
Ron Kaye: Thank you. And judging from the questions we got yesterday it seems that there's a lot of interesting human factors. So for all of you who are particularly interested in human factors today I hope I will quench that thirst here because we've got a lot of human factors to talk about.
I want to start by just providing a - an overview of use error which was a topic yesterday. And if nothing else, it will show you the agency's perception of what use error is and how it fits in context of what we're doing in terms of assuring safe and effective devices with respect to their use.
So this nice little animated diagram here shows use error in this - use - I'm sorry, device use occurring in the center. And the outcome of that we can simplify and say well, one of two things are going to happen, you know, we're going to get the desired safe and effective use.
Or sometimes unfortunately we have unsafe or ineffective use and we call that use error. On the left side there is human factors considerations.
And when I talk about human factors today and when we talk about human factors at the center, the terms human factors, usability, ergonomics, user center design and that sort of thing, we're talking about the same thing okay? So these would be usability considerations for instance.
But on the top left you see use environment. Of course the use environment, the amount of noise, interoperability with other devices, procedures, distractions, workload and that sort of thing can certainly influence the way people use devices and devices need to be designed accordingly.
And then you have the users themselves. And of course we know that - and this is - was stressed yesterday by Mary Brady and some others, particularly for home users. But users are variable. They have different knowledge.
They have different abilities, different expectations and limitations, that sort of thing. That's why it's so important to consider the user population when designing devices.
You want to have a good outcome which is up one - green zone on the upper right there. And finally we have the device user interface itself.
So for human factors the important part that we're considering as - in the FDA in terms of our regulatory purview is whether or not this device user interface consisting of logic and sequence of interactions, operation requirements, screen design and layout control characteristics and layout color coding alarms, feedback, etc.
So the question really is for what we're going to be talking about today, is we want to know that this has been optimized. So with respect to these other considerations - so that when they feed into device use that we're getting more of the green safe and effective use and less of the use error. Okay?
I think that's fairly straightforward. This is by the way, right, a graphic that is in our guidance that we published in 2000. So we're - we're - medical device use safety incorporating human factors into risk management.
So we are - the flavor of what I'm going to talk about today is fairly consistent with that document. There's a little bit more emphasis in - and specificity in some of the aspects of testing. Next slide.
FDA views use error as a serious source of risk for medical devices and particularly for infusion pumps right now. Equally dangerous - use error is equally dangerous in device failure problems.
If a device fails and somehow it injures or kill somebody, you know, or if somebody makes an error when using a device they can be just as injured or just as dead as a result of that. It's serious business use error. And it can - it is something to be avoided to the extent that we can possibly do that.
We've had recalls associated with use error and design issues. The design of the user interface was at fault. And we know that a better design will help because when those designs were fixed the errors went away. So it underscores and demonstrates quite vividly, the importance of this.
We have - we get (mod) reports of course and user errors often cited. You heard that yesterday. Sometimes called user error or - and in fact most (oftenly) - often caused - called user error and - as much as we try to use the term use error.
And I know that was a question yesterday. In other presentations I've done I have a whole slide on use error versus user error. What's the difference and why? I've got a lot of slides here so I don't have a slide like that but we - perhaps there will be more questions on that.
And not only that but the other error reporting codes - from my own analysis of some of these reports, I believe that use error is incorporated under other error coding codes at times in these reports. And a characteristic of the reports we do get in unfortunately is, the use error is often lacking in detail.
We know that it was - we knew it was a device used that let's say an under infusion took place but we don't know what the user was doing with that device, how they were interacting with that device and how that interaction failed what the user's expectations were or what they failed to do, that sort of thing.
It's often not contained in there. We'd like it to be and occasionally it is but often it's lacking. And so use error is often - I believe it's likely misrepresented in summary tabulations. It's already a common cause of reportable problems.
And that was presented yesterday as well. So that's some of the bar charts in terms of the frequency but it's probably even bigger than that and of my own perspective perhaps. But to sum up here, better design can reduce or prevent use error.
And so to make a better infusion pump we need to optimize the design of the user interface to the extent possible. So I'm going to be talking about testing that will demonstrate to us that we're going to require that will demonstrate that. Next slide.
Here's a - if you consider that there's maybe a flaw in the user interface device that will allow or perhaps lead a user into making an error that's going to cause some kind of harm, the existence you could legitimately call that a hazard, a use related hazard that exists in that device.
Now it's important to maintain this idea as separate from device failure hazards. Often in the past they were all lumped together, it was all confused and it was difficult to try to evaluate them all at once. The methodologies didn't work very well.
So use related hazards are kind of an entity onto themselves. You'll notice that these - this (VEN) diagram here overlaps and that is because there are cases for which the user interaction with a device will actually damage the device or perhaps the user inaction will fail to calibrate it correctly and it won't work right.
So that's, you know, device failure intersects with use related hazards in that way. Now another sort of overarching concept for this diagram that I think is important to keeping your mind is that for human factors evaluations they're a little bit different than other types of evaluations perhaps.
And that is that we assume that the device is going to operate correctly. A human factors evaluation is not about whether or not a device is going to work. Whether it's going to - in the case of infusion pump whether it's going to deliver drugs and deliver that accurately.
The question is when you get the user in the loop so to speak, or a sample of representative users, is that combination of user and device going to be effective at doing that consistently and safely or are there going to be problems?
So what we are - what we are talking about with human factors testing is we're not about looking for a success of the pump operation or even success necessarily of the users using that pump. We're looking for the problems. We're out to look for trouble. Okay?
So the focus of this - of the testing methodology that I'm going to talk about, you'll see that it is designed to detect trouble with respect to use of device if it exists. It is a powerful method - two methods really but in essence they do the same thing.
You're putting the device with the user, under scrutiny from sort of two directions and I'll talk about that, to see if there's problems. The magnifying glass and the spotlight are on whether or not there are problems there. And those are going to be recorded and evaluated and we'll talk more about that.
This - again, as I said earlier, we're talking about the user interface as the important part of the device for human factors and particularly for the FDA. And most of you are familiar with the infusion pumps and you know that there's a wide variety of user interfaces.
And this is just a - sort of a collection of some of them. It gives you an idea of the variability. I don't have a pointer but, you know, there's a (unintelligible) pump up there, a fairly simple mechanical device.
You've got the syringe pump, large volume pump and this mindboggling array down in the center, all pumps doing things and an insulin pump that is worn by a user (unintelligible) device. So that's the point of this slide. There's a lot of variability, a lot of intense interaction with these devices.
And very important because it involves drug delivery. Okay? And we know that too much or too little or wrong rate or failure to do - even be able to use the drug - the device when it needs to be used, they can only represent hazards or risks to the patient. Next slide.
I want to talk about from a high level, some infusion pump user interface attributes to promote safe and effective use. And this is to be generalized too, a variety of medical devices or technology, a lot of technology in fact.
But the user interface is good to the extent possible to provide users with sufficient, timely, clear and usable information to support their understanding of the operation of that pump while they're using it.
It shouldn't be so complex that it will unnecessarily delay, confuse or prevent users from operating the pump at all. It should be designed to optimize the arrangement of displays, controls, labeling and accessories to facilitate user interaction and error prevention.
Controls should activate and respond appropriately in response to user control actions. And they should also be resistant to wear from fingernails or inadvertent breakage and that type of thing.
Infusion pumps have little doors that close and we discussed yesterday, (Atali) talked about how those can be cracked. You know, and - and that's an engineering issue I guess.
It's also a human factors issue from the extent - from the perspective that the device should be designed so that in typical use users aren't going to have to break it or set it up wrong so that they force it and break, that sort of thing. Next slide.
Continuing with these desirable attributes increasingly important with the complexity of particularly, you know, screen based interactive systems, computer interface if you will, is the logic of operation. What is displayed to the operator when and how, when sequence? Excuse me.
And how that fits in with what they're doing and what they're thinking and what they're understanding while they're using the infusion pump. Feedback and response and control actions should help users understand pump operational status as they're going.
Alarms should be - should successfully alert them and then the alarm should not appear repeatedly for trivial reasons. Nuisance alarms - I mentioned yesterday, such that the meaning of the alarm has decreased. And that goes for warning screens as well.
I know we have - we heard repeatedly about instances where warning screens appear so much that the typical, you know, way that the device used - is used becomes - see the warning screen, exit out, proceed. Don't even read it.
The interface should warn users of incorrect control acts into input values prior to initiating or modifying treatments. To the extent possible that's a good feature. And we talked about (DARS), dose area reduction systems, yesterday a bit. That would be an example of that.
Sometimes there are specific parameters in fields that are hard coated that will not allow say, three digits or a value over a certain level. And the process of the pump use should be intuitive and consistent with user expectations to the extent possible.
You don't want to have a design that does something backwards compared to all the other devices the user has ever used. You know, a very simple example, you know, might be a light switch that turns on when you push down and off when you turn it up.
I mean that's trivial but that's called a population stereotype for design. And to the extent possible you want a design that when the user interacts with it they can pretty much figure out what it's going to do in response to their actions. Next slide.
So what are we asking for with this new guidance for new infusion pump submissions? As you - as we mentioned and you already know, we're asking for a clinical evaluation of pump use. And that's recommended as Alan said, for new pumps.
For major changes or modification of intended use of the pump and also for modifications made to the design of the user interface in response to usability problems. And I'm going to be talking about that third point in various contexts as we go along. Next slide.
So we talked about - Alan talked about assurance cases and human factors was one of the areas that required that.
So what I'm going to talk about next - it's actually - it amounts to two stages of validation to support assurance - the assurance case argument that use related hazards have been adequately controlled for an infusion pump submission.
That includes validation through simulated use and we've been asking for that fairly consistently. However it is part and parcel now of the other component which is validation with clinical - through a clinical evaluation or actual use. So the two go together so I'm going to talk about them both quite a bit.
But previously it was simulated use only. Now we have these two parts. Both approaches provide the opportunity to detect and announce, analyze potentially dangerous problems with the design of infusion pumps that could or - cause or allow avoidable errors, okay?
Some people, you know, there are questions about user error that say well, you know, how can you eliminate all of that? You know, sometimes users are just irresponsible or sometimes they do crazy things. That's true. But there are - most use is well intentioned.
The clinicians do not want to harm their patients. Users don't - that wear pumps don't want to harm themselves. And the problems arise when they're confused or if something that they do inadvertently is going to translate into a problem with dose delivery.
So again, we're talking about optimization of the interface to reduce those instances of misadministration of dose to the extent possible. Next slide.
So simulated use and clinical evaluation - why do we need both of these?
Well the two processes are complementary and I'm going to talk about that a little bit on the next slide. Each has unique strengths and weaknesses with respect to use validation or validation of use.
So given that we have now a requirement for clinical evaluation, simulated use has another new level of importance. It's necessary to demonstrate that the pump interface is sufficiently safe for use prior to performing the clinical evaluation. Why is that important?
That's important because we don't think it's a good idea to evaluate a new infusion pump interface by hooking it up to real people and delivering drugs. We can do that through simulated use.
And it's absolutely necessary that we go through that stage and get good results prior to the following phase of clinical evaluation. Next slide.
So this - this chart just compares for - in terms of validation need for testing it shows there are some strengths and weaknesses if you will.
And I said that the two approaches were complementary. Focus - being able to focus on critical tasks in your testing. Simulated use is very good for that. You can load up your simulated use testing session with a lot of very important tasks that the users might not typically do in that amount of time.
But because they're important you want to test them so you can script the scenarios if you will, so that you're evaluating that intensively. Clinical evaluation - well we suspect that that's going to be a lot of actual use. I mean the critical tasks are going to occur when they occur.
And you can't really force that. You don't want to force a user under clinical situations to - conditions to do a bunch of critical tasks just because you want to, you know, measure it. Realistic conditions - simulated use, you know, we try. You try. I know.
Often those are conducted in basically a - maybe a small conference room type of setting. And that's just the status quo unless you happen to have a high fidelity at your disposal. Then you can simulate actual use conditions very well with some of those.
And clinical valuation of course, much better for that. The extent to which users are representative to which you can detect unanticipated use problems is very important. Unanticipated use conditions - I never thought they'd use it like that or in this way under these conditions.
Force challenging or unusual use scenarios, follow to determine cause of (taps) failures. This is a very important component of testing. Both of them - you can do that very well. But you can see the comparison here.
Okay, so I said they're complimentary and this is - and I could also add the training component, I think for all of you have been on this chart. How trained are the people? Are they realistically trained? I'm going to talk about that as well.
And of course, you know, simulated use often - because it's done people try to do it one shot or maybe on two adjacent weekends or weeks. The training is fresh. And one problem that can happen is training can decay. Under clinical evaluation - we expect that to be a longer term.
We might see some of those affect the fear. It's important. Next slide.
Okay, these next several slides I'm going to talk considerations for validation testing under - that apply to both simulated use and clinical evaluations.
So in a lot of ways the intent for testing and the importance of the data collection is similar. And then I'll talk about each separately after this.
But a successful evaluation is something that demonstrates that users will be able to use the infusion pumps safely and effectively without committing dangerous use errors. But this only is - this only provides us with that understanding.
If the testing is sufficiently robust, to detect and explain weaknesses in infusion pump user interface design when they exist. Remember when I said that there are going to be detectors here and we're looking for problems.
Now if the vigilance, if you will, that is part and parcel of the testing protocol is at a fairly low level or if it's misdirected this - these problems are not going to be detected. So the testing has to be robust. It needs to be able to detect these problems if they exist.
So a successful test is a test where you set it up and it is a robust and well designed test and you get minimal or few or maybe no serious problems. But you could have detected them if they were there, okay? So that's - and again, we're talking about validation testing.
Now I should stop here as we're starting to talk about validation testing, and say that there's a whole lot of other human factors testing that can and should be done. It's generally referred to as formative testing. And validation testing is essentially equivalent to what's called summative or summative testing.
But by the time you get to this type of testing, the validation testing, hopefully you've done a lot of preliminary work, human factors testing. Now we don't really say - I mean there's - there are sources available that will tell you how to do it.
And keeping in mind the requirements of the validation testing that I'm going to talk about, it will provide you with a lot of focus and direction for doing that.
But what I've said in other talks and I think I'll go ahead and say it here, when I talk to you as manufacturers, I suggest to you that you cheat on the validation testing as much as possible. And that may sound like kind of a strange thing to say but here's how you cheat on that.
The way you cheat on it is you do the preliminary testing, understanding that this is what your validation test is going to be like so that if there are problems in the user interface you don't discover them in the validation testing. You've already worked them out.
By the time you get into this testing and you apply these methods you already are pretty convinced that that device when used by representative users, is going to be okay and you're not going to get a lot of errors. That's how you cheat on the validation test. Next slide.
Continuing for both types of testing - validation testing - a very important aspect of the testing. The risks and associated tasks are prioritized for data collection and analysis. This is sort of generic risk management approach to things.
But it's important to emphasize because if it isn't in the past a lot of tests have come in with a lot of trivial measurements and we don't, you know, how long does it take the user to unpack the device from the box? Or how long does it take them to plug it in?
Which are fine to measure if your risk assessment prior to the testing, shows that there's a critical task and if they don't do it right they're going to hurt somebody. But typically those aren't the type of things we want to test.
We want to see are they going to misprogram it or are they going to set it up so that it fails to deliver but they think it is, that sort of thing. Task use scenario - task or - task or use scenarios and in a test you do your analysis typically it's best to separate things into tasks, user tasks.
What are the tasks they do? What are the important ones? What are the critical ones? Where are the errors likely? In the testing - in the simulated use testing you turn those into use scenarios that contain those tasks.
But those tasks and use scenarios, especially with design features, modified a response to reported or known issues of - with use problems need to be prioritized as well. You don't just do a prioritization on the task and forget about that.
And I'm not just saying that for no reason. I'm saying that because that has failed to happen. So it's important to emphasize that when you're fixing a device and changing an interface in response to these types of problems you've got to focus on that. You have to prioritize that in use testing.
And this is great - this has to do with collection - data collection for clinical tasks, clinical evaluations. But it can really change the way you set up the conduct of your simulated use testing. Now there are two broadly defined types of data for each validation method.
And remember I said, there are two detectors and I'll talk about that later okay? These are the two detectors for use problems. And those again, broadly defined are performance. This is what you measure.
You measure success, measure failures, measure user performance in some way, some way that makes sense for the objectives of the test. The other, and equally important and sometimes arguably maybe more important, are the user based evaluations or the subjective part.
So you need to ask your users what they think about what they just did with the device. Were there problems? Was anything part curly confusing? Did you, you know, think that you might make an error at some point in interacting with this device?
And see what they say because errors are not always manifested in behavior. And again, these are still even the clinical evaluation, is a sample of actual use when you think about how broadly these devices are used. We're - it's a constrained sample.
So it's very important to get the user perspective on what's going on and not just some sort of tally of performance measures. Next slide.
So what are performance measures? Well they can - it can be done in a variety of ways. But the main issue - the main requirement is to record task failures.
We want to know about if somebody needs to do a critical task and they don't that needs to be recorded in the testing. And it's an important data point. Also, difficulties. I mean obviously I'm in trouble with something or not understanding how to do something that's something that should be collected.
As well as close calls. So a failure is an action or a failure to act that would lead to an undesirable treatment outcome to the patient or user, okay? So that's basically a use problem that you can see and detect.
A close call and sometimes these can be observed or sometimes you need to rely on the subjective part, but that's any instance of failure that was avoided by vigilance on the part of the user.
It doesn't mean for instance, successfully avoiding a problem by responding to device interface features such as warning screens. If a warning screen comes up and it guides the user back on the right path that is simply the interface doing what it should do.
However, if there's, you know, if somebody's ready to hit the start infusion button and they've got 88 in the field where there should be 8 and they notice at the last second, that could be significant and those types of close calls should be recorded.
Very important is follow up on each performance failure. Each performance failure and critical task - we're not interested in how many failures we had, okay? We had 250 uses and we had five failures so that's good. Well, maybe not.
That's kind of a high rate when you think of 20,000 users or 100,000 users a day perhaps. So the important concept here is to follow up on each performance failure to determine the cause and that can come from both observations.
Some people videoed the users particularly in simulated use testing. And it also involves a prospective user. This means go to the user and say okay, when you're doing - when you did that you ended up with this result, how did that happen? Let's walk through it.
Because we want all the failures that occur to be described to us. Not just a failure but what happened. Next slide.
Focusing on subjectives - subjective assessments. Again, that's the feedback from participants.
And it needs to be actively solicited from test participants or in the case of clinical evaluation of clinical users regarding aspects of device use that may be difficult or confusing. Actively means you need to go out and actually engage the user.
You don't leave a form sitting there that they can comment on if they want because a lot of people don't want. They want to get home and pick up the kid at the babysitter or do whatever they need to do. So that should be actively solicited not voluntary.
And when assessing use from the perspective of the users again, you want to spend most of your time inquiring about high risk aspects of use, critical tasks if you will. And this is a descriptive process. Ratings scales typically are only useful in a general sense.
It gives you a general flavor of what's going on. And they're fun to get a bunch of numerical data and you can do a lot of needs statistics with that - averages and standard deviations, etc. But what does that all mean in the general sense?
Because typically when problems happen with use it's a subset. Subset of users under certain conditions are going to make an error because of their interaction of the design as it is, okay?
So in the broader sense, ratings scale averages and that sort of thing it washes out. We don't - it doesn't give us that information. So - and so this aspect of the data collection this is a descriptive process.
You're asking for a narrative. Describe the problem; describe your difficulty; describe your impression, okay? So it's a verbal.
It's a piece of narrative data. So on those objective assessments patterns of specific difficulties reported by users is potentially as important as patterns of performance failures.
Again, since we're dealing with a small - a relatively small number of people, particularly in simulated use testing and in cases of use and you can also throw in ( Hawthorne) effect. You probably - most people know what that is. Does everyone know what that is - ( Hawthorne) effect?
Basically if you are observing somebody they're going to act differently, is all that means. If I know I'm being watched I'm going to be, you know, like - I'm going to do well hopefully. I'm going to stand up here and not stutter and use profanity for instance.
But users are the same way. You know, if they know someone's watching them, you know, they tend to - they tend to be, you know, on their best game. But they might say, you know, this aspect of users was really difficult. And not only me but user seven and nine and 14 say the same thing, you know?
Well that was difficult, confusing. Okay? That's note - not - that's noteworthy. You're going to have simulated use testing with all excess on the performance side. If you have a pattern of people saying - calling out the same aspect of the device as being confusing or difficult, that's significant.
And that should be looked at and we should - perhaps if we see it we might need to talk about that if you don't have an explanation for it. More on simulated use and clinical evaluations.
A very important point - a lot of people like to do a method of testing that involves usability performance objectives or success criteria. And these typically allow for a given percentage of failures for critical tasks but they're not appropriate for these validation studies.
Again, we want to know about every failure and we want to know why those failures occur. This doesn't tell us that. It might sound good if you say 90% of our users were successful on this task. And on this task 95%, this task 98%, the next one 100%, 100%, 100% and another 95%.
But what does that mean to us? Well that means that you had a 10% failure, you had a 5% failure, a 2% failure, 0%, 0%, 0%, 5% failure, 10% failure. So what does those failures mean, okay?
In a relatively small sample like simulated use particularly testing is, does that mean that somebody just was overdosed? So that's why - that's why that's a problem. So we expect that all critical tasks will be performed correctly unless satisfactory explanation of failures is provided.
And we can discuss that if there are questions on what that means. It's also very important in both types of valuations to be able to detect unanticipated performance failures. That means it's a very good idea to think of what failures might occur before the test.
But it doesn't mean that you list those that you thought up analytically beforehand and check if they occurred or not because something else might happen. Users being human beings, are unpredictable and they can do the darnedest things that a real smart bunch of engineers might not anticipate.
So unanticipated performance failures is an important part of this testing and data collection. And in your test plan you need to describe to us how you're going to go about being able to detect unanticipated failures if they occur. Next slide.
Okay, now I'm going to talk in the next few slides just about simulated use testing only, okay? This is - so this is not both methods. Test participation - test participants or the users - I call them test participants, they need to be representative.
They need to represent the user population. It's a relatively small number of them but in the past we've gotten tests that were all middle aged white male engineers on the design team for the device and that's not good testing. That's not - those aren't representative.
And the next point - they should not be employees at all. We no longer accept validation test results where the employees of the manufacturer are the subject in the testing, okay? They need to be from outside. How many? For simulated use it's a pretty small, minimum number of 15.
You can get pretty good results from a simulated use test with that number. I've done it myself. Multiple kinds of users - for device there was more than one distinct type of users. Perhaps there are professional healthcare providers and there are lay users that are going to be using it at home.
They do - they do different tasks on the device. But you want to test the device. Of course you need a minimum number for each of those distinct types of users. So 15 in each case for simulated use testing only. Training - and this is a dicey issue, training. It's difficult.
But we would like to see test results where the training given to the participants in the simulated use testing is approximate to what can be realistically expected for actual - the training of actual users and that depends on your analysis of what that's going to be.
The idea - I mean we - again, we've seen plenty of tests where the test methodology involved training the users. We trained them for 2-1/2 hours with our expert staff - our expert trainers and then we tested them. And they all did real well.
Again, we're trying to sample - in simulated use testing we're trying to sample as best we can, actual use. So being trained and being tested, you know, I would like that. A lot of people would. But obviously your performance - you're going to be freshly trained.
So there needs to be some type of lag time in terms of the training kit. Also the extensive training - I mean if you're going to train everybody for 2-1/2 hours can you reasonably say that every user of your device is going to receive that kind of training?
Or should there be some variability based on how much training people get? Next slide.
Testing should represent a sample of actual use. The set of tasks evaluated should be comprehensive. That doesn't mean don't leave important things out.
Testing should include all tasks that are critical for safe and effective use of the infusion pump. Don't leave important tasks out.
Prior experience with known device problems in your analysis of how you're going to construct your test - your prior experience with known device problems, adverse event recalls, literature, wherever you get that information should inform - and your own risk analysis should inform how you're going to select the tasks for your simulated test and how you're going to emphasize that in your data collection. Next slide.
Use conditions should be created as needed to provide challenging use conditions when anticipated risks indicate a possible challenge to the user arising from an environment of use.
And just to give you a brief example it might be - you might want to detect - you might want to measure or need to measure whether users are going to be able to detect an alarm.
So you might want to have background noise consistent with an OR or roughly consistent, to see if they're going to be able to detect that. You're going to - you can bend time with simulated use testing, okay?
Again, you can force the challenging tasks that users wouldn't typically do in that amount of time. But since you want to test that you can script your scenarios so that they're doing the critical tasks more in the amount of time that you're doing your testing.
And testing should allow a natural flow of user interaction. Telling a user okay, now do this, check; okay now do that, check. You know, it sounds ridiculous but I review these things, I see this, okay? That's not a naturalistic use, you know, in simulated use testing.
You know, it provides a lot of cues for what to do next when you're telling them. So it should provide a natural flow of user interaction to the extent possible you can do it. Next slide.
Again, for simulated use testing, mitigation strategies.
Often - often problems are found in the testing and some strategy for mitigation will come up. We changed the labeling or we fixed it. We enhanced the training. So everything's okay.
Well no, if it's a problem that you detected in your validation testing which is unfortunate, and you take these tests and say well we mitigated that and now we say that the risk is acceptable.
Well, if it looks like the problem was on critical tasks, mitigated with let's say testing - labeling you need to validate that. You need to do some sort of - perhaps a mini study to show that for whatever that problem was, whatever the users were misinterpreting you fixed it with that labeling fix.
Unless in some cases mitigation is a design removed the possibility of failure. Next slide.
If the user requires assistance from a test administrator during the testing - the simulated use testing, that is a fail and you need to evaluate the cause of that.
In your final report for simulated use testing you'll need to describe all performance failures for critical tasks, summarize all subjective comments and provide a root cause for each failure. And root causes aren't general statements like user did not follow instructions or the user is confused.
It's what happened with the interaction. And you need to provide an analysis of your results leading to your conclusion that due to what you found in your testing the device is reasonably safe and effective for the intended users. I'd like to see that in the report. Next slide.
Clinical evaluation specifically - again, as you know, the emphasis is on human factors and usability. It will involve detection of episodes of the dose, misadministration and other difficulties with use such as close calls that we already talked about.
Test personnel under these conditions, will need to follow up with clinical users to determine the nature of the device use that was involved for each misadministration, close call or significant difficulty and record the follow up results for each as part of that testing.
And that needs to be done properly. They - whatever the, you know, is it built into the protocol? It should be a method for being able to find out that information when those things happen, as soon as possible so that the user doesn't forget what they did and that data won't be obtained. Next slide.
Similar to simulated use testing on a slide like this, the users of course should be representative. I would think that they would be but of course we don't want to screen them in some way so that they are all very highly trained or something like that.
Again, they shouldn't be employees. I can't see how that would happen. But I don't know. How many users? Well at this point we say a sufficient number for comprehensive assessment. And, you know, I'm sure we'll - we will be discussing some of this and we're open to input on that.
Multiple kinds of users - again, as in simulated use testing if you have different types of users you need to sample both. And training again, similar to simulated use testing it should be representative of what typical training is going to be for participants in this clinical evaluation. Next slide.
In summary, FDA is requiring additional testing for infusion pumps to include a clinical evaluation in addition to validation through simulated use testing. Simulated use testing is necessary to insure sufficient safety and effectiveness of use prior to the clinical evaluation.
The two methods are complementary with respect to their ability to provide powerful validation of use safety. They're overlapping as well as unique considerations for testing inherent to each approach. Thank you. Okay, I'm finally done.
Melissa Eackle: All righty. We just heard - this has been a pretty heavy presentation. It took a while. We're going to be taking a 15 minute break. Again, we're going to have one more presentation before the questions and answers.
But I would ask that on your way out you pick up cards and you return them to the registration desk. We will have the microphone open when we do get to the question and answer period. And we do have some leftover questions from yesterday and we will be attempting to answer them for you.
Thank you very much. See you in 15. Oh, and those on the line please do not hang up. All right. Could - hello again. Could you all start walking back to your seats? I know you have a lot of questions but could you please start taking your seats? Thank you.
FDA people that means you too. All right, I've had several questions and some recommendations made. I did not ask anyone to silence or turn off their electronic equipment. I'm going to do so now.
I've been told it's distracting. So if you could make it silent, off or vibrate your fellow participants would really appreciate it. I'm a grandmother and a mother and also an employee and I understand we have to keep in touch. But that would be helpful. Thank you.
I just want to go over again, if you have not handed in your cards for your questions they go to the registration desk. We also are - after Mr. Chapman's presentation we're going to go right into questions and answers. And there is also a microphone in the middle aisle.
If you go to the microphone at the question and answer period please remember to state your name and your affiliation and to speak into the microphone slowly and clearly. We have a lot of people on the line who are very interested in what you have to say.
I would also ask the people in the back - I've been informed that there may be some high pitched tone frequencies that are coming from the mics. And if you continue to experience that please let us know. Our intention is not to cause you any more pain than necessary.
All righty. We're now going to be doing - continuing the overview of the new guidance steps on the assurance cases. And our presenter is Mr. Rick Chapman who joined the FDA in 2005.
Currently he's a software engineer in the Office of Science and Engineering Laboratories at the Center for Devices and Radiological Health.
He specializes in software based medical devices, medical devices with embedded software, infusion pumps and new methods for regulatory review of premarket submissions. He's also worked in the Office of Device Evaluation for over two years.
He joins the FDA after working for 30 years in the medical device and other industries such as software and product development and project management.
His medical device industry experience include software development for infusion pumps, implantable defibrillators, heart rate monitors, respiratory test equipment, portable blood gas analyzers, dietary assessment software and wild animal telemetry devices.
He has a Masters Degree in Software Engineering from the University of Minnesota and is currently working on his PhD in Computer Science. I ask you to give a warm welcome to Mr. Rick Chapman.
Rick Chapman: Thank you very much. Can everybody hear me? Can you hear me now? Okay, so I was at a conference yesterday in San Diego and it was for software engineers so I basically gave the same presentation but it had a little software flavor.
So I walked into the conference a little bit late and the speaker that was up there had asked the question are all of you in compliance with the FDA regulations and not a single person raised their hand. And so when I got up to speak I asked for their names and addresses.
So you're all in compliance, right? Okay. I just want to know what my audience is here. So who are regulatory people, non FDA regulatory? Okay. How many are scientists and engineers? How many are FDA? How many want to hear about assurance cases? Oh my god.
Okay, so now I had four (Hospira) people talk to me. Two out in San Diego and two here already, okay? In San Diego they got me in a corner at the end of the presentation and I didn't know I was going to get out of there alive.
So anyway, okay, so how many of you think that assurance cases is going to be a daunting prospect for you? Okay. I hope by the end of this talk that it is something that you realize you do every day. And that if you start from the very basics you can get this thing accomplished.
If you look at the end point which can be a lot of documentation in an assurance case report, you're going to look like you're not going to be able to make it.
But if you start from the basics and we'll go through that, I think you'll realize that this is something you do every day and you just need to write it down.
So the first part of this - I'm going to talk a little bit about the background and then we'll - the last half will be about assurance cases and what we expect from you. So next slide. So we're trying to guide you to the high level safety claims for the infusion pump.
And we're doing this by recommending the assurance case approach and we can really talk about a safety case here because we're really going to focus on the hazards. So we're going to use the assurance case approach to organize and dictate the content of your 510K submission. Next slide.
So the current system, you know, your devices have to be safe and effective. There's a 510K clearance process and there's that thing about substantial equivalents to a predicate device. And as we talk about arguing and logic and all that, this is an argument by analogy.
And unfortunately that can be a logical fallacy. It depends on what you're actually comparing whether it's a valid, logical argument. If you say that my device is red and therefore it's substantially equivalent to this other red device, well red may not be an important factor.
The important things for us at the FDA, you know, by law basically are that your device have to be safe and effective. So that's really what we're - where we're headed here. Next slide.
The challenges that we face are a lack of clear definition of the evidence that you submit and really how to evaluate that.
We have guidance documents, we have standards out there. There are checklists. Checklists usually get into the presence versus the quality of the information. Domain experts - expert opinion, we look at that. So I'm going to use the word evidence a lot, I'll probably overuse it.
But really what I mean by evidence is really your test data, your results from experiments, historical performance, compliance with standards, analyses that you do. This is a really important one is your analyses.
Some people - I'm sure you have scientists and engineers and other people that just analyze a problem and write it down and do calculations and all that. There's no testing involved or anything like that. That's an important piece of information that we may want to see in an assurance case.
And then there's scientific and engineering information from the literature. So when I say evidence it could be any of those things and there are probably other things that can come in. Next slide please. We have - we face poor documentation of requirements and environmental assumptions.
Environmental assumptions is a big deal here. Your device is only going to be safe and effective within a certain space that it's used in. If it's geared for the hospital it may not be safe and effective in the home.
I mean you really have to have your assumptions clear on exactly where this device is safe and effective. And those have to be spelled out and that's actually part of the assurance case.
A lot of people don't do a very good hazard analysis and so we've actually - in our guidance document we've given you a leg up. We have identified from, you know, all the submissions we've had on infusion pumps, literature reviews, engineers talking back and forth from academia.
We have come up with this basic list of hazards that you should look at initially and see if your device actually may have one of those hazards and then you need to address that. And what we're going to talk about is that really the assurance case starts with the hazards.
We have the human factors thing so that's another really important thing and Ron Kaye talked about that. And then above and beyond that if your device - we look at the device right now but eventually we have to look at a system. And so for an infusion pump it's not just the right rate.
And really when you give us the 510K we're looking for the right rate. That you've got - you can deliver the right rate of fluid over time. That's really what we - we can only evaluate. But really it's whether the right drug is in there and whether it's the right person that's getting the drug.
So when the hospital system - that's the system that we really have to start thinking about in the future. You know, systems of systems and that kind of thing. Next slide please.
Another challenge that we face is an incomplete understanding - and you face it too, is an incomplete understanding of the appropriate use of inspection testing and analysis. There's no real overarching theory of coverage that enables coverage to accumulate over multiple verification techniques.
Now I'm a software engineer so a lot of my examples are going to be software. But if - so who knows about software out here? Okay. (Unintelligible). You do static analysis. You do code reviews and you do testing. Really, how do those complement each other?
And even active (unintelligible) can't tell us how those might theoretically complement each other and what kind of emphasis you need to place on each of those methods of testing your software. So that's another challenge that's faced.
On standards - next slide please. Standards themselves do not provide assurance compliance with (MCAN). I'm going to skip most of this and go right to the last line there. Most of the standards we see are process standards. And that is not that useful for us as regulators. Next slide please.
The software standard that is currently out there, IEC 62304 is a process standard that talks about the lifecycle processes used in software development. But there are no requirements for the actual software itself.
So we actually had an internal battle within the FDA between a couple - a few of the software engineers and - because they wanted to recognize that standard and basically if you declared conformance to that standard you wouldn't have to submit any documentation related to your software.
And most of the software people in the FDA were up in arms over that because what's in the document is really what pertains directly to your product. So just conforming to that doesn't get it.
So what we did is we looked at that document and we searched for the words document or analyze, those two words. And really that's - so now we recognize that standard. But you only get by with not having to give a complete description of your software development process.
That's because you've got 62304. But the document still has to be sent to us. And really as far as you're concerned, that's really a photocopy experience, right? Because you're following 62304 and you've generated these documents, they're in your files.
All you need to do is photocopy them and send them in with your submission. Next slide please. IEC 60601-1 third edition - this is another standard. And you look at here like Item B sales space functions, redundancy, diversity, partitioning, functionality. Next slide please. And some more on this one.
Common cause of failure is systematic failures. Next slide please. So here we see that there are some concrete examples of implementation which a third party can observe to assure safe and effective products. So how do we know whether the design implementation decisions that you made were well made?
And well made in the sense of a good technical judgment was applied, there were correct trade-offs made in the design, et cetera. Risk management should tell us this. Next slide, please.
So in a pre-market review, what do the regulators really look for? We're looking at design trade-offs. Are they safe? Who made them and stands by them? Why they were made this way, on down the list. And will they persist in continued use? And you'll - you all can read these slides later. Next slide, please.
So what are these trade-offs that you make every day as a development team? They're essentially requirements, get translated into specifications, standards get translated into controls, manufacturing translates it into a product and then your manufacturing controls are translated into a quality system which tend to comply. Next slide, please.
But did you see any process standards there? Probably not. These things are the decisions made by real engineers every day. You don't have to be perfect. The law says you don't have to be perfect. It just has to be good enough and it has to be transparent to the regulator. Unfortunately there is no process standard for good decisions. So we all face that battle. Next slide, please.
So risk management - I'm going to use a software example here again. So there's different ways in which the software might fail in use. You can have some hazardous situations. Keys are pressed too quickly. You can have metrology errors, logic errors, semantic errors.
Compilers may not catch many of these things and the industry has traditionally relied on the marketplace to inform us of these errors and this - a lot of the recalls are software errors. Unfortunately this is not good enough in the industry any more as you obviously see from the new guidance that we just put out. So next slide.
So how do we avoid these types of problems? Again, this is just to continue the software example. You can do simulation, user interactions. You can do state machine modeling and analysis. We have a lot of research going on on model-based development and model checking.
You can do design for verification. An example of this is the restricted syntax rules. There's MISRA C which is a subset of C that could be used and it prevents you from making some serious - having some serious problems in your software.
You can do static analysis and we've done research on that for the last four years and we highly recommend that. And that's an automated (unintelligible) of the (unintelligible).
And finally you can have experienced engineers and so that's one thing where you can avoid these types of problems. Experienced engineers have seen these things and can solve them. Next slide, please.
Okay, so can auditing help? It can if it's technical. Auditing quality processes won't help because the trade-offs, the ones that we're often sorry that we made, are often underneath the quality management system control layer. Next slide, please.
So how can a manufacturer maximize compliance in a QMS? Okay, you got to recognize that the QMS does not usually expose a bad decision or a bad design, but when overly burdensome, can actually hide that mistake. Just recognize that the quality management process is necessary but not sufficient on its own. Next slide, please.
So how can a manufacturer maximize compliance in a QMS? They can incorporate risk management and processes into their quality management system. And they can prepare design documents which explain why - now we're starting to get into why we want to go the assurance case route here.
You can prepare design documents which explain why choices were made and the way they were made that will explain why the requirements have changed during the development life cycle, because requirements always change. It might even be part of the requirements documentation itself.
Legally marketed devices, again, don't have to be perfect. So you've made these design decisions. Some can be, you know, financial reasons you're picking it, but at least you've got to have a rationale for why you made a design decision. Next slide, please.
Okay, so one possible answer that we've come up with is the safety assurance framework. This has been used since the 1950s. This methodology has been used in the 1950s - since the 1950s. It's been used a lot in Europe in the avionics and defense industries and in the nuclear industries.
In the 1990s, there was a resurgence of academic interest in assurance cases and there's been a lot of work from the academic point of view since then. But it's not a new thing. It was really driven by industry initially so they wouldn't have to give a pile of documentation. You just have to focus on the hazards and go from there. Next slide, please.
So how does a assurance framework relate to the FDA free market processes? This is a little repetitive. So a 510(k) is mostly a checklist. The FDA asserts that we know what we want in the 510(k) and we are actually asserting what we want in our new guidance because we have listed the hazards.
And we want more (unintelligible) factors information. If the sponsor follows that checklist, they can get by with this. But the systems are becoming much more complex over time.
And as they do - next slide, please - and I'm not going to read this slide but as they - basically this just says that as the systems become more complex, I think you might even experience - have experienced that you're getting more AI letters.
You're getting more questions. It's taking longer for you to bring products to market after interaction with the FDA because there is such complexity. The reviewers see a pile of documentation and there's nothing to tie it all together.
And so they come back with an AI letter with questions to try to get you to explain, you know, how you did something or how you mitigated a risk or whatever.
So that's basically where we are now. The things are just so complex that we just don't know how to tie all this together and you're getting AI letters and you're not getting in the marketplace.
And then when we do get you in the marketplace, even though you've given us a stack of documentation, you have evidence that you have great processes, well-written documentation, you still end up in recall. It - this can't go on like this. Okay, next slide, please.
So where do the assurance cases come from? The guy was named (Steven Tullman). I think he's still alive at USC right now. In 1949 he wrote a couple books. They didn't sell very well, but now they're selling actually quite well because litigators and safety engineers and scientists are all using it. Next slide, please. (Unintelligible).
Okay. Now this is a - I can (unintelligible). Can everybody hear me in the back? Okay. I'm going to talk about two systems here - the legal system and the scientific method. And this is why I think if you look at the assurance case for what it really is, you're familiar with the system.
The legal system is a risk-based system. You have reasonable suspicion and this is a temporary loss of liberty. They can only detain you for 30 minutes. Okay, so there's a minor - and they don't have to have much evidence to actually detain you for that 30 minutes.
There's probable cause. And they have to have much more - like if there's - it may have to be actually witnessed by the police officer and then you can be arrested. So there's much more loss of liberty there.
Beyond a reasonable doubt when you're in a court case, this is loss of life or liberty. So this is pretty substantial here. It's got to be beyond a reasonable doubt.
And the final - finally there in the civil court, because we're only talking about money, it's only the preponderance of evidence. The - it only has to be 51%. There's a 49% chance that you're actually right, but because it's only money, it's only 51%.
Then there's the scientific method. And this - we're all familiar with with clinical trials. You generate a hypothesis. You do experiments and you have methods to test the hypothesis. You have statistical measures of the reliability of the data and finally there's a discussion and conclusion.
The discussion is really the argument that you make on why that data and your statistical information about that data, the confidence you have in it, actually supports your (unintelligible) future hypothesis and that's your conclusion and that's your claim in an assurance case. So you're familiar with both of these methods of reasoning about what we do in real life.
And then on a third aspect - next slide (unintelligible) slide. When we talk to each other daily, we're making arguments and we're making a claim about something and we're - we have some evidence. He did that.
We do this all the day. We argue in this framework all the time. There's a claim, there's evidence and then there's an argument that ties that all together to show how the evidence supports that claim.
And the assurance case is a method for reasoning about systems that's appropriate for scientists and engineers and has been very successfully used in the industries that I've talked about in Europe.
And - even in this country, a lot of major operating system companies, like - well, I won't mention any names, but the major operating system companies, of which there is probably one, they have generated safety cases, assurance cases for their products because they are delivering these products to manufacturers in Europe and the Europeans require a safety case. So this is not that new a thing. Next slide.
The term assurance case is generalized. There's a safety case. You can have a compliance case. You can have an effectiveness case. You could actually pull that together. But again, on the effectiveness side, we more often use the scientific method which is really pretty much the same thing. And then you can have a business case. Next slide.
So what is a safety case? Now we're going to get into the meat of this thing. It's a structured argument. It's supported by a body of evidence - so that's your data, your analyses, your test reports - that provides a compelling, comprehensible and valid case that a system is safe for a given application in a given environment.
And that given environment is assumptions you make about that environment, that this thing is going to operate in this environment. We don't get that a lot, but you need to realize that out - if the device is used outside of that environment that you specify, it may not be safe or effective.
Okay and then the argument part of this is commensurate with the potential risk and the systems complexity. So if it's a low-risk thing and it's not going to lead to, you know, injury or death, you may not have to make much of an argument.
You may not have as much supporting evidence - or data to support that claim. So it's very risk-based system and that's what we want to see. So the high-risk stuff gets the most emphasis. Next slide.
This is probably the most important slide you need to take away from here. Again, if you jump to the end, it's going to be a very daunting process for you. But you need to do the hazard analysis and you start with the hazards. So you got to go hazard by hazard here.
The claim that you make - your top level claim is that you have a safe system and then you jump down to the sub-claim which is that I have mitigated this particular hazard or the hazardous situation.
From that, you determine the properties of your system that will actually achieve that goal of mitigation and then you generate your safety requirements from that.
And there's a very good description of this by Chuck Weinstock and John Goodenough from the SEI and you can download this off the Internet. It's towards an assurance case practice for medical devices and I have a reference on the last slide to this.
They talk about this structure of the assurance case and they use infusion prompts as specific examples in here. So this is something you all want to read. So it's something to actually get started on. Next slide, please.
Okay, so we'll talk a little bit about evidence again. And these are qualities of the evidence that our reviewers are going to be trained to look at and so you need to consider these qualities, these attributes of your evidence when you're generating it.
There's requirements validation and this is just a demonstration that the set of safety requirements is complete and accurate. You'll never know if they're complete. That's an unsolvable problem, but they can certainly be accurate. You hope they're as complete as humanly possible to find out, to discover.
Requirement satisfaction - this is a demonstration that all safety requirements have been met. And then there's requirements traceability, demonstration that all safety requirements have been tracked throughout all stages of systems development and safety analysis.
One way to look at an assurance case is that it's a traceability matrix on steroids. You're going to have the argument part in the traceability matrix that you normally have. So if you think about a traceability matrix, we're getting really close to what an assurance case is going to look like, except there's a little bit more interconnection in the actual case. Next slide.
Okay, at the system level, evidence must be provided for each of these categories. Now, so that was requirements validation, satisfaction and traceability.
If the set of identified system safety requirements can be shown to be valid, satisfied and traceable, then it can be argued that the system is acceptably safe.
If you do that with the requirements and you started with the hazards to generate those requirements, you can't just come up with a bunch of safety requirements because of you need to link it back to the hazards to know that you've mitigated the hazard.
But if you do that, by definition and acceptable to us at FDA, you have a safe system, okay? And again, I just reiterate what we consider evidence to be. Next slide.
Now there's a suitability of evidence. This is assurance of a requirement of the evidence. So the evidence has to be relevant, it has to be trustworthy and it has to be independent. Next slide.
As far as relevance goes, the directness of that evidence - so for direct evidence you might have timing data. Indirect is the confidence of your personnel. It's not directly related to the relevance of the evidence but it plays a small part.
Coverage - this is the proportion of the requirement for evidence which the (unintelligible) the evidence addresses. So for instance, if you do testing, what percentage of your code did you actually test with those - with that test plan? So you might have only tested 25%. So a thorough test from a static analysis point of view, if we're talking about software.
A technique which provides evidence of the handling of all runtime exceptions, which most (unintelligible) tools will do for you, is a - would be a thorough coverage - would provide thorough coverage and a less thorough coverage would be if there was just a technique that showed you where there was a divide by zero. So that's one of the runtime errors that's possible, but - so it's not a complete coverage of all the runtime errors.
The suitability of evidence - next slide - we get to the trustworthiness and this is where the process comes in. It's the perceived ability to rely on the character ability, strength or truth of the evidence. The trustworthiness of evidence is an expression of the process evidence related to generating the evidence.
So - and these factors might be the buggyness, the level of review that you've done for tool-derived evidence. I's the tool, qualification and assurance that you've done, the validation of your tools and it's the experience and confidence of the - of your personnel. Next slide.
And finally we get to the independence part of your evidence. This is the extent to which complementary items of evidence follow diverse approaches in fulfilling the requirement for evidence. So next slide.
These are some support patterns for evidence. This is a single support pattern. Now if I were doing - if you do static analysis with a - one of the modern static analysis tools, you can claim that you've mitigated a large number of the software hazards immediately if you use that tool.
So that would be almost a direct one-to-one static analysis tool. You've mitigated those risks if you have eliminated those - or changed the code to eliminate those problems. Next slide.
Then you might have a situation where you have more than one piece of evidence that would support a claim. So this might be that you do code reviews. You might have incomplete test coverage on your - from your test plan.
All of these are now together supporting that one mitigation of the - the one claim that maybe you've mitigated that you have defects in your code, okay? So that's a support pattern of (unintelligible) support items. Next slide.
And here's the convergent support pattern where you have different pieces of evidence that directly support the claim. This could be different methods - well, again, static analysis is one way. It directly supports this.
If you do code reviews and you're looking for potential runtime errors and buffer overflows and divide by zeros, that's a separate, independent means of getting that information. Next slide.
I'm going to go quickly through this, but this is just a little bit about argumentation and we're not using the same verbiage here as we're going to use in the assurance cases so I'm going to just leave that one there. Next slide.
There's different methods of arguing and you need to be aware of the logical fallacies that you may get. And there's some papers out there. Another good source for assurance cases is John Knight's publication site at the University of Virginia.
And he and one of his students put out a paper on fallacies used in generating assurance cases. And they took an assurance case from the avionics industry in Europe that was published and they discovered the different fallacies that the - they had two people independently rate the arguments and whether there was valid logic or not.
And you can have argument by analogy. Again, if you're not comparing the right things, it can be a fallacy. You can draw the wrong conclusion from your argument and you can have omission of key test data.
You could be arguing one thing and then claim something completely - I think it's a red herring if you argue this and then you claim that. So our reviewers are going to be trained and looking for these fallacious arguments so you should be aware of that also. Next slide.
As far as arguments go, the strongest arguments are both valid and sound. A valid argument is if the premises are true. That's your data and your evidence and your argument and your - so if that's true, then your conclusion is true. A sound argument is an argument which is valid and has true premises.
The weaker argument is a consistent argument. If the premises are true, the conclusion may be true and it's true with some probability. Most of your arguments will be consistent arguments, okay? So there's some probability that what you tell us may not be true.
And that's very reminiscent of the scientific method. You're not, you know, you're at the, you know, 95% confidence level that the hypothesis is true. So there's a little bit of probability that you're concluding the wrong thing. And we know that, we recognize that. Don't tell us it's going to be 100% when we know it's not going to be 100%. Okay, so next slide.
So when we see a safety argument from you, the worthy cause should be just riddled throughout your report. The - this infusion pump is safe because. The safety requirements are defined in my. The safety requirements are met through this.
Safety management continues to be adequate because we have these things. So that's what - that's kind of verbiage you want to use. We want to know your - why you made your design decision. Next slide.
The format of the assurance case - it can be a simple narrative. It can be a tabular thing. You can - you could actually start with a traceability matrix that's geared towards looking at the hazards and generate your evidence.
Add pointers to your evidence which is your documents that you normally submit to us and then you can have another column where you actually argue the case that that evidence supports your claim.
And then there's a graphical type and there are several tools out there. At this point, all of these formats are acceptable to us, okay? We don't care which format you send them in.
There are tools out there, like the (Addilar) tool which gives you pretty pictures. It provides linking to all your documents. It's a very, very nice tool. There's other tools that you can get, Visio templates to generate the - a graphical assurance case.
If you happen to come in with an assurance case and it's using one of these tools, we're kind of committed then to work with you to see if you can output one of these other methods, either a tabular or a narrative output from that tool.
Or we might actually be willing to get a (unintelligible) of that tool and - if we think it's going to be in common use and actually use that to review your assurance case. So the tools and the format is pretty much freeform right now. We'll work with anything right now. Next slide.
This is just a picture of a graphical representation. You see on the upper right that you have some top-level claim. You work your way down. You have an argument in between, pieces of evidence that support that claim.
The next three slides, so next slide, is just some things that as you get into the assurance case, you'll realize that it's a very logical, structured thing and that if it doesn't seem logical and it's not really well-structured, like you have some nodes - you have some things that are sitting out here and they're not really connected to anything, then you probably have a problem and you need - maybe there's some sub-claims in there that you need to make to bring it back up to the top-level claim.
So the structure of it is a benefit. It lets you know that you're on the right track, especially if you start from the basics. If you start with the hazards, do the basic things. Start with one hazard. Don't look at all 60 hazards that we have listed and the 30 other ones that you found in your hazard analysis. Take one and work your way through.
And a lot of these hazards can, you know, some of your test methods may actually address and mitigate several of those hazards so you need to play that by ear and so you'll have links cross-linking some of your evidence to several claims, okay? Next slide.
So this is just to go on it. This is just some academic stuff and as you get into it and you - I'm sure you're going to have software engineers that get into this. They'll realize that this thing can be proved, that you have a logical structure. It can be recursed. You can find things on it. Next slide.
The safety case can be exported and it can be displayed graphically to support the big picture. Okay, next slide.
The thing we want to see from you then is a safety case report. You have this - the safety case is really everything you've done to mitigate your hazards. That's your safety case.
The safety case report is not a stack of documents 15 feet high. It's a document about, you know, a few pages, maybe up to 100 pages, where you lay out your case and you point to the pieces of evidence that we can then either choose to or not look at.
So if you make a claim and you have pointers to this evidence and then you have an argument that ties all this together and you convince us, we're done. That's a simple thing. And so we think it's going to be easier on you. It's certainly going to be easier on us and that means you're going to get in the marketplace sooner.
And it's going to be easy to write an AI letter because the argument wasn't - it wasn't a valid, logical argument or you'll see that you - you'll see that you are missing pieces to your whole development process. Next slide.
Let's go back - can I go back one slide? Okay, so a safety case report is really a snapshot of the rationale and content of a safety case at an appropriate milestone. So you need to generate this safety case. You can backfill it, but it's better if you start from the beginning because you'll have the structure of that safety case.
Again, you've done your hazard analysis. You've started with the hazards and it's worked its way down. You may have a test plan in the evidence spot. That'll be replaced at the end by the test results with an argument then of why you think it supports your claim of mitigating that hazard.
And really that safety case report will show that, again, as you move from the hazards to the safety properties of your system to the safety requirements, it'll really indicate that you've met the safety requirements that you've outlined. Next slide.
This is reviewable against the project expectations at milestones. You may expect to have certain things done so this is kind of a project tracking thing. You may need several report types for various stakeholders. So we'll want to see the one near the end, just, you know, as you submit the 510(k). And it may need several updates over time as you're doing your development.
It's okay, especially right now, if you are in an early stage to, you know, come in with a pre-IDE where you can ask questions about your safety case and whether we think it's a valid one that we can work with. Okay next slide.
We're almost done here so. So who reviews this report? Well, you, the manufacturer, will review the report. You can have a third party come in and review this.
And finally the regulator, us, will review it. And we already know what the claims are. I mean we just published a bunch of those. We can see the evidence any time that we want. We - we're not, you know, we don't have to see that evidence.
You might have made a good argument that you've mitigated that - like if you - for instance, static analysis. If you state you've done static analysis and you used one of these tools, we pretty much will figure that you have mitigated a number of software hazards right there.
And we're not going to go any further than that. It's just that simple. We may not even have to see the report from the tool at that point. But you should plan for multiple reports as you go through development.
Okay, so I - that pretty much wraps it up. The next slide is a link to this. Next slide.
Okay the - this just a link to this report from SEI and that's it. I'm sure you'll have questions.
Melissa Eackle: Thank you, Rick. All right, I'm going to ask the FDAers from yesterday and today to return to the (unintelligible). And Mark Barnett, who you met yesterday, is going to be joining us as our moderator. You can - Mark (unintelligible). (Unintelligible) repeat of yesterday.
Melissa Eackle: Hopefully. Okay. All righty, before we start, I want to put some, I guess caveats here. We have received this morning over 100 questions. And in light of - and we're pleased as punch that we have. However, unless we're planning to be here through the weekend, they probably are not going to all get answered.
What I will reassure you is that if your question that you submitted does not get answered, we are going to make every effort to get those questions and the answers put up on the infusion pump Web site.
Now it will not be tomorrow. It may not even be by Tuesday when the webinar - because we have a holiday weekend and it's hard to believe but federal employees actually do take weekends now. And so we will get those questions and those answers up.
So I don't want you to feel that we're going to ignore them or we're not going to answer them. We will be answering them but - in 100 and we are sure there are people who are going to stand up also to ask questions. It's just not possible to get to them all, okay?
Once again, we're going to try to use our handheld mic. We've been told they've been fixed and so we all should not be blown away. But I will tell you if they do what they did yesterday, we will not be using the handheld mics.
For those of you who come to the center aisle to ask questions, please remember to speak clearly and slowly, to put your name and your organization for the people who are on the line and also for the rest of the participants.
And I am now going to turn this session over to Mark Barnett, who you know is our moderator. Thank you very much.
Mark Barnett: (Johnny) would have remembered that. Anyway (unintelligible). All right. I - yes, we're on. All right. How we doing now? All right. I'm going to read these just in the order they came in.
I was tempted to try to prioritize them or to eliminate the ones that seem to be somewhat redundant but, you know, what's not important or important to me is not the same as what's important or not important to the person who answered it - who asked the question.
So I'm just going to get through as many as I can without any kind of editing. Okay, so here we go. Oh, well, let's do a live one first. Save your feet. Go ahead.
Robert Bard: Good morning. My name is Robert Bard. I'm the Managing (unintelligible) for HealthCare Technologies Consultants. And question is sort of across the board. So it's - I did talk to (Unintelligible) before and I'll try and ask this question as some - from the discussion on human factors, some simulated use and a pre, I'm sorry, the clinical evaluation.
Can we talk about it in terms of home care-specific infusion devices? When I listened to the information, I had a hard time trying to grasp how somebody gathers the information that the agency's looking for.
I had a hard time understanding the number of patients that we use. How we would be looking over somebody's shoulder that's in the privacy of their own home. A host of other things. And if you take a second if we could talk about infusion pumps in home care.
A lot of times simplicity is the order of magnitude which can be a simple elastomeric or mechanical, non-electromechanical device to the complexity of a insulin pump to a PCA pump - a host of things, all ambulatory, all taken on the road with them.
And I guess I'm just trying to figure out when you're looking for patients, for the clinical evaluation, I assume you're talking about doing them in the home setting.
Are we talking about a one-day study? Are we talking about a six-month study? Are we talking about 15 patients? Are we talking about 10,000 patients?
At one point in time I heard that maybe this is a statistical analysis. Well, I can't for the life of me figure out how you'd get a statistical analysis for an undefined term without doing just tons and tons of patients. Maybe I could listen and see if I could clarify it. Could we start with maybe (Richard)? I mean, Ron?
Man: That's this one? Oh, it is on, okay. I agree with you that's a - that's quite a challenge. As far as the statistical analyses and the way that would bring about a - some idea of a minimum size for your sample, that is - it is difficult question because of the way we're going about the evaluation for human factors specifically.
And I don't think - I don't want to speak for the other disciplines because there, you know, there may be some interest in other aspects of the device performance.
But with respect to human factors, again the - and admittedly we're obviously new to this since it's a new program. So the best answer I can give now is that we're going to have to think about that carefully and we are open to input and suggestions about how extensive such an evaluation would need to be.
Now, as far as collecting data in the home environment, yes, that would be a necessity in this case because, you know, we need data to demonstrate and support the assurance case argument that the device, in this case is safe and effective for those users. How you'd go about that - well, of course, yes, you can't hover around somebody 24 hours a day as they use the device.
It would be - need to be some sort of a mechanism for sampling that, perhaps including the home user, you know, the - who volunteered to participate in the study, filling out some sort of a record of their interaction with the device or perhaps it might be a family member that could do that better as one source of data.
There may be ways of logging the interactions with the device and looking at them better - later and perhaps comparing the two sources of data. But it is a challenge and it is something that I think lends itself to creative and innovative approaches for doing this type of evaluation. And I think that's about the best I can answer your question right now.
Robert Bard: Back to the (unintelligible). If we were looking at submitting in the near term, say anywhere from maybe within the next month to three months, are you prepared for submissions under this guideline or are we talking right now and not doing? What is the plan?
Alan Stevens: Hi. Alan Stevens. So, you know, we are in a draft period. This does represent our thinking on infusion pumps and how we need to be reviewing infusion pumps.
Certainly we're evaluating submissions on a case-by-case basis as they come in. And if you're looking at something such as a clinical evaluation and you're conducting that, I would recommend that you contact us as soon as possible if you're thinking about conducting such a study to get our feedback.
Robert Bard: Before submitting - not as a pre-IDE but just for a starting discussion point, is that what you're saying?
Alan Stevens: Yes, you can call me, or email me. The email, you know.
Robert Bard: Okay, I got your number. All right. Thank you.
Alan Stevens: I (unintelligible) better.
Mark Barnett: Go ahead. So we're take another live one. Let me just mention to our FDA panelists, to keep your answers as short as you can while still covering the topic so that we can at least get through as many of these 100 and some questions as we can. Okay.
Andrew Sealfon: Hi. I'm Andy Sealfon with RMS Medical Products. We're a domestic company that designs, engineers, manufactures and produces medical devices in USA, last of a dying breed.
I had a question. I - first I wanted to thank all of you for presenting all of this to us. This has been enormously helpful and I know it's quite a - an undertaking. So I'm sure everybody here as an industry appreciates the effort.
My question relates to something that was covered yesterday and today as well and gets back to the question of the broad brush stroke - approach to regulating pumps.
And I was going to try to jump the discussion up to more of a global look at it, a view from 20,000 feet. From the FDA perspective - and there's not much guidance in the guidance document on some of these questions I'm going to try to go down - what is needed or wanted or desired in an infusion pump?
And what I heard as an engineer was a wide range of different requirements for an infusion pump. The anesthesiologist who spoke yesterday needed a fairly complex pump in the operating room to deal with the multi-drugs that he had to handle and to look up references in the drug library.
And yet many of the nurses who spoke said they wanted something very simple. And we heard from home care people and talked about the '57 Chevy. Now clearly as the complexity of a pump increases, your error rates are going to increase, all other things being equal.
By the same token, your failure rate is going to increase proportionate to the number of components that are in your system. So as we make pumps capable of doing multi-functions, doing high versatility, we're going to naturally increase our problems. And again, there's not much guidance on these subjects.
If I could just break this down a little bit further. I broke it down into three quick areas which was the basic pump mechanism itself, the part that does the pumping.
For example, what is the maximum pressure, safe pressure that the human body can take in an infusion? Pumps, if they have to do intrathecal or epidurals, need fairly high pressure to drive fluids through small tubes. But if you're doing an antibiotic, for example, you would not maybe want that kind of pressure.
So if you make one pump fits all - again, that's a broad brush that we talked about yesterday - that's going to create the potential for problems to appear. If you make a pump that can go from (unintelligible) ml per hour to 1000 ml per hour, you'll likely have an order of magnitude error somewhere along the line and that puts a lot of pressure on the human factors that we heard about today.
So my question is how will we - in industry get guidance on such things as pressure, accuracy? What accuracy is needed? What does accuracy mean? Today we heard it meant rate of flow.
In some cases it means rate of flow. In some cases people talk about time. Some people talk about volume. Are they the same? What really matters? What - should we all be quoting the same thing when we make a claim about accuracy.
When we're talking about things like the complexity, the drug library, how important is that? For the anesthesiologist, very important. For the routine antibiotic - so what percentage of all infusions, for example, are in anesthesiology versus antibiotics?
My last reading was about 70% - or 65%, 70% of all infusions are antibiotics. How important is the drug library and all the complexity associated with that? Would we - because when you're talking about the stewardship program - the right antibiotic for the right patient, right route of administration - those of you who are familiar with that can appreciate this.
You're going to need pharmacy. You're going to need a lot of other involvement. All of that information is not going to be in this drug library of a pump.
So what guidance could you give small manufacturers in designing equipment for the market? Do you want to see more complexity? Is that a requirement or is simplicity better and would be reviewed better? What do we do to get more reliable pumps into the marketplace?
Mark Barnett: You know, I reminded our FDA panelists to keep their answers short. I'm going to give the same reminder to the people questioning. We want to - we want to hear from as many people as possible. So keep them brief. Let's have a brief answer from the FDA. If you - which question do you want to answer? There were 12 of them?
Man: I'll start and I'll be very brief. It starts with what you say your device is going to do. What are your indications for use? I don't think we - I don't - I hope we didn't say we need a one size fits all because that's not what we're asking for.
We're saying what is it you want your device to do? What is the operational environment and then prove it to us that it works. That's the simple answer. How it's done, we're not going to dictate. There are standards. We're working on getting better standards. You heard that from (Amy) and I think the people here are committed to do that.
So those questions about - I don't know if that'll be specifically addressed in any future standard about body, you know, the pressure for the body. I don't know if that'll be addressed specifically.
But there is already something about dose accuracy in (12-24). I think that's it, right, (2-24), the (IC) standard. There's something about that - dose accuracy. The rest of it still has to be worked out though I think. I'm done. I hope that was brief enough. I'll let someone else speak up.
Woman: Can you hear me? Okay, good. Well, I think that - I think that you're right. I think that there potentially needs to be different types of pumps for different environments of use.
As of right now, for external infusion pumps we have that. We have elastomeric pumps because you, the manufacturers, deemed it a necessary - a - you're fulfilling a need.
You saw that there needed to be a disposable, simple-to-use, easy pump that can, you know, be filled and drugs that are stable can stay, you know, shipped out to the patient's family for up to a week at a time. You've already identified that. You have those on the market.
You've identified that you need pumps that can do continuous infusion. So I think what we've, you know, and intervals and so forth. So we do have various pumps on the market.
My challenge back to you, manufacturers, is besides obviously trying to get, you know, the best quality pumps on the market is to look to see what your market is. What is it that you need? This is your business. Who are your patients? What are their needs? I think it starts there
And I think we heard yesterday and I think we've heard from the stakeholder calls that we have patients who do want simple pumps, that do not need the bells and whistles. They want something that's reliable and durable. And there yet we heard yesterday that we need pumps that are incredibly sophisticated for our very complex patients, so it's the whole gamut.
Mark Barnett: Thank you. While we're on the topic of all pumps are not created equal, I noticed that there were a couple of questions deep down in the pile from people who wanted to know about pumps for enteral feeding.
And the theme of those questions seem to be that those pumps have different characteristics and would they be subject to the same requirements as all other infusion pumps? So I thought I might as well mention those now. Anybody want to comment on that?
Melissa Eackle: As a clinician - and all pumps can kill you and very often they're used in - and I say that - that's not tongue in cheek. That's absolutely true. If there is a flow problem with an enteral pump and you have an over-delivery, it can cause a death. By the same token, if someone is extremely dependent on that feeding in a consistent manner, you can have as serious consequences.
So I guess my message is - and I'm not my (ODE) brethren - but knowing them a bit, is these are critical devices and that you would expect the same kind of behavior, performance and expectations of an enteral pump that you would of anyone - any other.
Alan Stevens: Yes. This is Alan Stevens. Just to tag team on what Melissa said, you know, we go back to the analysis we did of the recalls and adverse events. That included enteral pumps.
We have seen problems, maybe not to the scope of, you know, say, some of the other (unintelligible) pumps and maybe some of that's due to the widespread use of some of the other pumps and, you know, some of the factors there. But the guidance document does include enteral pumps and so our expectation is that the evaluation methods that we're putting forth are going to apply.
Melissa Eackle: Can I just finish by saying, I was an analyst and I will tell you that as a clinician I had no idea what people did with enteral pumps in the home and I found out very rapidly and in other countries.
So when you're doing human factors and (unintelligible) this is to you, you really need to know the environment. How are they going to store and what temperature is that fluid? What is the fluid? What the temperature is and, gosh darn, what possible in the home can they be adding to that because I've seen any number of additives to the tube feedings.
And the manufacturer's like, well, who would have expected, you know, that that would have happened? And what we're saying to you is you got to expect it because in many cases that's just what people do when they get it in the home.
Mark Barnett: Okay, thank you. A couple of the questions here had to do with software recommendations. Let me read you two of them quickly. One says, "What tool can FDA recommend for software static analysis."
And the other one says, "Any guidance from the FDA on using middleware software in operating systems for infusion pumps, e.g., Linux, (unintelligible), Java, Android. How do assurance cases apply to software and hardware components, static analysis for middleware?" Anybody want to jump in on that?
Mark Barnett: Okay.
Rick Chapman: Okay, for the static analysis tools, we don't really care which one you use. In our lab we have PolySpace for MathWorks, CodeSonar from GrammaTech. We have Coverity's tool. We have Parasoft. We have Clockwork and I think we're getting or we may actually have a company that puts out a product called ( QC++) or something like that.
Any of those tools are fine. If you come up with another one that's good - now each of them have their own strengths and weaknesses. At this point, you're so far ahead of the game by using any static analysis tool that they're all acceptable.
Kevin Durand: Hi, my name's Kevin Durand. I'm an engineer for DEKA Research and Development. My question is for Richard I believe. Regarding the assurance case, so when you were telling us who's going to see it, who's going to review it, at the end the review is within the agency.
Were you saying that we could submit with a work in progress on the assurance case and with the expectation that when you are on the market it is complete and buttoned down? Or were you saying - were you saying it needed to be completely finalized before we submitted it?
Rick Chapman: Right now and this could change over time, right now we would recommend that you come in with a pre-IDE showing us the structure of your assurance case. For your 510(k) submission, it would be a complete assurance case with test results and all.
Kevin Durand: Okay.
Rick Chapman: But definitely right now you can come in early and we'll take a look at it.
Kevin Durand: (Unintelligible). Thank you.
(Natalie Morgan): Hi. My name is (Natalie Morgan) and I am here from the law firm of (Trusiere, Durand and Husier). It's a firm in Houston, Texas. And I'm also here on behalf of the American Association of Justice Insulin Pumps Litigation Group. And along with our co-counsel law firms, we represent a number of patients who use the insulin infusion pumps or were using those pumps.
And they have - a number of our clients have all reported to us that, even though they're using these pumps correctly, which I know the FDA's aware of this, but even though they're using these pumps correctly as they are trained and as they understand the instructions, there have still been unexpected and severe blood glucose levels high and low.
And so because of their traumatic and emotional stories, we're here today as their advocates. And during the past two days and all the presentations, I've - one thing that's become clear is that the insulin infusion pumps are different and have different qualities than some of the other infusion pumps in that they are at home.
Their dosage has to be specific and at a specific time or it can result in serious injury and death. And it is not used in a clinical setting so it's a challenging and complex environment.
And so with that in mind, my question is does the FDA consider a patient using an insulin pump at home potentially equivalent to an insulin pump that is being used by a trained professional in the clinical setting? And would the FDA ever consider imposing different and more stringent standards on the insulin pumps that are being used at home?
Man: (Unintelligible) answer that question. I guess I will start off. Well, I think the insulin pump user population definitely poses challenges as you mentioned. And the question earlier about home use gets right to that.
Obviously the parameters around evaluation for home use patients are going to be different than those in the clinical setting but, nonetheless, equally valid. So we have to figure out a way to do this. I mean we can't say, well, the home users are just - they're out there. We just can't worry about them. Obviously we got to figure out a way to do something about that.
So I think we definitely know they're - they have a different challenge. I think that's probably true and I'd have to leave this up to the commission to answer - I think that's probably true for a lot of the different segmentations of pumps.
Enteral pump patients are probably different than insulin pump patients are probably different than, you know, the folks you'd see using large volume infusion pumps. So I think they all have their own challenges. We do consider them to be unique in a number of ways because they are primarily home users and primarily lay users of the pumps.
But I think a lot of the - a lot of the information that we're requiring is still valid, even for insulin pump users. The challenge is going to be how do you assess those folks compared to the other people? And I'd love to have someone else chime in here as well. So, you know, both (Marion) and Alan have (unintelligible).
Woman: Can you hear me now? Yes, good. Yes, I definitely think that there are challenges with the insulin pumps in the home setting. And we do know that those insulin pumps - we did have an insulin pump panel, or excuse me, workshop about three months ago looking at the recall issues with insulin pumps.
Most of the - you were trying to compare patients in the hospital setting receiving insulin as compared to the home setting. And as a general rule, clinically it's a different patient in the sense that the pumps are - generally they're trying to stabilize the patient in the hospital setting and they're not using the home-based insulin pumps for that.
They're generally using the larger volume bags of insulin and they're titrating up and down. So it is a little bit different of a scenario - clinical scenario, home versus hospital setting.
Man: Yes, I just want to, you know, you asked a question about how we evaluate substantial equivalents. You know when we look at insulin pumps, the ambulatory insulin pumps, you know, in my presentation I noted the separate product (unintelligible) that we have.
And one of those is specifically for insulin pumps. So when we're looking at substantial equivalents, we're not comparing an insulin pump for home use to a hospital pump. It's compared to another insulin pump that's already been cleared for that use.
(Natalie Morgan): Okay and I have one more question. What does the FDA foresee that it's response will be if and when the existing insulin pumps or new revisions of the pump continue to exhibit the adverse events of death and serious injury?
Man: Could you repeat that again?
(Natalie Morgan): Yes. What would be - or what would the FDA foresee as their response if and when the existing insulin pumps or the new revisions of these devices continue to have reported incidents of serious injury and death?
Man: Well, I think - I was going to talk to this later. So our goal is to have better pumps and better is a goal that never kind of - you never reach, right? You're always trying to get better.
So we're hoping and we believe that we will see an improvement. If we don't, we have to re-assess our strategy. I think that's what you do. When you make a plan and it doesn't work out the way you hoped, you try to reassess and see if it works.
And I truly believe and, you know, I'm - I may be the only one who's naive enough to believe this, but I truly believe it will be better. And if it doesn't, we'll have to reassess our plan. Now I don't know, you know, maybe five, ten years from now, but hopefully sooner.
(Natalie Morgan): Thank you.
Woman: And from - can I just answer this from the compliance standpoint, is that as we receive reports about an increase in adverse events with the insulin infusion pumps, then the office of compliance will become vigilant and make sure, because that's one of our ways that we can determine whether we have to inspect a facility is if we see increase in either adverse events.
Then we go in and inspect to find out what is going on with the firm, so that would definitely trigger an inspection.
(Natalie Morgan): Thank you.
Woman: We got one more person here.
Al Taylor: Al Taylor. And, you know, over the last year, we got a new president - a year and a half now - a new secretary of HHS, a new commissioner, new center director and most of us have new office directors. So in my case, in a matter of months everyone in my management chain all the way up to the top changed. And that was a lot of new people to educate.
Early on when the Infusion Pump Working Group was talking with some of our senior management, one of the senior managers looked at me from the lab and said, "Well if industry can't design an infusion pump that works, why don't we do it?"
And, you know, I don't think that's the problem. I think that the industry is designing good infusion pumps. These are mature products, we have a lot of experience out there and I don't think - the last thing we want is for the Government to be designing our medical devices, right? We have a system that works in this country and in this world because it's a global economy now.
So we don't want an FDA to - the questions are being raised here this morning, these are the difficult questions for all of us and the answers are coming from you all, not from us.
And we're doing our part to provide a second opinion after you've done your own design review and decided we have the product that is going to fill a gap in the market. We're going to provide a second opinion that you've done the things that you need to do and that's really - it's really not much more than that.
Coordinator: Okay, thank you. Let's have another live one.
David Vogel: Yes, thank you. My name's David Vogel from Intertech Engineering. We're a - an independent service provider that provides verification, validation activities primarily with software and a lot to do with infusion pumps.
My question is about static analysis and a concern I have, it seems that static analysis is becoming a cure all. And in particular, (Mr. Chapman)'s presentation, several times, when we referred to hazard analysis and the current state for the software hazards you said that if you used static analysis tools you may be almost done.
And yet when I look at the software hazard tape on the guidance, I don't see the static analysis tool would really address even half of the hazards that are listed. So could you help me connect the dots and understand a little bit better your comments?
Rick Chapman: I think I said some of the hazards, software hazards would be mitigated by using a static analysis tools, but not all. It really, your software, you're looking at the requirements all the way to the implementation of something that meets those requirements, so static analysis tool can only address some of the problems in the implementation part.
You still have other types of problems in that implementation and then you have that whole requirements aspect of the software. So - but it does address the run time errors and that kind of thing.
David Vogel: Would it be fair to say that it's most addresses the causes for some of the hazards?
Rick Chapman: Well yes, it's the cause is because you have a line of codes that has...
David Vogel: (Unintelligible).
Rick Chapman: ...has an error in it, yes. So you'll change that line of codes. That would be the cause, yes.
David Vogel: All right.
Coordinator: Okay, one more.
(Sid Desai): Hi, I'm (Sid Desai). I'm with ISO Corporation. I'm an engineer by heart. Well we make a lot of money from - and we had a chance to discuss this with Mr. Watson a little bit. But it seems like the guidance document is kind of a broad page.
And what we are hearing is that there are different kinds of pumps and the requirements or the details may be different for different types of pumps such as an (unintelligible) pump. So is there a reception on part of FDA to modify the document so that the different categories and different requirements can be expressed in some different way? Thank you.
Al Taylor: (Unintelligible) get to the microphone. I guess what I will say is that comments, provide your comments and we will consider any reasonable arrangement of the guidance document.
I don't think we were - our intention though was ever to segment the different pumps out and the requirements for the pumps. We more took what we had seen across the industry and said these are the kind of problems that we see.
These are the things that we think we want to see in the submissions to help address those and we'd like them in this format. And we think these are the types of things that will help us ensure that the things that you think are important to those particular pumps have actually been implemented properly.
So, but, you know, as far as rearrangement of the guidance document, we're receptive to that, I think the comments would need to be clear in what you think it ought to be and an example of that would be helpful.
Coordinator: Thank you. Yes, please.
Anna Nowobilski-Vasilios: Thank you very much for this venue to ask questions. After having reviewed the guidance, there certainly are a lot of points that we can...
Coordinator: Can you identify yourself?
Anna Nowobilski-Vasilios: Pardon?
Coordinator: Did you identify yourself?
Anna Nowobilski-Vasilios: Oh, I am sorry. My name is Anna Nowobilski-Vasilios, I'm a clinical pharmacist currently working as an independent healthcare consultant and I hope to bring a question from the clinical perspective that's specifically related to drug stability and compatibility.
In reviewing the guidance, my question is, actually I have three - a three part question. What is the value of requiring drug stability compatibility of just one drug for each product, or one drug for each root of administration, will that perhaps cause confusion on the side of the clinician?
The second part of this question is what role do current published clinical reference books related to drug stability, which include diluent concentrations, selective medical plastics and selected infusion containers have in this guidance?
And my third question is if duplicative stability and compatibly studies are required, what will the economic impacts be? What studies could have been done for that money that would have maybe made a better impact in patient care?
I do want to say that this is always been the purview of the patient care team, which includes at the patient setting, be it in the home care environment or in the hospital environment, the pharmacist, the nurse and the physician caring for this patient. Thank you.
Woman: Before you start, as a clinician myself and also as an analyst and a network leader, I'd like to address at least one point that you brought up, which is a very important point. There is a lot of information about our medication that is available to us.
What we have found to be true, at least as an analyst - and this is anecdotal because it was at my desk and talked among the team - is very often there are approved medical grade plastics, but other things change.
So there might be a very slight change and no one thinks it's big enough or close enough to the edge of the specification to make a difference and it makes a difference. Or your supplier changes something and they don't think it's a big enough change so you - it's within the agreement and so you don't know until afterwards.
And maybe your drug company comes and tells you, you know, there's leaching, it's turning, when we use it with your device, we're getting black spots. And so that's all well and good, but at the end there's a patient attached who may be getting those leaches may not get the appropriate dose, there may be other problems.
So one of the things we talked about in the group - and I'm - I've been in (Swiss Group) since the beginning - is that is a very important thing, because in the end the patient is the person who pays the price for what we don't know. And so with that I'm going to turn it over to Alan.
Alan Stevens: (Unintelligible).
Anna Nowobilski-Vasilios: However, as a plastics practitioner, most of that information is already available. And perhaps what is necessary - and I'm just speaking outside of the box - is that we know exactly what the plastics are that are used within the product, rather than putting the onus on the manufacturer for something that's in the drug labeling.
Woman: You don't always know what's in your components and that's the truth.
Alan Stevens: So I'll try to address your questions. With respect to the value of testing one drug, you know, we started this - we've been asking for this, even prior to the guidance, for the past two or three years we've been asking for this information.
And this is generated out of some 50 alerts that we're provided and we were also having conversations prior to safety alerts going out and we felt like we needed to ask for this.
Since that time and we've been getting this information we've actually identified submission where the materials were actually adulterating the drug products. So it's not as straight forward - from my perspective - on the review side as saying, "Well, we know something about the plastics and how they're going to interact."
Now certainly, you know, in this question about, "Well what's the value of only doing one drug when there are thousands of drugs?" Particularly for general use products it's relatively straight forward, say for insulin pumps, because we've already narrowed down what drugs we're talking about. For infusion pumps that are generally indicated it becomes much more of a challenge.
So when we started to do this we recognized that this is somewhat of a compromise between (unintelligible) we make the pump list hinder indications and they're only indicated for whatever drug they've had so do we let them keep the general indication and provide some information about compatibility?
And we're still continuing to evaluate this now with respect to is there any potential to identify representatives, like a representative for antibiotics and, you know, whatever class of drugs we're talking about. And I - any comments that you may be able to provide on that aspect I think are greatly appreciated.
Anna Nowobilski-Vasilios: If you look at the chemical structure of each antibiotic they may be some - maybe if you looked at the - if you looked at all penicillin's, you may be able to make that generalization, but if you start looking at some of the semi-synthetic drugs that are outside of that category, outside of the penicillin's, outside of the cephalosporin's.
If you start looking at all the - you have to look at the structure of each particular drug. That's why I'm asking this question, this is just such a complex area.
Woman: I would just like also to say that I know (ODeac) works very closely with the Center for Drugs and there are pharmacists on the team. So we are looking at that, in fact that's one of the reasons it made it to the guidance, because we're working much more closely with our sister centers now, when we have systems that can interact with each other.
Alan Stevens: Yes. It sounds to me like you have some really good comments to make and I would ask that you submit what you think (unintelligible).
Anna Nowobilski-Vasilios: We will, thank you.
Alan Stevens: Thank you.
Coordinator: Thank you. Next, please.
Steve Wilcox: Steve Wilcox with Design Science and Human Factors Consultants. It doesn't seem like a bad place to be at the moment. Just a - I have a - I want to probe Ron a little bit on this notion of something you told us we've all got to do now.
So we're all going to do (unintelligible) testing. We're going to end up with used errors you can't get rid of all of them. And you've told us that we can't use the argument that, "Well - to the explanation, it's not satisfactory to say, "Well, they're only X numbered." So we've got to construct a satisfactory explanation.
And I can only think of two. One is we've done everything we can, we've already mitigated and the current technology doesn't allow us to get rid of it, number one.
Or number two, it doesn't have safety implications, yes people are going to make errors, but it won't hurt anybody. Can you give us any other examples of what would be a satisfactory explanation?
Ron Kaye: A satisfactory explanation for...
Steve Wilcox: For use errors.
Ron Kaye: ...for use errors. I mean there is - I know there's a satisfactory explanation for use errors, you detect in the testing situation could be called experimental artifact. And I'm not sure if this is what...
Steve Wilcox: (Unintelligible).
Ron Kaye: ...you're asking about but, you know, when you are - and this applies to particularly to simulated use testing, but there are cases where the participants in a test, they want to do what they should do.
They want to participate but they don't really understand the test, they have instructions. They can be, you know, the test materials and instruct them on what to do, it might be misinterpreting themselves. Those, you know, those materials have human factors problems.
Steve Wilcox: Okay.
Ron Kaye: And you do a test and you end up with some people thinking it's, you know, thinking well I though you meant for me to do X and you really wanted Y and it's counted as a failure and of course, you know, that's understandable and that's going to happen and it happens, you know, particularly in an experiment where you're dealing with people and human specific behavior.
Now beyond that, you know, that is another example. But what you said is absolutely right in terms of there being a, you know, a residual error likelihood. Any time anybody interacts with a device there's going to be a chance of some kind of error, just like any time anybody, any one of us gets behind a wheel of a car, there's some residual chance there.
The idea is that the interface should be optimized. The design of the interface should be optimized such that those types of errors are minimized.
Steve Wilcox: Sure.
Ron Kaye: Okay. With understanding that there's going to be some residual, but there's pumps on the market where, you know, you don't see a certain type of error and then if another pump has that you have to wonder well why, you know and the argument is, "Well the way we decided to build it, we can't eliminate that because it's the way it is.
Well, you know, if there's are of serious concern, you know, maybe the conclusion would be that, "Well you - looks like you didn't build it right and we wouldn't want to be stuck with that on the market where people are going to be committing errors of that level." So did I address your questions...
Steve Wilcox: Yes.
Ron Kaye: ...pretty well?
Steve Wilcox: I mean, now you're given me a third. So is that - still sounds...
Ron Kaye: Okay.
Steve Wilcox: ...like there are basically three things we can set it for those there if we can, that - can no longer be mitigated.
Ron Kaye: That - what's the last part of that?
Steve Wilcox: I said that, you know, the errors, of course, that can no longer be mitigated. There's basically a satisfactory explanation can be one of three things.
Ron Kaye: Right.
Steve Wilcox: It doesn't have safety implications, we can't do any better, or it's an artifact of the testing situation.
Ron Kaye: Right, but I don't think it's the idea is to, you know, once you have an error to pick one of those three.
Steve Wilcox: No, I'm saying if you can no longer mitigate.
Ron Kaye: Yes.
Steve Wilcox: Of course you want to pick one.
Ron Kaye: Right, right.
Steve Wilcox: Okay, (unintelligible).
Ron Kaye: Right, okay.
Coordinator: Thank you. Next, please.
(Stacey Wells): Hi, I'm (Stacey Wells), I'm a medication safety coordinator for nursing at Cedars-Sinai Medical Center and we have 3800 pump patients at this point, a little over and that doesn't include our (PCA)'s and (PCEA)'s. One of my job roles is to look at all the medication errors that happen, the potential errors that happen. And we have some specific ideas on what we would like for pumps in the future.
It's - maybe I'm missing something, but in your guidance document it sounds like there's maybe a disconnect between the manufacturers and end users. Mary was alluding to knowing, you know, what it is that we need.
And because we've done all these root cause analysis we have some trouble with we think are wonderful ideas in mitigating the risk to our patients. So is there anything that you're going to encourage or make the manufacturers engage with the end users?
Anthony Watson: Hi, I've been neglecting to introduce myself. I'm Anthony Watson. I will say that this again, I think this is one thing that has been thematic - at least for the past three to four years with respect to a number of different products in the areas that I particularly deal with - and that is we can't approach this from one dimension. It's got to be a system wide approach.
When I was at the AMEA meetings, I was very proud to hear that they are looking at an infusion safety summit, which is about, you know, infusion devices in general not just - I hope I'm not spilling someone - some beans I'm not supposed to spill, but the idea is that we're going to look at what are the touch points of - and it was centered around infusion pumps, but also infusion safety in general.
But what are the touch points around this type of situation that we can improve totally, because as I mentioned yesterday, users, regulators, hospital administrators, everybody has to fix this problem.
We are responsible for the device piece that we allow to go on the market. The manufacturers in this room are responsible for the product they provide to the users.
The users are responsible to doing what they're supposed to do to make sure the devices are bought - that they understand from at least some standpoint how to use the device how the manufacturer intended. And then we all have to figure out how to get rid of the residual risk. That's going to be a group-wide approach.
So I absolutely think we have to engage the users. I think there's a way to do that. I think (Amy) is, you know, the group that's going to at least start that, but I would encourage somehow, you know, this could be a grassroots thing, it doesn't have to be, you know, we're sort of set the bar, but everyone else can jump in.
(Amy): We're ready.
Mary Brooks: Yes, I have to jump in on this. I want to applaud you, I really do, for coming up and answering - asking this question and also looking at your facility and what your needs are and doing the root cause analysis, what I will ask is that you publish.
As nurses, we don't publish enough, so please write it up. I know that manufacturers want to hear from clinicians and one way of doing that is to get the word out.
You're doing it today and that's why this is a workshop and we're trying to keep it very comfortable and so that people feel that they can come up and ask us questions and we're receiving a tremendous amount of really good input and feedback from everybody in the room.
So I would encourage you to publish so that you can get your word out to the manufacturers either in the Infusion Nurses Society or Critical-Care Nursing. And you can also publish to Clinical Engineering as well.
And I don't know, Melissa - and forgive me for putting you in the spot - but I don't know if on our Web site, when we get good information like this, if there's a way, a venue that we can get this information out. That's fine. I apologize for putting her on the spot, but...
Melissa Eackle: No, I was getting to - I'm a mother, but I still have trouble multi-tasking and also a nurse, which makes it a little easier. It's a great question and what Mary said is true. And I want you to submit your comments because, before I say anything make sure that that gets in the docket to the Federal Register notice for comment.
My question back to you - and I just want to take a minute here, Mark. What are you doing now that you find is not successful? Who are your sharing that information with and what worked and what doesn't?
Mary Brooks: Well, right now a lot of it's internal and we have, you know, we have a lot of huddles and discussions and what not. And we just went through our MERP Survey - which Medication Error Reduction Plan - and so all of this has sort of got us where we have quite a bit of information.
And I'll give you an example of something that we would like to see in the future and that would be for instance, our high alert medications. We're having a real trouble making sure that the nurses are doing independent double checks. We would like to see the pump allow the nurses to electronically sign in the pump and then, of course, that would have to interface with the electronic medical record.
Melissa Eackle: Do you all know what a high alert medication is, manufacturers? You're all aware? Okay. Wonderful. As a nurse, I'm speaking directly to you. We tend to do our work and not tell anybody about it, so publishing is a good idea.
The other thing is we have professional organizations, American Association, you know, The Nurses, the Critical-Care, the separate part, they need to start stepping up and they do have the new bedside initiatives. And they need to start reaching out to the people who make our devices that we're going to use.
And so I would encourage you to start telling your manufacture reps - who I know are coming into the office and trying to sell you things - and also publishing and getting us the professional organizations.
And manufacturers, I would ask you to start asking - and this is me, personally, not as an FDA, but as a clinician - that you start engaging with professional societies to try to get that input.
If you're looking for what your use population looks like in the environment, you need to start reaching out. Don't wait for them to come to you, because I will tell you, being a nurse you'll wait until the cows come home. We will not be coming just for any number of reasons.
So thank you, it's very valuable information, please put it in the docket. And I'm also writing down that maybe someone should start a blog or a Listserv, because I've been writing down - people have been giving me a lot of recommendations about what you'd like to see going forward after the start of this conversation, so I toss that idea out. To have an honest conversation, you know, outside, in the net and start the conversation. So thank you.
Mary Brooks: Great ideas, so thank you so much.
Man: Thank you. Well, we haven't made the pilot cards any smaller, so let's start working on that a little bit. This one says for existing pumps on the market that are used in the home, do these indeed to be labeled home use even though they were not designed and tested in the home? And if they answer is yes, does it require a new 510(k) submission?
Anthony Watson: I think Mary Brady talked about that couple days ago, somewhat - yesterday.
Man: Yes, seems like a couple days ago.
Anthony Watson: What, I'm sorry? No, I was talking about Mary Brady, Mary Brady from the Home Use Initiative. And I don't want to talk out of turn, because I don't know everything there is around that. Mary Brooks is part of that Home Use Group and I think actually Ron is as well.
But my understanding is that it's the devices that are specifically intended for the home use definitely have to be labeled that way and those that are likely to be used in the home, I mean that they have a high potential to be used in the home, also have to have some type of home use assessment.
That's the extent of what I know about that, so I won't go any deeper than that and I'll leave it up to other people who know more about it to speak to it if they feel comfortable doing so.
Melissa Eackle: Yes, at this point, I know from the (Home Care Committee), we're exploring whether or not home health devices need to be labeled for home health use, especially if they require a PMA or a 510(k), or a regulatory decision. We are exploring that to be determined at this point.
We're basically seeing what the needs are. If you already have a marketed device, it's already received FDA clearance, I would not think, from the premarket world that we're going to require you to come back in under a 510(k). Back to you, Tony.
Man: I'll leave - it depends on (unintelligible) get my...
Anthony Watson: What the change to the labeling is. I - we do have - by the way our guidance that is being, I understand, being revised somewhat about when to submit a 510(k) still was valid. I'm not going to say anything here other than that is your first place to go. It may - I've heard how somewhat useful and not so useful it is because of the calls we get to clarify it, but I understand that they are looking at being more specific about that.
Basically though I mean, if you were to say, yes - but I can tell you that if you have a pump that is generally indicated right now that you specifically indicate for the home use, that would require a new 510k, because that's a new specific environment to which you are claiming and then we would expect to see the evidence to support that it's safe in that environment. So I can say that for sure.
Man: Okay. This one says, "How is FDA evaluating infusion funds currently under review while this guidance document is under review and discussion? Is that they requiring devices - and I think this must mean requiring devices currently being evaluated to conduct a clinical evaluation?"
Alan Stevens: Right now...
Anthony Watson: I think Alan, this is Anthony Watson again.
Alan Stevens: Right.
Anthony Watson: Right now I think Alan already alluded to the fact that it's being reviewed on a case-by-case basis. That is the requirement and what that basically means, when the device comes in we'll assess whether or not there are parts of the guidance that we would apply at that moment.
The expectation though is that the guidance is in effect, it's our current thinking. The most - and we basically believe that the majority of that guidance document is just a presentation format difference. We are asking for more information, but it also is a presentation format. The two biggies in there that really are the ones that obviously have people's attention are the pre-clearance inspection and the clinical evaluation.
We've discussed the clinical evaluation in detail and you'll hear about the pre-clearance inspection later this afternoon, so I won't get into any detail about that. But we, I mean, as far as the assurance cases and the expectations of the different hazards, those are valid. We expect those to be part of your engineering decisions anyway. We're only asking that you provide that information to us in a detail that you haven't done so before.
Man: Okay, this one says, "Clarify what it means when the guidance states that most changes would require a submission of a new 510(k)."
Alan Stevens: Can - sir, is there a way we can clarify?
Man: Pardon me?
Alan Stevens: What part of the guidance, I mean I don't have the guidance in front of me.
Man: Well, we can...
Alan Stevens: Is that in respect to software?
Man: It doesn't say, we took - I think the key word here is most changes.
Anthony Watson: Right, right. I remember that. I remember that, yes. I think what it basically says is that the types of changes that we expect to see in most cases do require, as of today, a 510(k). There are probably some minor changes out there that don't require a 510(k).
If you look at the - and I'm referring again to the existing When to Submit a 510(k) Guidance, which everybody, you know, has provided various comments about how useful that is And I just mentioned that earlier.
The - we review them quite conservatively. Other people don't necessarily view them as conservatively, but most of the changes that occur in infusion pumps do require 510(k)'s. And I think it's evident by the number of 510(k)'s we get for changes to infusion pumps. So I think what we're - well we're just re-emphasizing that in the guidance document, that most changes do typically require a 510(k).
There are some very minor changes that don't, but I think what the point was of that statements was just to emphasize that, you know, if you have any doubts about that - and unfortunately I hate to ask for, you know, more questions if you - especially if - we would love to be able to say this is A, this is B, this is C and as long as you do A, B and C you can just go and have fun. We can't necessarily pin that down.
A lot of the changes typically are to the interface. We get a lot of interface changes. Interface changes generally rule require some sort of human factor for evaluation. So that is why most changes that we get typically do require some type of 510(k).
Man: Okay, another live question.
Ricardo Alphonso: Yes, good morning. My name is Ricardo Alphonso, I'm with Pall Corporation, a provider of critical inland filtration devices to actually many of the companies that are here today. I'd like to also add my comment, congratulatory (unintelligible) to the agency for having this forum.
And the question is, obviously, there is a guidance document out there and eventually will come into some regulations that would apply to the new devices, I would imagine, what happens to the millions or hundreds of thousands of pumps that are in the market already? What's the agency's strategy to addressing that?
Anthony Watson: Hello? Okay. I have to - there's a delay there I - I'm always trying to speak before it's ready. This is Anthony Watson again. The - and this is a regulatory answer essentially.
The way we understand it is that when the guidance becomes a special control guidance, which we've said we intend to do, at that point the expectation is that manufactures will meet the requirements of the guidance.
Now, the big question I think most people have is, "Will you need to submit a new 510(k) in order to meet those requirements?" That, I think, from what I understand, at this point the ones on the market may not require necessarily a new 510(k) to meet those requirements. However, if you come in with a new 510(k), you will be immediately expected to meet those requirements.
So the devices on the market will eventually get phased out. This is a high turn over industry as far as I can - I mean, from our - from where we sit, it appears to be a high turn over. We see a number of manufacturers coming in with changes quite frequently. So the next time you come in with a change we'll expect that you will meet the requirements of the guidance document.
And eventually, those on the market will be phased out that are meeting the old requirements, but you still have to make sure that you meet them. You go back and do the analysis to make sure that at least the pumps that are coming along the line will have to meet that guidance.
Man: Okay, this next card says, "Are changes to pump sets that are not classified as infusion pump accessories still considered part of the system? What changes to sets would require a new 510(k) filing for a Legacy pump? Do you have any specific examples?"
Alan Stevens: Hi, this is Alan Stevens. So my understanding it's a question as with respect to the administration sense for the infusion pump changes. And I think we've already addressed this. When a modification requires a new 510(k), we already have a guidance document for that, we're not changing those criteria. What was the second part of the question?
Man: The second part of the question is, "What changes to sets would require a new 510(k) for a Legacy pump?"
Alan Stevens: I think I've already...
Anthony Watson: I mean, I think I've already addressed that question.
Alan Stevens: ...(unintelligible) Legacy pump.
Alan Stevens: ...examples?
Anthony Watson: Yes.
Alan Stevens: Like if you change the materials, construction, I mean that's...
Anthony Watson: If you change the materials, through the administration side we would expect to see new 510(k).
Man: Okay. All right, "What are the objectives of the clinical evaluation? Is it only for assessment of human factors or safe performance in general?"
Alan Stevens: Hi, this is Alan Stevens, again and I'll comment and Ron, if you want to say something.
Ron Kaye: Okay.
Alan Stevens: So in the guidance document we outline that, you know, the objective of the clinical evaluation is primarily to look at human factors (unintelligible) human factors and device performance, you know, certainly safety is something that we're interested in.
And if adverse event were to occurred, you know, we're interested in understanding how this occurred and how the device may have contributed to that. So that's primarily what we're interested in is device performance and human factors.
Man: This one says, "Does FDA have plans to affect changes to standards like alarms, electric and safety standards, based on the learning from recalls related to infusion pumps?" I think this means does this get extended beyond - does what we learned from infusion pumps get extended beyond that to other devices?
Anthony Watson: The answer is yes. We - as I mentioned, we're already engaged in some standards that (unintelligible) around infusion pumps, but there's also alarm standards. I know Ron participates in the alarm standard development. The human factor standards, I mean, these are very horizontal type issues.
Infusion pumps are not the only device that has these problems. And I think Al Taylor will - I say he - Al Taylor has preached this over and over again that, you know, there's not just an infusion pump problem.
It's the scope of the problem that we're trying to address for infusion pumps, specifically with this initiative, but the types of problems we're seeing come in other device areas as well.
So it is something that we participate actively in for other standards. As I - there's specifically a lot of things I mentioned and were mentioned in that things I mentioned and were mentioned in that question, alarm, human factors, those are all standards that we participate in. So yes, FDA has a very high interest in addressing those across the board.
Man: This one says, "If a company has a 510(k) application in and it gets returned for human clinicals, do they go to the front of the line?"
Alan Stevens: Can we clarify returned for human...
Man: Well, if a person...
Alan Stevens: I mean, we NFC'd the submission...
Man: Is the person who wrote that question here?
Man: Do you dare to stand up?
Man: Okay, that being said, let's - we (unintelligible).
Alan Stevens: I mean, we don't - I mean, we have - the way we review submissions is based on, I mean, if it's a 510(k), we have a 90 day clock. We try to have a first round review completed within 50 days of receipt of the submission.
And we're obviously managing not just infusion pumps, but we're managing, you know, all the products that we regulate within general hospitals. But we do try to keep on top of those in a calmly manner.
When we're talking about IDE's, certainly we give priority to those because we only have 30 days to turn them around. So I'm not quite sure, you know, I don't think we can expedite it any more than that.
Anthony Watson: And also, if you look at this guidance document, there are other documents that touch on the infusion pump guidance that are not just in the realm of ODE.
There's a lot of the reports we're asking for, we're also going to be asking other offices to look at those because they have expertise in evaluating those areas.
I don't know about the front of the line thing. We hear that pop up in different scenarios about when they should be moved to the front of the line. Those are governed by other regulations and other processes and policies that are not within my scope or anybody sitting at this table. So what, I didn't understand...
Man: (Unintelligible) have to repeat that question that - and she wasn't heard, I'm sure.
Anthony Watson: Right, I was going to repeat that.
Anthony Watson: So what I think I heard you say was that you understand the question to be, "If someone is - if someone gets say an additional information letter, as we would call it, asking for a clinical evaluation that wasn't done and they come back in, does that...
Anthony Watson: So do they have to go back to the start of the 90 day clock?" We don't do that. So the clock starts from where you come back in. So if we stop at day 30 we have a 60-day squeeze on us to get this done.
Anthony Watson: I'm sorry, what's that?
Anthony Watson: Oh right, assuming that all the other policy requirements are met, that you've met - you haven't gone over the whole timeframe - I mean, I don't want to go down that path - but as far as the review process is concerned, we have the balance of 90 days, that's the way the clock works.
We try to get that done. We don't always meet that. And it'll be even more of a challenge to do that, but hopefully, you know, we'll - we're - that's the goal, that's' what we're trying to do.
Man: Okay, let's have another live one.
(Jeff Hutchings): Yes, my name if (Jeff Hutchings) from (unintelligible) Associates. We work with a number of medical device manufacturers. I wanted to expand on the question before last about the applications of some of the principles that we've heard about today, more broadly across medical devices.
With the 510(k) submission is an expectation to see some of these techniques sensibly simulated use of (unintelligible) paces and/or would there be an acceptable approach more broadly in the medical device industry?
Alan Stevens: So can I - this is Alan Stevens. So if I - maybe I can repeat what I think you're asking and you can clarify.
(Jeff Hutchings): Okay.
Alan Stevens: So you're asking if you use an assurance case method for another medical device, not an infusion pump, would the agency accept that?
(Jeff Hutchings): Yes.
Alan Stevens: Well, at the risk of answering for other branches, you know, certainly, you know, we're looking at assurance cases - and our center director has actually indicated that this is, you now, the initiative that we have for infusion pumps.
There's a potential passport for other high risk medical devices where we see problems and, you know, certainly those areas are being looked at on a case-by-case basis right now.
I don't know why you wouldn't be able to submit an assurance case for any medical device. I mean, you could use the assurance case if you, you know, the review of any medical device, you know, whether the, you know, certainly we within our branch would be prepared to review that.
Coordinator: Okay, thank you.
Anthony Watson: I am curious, though, did you have an example of an insurance case, that you know of, that was submitted to another group? I mean, you don't have to name this company, but did you...
(Jeff Hutchings): No, I don't...
('Anthony Watson): I mean, did you see them?
(Jeff Hutchings): No, I don't have a (unintelligible).
Anthony Watson: Okay, because I thought...
(Jeff Hutchings): (Unintelligible).
Anthony Watson: ...I don't think...
(Jeff Hutchings): (Unintelligible).
Anthony Watson: No, I don't but we're working with a number of companies where...
Anthony Watson: Okay.
(Jeff Hutchings): ...where we're looking at 510(k)'s and obviously interested in this as an approach. So that's why...
Anthony Watson: Okay.
(Jeff Hutchings): That was the point to the question.
Anthony Watson: Okay, great, thank you.
Man: This next one says, "Is the clinical evaluation expected to focus on clinical advocacy like the PMA, or on - excuse me - or on safety from the use perspective?"
Alan Stevens: Hi, this is Alan Stevens. So we're not primarily - we're not looking at clinical advocacies, so we're not looking necessarily at disease treatment outcomes, we're looking at design validations of device use.
As I said before, certainly we're interested in adverse events as they relate to safety that may indicate use or performance problems with the device. Certainly, you know, this is complex, once you actually put it into a use environment where the root cause of problems are actually occurring.
And so this is why, you know, the development of the protocol is particularly important, so that we can identify and understand how the device is performing in use.
Anthony Watson: I think there's also, you know, a lot of the questions around a clinical evaluation and what is the purpose of a clinical evaluation, I think Ron touched on it. Actually, Ron spent some significant time on it and Alan touched on it.
I think one thing to, you know, I think the key words about clinical evaluation or residual risk, you go through the evaluation, you do a design and you do, you know, you do the design work, you do a simulated use and then there's that little piece, that nagging piece that seems to pop up in the (NDR)s that happens when you go in the environment.
And then, we're trying to say, let's mitigate that. Let's take that down another notch by putting it in the environment and see what happens. We're not - like Alan said - we're not looking to see does the drug work, we know about the drug.
You've done some testing that we've asked for to look for the capability of the drug, at least in the sample and we're going to get some comments about whether they think that - whether people think that's valid or not.
So this is about residual risk. What happens when you put it in the environment, let's try to mitigate that. That's what the clinical evaluation's about.
Al Taylor: Okay. And this is Al Taylor. It's worth pointing out that the quality system regulation, you know, defines validation as subjective evidence that the product satisfies as an intended use.
And in the preambles of the quality system regulations, back in '96, we said that validation will almost always include testing of the product in an actual or simulated use condition.
And so we're really just saying that in many cases, the simulated use conditions are not - in (unintelligible) and the adverse events that we've investigated, the testing really didn't get to the roots of the issue.
And so that's why we're putting more focus on - when it makes sense, let's get out into the real world and see what really happens. And that was always the intent of the quality systems regulations.
Man: Yes, another live question.
(John Yeager): Yes, it's (John Yeager) from (Assist) Medical Systems in Minneapolis, Minnesota. There's been a lot of discussion on the clinical assessments that have - that you're requesting under the guidance document. Further, you're suggesting the use of these in a significant risk format, therefore requiring a formal (IDE) with you first to be approved.
The question that I have is, have you considered other stakeholders in the clinical study process, particularly the human subjects committees that have to approve these studies once they do go to the field?
And what's the justifications or the points of consideration might be for the sponsor of the study to be able to engage those committees appropriately, given that the types of studies that you're proposing are validation studies?
Anthony Watson: I guess what I would say, that is you always have to go through an IRB if you're going to use a certain type of a, you know, if you're doing a human subjects study. And what - the IRB's role is going to be primarily the same.
The difference is going to be that now they're going - that there's an IDE involved, perhaps. But, you know, I want to point out that we have a guidance document on significant risks and non-significant risks studies and the infusion pumps are listed as significant risk devices in that guidance.
It hasn't been completely followed. It hasn't been completely, I don't think, understood even. I'm not even sure how many people have even seen that, but it's out there, it's been out there for a while. I forgot when it was actually published, but it's been out there for quite awhile.
So, you know, as far as the stakeholders are concerned, I mean, at this point I guess we haven't really considered that it would be much different than those folks, especially the IRBs at that point.
But if you have another opinion on that, I would suggest that you comment back that you think that there might be some differences in the way they have to deal with this and how you think that might be affected, either adversely or otherwise.
(John Yeager): Well, it's a concern. I believe that if you look at these studies with softer endpoints, more subjective kinds of considerations, exposure of different patient groups, particularly perhaps home-use groups to a device that is under investigation to prove safety, that they are going to be concerned significantly for the well-being of those patients participating in that study and perhaps even reluctant to want to allow that to occur.
Melissa Eackle: (Unintelligible).
Anthony Watson: I - and I - Melissa wants to comment. And I would just say that, that's the point of an IDE. We - our job is to make sure the study risks are mitigated. That is the point of an IDE and that is why they are under IDE, they're supposed to be under IDEs.
So I would say that if they're worried about patients being exposed to investigational devices, I would say are we sure of that's a concern, that's why were making it clear that it needs an IDE.
But by the same token, we will probably get more information up front, because as we said, before you even go into the clinical environment you're going to do some simulated testing to show that at least you believe that the process works.
And we're not going to allow it to go into the clinical environment, that we believe it does, at least it minimally says it does. So there's going to be even more protection in this case, than they would get under say, a non-significant risk study.
Melissa Eackle: I belong to several state (quota) groups both the FDA and a user and also as a clinician. Actually, clinicians are the ones who fought for the clinical evaluations, because we see what happens.
You basically do the clinical evaluation now after it hits the market and we find out what the issues are and how people use them. And very often, it's very hard to go back and fix the problems, which are expensive.
So after we became aware of the problems and the scope of the problems and the number and the likelihood that they're going to continue to occur, we felt that it was very, very important that these devices, after they've been through the appropriate assurance case, simulated testing, actually get out into the hands of the people that are going to be using them.
In the environment they're going to be used, because it seems more and more we're finding it out that is that there are things that we were not able to think ahead about.
People are using them in different ways with different things and it's always like, "Well, we would never have suspected," or "why are they doing that," you know, type of things. The reality is, people do do that and it's really important that you know it about your device, so none of us are caught off guard.
Man: Okay. This one says, "Given that the guidance document has a primary focus on risk management, will the FDA be working with the industry to develop a model for risk management to be used in pre and post-clearance situations?"
Woman: You want me to write that down?
Al Taylor: Yes. Yes, I'll take that one. We actually have a guidance document on risk management that's close to publication, it's a draft.
They - we have recognized ISO 14971, which is titled, "Risk Management for Medical Devices," as providing a risk management process that satisfies the agency's concerns for things that should be done during the development phase, during production and during post-production, you know, servicing, maintenance, complaint handling, all, you know, it's a life cycle process standard.
And as Rick said earlier, it's important when - to have a process to have some structure for managing complexity and it's also important to make good decisions. So we want you to have a process and follow it and we will also be reviewing your decisions at appropriate junctures.
Man: Before we go any further let me check with our moderator. I think we're supposed to stop this session at 12 o'clock, is that right?
Woman: That's correct.
Man: Okay, so we have a little more time. This one says, "If we have a 510(k) submitted through a third party reviewer and human clinicals are required, can we change the submitter to us, exclude the third party from human trials and continue with the submission?"
Anthony Watson: The rules for third-parties are defined. And your question is about one under review, which is a question that you would have to go through our third party person in 510(k) staff, that's (Eric Recken). I'm calling him out, his name is (Eric Recken).
Some of you probably know (Eric), you've probably dealt with (Eric). That would be a question probably for him, that's a third party question that I - even though I have some ideas about what that means.
Generally speaking, the decision point would be whether a clinical evaluation is a clinical study or requires - meets the definition of a clinical study under the definition for a third party guidance?
I think that question is probably best addressed by him. I don't want to go down there, so I'll just leave that for him. I wish I had a better answer at this point. That's a question that I think we need to go back and get some answers about.
Man: Let's close out this session with several questions about the issue of modification, so let me read those. This one says, "For pre-clearance inspections what's considered a new device? Most pumps are not new and that there are gradual software features added or minor modifications made like new LCD screens, would these be considered new devices?"
Alan Stevens: Maybe - can we put (unintelligible) after the pre-clearance?
Man: Until when?
Alan Stevens: We're going to have a session this afternoon talking about pre-clearance, I...
Man: Sure. Okay, we'll put that one aside. That's fine. Okay.
Alan Stevens: It's a good question though.
Man: Yes. Well do you want us to put all the change questions aside or should I read you another one?
Alan Stevens: No, that was about pre-clearance (unintelligible).
Man: Okay. All right, "The guidance document states that FDA considers most changes to be significant requiring 510(k) submissions, can you provide details on where to draw the line between significant and insignificant changes? For example, would a component change like a capacitor be considered significant, or let's say an LCD screen for that matter?"
Anthony Watson: I think we answered that earlier when we talked about most. What we've considered to be most changes. Significant and non-significant is not a bright white line, it doesn't exist. It really is a judgment call.
And what we're saying is you generally know when you're close to the line, that's when we get the call. When you're, you know, farther out on the ends it's an obvious decision, but I think what - there is no bright white line.
And I don't want to make it sound like there is, this is a subjective - unfortunately somewhat of a subjective assessment. But, you know, we've given some general guidelines in the guidance about what we considered to be changes that would require say a clinical evaluation or the conditions under which a pre-clearance inspection will be given.
So I think in the case where, you know, significant and non-significant risks - non-significant changes, that is not an easy answer, I don't think we could sit up here and rightfully tell you what that is.
Man: We have a couple more that are pushing you into defining that bright line. So I'm just going to - you can have the same answer for these, I'm just going to put them aside. Let's - we have two live ones, let's finish up this session with the two live ones.
(Lee Grant): My name is (Lee Grant) I'm from West Virginia University Hospital. I'm the Director for Biomedical Engineering there. My question is, is there any plans to make any of this - the manufacturers are doing an awful lot of research, is there any plans to make any of this public so that we can - as men use our facility, we can analyze what residual risks might be out there and make the assessments, is that acceptable for our environment?
Alan Stevens: Well I will try - this is Alan Stevens, I'll try to answer this, the best I can. From our, you know, from the agency perspective, either what we've publish for 510(k) clearances are found in - it's not even really our documents it's similar to a PMA where we publish a safety - some of the safety and effectiveness data.
The 510(k) summary is the manufacturer's document. However, you know, we have been increasing not just for infusion pumps, but every device that we review under 510(k) over the past, I think, beginning of April or February.
I can't remember the date, but we've stepped up our analysis of the information that's actually in the 510(k) summary and how useful that is for describing what we thought was important and what aspects of performance we thought was important when we were reviewing submission.
So hopefully going forward the 510(k) summaries are going to be much more informative about what the agency looked at.
Man: Thank you.
Tim Vanderveen: Hi, Tim Vanderveen with CareFusion, San Diego. Over the past ten years or so with the introduction of smart pumps we've made a lot of advances in safety that hospitals can take advantage of, but they're not always used.
And just one data point, after a recent highly publicized death with Heparin we took a look at 470 hospitals and how they had set up their drug libraries. Surprisingly 51% of the hospitals had no hard limits for Heparin anywhere in their hospital.
There were multiple other issues, multiple drug dosing units et cetera, does the FDA have any plans to help hospitals understand and better use the safety that already exists in these devices?
It really is kind of sad that a lot of these safety features that are available aren't necessarily always used for a variety of different reasons which we don't have manufacturer's control.
The capability is there but the hospitals choose basically whether to or not to use many of the safety features that are built into the devices. So I'd like to hear if you have any plans to maybe help us address that.
Melissa Eackle: Thank you for the question, it's a great question. At the very beginning of this workshop and frequently throughout we've said it's a conversation and there are multiple stakeholders and he, Tim, I believe it is, just elucidated one of the problems where the conversation has broken down.
What I would say starting with is, obviously when you market your product and you have a smart pump and there's plenty of literature on the reduction of errors with smart pumps, but then there's also literature that says how smart are the smart pumps and that always hinges on the person who's using it.
So what I would say is as manufacturers, part of the issue is you're going to have to go out and really sell the use and why it's good and educate people.
We clinicians ask the FDA often, publish on when we find issues and very often you'll see them in nursing magazines, I think Nursing 2009-2010, when we see an issue where there are safety mitigations that are not being used that are commonly available and we publish to try to remind people that is happening.
Again we also reach out to professional organizations, we go and speak at conventions and workshops and talk about these kind of issues. That's what we do now. There's a lot of things we could be doing.
And what I have been doing while everyone has been coming up with recommendations is writing them down to say, "Maybe we should take a look at. Maybe we should talk to certain people with.
Maybe we should do this or that." And what I would say to you is I'm going to give you my email address - and I do it with sort of a little bit of a twitch - but if you have things, recommendations, not on the guidance, because you should be sending those to the docket, but if how to enhance the conversation.
Things that we should be talking about perhaps that we're not, groups that we should be including that perhaps that we're not, those kinds of things. My email is Melissa, M-E-L-I-S-S-A, Eackle, E-A-K-L-E,@fda.hhs.gov. And since I am the chair of the workgroup I will bring it back to the workgroup to come under consideration. And now over to Tony.
Anthony Watson: Well I just want to say - and I'll be very brief - I think this is a revolving role a little bit for FDA. I mean, for the most part we deal with the manufacturers, the devices themselves and various roles. I think what you're - what maybe you might see more of is the sort of promote piece of the HHS mission coming from FDA, to somewhat be more of a facilitator in these discussions. But we are, in the end, regulators, still.
And, you know, we aren't always the right people to be facilitators, because, even though we're regulators and we can, you know, sort of impose certain things, which is not, you know, always welcome.
There are other agencies, the other organizations that we work to engage other folks. The standards of organizations, some of the organizations, the health care organizations, that Melissa mentioned and other science think tanks that (OSO) engages with.
So I think we're going to be a much more rounded agency, I think that's the right word, to engage these conversations. And I want to put another plug in, for the (Amy infusion device) and I'm throwing that out there and I don't know if we even have a date for such a thing.
But, you know, I'm putting that out there, again, because that is where we hope to get everybody to the table to start talking about this, kind of.
Melissa Eackle: And I'm going to finish up, since we have a minute. Sorry and while he's putting in the plug for (Amy).
I'm going to say please reach out to healthcare provider organizations on a more regular basis. And also actually to universities and the colleges because they're trading up the next group of people.
And it is really important that we start getting them involved in this conversation because they'll be taken care of us when we retire.
All right we have lunch until 1:30. Whoever is on the line, please stay on the line. And when you come back we'll be hearing about pre-clearance inspections and the last part, MDR reporting. Thank you.
Melissa Eackle: There's going to be a slight change to the agenda. From 1:30 to 2:30 we have the pre-clearance inspection and the MDR reporting. These are going to be a little briefer than the hour that we've assigned because we want to handle more of the questions because there are so many of them.
So after we finish the brief, well not so brief but briefer, pre-clearance inspection and the MDR reporting section, I'll be letting the FDA put your systems back up, and our wonderful moderator, Mark Barnett, to take some more questions. All right, thank you.
Our first speaker is going to be Lieutenant Commander (Nick Thacker). I got your, that's the right rank right?
Melissa Eackle: All right, I'm never sure. I'm not really good at those and stripes. So our first speaker today is Lieutenant Commander (Nick Thacker) who is in the Office of Device Evaluation.
He is the cheerleader for combination products and is responsible for managing the combination product console or gold within the general hospital's device branch.
He's also a member of the pre-market infusion pump team where he reviews infusion pump submissions and works on policy associated with the pre-market regulation of these devices.
Previously, Lieutenant Commander (Sacker) was a team leader for the post-market infusion pump team within the Office of Compliance and was responsible for all (personal) market advisory and regulatory activity associated with manufacturers of infusion pumps, insulin pumps - infusion pump accessories and components.
Before joining CDRH, Lieutenant Commander was a Senior Compliance Officer with the Center for Drug Evaluation and Research, managing several high profile and precedent-setting regulatory actions.
Also he was a consultant serving prior to that the pharmaceutical industry where he managed engineering process development, facility planning, GMP and remediation activities and validation efforts for pharmaceutical firms.
He has a Bachelor of Science in Chemical Engineering and a Minor in Economics from Rutgers. Please welcome Lieutenant Commander (Nick Thacker).
(Nick Thacker): Hi. (Unintelligible). Can everyone in the back hear me? So I have a big I guess booming voice. Somebody told me I have a face of radio so.
So prior to joining FDA, you know, when I was a consultant there was a joke that we had about the FDA. And that's the biggest two lies that the FDA will ever tell you is that they're happy to see you and they're here to help.
Now joining the agency and working at the agency for six years, I can absolutely tell you that I'm happy to see all of you here and I'm here to help. But if you have any questions you need to go ask Alan, Alan.
So I'm giving credit to Mrs. Valerie Flournoy because Valerie Flournoy being the Brand Chief of the General Hospital Devices Branch for the Office of Compliance will be working closely with my colleagues on the pre-market side to, you know, enforce these pre-clearance inspections.
So what about these pre-clearance inspections? By the end of today's presentation you will understand why FDA believes that these pre-clearance inspections are necessary for external infusion pump 510(k) submissions. And what elements will be included in that pre-clearance inspection. Next slide.
So why have an inspection? The question may be asked why are pre-clearance inspections necessary for 510(k) clearing products? Moreover, why have this kind of inspection for 510(k) clearing products at the Class 2 device?
Now I stated in my colleague's presentations over the last day and a half, and as we will continue to present in the subsequent presentations and discussions that we have. The FDA has seen an increase in the number of Class 1 recalls associated with the external infusion pumps.
We have also seen an increase in the number of MDR reports and consumer complaints regarding these products. And again, I'm echoing what my colleagues have already told you. And maybe that was just obvious what you may already know.
The increases in MDRs and complaints are important because as we all know, there's a significant under reporting of MDRs when it comes to medical device error. In fact, you can say that there's a significant under reporting for drug or medical device initiatives.
So in the case of infusion pumps, when the agency sees the increasing trends that we have, we become concerned that this is a problem that may be far larger than these first market signals may tend to portray.
These increasing trends triggered our colleagues in the post-market side, post-marketing meaning Office of Compliance, to perform inspections of your facilities.
And those revealed common - a common theme of systemic quality system deficiencies. And these deficiencies in my opinion indicate a fundamental problem along with the total product lifecycles of external infusion pumps.
Now whether it's a breakdown between developing and design, or between design and manufacturing, the FDA is concerned that ultimately it is a public, and that includes all of us in this room, are paying the price for product defects which I think could have been prevented.
So FDA had anticipated there may be a question about our authorities that require pre-clearance inspection or 510(k) submissions associated with external infusion pumps.
The FDA's authority for these inspections that are derived from the Federal Food, Drug and Cosmetic Act under Section 513 F5 as shown here on this slide.
I'll give you a second to absorb that. Good enough. Next slide.
The (unintelligible) do you guys like that? That was totally me, okay.
Yes I have to deal with all of you eating lunch and now trying to fall asleep. The, okay so the (unintelligible). There (unintelligible) highlighted in red represents why the FDA is seeking these pre-clearance inspections.
In a typical review of the 510(k) submission we do not assess the sponsor's adherence with the quality system regulation as stated in 21 FR Part 820. This assessment is performed as part of FDA's routine bi-annual inspection process or cause inspection, whatever, but it's not performed in the pre-market area for 510(k) submissions.
However, given the increasing trends of the post-market signals, we believe that there is a substantial likelihood that the failure to comply with such regulations will potentially present a serious risk to human health.
And I will repeat this a couple of times during my presentation to drive that point home because that is our regulatory hook. And that is what gives us the ability to ask for these pre-clearance inspections as part of the pre-market process.
So one may argue that the quality system regulation is a post-market regulation. I think a submission process is all part of the pre-market process, where it clears it based on substantial equivalents.
And really, the quality system doesn't really enter this discussion regarding substantial equivalents. So why have this seemingly post-market inspection as part of a pre-market process?
Well the FDA's response to this argument is again that the post-market trends indicate to the agency that there is substantial likelihood that the failure to comply with a good manufacturing practice regulations will potentially present a serious risk to human health.
Now under this scenario, Section 513 F5 of the Act, allows FDA to assess their sponsor's compliance with a good manufacturing practice regulations as it's stated here in green.
And the mechanism for FDA to verify the GMP requirements are met by the sponsor is the inspection. So therefore, as part of the pre-clearance process for 510(k) submissions associated with external infusion pumps, the FDA will require a pre-clearance inspection to demonstrate that the sponsor is in compliance with 21 CFR Part 820, the quality system regulation.
So what triggers a pre-clearance inspection? Well according to the guidance, pre-clearance inspection is required when the subject is a new device.
Now a question with that this morning, what is a new device? What if I'm just making a change to my existing device? What if I'm changing everything in my infusion pump except the motor or the pumping mechanism, new screen, new user interface, whatever?
Well in that case you've submitted a 510(k) for a change to your device. So then we'll look at the other categories and see if it fits in there. So if you submitted this 510(k) and an inspection has not occurred within the last two years at the facility where your device is being manufactured, pre-clearance inspection is required.
If an inspection has occurred within the last two years but the inspection was classified VAI or OAI, voluntary action indication, official action indicated, so basically you received a 43 - FDA 43 inspection is required.
The last scenario is if a 510(k) was submitted for changes to the subject device as a result of pump failures. So essentially we're talking here you submitted a 510(k) for changes to your device as a result of a voluntary recall you need a pre-clearance inspection (expression), I'm sorry.
Yes, let's hold off on the questions please. Next slide please, thanks. What will (U) section cover? So the (U) section will be a Level 2 comprehensive inspection.
This type of inspection you have had in the past. It's identified in FDA's compliance program guidance manual, 7382.845, entitled Inspection of Medical Device Manufacturers.
So essentially the (unintelligible) CPA, corrective preventive actions and (four) other sub-systems of the quality system. And (unintelligible) sub-systems (unintelligible) covered.
But in this case, the (unintelligible) involved in the development of the inspectional guidance. So you should expect that design controls, complaint handling and production process controls to be part of the sub systems that FDA focuses on. Unless of course our post-market signals and review of your submission indicate that we should be focusing on something else.
You should also expect that specific elements from the review of the 510(k) submission will be in this inspectional guidance. And if there are any trends for - from MDRs or complaints that indicate to us that we need to further investigate that, that will be part of the 510 - of the pre-clearance inspection.
So what's the FDAs process? When an external infusion pump 510(k) arrives at the agency, Office of Compliance will schedule - begin the scheduling of the inspection and drafting of the inspectional guidance.
And the Office of Device Evaluation will begin review of the submission. Now all of the other offices, OSB, surveillance of biometrics, science and engineering laboratories, our shared OD, will be providing input into this inspectional guidance.
This inspection will be scheduled through the district office that has jurisdiction over your region. So I mean that's something, you're right, everybody's done that.
Now the important thing is that the clearance of your submission is dependent on resolving the reviewer's questions regarding your submission and resolution of the FDA 43 (algorations) as presented to you at the conclusion of this pre-clearance inspection.
And your submission will be placed on hold until these questions and the inspectional concerns are addressed.
So what does that mean in terms of impact on your 510(k) submission? Well basically it's time prints, you know, you're at the conclusion of the inspection. You're issued a 43. You know that you have quality system deficiencies.
You know that you're going to have to respond to them prior to the DIR or whatever inspection being closed. And prior to your submission being cleared.
Well, you know, everyone knows that sometimes quality system deficiencies take a while to resolve. So there will definitely be an impact on that respect to your timeframes.
Now from our end, (unintelligible) workers, our district offices with the Office of Regulatory Affairs and also work with well the Office of Compliance will work with the other offices within CDRH to expedite the review of these inspections and to make sure that they have a systematic process in place to expedite the review.
What I wanted to point out is that FDA is taking a total product life cycle approach to the review of external infusion pump 510(k)s. So in other words, the investigator, compliance officer, pre-market reviewer are working in concert to ensure that the inspection and the pre-market review are aligned with each other.
And to make sure that you guys get a thorough yet fair look at the 510(k) submission regarding infusion pumps, the external infusion pumps.
That's about all I have to say. Does anybody have any questions? You're holding questions. So that's all I have to say. I mean it makes sense right? I mean I had like, I mean this section is like less than a page long in the guidance. Thank you very much.
Woman: Thank you (Nick). Okay, our next speaker is Lieutenant Commander Mary Brooks who was introduced yesterday. So please give a warm welcome to Lieutenant Commander Mary Brooks.
Mary Brooks: Well my voice is now nearly as big. So I (unintelligible) stay away. We've heard a lot of good information over the last two days. And for those who were here on Monday for the home care environment workshop, we know that reporting is important.
We talked about the 3500A. We talked about who needs to report and why they need to report. This is the first time a guidance document, a pre-market guidance document has incorporated post-market activities.
This is monumental for the FDA. We are working together, collaboratively networking with the different offices.
It was a hard concept in the beginning, making sure that TPLC included the post-market because it's such regulatory and it requires the pre-market clearance of the sites.
But like I said, through communication and collaboration among the offices and networking, I think that we've come up with a document that hopefully will produce safer products on the market. That's why we're all here today.
That's why the room is still full on Day 3 in the afternoon after lunch. And everybody's awake. We want safer products on the market.
I (unintelligible) we've received several, several, how many hundreds are you up to, questions today? To talk more about what needs to be in a report? What deems a quality report? We have several hospitals that have come up to us and asked us questions as to what needs to be in a report.
Well we need to make sure that there is patient information. We need to make sure that you provide (unintelligible), make sure that you put in the age and the gender of the patients.
(Unintelligible) doses and medication and the drug of the medication, what medications are recurrent going on in their medical history? What happens to the patient when the device fails?
Give us details about their (lungs). Was it a visual one? Was it an auditory one? When did it (unintelligible)? How did the vital signs change with the patient?
If you have multi channels, how many channels were affected? Did you need to remove the device out of the care environment? What happened to the patient when you took them off the pump? Did they need a second pump? Did they have side effects of not receiving the medication while they were waiting for the (medic) infusion pump to be loaded up and ready?
We ask that you make sure that you save your tubing and you give the tubing back to the manufacturer. We also want to re-emphasize that hospitals need to be working with the manufacturers.
You need to let your manufacturer and the reps know what's happening with the device. Feel free to file a report with the FDA, you know, under the regulations you have to if it's a death.
And if you don't know the manufacturer, if it's a serious injury, you need to report that to the FDA. But you can also send a voluntary report. When you do send a report to the FDA we're going to work with the manufacturer to (unintelligible) them up.
But make sure that the contact person from the hospital facility is available. A lot of times, and I'm going to have (Katie) shaking her head. A lot of times it's the night shift that had time to write up the report.
And you need several messages on the phone and you're the night shift nurse, you know, calling me back at 2 a.m. and it's really hard to follow-up with them.
So it's, you need to ensure that you have a point of contact that's available at reasonable hours for us to be able to contact them.
Again let me emphasize that hospitals need to be working with manufacturers. And manufacturers need to be working with hospitals.
And I'm adding that this better reporting, we're going to get safer products on the market and there will be less patient harm. I think that's it for me.
Woman: One last thing about MDR, Mary and I when we were talking about making these session a little briefer, when you send in a report, whether you're a manufacturer, whoever.
If you have taken certain actions, for manufacturers this might be you have a process in place for follow-up information, to call every week or to send an email or a letter.
When we get the report we don't know that you're doing that. So you end up getting an email, a letter or a call from us saying we - you need to follow-up on this. And you're like well we're already following up on this.
We can't assume that you're doing that. You need to tell us what it is you're doing. If you're not going to do any further follow-up, you need to let us know why.
If you're going to be doing follow-up then we probably are going to wait unless there's really good questions in our mind or the timeframe seems a little lengthy. We're going to wait for your follow-up report.
So tell us what you know when you know it. And what your plans are so that we can work with you to make a good evaluation. And that can only help everybody involved.
So I'm going to ask Mark Barnett and all of the FDA, the people who presented yesterday and today, to come up to the dais so we can go back to those wonderful, Mark tells me now more than a hundred questions, and start going through them.
And if you wish to come to the mic in the center, please remember to talk clearly and slowly into the mic and to announce your name and your affiliation.
All right, FDA would you please join me? And I know where you live so you can't hide. Come on up.
Mark why don't we take the question releasing to FDA who I understand who are in the hall. But...
Mark Barnett: Well I'll, I can - I'll just start out I'm thinking while they're waiting.
Mark Barnett: Can you hear in the back? Everybody okay? I had the (shaling) for lunch. And, you know, fish is brain food. And so I started to dig down into my very (rigor) knowledge of Greek Mythology to try to figure out, you know, which list most reminds me of this pile of questions.
And I thought maybe it was the one of (stish a fres), remember a guys he's gotten just pushing a big rock up the hill. And when he got near the top it always rolled down.
So I though maybe that was a pretty good analogy for this never-ending pile of questions. Then I thought maybe it's the myth of Hercules. Remember he was ordered to clean out the (unintelligible) stables in two days, except the more he cleaned, the more he got.
And I thought, you know, that - but I think the one that fits best, probably the most appropriate one is the rock. I figured about the (unintelligible) about this.
(Unintelligible) until the (NCA) people got here. But are we all here?
Woman: (Amanda) please go ahead...
Woman: Just start the questions.
Mark Barnett: Okay, all right. All right, let's go to a live question.
(Bob Barnum): This one, I'm sorry, my name is (Bob Barnum) and I'm with Healthcare Technolgies Consultants. This is for Mary Brooks. I dealt with MDRs for (unintelligible) I think that's (take effect) about 83.
That, the question I that I think is really interesting is the interaction between hospitals and manufacturers and it's tremendously important for a unity of for both to work together.
The biggest problem I've seen in (unintelligible) devices is one getting materials back to do testing. And I put that blame on the risk management within hospitals that they won't release.
Sometimes if there's any investigation that you have to go to the hospital, take a look and then they immediately put it back to a (dorm).
In some cases, some of the products have been taken apart to examine them. And within (a 30), I'll tell you, I had nothing left to use as evidence for any purpose.
But more importantly, it's working together. This is not a blame game. You know, in some states it's as simple as an industry blaming the user and the user is saying that it's got to be the manufacturer.
You know, in some cases, hopefully in all cases we're trying to get to an end result. And hope that (unintelligible) that you'll understand that the manufacturers truly are trying to get to the end result. And the hospitals will try and cooperate and provide the information.
And there is one specific part when you said what is needed from the hospital. If the device is connected to another device, so if you're using a last minute pump with a pediatric (tick) lines, you will get a half, a tenth before we change this.
And you have just created a different flow in to the labels on the pump. Thank you.
Mark Barnett: Thank you. (Unintelligible).
Mary Brooks: I appreciate you coming up and giving us a live comment. I (realize) I think about three times in that quick little talk how important it is that manufacturers and the clinicians in the hospitals speak to each other.
We have to figure out some lines that are (unintelligible) of communication, more outreach because your products are helping Americans and they're helping them save lives.
And the clinicians at the hospitals who are doing their part as well. But you got to figure out where that gap is. And I do appreciate you bringing that to our attention.
And we do acknowledge that it is difficult for the products to get (unintelligible). And we do ask that you communicate in advance to your hospitals. This is where your reps come in that are servicing the hospitals.
Let them know, have them communicate with the hospitals, you know, that they need to ensure that if the device fails, what should they be doing and how to get the devices back to you so that you can evaluate them.
Mark Barnett: Thanks. We have several similar questions as a matter of fact. So it seems to be something on some people's minds.
Mary Brooks: Yes that's specifically why I spoke to it.
Mark Barnett: Right thank you. This one says the FDA has placed an emphasis on static analysis. However, dynamic analysis is also a well-known technique. But it's not dimensioned. Why has it been excluded?
Dynamic analysis provides additional information not found in other techniques.
Man: Play it on. We kind of lump all the tools under one name. We tend to pick a name and stick with it. But the commentor is exactly right. And we should be talking about all of these tools.
Mark Barnett: All right this one says will FDA be introducing a new home use infusion pump classification with new product codes to ensure accurate post-market surveillance analysis?
Yes, the question says will FDA be introducing a new home use infusion pump classification with new product codes in order to ensure accurate post-market surveillance analysis?
Mary Brooks: Hi this is Mary Brooks again. At the time we - the home care community (unintelligible) those (enrollmenting) at this time. And as far as I know from the pre-market we haven't notice for that add in there.
Mark Barnett: Okay yes.
Man: That, you know, I just want to follow up with what Mary said. If we have a pump that was indicated for home use, certainly we could consider creating (fire) codes under the infusion pump regulations to track (unintelligible).
That's my comment. Thank you.
Mary Brooks: I think just to clarify that it would only be used for home and not hospital use because there is that definition between the devices that go back and forth. So if they indicate, if the person who has the question, are you saying for a device that's made specifically for home use and not hospital use? Is that person here to clarify? (Unintelligible).
So if the device is cleared for (unintelligible) right now then that is, then that's something that we'll have to discuss amongst FDA. But right now we do have devices that are in the home setting.
Also on the existing 3500A there is a box, a field for you to indicate what are the environment, if you say whether it's hospital or home. And (unintelligible) an unfilled field.
So we have no idea where the adverse event took place until we start asking questions. So it is there to use. And we can pull that data from our systems if it's provided.
Mark Barnett: We have at least two more questions here that go back to the idea that facilities are not returning their devices to the manufacturer for subsequent testing.
And don't want to dwell too much on it except this seems to be on people's minds. One of them says can the FDA help to facilitate these (migid) returns? I just read part of the question.
And the other one said will the FDA or can the FDA revise post-market requirements for the user to submit this information? So you may not want to comment on that if you already have. But it's a sign that people are thinking about this. Anyone, any additional comments or should I go on?
Okay. This one is for Alan. Isn't that sweet? It says regarding clinical evaluation, are you looking at measures of patients outcomes? In other words, if the pump is infusing morphine, are you looking to see if the pain is reduced or simply if the drug is delivered as programmed?
Man: Hi, this is (unintelligible). Excuse me, I think I addressed this earlier when we talked about clinical efficacy outcomes. And that is specifically what I was referring to.
Certainly clinical efficacy is important. But that's not what we're evaluating. What we are evaluating is device performance and (in factors). So if there's an adverse event that occurs or data failure of the device that occurred, we're interested in what was it about the design or the user interface.
Or what was it about the performance of the device that contributed to the failure?
Mark Barnett: Okay any comments on that. All right. This is - will clinical evaluation be required for other devices of the FDA? Is this the model for other types of devices in the future?
Industry leads predict that (unintelligible) would like to know now, not after a 510(k) submission.
Alan Stevens: Hi, this is Alan again. So our center director, (Jeff Sharon), in the announcement of this initiative I think has already indicated that we are looking at the model that we have put in place for infusion pumps as a potential model going forward in other areas. So it is certainly a possibility.
Mark Barnett: All right. Okay this is for (unintelligible). This one says when on clinicals have ever been run, what's going to be the clinical criteria for clearance? Will it be standards? Will every manufacturer pick their own clinical criteria?
Man: I - we're not going to dictate the clinical criteria. I think it's - it depends on your design and what you're trying to demonstrate. We sort of laid out what our end game is.
We're looking for the contributions to the device. We're looking for the performance of the device in an environment. And any contributions as Alan said to adverse events or device failures that can be mitigated.
That's really about as broader or about as specific as we want to get about - around that because as you design your product, we want you to sort of think about how your device fits in the environment. And what specific features would be important to examine in that type of a study.
So I don't think we would want to specify end points, even though you can, I mean I think some of the very clinical, some of the end points that you would see in other clinical studies are important, but not the primarly evaluation point.
We're looking at the device, the performance of the device in the environment. And particularly around the human factors and other things that are there. So your study is going to be end point, you're going to be very much related to the human factors and aspects of your device.
Mark Barnett: It was just pointed out to me that that card had a question on the back. So I'll read that one too. It says are we going to have to submit user evaluation results for review and then submit the clinical protocol?
Anthony Watson: I got to remember this (two way). This is Anthony Watson. I keep forgetting to add Part 2. We, the clinical protocol is probably important to get right up front.
So if I understood the question correctly, it was do you submit the user evaluations and then the clinical protocol. And the answer is...
Man: Have to submit user evaluation results for a review and then submit the clinical protocol?
Anthony Watson: I think you want to get us a clinical protocol. We can comment on that and see how the user evaluations fit into that environment to assess all of that together.
So for the clinical evaluation like we said, we're looking at that, those human factors, pieces. And the user evaluation as Ron mentioned, if you're - if that is important part, that's the more of a subjective part.
If you want to, if you're going to include that in there, we would want to see how you can, you know, include that piece into the evaluation.
So I would say when you're on your clinical protocol, (unintelligible) the manufacturers that have actually gone through this process actually know that we want you to come in early and we can tell you how, you know, what you're proposing fits into our - the overall plan.
And sort of our thoughts on how that can work. So I would say all that comes in at one time and let's just work our way through it to make sure that that happens seamlessly as much as we can.
Man: I'd add to that. I really think that we need to emphasis if it's clinical evaluation we're talking about, it's in the context of your design validation. And it should be reported as part of your design validation.
And it's something that the quality system regulation has required since 1996.
Mark Barnett: Okay. And this says we heard from (unintelligible) yesterday that collective actions take four to six months for a plan to be communicated from the manufacturer.
Often this delay is to obtain 510(k) comments. How does the FDA see these timelines impacting healthcare facilities when the new guidelines requirements add additional requirements like clinical trials for modifications?
Anthony Watson: This is Anthony Watson. I guess I'll start off. I don't know if our colleagues from compliance will want to have something to say about that as well.
This is another piece of the puzzle. So when we have a recall, compliance will often ask, our colleagues from compliance will also call us in and talk about, you know, whether it needs a new 510(k) and what would be the process to get that 510(k).
If there are other (mediated) requirements that companies may have to agree to. So we look at that whole package and decide what is that overall impact. And compliance does a very good job of examining shortage issues and, you know, talking with the companies about what the issues are regarding shortages.
So it is all part of our discussion. The way we view this is that whatever you put out there, if it's going to correct the problem, you want to make sure it corrects the problem.
So the information that we'd be looking for in the 510(k) would be targeted toward whatever the correction would be. So it's a validation of your correction basically.
So it basically is a necessary, in our opinion, a necessary step to get it out on the market. It's not an additional step, it's a necessary step. Do you have anything?
Valerie Flournoy: (Unintelligible) when there are corrective actions that are going on, we do work with the district offices as far as (unintelligible) district is the one that audits the firm through a recall process.
If we get information that is taken (unintelligible) normally you'd got through the district offices and we call the firms and say how to (unintelligible) be able to (confeature) corrective action.
And then you tell them, you take this (in law). Usually in talking with the firms we ask them what is your, this is what you saw in my slide yesterday regarding the proposed strategy.
So what is your strategy? And get the corrective action out as soon as you can. So based on that then we talk to the district and if there's been any type of complaints regarding that it's taking too long, then you get in contact with the district again.
And if we have to talk to the firm, we'll tell them it's taking too long. What can you do to get the corrective action out as soon as you can?
Woman: And by the way for those on the line that was Valerie Flournoy from the Office of Compliance.
Mark Barnett: Thank you. Okay this one says many aspects of the guidance are very specific to drug infusion pumps and don't - this goes back to a pump is a pump is a pump or maybe it isn't.
I mean we've had this before. But it's worth mentioning again. And many aspects of the guidance are very specific to drug infusion pumps and do not readily translate to (unintelligible).
For example, drug libraries, identification of FDA approved products to be infused. And someone else mentioned clocks as another questions.
So clarification of who the guidance is to be applied to (unintelligible) would be very helpful.
Alan Stevens: Hi this is Alan Stevens. So I think that the framework that we've outlined in the guidance document with respect to assurance case, the hazards that need to be addressed and mitigated while maybe some of the hazards that we have identified would not apply to an (eneral) infusion pump.
Certainly there are hazards that exist for an (eneral) infusion pump. And our expectation would be that the sponsor for that device would conduct a hazard analysis similar, you know, using what we have put forth as a framework. And use that to develop their assurance case.
Certainly a clinical evaluation would still apply because we're so concerned about device performance and human factors for the users. Pre-clearance inspection, risk management, I mean I think from my perspective all those sections are non-specific to the device that we're talking about.
I guess my sense of the question is that it's with respect to some of the hazards that may not apply. If the hazards don't apply, for example (unintelligible) infusion pump doesn't have software or maybe doesn't even have electronics.
Some of those hazards are just not going to apply. And that's okay. We would expect that. Our expectation is that you identify the hazards that do apply to your device and address those hazards.
Mark Barnett: I think that's a really clear answer. And I think the jist of all those questions was, you know, I've got a product and some of this stuff doesn't apply to me, what do I do?
And I think you're answer is perfect. If it doesn't apply to you, it doesn't apply to you. And you fulfill the rest of it that does.
Okay, this one says the guidance (unintelligible) infusion systems are to include IV sets. So if a new IV set is developed, will a clinical evaluation be required?
Anthony Watson: This is Anthony Watson. The guidance, this sort of gets a little bit, almost a little bit back to what we just talked about.
The guidance is about infusion pumps but also about the infusion pump system. So if you have a, we're not really targeting specifically the infusion set.
But as part of the, if you have an infusion set, you should look at that as the system that goes with the pump. So when you're evaluating the pump, you want to evaluate the infusion sets as well since that's part of the infusion system.
I mean we know that there are problems related to that as well. So you need to evaluate your infusion pump system. But we're not saying if you're going to (unintelligible) infusion line by itself, you need to do a clinical evaluation for that.
That's a little bit different. So infusion sets are actually handled under a different regulation. But if you're (smarketing) the infusion line with the pump, you need to evaluate that system in the evaluation.
Mark Barnett: I still got another question here that's similar. So let me give that one now. And the bottom line here (unintelligible) is it the expectation that we're going to specify the IV sets to be used with the pumps?
Alan Stevens: This is Alan Stevens. So I think the answer to that is that it depends. I mean if the - certainly the pump can only function with a specific type of IV sets. And that would be appropriate to include in the labeling because that is what's necessary for safe use of the device.
But if it's broad, more broad and it can be used with any infusion set then, you know, then the parameters of the infusion set don't have any impact on the safe use of the device. Then, you know, we would consider that, you know, slightly differently.
Mark Barnett: Okay we have several live questions. Let's go to the first one.
(Kevin Duran): Hi my name is (Kevin Duran), (unintelligible) research. My question is about the (unintelligible). If in a theoretical world we have a perfect submission, we comply with the (unintelligible) requirements.
The assurance case is in place. And we submit it in 90 says. The clock starts ticking. Will you be able to (unintelligible) support getting into a facility in that same 90-day period in order to wrap up that requirement for the submission?
(Miguel Patrick): Hello? Hello? Can you hear me? This is (Miguel Patrick). Yes the answer to your question is yes, the agency is ready to do that.
Now everyone knows that the agency is strapped for resources. Strapped is the wrong word. But there are some caps on the budget in terms of getting (unintelligible) to inspection, domestic inspections and especially foreign inspections.
So with that respect, I mean I can't say for sure that there won't be delays because there may be. But we will try our hardest to meet that goal.
But, you know, if we don't meet that goal, well that falls against us in terms of I guess the (unintelligible) agreement and whatnot. So we have an emphasis to meet these goals as well.
And there are people watching our ability to meet our timelines. So the answer is yes.
(Kevin Duran): So is there a caveat for the statute that requires a response to the responser in 90 days?
Mark Barnett: I'm sorry, what was that question?
(Kevin Duran): So my question is if you can't get it in, but the statute requires that you respond to (unintelligible), the response is because (unintelligible) inspection, you know, the time delay of approval occurrence. Is that (unintelligible)?
Man: Well the, let me make sure I'm answering your question correctly. This is (unintelligible) case situation. For those of you who know your number, Class 3, 5 and (k)s required inspections before they were clear.
And those came right down to the wire. We've often, we cleared them, but they were pending a GMPM inspection basically.
And so those weren't cleared until the inspection was ready to go, same thing here. The clock for us keeps ticking. We actually don't get a break. I mean if it's one of our, it goes against our (unintelligible).
So what (Miguel) mentioned, I mean it's of - it's our desire to get these things done on time. But the reality is the field has their own priorities to get to things done. This is one of them.
But as well as they have to still get to all the other manufacturers. So there is nothing, I mean this does not change the (madufinal) requirements on us. It doesn't change anything as far as that process is concerned.
We have to take the hit for (medufinal) if we go over. But obviously this is a priority for the agency. So we're going to try to get those manufacturers as soon as we can.
My understanding is that a lot of infusion pump manufacturers have been inspected within the last two years.
So maybe some of the smaller manufacturers haven't been actually inspected. But those that have been inspected within two years will be - will not have to, you know, wait for that clearance. (Unintelligible).
Man: I just want to make sure (unintelligible). We put a lot of resources and many of those are business milestones are set upon, our case submission.
Your first response to the agency's questions and then your response in clearing up that (unintelligible) requirements. So it's pure business planning. Thank you.
Valerie Flournoy: This is Valerie Flournoy. I'm going address this question in relationship to the Office of Compliance. We are (unintelligible) sections. But however, currently we are (unintelligible) to inspect all the infusion pump manufacturers, especially the ones that have had (unintelligible) inspections in the past.
So if by the time that you submit your 510(k) and you have not been inspected, you need to rise to the top of the list that you have to be inspected.
And so, but if you've had an inspection within the two years prior to the submissions (unintelligible). And it was an adequate inspection, you don't have to be inspected again.
It's only if you have not - if you don't get a chance to get to you. But all the manufacturers did need inspected very soon.
Mark Barnett: Okay next one.
Man: Yes please.
Woman: (Unintelligible) Medical Device Consultants Incorporated. And I've been helping the project teams that I'm involved with work with the (tim tack) of those guidance document on their project timelines.
It occurred to me that, an unintended consequence of what we're doing here might be that we're lengthening that time that the pumps on the market will stay on the market. Because it's going to take folks a little longer to do the testing that you're asking for to get the next gems out there.
And I was wondering if you had any thoughts on that?
Anthony Watson: This is Anthony Watson. Absolutely we thought about that. And the dilemma, just put yourself in our shoes for a moment. The dilemma is if you truly believe this is what you have to do to get better pumps on the market, you then not require that for the next generation going out.
It's a dilemma. Do you say make the bar where it is now and have everybody stay to that bar? Or do you say we realize this is what we think we need for the next generation to be better.
But we're going to allow something less than that. Something in between what we want now, or what we do now and what we want for just to make sure that products get out in the market.
And if you do that, then you take the chance that there really is nothing better getting out the door today in that interim then you would if you actually followed the guidance.
So it is, we did think about that. And we just decided that this is where we have to start. There has to be a starting point. And, you know, it will probably be a little bit more difficult now.
But hopefully as this moves along and we start, you know, get a little bit better momentum going, figuring out exactly how this thing works and having some examples of success.
Then, you know, that won't be so much of a barrier. There always is this in the beginning. But we decided that we either do it now or we don't.
Mark Barnett: Okay anymore comments on that? Okay good.
Man: I once worked for a company, not a medical device company, where the unofficial motto was we never have time to do anything right, but we always have time to do it over.
And since coming to the FDA I found out that there are some of those companies in our world. And, but there are plenty of companies who are doing things right.
And believe me, the learning curve, having been a part of a company that went from the bad kind to the good kind, the learning curve was steep, painful and took a lot of extra time.
But the end result was that it didn't end up taking more time. It took less time. And that's what I think we want to strive for.
Mark Barnett: Thank you. This next card says the guidance document calls for human factors testing early in the design process. But the FDA is typically involved at the end of the design process.
Additionally no one is exactly sure what these studies should look like. Are manufacturers expected, excuse me, are manufacturers expected to submit such studies to the FDA in advance of the normal 510(k) process? And if so, how will that work?
Woman: Ron Kaye could you come forward to answer the human factors question. Just take us a second. He's coming up.
Ron Kaye: Could you repeat the question.
Mark Barnett: Yes, okay the question says that the guidance document calls for human factors testing early in the design process. But the FDA is typically involved at the end of the design process.
Additionally no one is exactly sure what these studies should look like. Are manufacturers expect to submit such studies to the FDA in advance of the normal 510(k) process? How will that work?
Ron Kaye: Okay well what I emphasized today was what we look at primarly and the point of my presentation was about validation testing. And those - that testing will occur at the end of the design process.
Now you are encouraged to do human factors work earlier in the design process. However we don't specifically review that. It could be that if you were good enough and fortunate enough, you might not do any human factors work at all.
I mean you might be so talented you could make an infusion pump interface that would pass the validation test. And more power to everybody who can do that.
Unfortunately that, usually or quite often when the first human factor's involvement is the validation testing, there are problems found in that testing.
So we recommend that you do your human factors work at an early stage. And preferably iteratively as your pump is being designed. So that by the time you get to the validation, which I did talk about, I mean that was all the methodology I was talking about earlier this morning, that you'll have a good result there.
Mark Barnett: And there's another human factors question. So I'm going to give you that one now. It says human factor assessing typically involves observers observing users and documenting the results.
If this is expected in clinical evaluations, has the FDA considered limiting clinical evaluations to a few sites and a small number of users so that adequate numbers of observers can be deployed?
Ron Kaye: Yes that's a practical challenge I believe to doing this type of testing. And I'm, you know, quite frankly that's going to be a subject of some discussion as we proceed here.
(Donald Sears): Yes I just wanted to, I wanted to touch back on the other question with respect to the iterative validation and my interpretation of the question, I'm sorry, this is (Donald Sears).
My interpretation of the question is that this, whoever asked the question is concerned about doing iterative testing. And then by the time they get to us at the validation stage, that if they haven't done that correctly, that they've spent a lot of time and effort in testing inappropriately so that they haven't correctly identified the user requirements and the issue of developing those protocols.
And so, you know, I want to tie back to the second part of our infusion pump initiative which is proactively facilitating device improvements. We have a process in place whereby we are available to provide feedback through the pre-(ID) processing.
That, you know, I only say that don't be alarmed in a sense that it's called pre-(ID). That doesn't necessarily mean that we're talking about an ID submission. We use those all the time for feedback on human factors protocols for so many other purposes.
So certainly there is that mechanism available to provide you feedback on protocols.
Man: Yes and in fact that's true. However the pre-(ID)s have to do with validation studies. So I think the question I think was for preliminary work before that point.
You know, this would be the more - what could be called the formative work. And that is very early in the process. And it can be done much less formally as you wish.
The idea is that you're not trying to create some sort of a test document that the FDA is going to review itself. You're trying to find out well where - how good your device is with respect to it's use by the intended users.
And if you think your employees represent the user population and you think you can get good information that way, you can use them for that testing.
And but there are, you know, there are methodologies available for formative testing. FDA has a three-day course on validation testing. (Unintelligible) it also contains a lot of good technique on human factors testing in general.
So that, I mean that's one source. And there is, I mean I would suggest that what that testing would entail would be to focus on the requirements of the validation testing as your ultimate goal in being able to achieve good results there.
And then it's just a matter of, you know, doing what sort evaluation is reasonable leading up to that point.
You know, the idea is not to go into validation testing, you know, essentially blind with respect to what types of problems you might have at that point.
Mark Barnett: Okay. All right now I think (Willis) I think you said you wanted to be done here at half past two, is that right? If that's the case, we do have one live person. Do you want to do that first?
Woman: All right, is this a brief question or, okay because we can take you first when we come back. I wanted to give people an opportunity for break. All right well why don't you ask your brief question.
Man: Yes just ten minutes.
Woman: All right well, you just start and we'll be back.
(Brad): No I just want to clarify, I'm (Brad) (unintelligible). I just want to clarify the answer to the last question. So the formative study of the guidance document as a suggestion, just as a requirement.
Or if it's done, the focus studies don't have to follow any specific standard. Device won't be fund not (substantially equivalent) because of the formative studies. Am I understanding that correctly?
Man: Which guidance document are you referring to?
(Brad): The one guidance document that, sort of that we're talking about today.
Man: Okay, it doesn't really have a formative study section to it.
(Brad): It's a paragraph. Is the two paragraphs then.
Man: Yes that's a paragraph. It recommends...
Man: I mean that is a recommended course to take. And the whole idea is that - is to, you know, facilitate the design process such that by the - when you do your validation study which is a formal study, then you'll get good results on that.
(Brad): Okay thank you.
Man: Yes okay.
Melissa Eackle: Thank you very much. That was very quick. This is Melissa Eackle. We're going to take a 15-minute break here because I know there are some people who have questions that we didn't get to write down.
Is anyone else warm in here?
Melissa Eackle: Okay all right. I'll go see what I can do about that. We'll see you in 15 minutes. If you have further questions please get the cards and submit them. Thank you. People on the line please do not hang up.
Melissa Eackle: Hello. If you could start moving towards your chair, we're going to be starting (begin). FDA people, if you would return to the (unintelligible), we still have more questions to answer and we just get handed more. Yes, you did, and I'm tired of it.
First of all, I want to - giving you some hopeful news. There's actually - they tell us they're working on the thermostat so they're hopeful it will not be so stifling.
But I want to give everyone permission, if you feel comfortable, to take off your jacket or your sweater as long as there's something underneath, so we maintain the proper decorum and get comfortable so we can answer these questions and not be sweltering.
All right, I'm going to turn it - oh, there was a point of clarification, Mark, before we go to the questions that was asked of Valerie and I during the break. So, Valerie?
Valerie Flournoy: I have to remember the thought. This is in regards to the health care facilities not returning the device back to the manufacturers.
You have to remember that the (unintelligible) manufacturers (unintelligible) not determine that the root cause (of the failure) is unless you return the device. Otherwise, their corrective action cannot be implemented as quickly as you would like it - as you've told us here today, and (unintelligible) with you some other options healthcare facilities (and what you can do).
Woman: And just to clarify, the FDA - we do not want to receive your used medical devices. They need to be (unintelligible). Please do not mail them to us. They need to be returned to the manufacturer. So that has been brought up. We'd like to clarify and make that clear.
Do not finish your broken medical devices. Please submit it back to the manufacturer who manufacture them so that they can take that information back into their (TPLC) process.
Okay, two things. As an analyst I remember my favorite memory of opening a FedEx envelope just finding (a little) (unintelligible). So I want to reiterate, please do not send us (gifts), because I (unintelligible) very careful that (I could) very easily been stuck with a (unintelligible) needle, and that would've been very unfortunate.
The options as an analyst - when I would talk to facilities when I was an analyst and they would say, “It is not our protocol to return the device to the manufacturer for any number of reasons.” And I do understand that.
What we would do is we could talk about options for this. Some of the options are if you're set up to do your own evaluation to invite the manufacturer to be present with their experts so that they can discuss the protocol and actually absorb the evaluation. So, that's one thing.
Another thing that you can do is use a third-party forensic organization. I'm not going to mention anyone, but at least one member of one is here present, and they do objective evaluations of malfunctioning devices or adverse events if you ask them to.
So, the one thing I want to ask you not to do is I understand many hospitals (unintelligible) by manufacturers sequester devices for lengthy periods of time, a year and longer. And I understand that you have reasons for wanting to do that. But the real thing that happens is if there is a critical error, any pump that's designed like that can (fail) in that way.
So the person being (affected) is not just the person you're concerned with, but every other person who uses that pump. So it is really critical to us that when there's a malfunction, especially with a severe injury or death, that that pump is evaluated correctly and as quickly as possible so that if a correction needs to be made, it can be made quickly and that we limit the risk. Thank you.
Mark Barnett: (Unintelligible) questions that - oh, go ahead. We'll take one right (unintelligible).
Man: Oh okay, this is actually just a comment to the commentary that's been going on about returning the devices to the manufacturer.
The suggestion of having a forensics firm or whatever look at the device or looking at it at the institution is also suboptimal to the manufacturer because they have test equipment and other bench equipment that will allow them to investigate beyond the normal use of the device in their own laboratories, which may not be able to be brought elsewhere.
And therefore, getting that device back in tact after use is very critical, as you pointed out, for those - the speedy result resolution of whatever issues may occur.
Mark Barnett: Thank you.
Woman: And let me just say I do agree with that. That's the ideal situation, but unfortunately many facilities' protocols does support that. What we don't want to happen is the device not to be evaluated. If you think of other options, please feel free to share those. But the important thing is to find out what happened.
Mark Barnett: At least three of the questions, and maybe more down deeper in the pile, have to do with FDA staffing and concerns about whether the FDA - (unintelligible) - and concerns about whether the FDA is going to be able to perform these new requirements and still do it in a timely way. Let me read one of those for your consideration.
Says, “How is FDA staffing to address increased volumes of submissions, increased amount of data and submissions, and more facility (environment) inspections? Is this staff being trained? Timely reviews and inspections are critical to innovation and advances in patient safety.
Currently, FDA timelines for submission reviews are increasing significantly, and without appropriate staffing, this is going to get worse. Anybody want to comment from the FDA?
Anthony Watson: (This is) Anthony Watson. This sounds like a management question, so I guess I'd better take it on.
The reality is we work with what we have, and we have great staff that work as hard as they can. We are - we do have increasing timelines in some areas. Not - that's not true across the board.
You're looking at summary data and we have hills and valleys. You know, some groups have it -- have it hit real hard and some groups don't. And ones like our group, which have initiatives, are going to get hit hard.
So, you know, we have great staff that are working very hard, and that's, you know, it's an office - the whole center's involved here. So the workload that was now primarily in the pre-market area, and even in post-market stuff is going to - everybody's going to be involved now.
So, I don't think it's going to get any easier, like I said before, but we are - we've talked about it. All the offices have bought in, all the office directors were there, they've seen this guidance document. They all know. So everyone has said, “Let's take that step and let's just do it.” So, that's what we're going to do.
Mark Barnett: Okay. This is a - we're - and I wanted to...
Mark Barnett: ...repeat - there were several questions with that theme, and I read one of them. Yes. I'm sorry.
Man: You know, the other thing is what we may lack in - well, appear to lack in resources, but I think we'll just make it up in sweat. So, a lot of us are putting a lot of time in to make sure that you guys get the full review that you deserve.
Mark Barnett: Thank you. As I said, there were several questions like that. I read one of them, which is representative of the others.
This one says, “The requirements of a five-point case submission closely mirror the requirements for a PMA. Is this the direction the FDA's going to be going for - with all 510(k)s in the future, or is there going to be a further breakdown of Class 2 devices similar to the Class 2A and 2B, which will determine which of the devices will undergo this greater pre-market scrutiny?”
( Murray): Hey, I'll tackle that one. No one was ever carried by our feeding back sound system. It's a low risk device from a clinical point of view, but it can definitely carry you if there's an electrical fault in the system.
And what we're talking here is about the engineer - reviewing the engineering of the devices, not their clinical utility. I think we pretty much understand how infusion pumps work. We want to make sure that they do work.
And so when you see that the requirements are (unintelligible) for PMA, we're talking about the requirements to submit information about the engineering, but we're not asking you to do a clinical study of the - of clinical effectiveness.
I know there were a number of comments during the break along the lines of, “Well, instead of 2100 pages we'll be submitting 3100 pages.” I think there are many times when the reviewers would be happy with 50 pages or even 10 pages of cogent arguments supported by evidence from your design history files in lieu of 2100 pages that don't tell us very much.
So I think that's - this really is difficult. I don't want to make light of the effort to substantiate these claims of safety and effectiveness. This is very difficult, but it is something that a lot of medical device companies are already doing, and I - it's just where we need to go I think.
Anthony Watson: You want me to answer that?
This is Anthony Watson. The only thing I would add to that is, there's nothing that we're asking for in this guidance document that cannot be asked for under a 510(k). It's just more of what we need to make the decisions - better decisions. That's really what it's about.
For sure we think this question went I think beyond infusion pumps and asked about really - are we looking at a trend in the FDA with more than just the infusion pumps (unintelligible) devices in general. Is - that make sense to you guys?
( Murray): Well, I think Alan pointed it out earlier that what we are doing here with the infusion pumps is being looked at as a model for other devices. Not necessarily in the requirements per se, but in the method - the analysis that we went through, the way we rolled out the process. Not just from what has been externally rolled out, but from what happened internally.
I've been talking about the way we got the guidances - the guidance document through. That was actually quite - I think I'll say innovative in the way we handle guidance documents. It was - involved a lot of people and (it had to get passed very quickly) for - at the higher levels.
And, you know, when you try to pin five office directors in an office and say, “You can't leave for three hours ‘til you go through this and tell us what problems you have, whether there are any questions (unintelligible) we're going to answer you right now,” just like we're doing with you right now. You get some interesting responses.
So it was worth it. It was a lot of work, and it's just staring for all of us. But, you know, the real bottom line is it's a model. Other people are looking at it. And as far as the two-way (unintelligible) I'm really not qualified to answer that, to be honest with you.
Mark Barnett: Okay. This one says, “When a contractor or supplier helps to design or manufacture parts of an infusion pump product, what expectations for risk assessments, design, and process controls is required for a contractor or supplier?” And it says here in parens, “Assume the contractor and supplier does not provide clinicians of similar - simulated systems or MDR interface.”
Valerie Flournoy: This is Valerie from the Office of Compliance. Just remember that when you all - (unintelligible) the 510(k) so when you're working - let me back up - during an inspection and you're saying that you have a contract manufacturer, remember you're supposed to have agreements and you're supposed to audit your supplier so you can inspect it, investigate it (unintelligible) to look at those types of records (if there's an) issue.
(Unintelligible) be talking to you. The same thing applies however (unintelligible) but now we just (unintelligible) inspection. Process (unintelligible) similar to (unintelligible) focus now (unintelligible) on design control, your complaint handlers, your (unintelligible) and your MDRs.
Mark Barnett: This is another one about logistics. It says, “For pre-clearance inspections, if the design occurs at one location, the manufacture occurs at another location -- in this case, international -- and compliant handling is at a third location, will all of three of them required to -- something - -510(k) clearance?” Yes? Right.
Man: Well, I think...
Man: Yes, exactly...
Man: ...the short answer's the - yes. But the more complicated answer is it depends.
The - I mean if, you know, the fact that - there is one sponsor for the 510(k)s for the submission, so somebody's ultimate responsible for the design of this device. So in that respect, we're obviously leaning on the - on that person or that entity to demonstrate that the device is safe and effective.
But, at the same point, yes, you know, actually I'm - Valerie, if I'm not mistaken, there is a regulation in the CFR - in 21 CFR that says that if you have varied elements of your quality system in foreign locations, it's the responsible of the sponsor to make sure that the information is accessible at the time of an inspection. Is that correct?
Valerie Flournoy: This is Valerie. That was correct. So the - what I'm going to say is that, remember when they fill in and they inspect those forms, manufacturers, they're expecting that when they look at the complaint handling, those procedures of how they process the complaint handling, what is - what type of (unintelligible) analysis (unintelligible), that should be provided to the investigator. They shouldn't have to come back to the US to look at what was done in another location.--
Mark Barnett: Okay. This is one says, “Can you explain why externally-trained health care professionals that are company employees should not be able to participate in the - (unintelligible) for the first time.” Help me out here.
Man: Simulation tests.
Mark Barnett: Oh, (unintelligible). Okay. I thought it was (unintelligible). Some more (unintelligible) testing. Okay.
Man: The idea is that employees were not considered to be representative users for a variety of reasons. I mentioned familiarity with the product, which is often the case. They also tend to not be as diverse as the user population for various reasons.
But, you know, for - probably the main reason is that the expectation is that those employees will not feel as able to be critical about problems with the device that their company is manufacturing. I believe that's a legitimate fact myself.
When I was in the private sector, it was very difficult as a human factors and (unintelligible) to critique the system that we were developing, which was (unintelligible).
But that (unintelligible) is not based on my personal experience only. That it - the concern now - it could be argued that in any given situation that may not be true (unintelligible) in such a reason.
However, you know, there's so much to this testing and so many aspects of it, and we get so much of it, you know, coming through the door at the FDA, that have - when we have a requirement like that, that makes sense in general, we pretty much have to stick to it. We can't make exceptions here and there.
Is that okay? Hopefully that answers your question.
Mark Barnett: Okay. This one says, “Is there are assurance case reports on method of documenting the risk? How does assurance case reports link with risk management summaries?”
( Murray): Yes, this is ( Murray), but I have to say that I wish Rick Chapman were here. He's actually our expert on assurance cases, but I - you know, my understanding of the assurance case report is that there's - I mean all of this has a threat of risk management to it.
At some point the assurance case is going to have some evidence to support how you mitigate the risks. So - and maybe I'm misunderstanding the question, but there's this thread of risk management through all of this, and so the assurance case is going to have some impact on the risk management report and vice-versa. I think they're complimentary pieces of information.
Beyond that, I would not want to get into any other technical details just because I'm not an expert in that area. I understand from a high-level what it means but Rick is our expert in this area.
Man: I just want to make, you know, we do say in the guidance that (unintelligible). We do mention in the guidance document that the risk management activities and the documents therein are to be included, and we do consider that they're complementary with the assurance case, and they would be included as either evident or part of the argumentation for the claims. So there certainly is a link between these activities.
Mark Barnett: All right, this one says, “What are FDA's expectations for the scope of a clinical evaluation from the infusion pump? In other words, the number of users, the number of uses, the number of devices and so, on?”
Man: Asked and answered. Think we've already answered that question.
Mark Barnett: All right. Couple of drug questions. “For infusion pumps intended to be used with a specific drug, how will the agencies ensure consistency in labeling between the device and the associated drug?”
Alan Stevens: So this is Alan Stevens. I'm going to address the question as best I can with my interpretation of the question. So, the way we look at - so we're talking about a situation where the device is indicated for a specific drug. Let's take insulin pumps for an example, because those are indicated for insulin, which is a specific drug.
And so, what we do is ask that the sponsor or identify or evaluate - in this case, we know if - we already know ahead of time that there are insulins that are approved for infusion so - for an infusion pump, namely Apidra or Humalog and NovoLog. And those labels allow the device - the language on those labels allow the device to be (unintelligible) cost (unintelligible) with the drug.
If you look at the labels for insulin, that it will say - (I don't know) the exact words off the top of my head, but it said something like, “For use with these following three devices that - before using another infusion pump, please check your infusion pump labeling.”
And then the infusion pump label will say, you know, “We have evaluated our device with, Apidra, you know, and blah, blah, blah, blah, blah” or NovoLog. So this is the kind of framework within which we expect that that information's going to be presented.
Certainly if the labelings in the drug would need to be changed so that it's a, you know, if it's exclusive, it's a combination product, that would probably require - or would require a cross-labeling, and the device would be considered a combination product in that case.
Man: ...answer the question.
Mark Barnett: Thanks. What - I want to keep my eye on the clock. What time are we supposed to be done? Okay.
Woman: (Unintelligible). Three-forty-five...
Woman: ...and (unintelligible).
Man: Okay. So - okay, I'm - I'll watch my watch. It says, “If robust -- in other words, professional or third party -- simulation labs can be used, can I get in for clinical (unintelligible) to be made, and if so, eliminate the need for an IV in clinical testing?”
Man: (Unintelligible). The question sounds like is it - can an argument be made with a great simulation that...
Man: Yes, simulation labs instead of the IV in the clinical testing.
Man: Do you want to address that one?
Man: Yes, I think - yes, the answer's that that would - given the direction and the interest in the clinical evaluation, the answer to that would be negative. The - and that is because, theoretically, you could use a simulation lab to be very much like an actual - you (unintelligible) environment. And it (unintelligible) that's true enough.
However, still the fact that it - the simulation lab is a very high (probability), the simulator is - you - the quality of the evaluation has more to do with what you do with that lab.
You have to - and again, speaking from personal experience, you have to set up the scripts, you have to set up the scenarios, you have to know what your - you're going to evaluate and how and plan for that.
And it's really a lot of work. And the - and given that that all could be done - you know, the idea of a simulation lab is, you know, you want to be able to test in those conditions for specific reasons because you think certain aspects of that environment are going to be important to consider. And a lot of that has to do with realistic use, distractions, shift changes, that sort of thing.
And you get a - at the point that you're at the level of really emulating a - an actual clinical environment, it (unintelligible) critical amount of effort with that lab, and a lot of time, you know. I know we haven't give you a good specific answer on some of the questions about the resources for the clinical evaluation.
However, simulation labs are expensive, and I, you know, probably would not want to run, you know, test participants in a simulation lab that emulates an actual clinical facility for - in a period of weeks, for instance. It'd probably be more cost effective to do an actual clinical evaluation.
And because of the - and then getting back to the first point, because of the uncertainty about how well that represents the clinical environment anyway, it's not clear cut that it really would.
However, for simulated use testing, if you have access for - to a simulation lab and you want to use that for your simulated use testing, I mean that is simulated use testing.
You could certainly use that, but I would advise that you would want to know why you need that level of fidelity. And you very well could, based on your analysis. But a simulation lab is not a magic thing that's going to automatically provide you with good test results. It's- there's a lot of effort that needs to go into those to do a - to run a good test.
Mark Barnett: Okay, this question seems to be kind of a reverse of the previous one. It says, “During the analyst presentation you mentioned that IDEs will now require a simulation testing for support. If the verification and validation suite is robust in its inclusion of system testing, will there still be a requirement and may not be appropriate for early stage development?”
Alan Stevens: Hi. This is Alan Stevens. I think the point of what I was trying to communicate in my presentation is that when we're viewing and we're reviewing the (IDU) submission, the expectation is that we're going to be looking at all the, you know, we need to ensure that the device is safe for human use.
So, what we do to - and we need - that means we need to look at (bench dealer) software validation, you know, for all the other information and bench data that we're already going to be looking at.
In addition, we also want to see the simulated user testing so that we are assured, to the extent that we can be, prior to studying it on humans, that the device is as safe as it can be.
Mark Barnett: Okay. This one says, “For system components that are either procured or internally developed, do we need to just fully characterize them in the context of the infusion pump system, or do we need to go back to the design intent and requirements of the component?”
Man: (Give me) that one again?
Mark Barnett: Okay. For - some of these are long sentences. I mean, there's a lot of commas and semicolons and - “For system components that are either procured or internally developed, do we need to just fully characterize them in the context of the infusion pump system, or go back to the design intent and requirement of the component?”
Man: (Unintelligible). You need to show that the component is suitable for the intended use in your product, that it fulfils its requirements in your design.
Mark Barnett: Okay. This one says, “How do you expect drug compatibility to be dealt with, considering new drug approvals? Are you expecting manufacturers to specify all (approximal) drugs (unintelligible) infused, or are you looking at the material compatibility?”
Alan Stevens: Hi, this is Alan Stevens. So (any of the) new drugs are going to be approved, you know, after the submission is cleared. Right now the labeling - it doesn't say anything with respect to drug compatibility.
And, as I mentioned earlier -- and this is kind of an evolving requirement, and we're looking at tidying up this requirement a little bit more and, you know, any comments that can improve the evaluation of the compatibility of the drugs are certainly appreciated.
From our perspective, given the information we've seen and the affects that the device can have on the drug that's being delivered, and some of the safety regulations that we've seen, we do believe that it's necessary to at least minimally evaluate the drugs - a set of drugs. And, you know, we touched on that a little bit earlier.
So, from our perspective, there is a safety issue that needs to be addressed.
Mark Barnett: This one says, “Will FDA provide a detailed guidance or remove assurance case report requirement in order to ensure consistent and expedient results and approvals?”
Anthony Watson: Hi, this is Anthony Watson. There's no plan at this very moment to develop a guidance document specifically around assurance cases for infusion pumps. That can be something in the future.
I think it would be helpful if we can get something that seems to be working, a model of an assurance case, to publish that, to get it out there for people to use. I mean, I think we want to be able to do that. And you guys are the pioneers to make that happen.
So, you're going to figure that out, and then we'll get it out there in some form or another once we've feel like we've gotten something that we can say. We know enough about a good assurance case for infusion pumps that we can publish something.
(Nick Thacker): If I can tie (unintelligible) onto that, this is (Nick Thacker). A lot of the comment that you - questions that you've had and the comments that you've made are really good comments, which - I mean, I'd really appreciate if they were part of the docket or the response to the guidance document, because some of the questions are regard to our process and how we're going to do what we need to do, and then some of them are like the question that was just asked.
So, please remember these questions and submit them as part of the docket.
Mark Barnett: All right, this one says, “Since manufacturer letters are not mailed to any central location in a facility, is there any plan to have an electronic system in place for all manufacturer correspondence? In other words, recalls, alerts, hazards, notices, and so, on?”
Sure. “Since manufacturer letters are not mailed to any central location in a facility, is there any plan to have an electronic system in place for all manufacturer correspondence?”
Valerie Flournoy: This is Valerie from the Office of Compliance. Well, right now you all are aware that the recalls are placed on FDA's Web site. And we have been working with our Office of Communications staff as - what do I want to say - like question and answers as problems are brought to our attention, and we feel like some type of communication needs to be issued, it is put on FDA's Web site.
We are now stepping outside of our box, where if we need to go to some of the trade organizations and other medical organizations, we do do that also from (that watch).
Man: I'd like to get a clarification. What do they mean by manufacturer letter? Are we talking (forwarding) letter or what?
Mark Barnett: Let me go back to the question. Hang on. They just talk about manufacturer letters. It seems like they've got correspondence to manufacturers.
Man: Well I mean...
Mark Barnett: So does it end up on a loading dock, or does it end up in the risk manager's office?
(Nick Thacker): If the - so this is (Nick Thacker). If - well, I mean if the address that we have on file is your loading dock, then I guarantee that's where it'll end up. The pre-market letters go to the official correspondence for the sponsor. Post-market letters go to the post-market letters - I'm talking about two categories. One is warning letters and (unintelligible) letters. Those go to the sponsor where the facility or the inspection occurs.
And then there are correspondencers (OSD) called additional information letters. They're a follow-up to medical device report. Those go to the same place, like to the facility - to the MDR reporter. So I think that answers that question.
Mark Barnett: Okay, thanks. Mary, this one actually has your name on it, so I think I'd alert you in advance. It says, “How does FDA view the role of patient monitoring of ventilation and oxygenation as a risk control tool for PCA pumps?”
So they're talking about - I think it's what you - our or your opinion is about the value of monitoring ventilation and oxygenation in a patient that's getting - that has us - that's using a PCA pump. I'm not sure that that's relevant, but that's the question.
Mary Brooks: Yes, I - we received a couple questions. I think they have them marked as device to sell, but definitely - I guess I think that when you are - a patient is (receiving) medication of any kind, whether it's PCA or a blood pressure medication, so forth, you need to be looking at all of the monitoring systems that you have available.
So obviously if you have a patient on insulin, you need to make sure that you're constantly checking their glucose. Same thing if you have a patient who is on (apressa) or vasopressin. You need to be constantly monitoring their blood pressure.
So I guess I think with any infusion, based on what the medical is, you need to be looking at alternate monitoring systems.
Man: Okay, thanks. We're almost...
Mark Barnett: Well, let's get one more in. I know people want to go outside and get cool. It's 88 degrees out there, but you'll cool off when you get - going out there.
“Do you think the large discrepancy between industry MDRs and end user MDRs is due to over-reporting by industry?” My - I would add to that. It could be under-reporting by the users, but anyway. And it says - well I guess that's (unintelligible)...
Woman: (Unintelligible) Mark, I'm sorry. It's incredibly hot under this light with this uniform on.
Mark Barnett: Yes. It says, “Do you think the (large) discrepancy between industry MDRs and end user MDRs?” In other words so many more MDRs in the industry...
Woman: Oh, from manufacturers...
Mark Barnett: ...from manufacturers, yes...
Mark Barnett: ...than from users, okay.
Woman: No, when we cross-compare voluntary reports and new facility reports, we look to see whether or not the manufacturer is also sending in information based on those MDRs and not.
So we absolutely do not feel that the discrepancy is based on the manufacturer's over-reporting. We think that manufacturers are underreporting, and we think that user facilities - we know that user facilities are underreporting, and we absolutely would love to receive more voluntary reports.
(Unintelligible) the information to be as much information in there as possible with the (unintelligible) device, what happened to the patient and so, forth.
Mark Barnett: All right. Everybody's fanning themselves. You want - (who is) the moderate? Who do you think?
Woman: I'm ready.
Woman: (Hit the lights or something).
Woman: All righty, we - as I said previously, we are going to take the questions that were unanswered back to the shop and we're getting them up on our infusion pump Web site.
Going forward, don't expect them tomorrow or the next week -- so two weeks -- but certainly we're going to put them all together and get you answers so that you can see what it is.
I'd like to call (Tony Watson) back to the podium. He has a few things he'd like to say. But before he does, as the chairwoman of the infusion pump working group, we are extremely grateful to those groups on the phone and everyone who came and participated to start the conversation in getting it moving forward. We really do appreciate it and we are looking forward to working together. It wasn't me.
Anthony Watson: (Unintelligible). This thing doesn't seem to be holding on. I'll try to do it by holding on. Can you hear me back there? Okay. (Unintelligible) when we look back at these two days, those of us who were here will have to be our own harshest critics.
But it will take some time to see the fruits of our efforts, however, in a really short period of time, for you to start seeing some activity, tangible and meaningful activity, leaning towards better infusion pumps.
Imagine that we will do this again in the future in some form, to asses where we are as well as what we need to do next. Some of the things I have seen, are is it possible - possibility of another workshop, some other type of conference, partnerships. We've heard all of those and we think they're all very valuable things to take back and think about.
Clearly, we have our own clarifying to do. We hope some of the comments about the guidance document. We also want to emphasize to please send those, as others have said, and as I have said before, that (Tom), please send your comments to the docket. That's where they're officially recorded, and that's where we will respond to them.
Even though we gave you some ideas into our thinking, those are temporary ideas and some thoughts. But until we go back and actually digest the questions and really get into a discussion, they're still very preliminary.
I personally - and I know others of us here like to keep the lines of communication open as we move forward. So as we start to look down the road -- and this came up earlier -- we need to start asking the questions that will measure whether we have attained our goal.
Although I believe that measurable goals are a critical element of assessing our success, I mentioned this earlier, but I propose to you that we can never reach an end point - truly reach an end point with a goal that includes in its mandate the word better. (Unintelligible) goal and it's an never-ending effort.
So you get better then you start, (whether or not) you (move) baseline, then you get better from there. Eventually, hopefully, the gains become minimal because we've reached the limit of our success. I refer you back to the airliner that lands the most - no passengers injured and, you know, the news - it's all over the news.
There it is. So now I don't have a problem with history judging us. I'm saying we didn't - we failed to reach our end point. I don't have a problem with that. I do have a problem if they say we failed to try. And that's what this is about. Let's try. Let's just do it.
I make - I may, you know, I may be oversimplifying it. I (unintelligible) too. I know it's a big task, but we ought to try. There's no - this - these are important devices, patient safety-critical devices. We have to try.
I'm willing to admit that this sounds lofty and a little bit pie in the sky, perhaps unrealistic, but as I said earlier, I am an optimist. Believe it or not, I am an optimist. I don't have everything else. I want to thank you for being here.
I've had an opportunity to talk to some of you in other venues, and I've met some of you for the first time. I really enjoyed talking with you, and the ideas are coming. My brain is full of them, and I'm hoping to get some more and...
I'm sorry what was that?
I'm not very good at reading queue cards, so I have to - oh yes. I'm sorry. How could I possibly forget to thank the folks who are involved in the work group?
So I'd like to ask the infusion pump working group to stand up and acknowledge them, please. (Debt) to you folks that are involved in this (unintelligible) work group. Please stand up. Thank you all, okay? And I want to ask you please travel safely. Thank you and goodbye.
Coordinator: Thank you. That does conclude-