[CONVENE 8:34
A.M.]
M. STOTO: There we go.
Everyone, please remember to turn your microphone on before you
speak. I’m Mike Stoto, the Chair of the
Committee. I’d like to call the meeting
to order. David Butler, who’s going to
be presenting the report from the Institute of Medicine needs to leave by a
certain time, so I want to make sure we have that discussion and the subsequent
discussion from the Air Force that relates to it as soon as possible. So I think we’ll try to keep this first part
relatively brief and come back later to finish up things that we might have to
do otherwise.
So let’s begin, if we could, by just
going around the table and then the room with people introducing
themselves. I’m, as I said, I’m Mike
Stoto, the Chair of the Committee and I work at the Rand Corporation.
P. CAMACHO: I’m Paul Camacho and I’m at the William
Joiner Center at the University of Massachusetts.
L. SCHECHTMAN: Okay.
I’m Leonard Schechtman, FDA’s National Center for Toxicological
Research, Deputy Director for Washington Operations, and the Executive
Secretary of the Ranch Hands Committee.
S. LEFFINGWELL: Sanford Leffingwell. I’m a consultant in occupational and
environmental medicine, formerly with Centers for Disease Control.
R. TREWYN: Ron Trewyn, Kansas State. I’m here to be as far away from Paul as
possible so we behave ourselves today.
D. JOHNSON: Dave Johnson, Florida Department of Health,
Executive Medical Director for the Division of Environmental Health and Acting
State Epidemiologist.
E. HASSOUN: Ezdihar Hassoun, Professor of Toxicology, the
University of Toledo.
K. OSEI: Kwame Osei, Professor of Medicine, Chair of
Division of Endocrinology at Ohio State.
R. SILLS: Robert Sills, head of molecular pathology,
National Institute of Environmental Health Sciences.
J. ROBINSON: Julie Robinson, I’m the Branch Chief for the
Air Force Health Study.
K. FOX: Colonel Karen Fox, Principal Investigator for
the Ranch Hand Study.
K. CAMPBELL: Kim Campbell, Food and Drug Administration.
D. BUTLER: David Butler, National Academy of Sciences,
Institute of Medicine.
M. OWENS: Maurice Owens of Science Applications
International Corporation. I’m a Program
Manager for the Ranch Hand support contract.
J. MINER: I’m Jay Miner, Operational Technologies. I work in the Air Force Program Support
Office.
W. GRUBBS: Bill Grubbs, also with SAIC. I work with Dr. Owens.
M. YEAGER: Meghan Yeager, SAIC.
M. PAVUK: Marian Pavuk, SpecPro Corporation. I work on the Air Force Health Study.
S. LEVY: Lieutenant Susan Levy. I’m the Deputy Program Manager for Ranch
Hand, the Human Systems Group, at Brooks City-Base.
M. STOTO: Okay.
Thank you, everyone. Okay. Did everyone get — introduce themselves? Yes.
Okay. Len, you want to begin with
the household — housekeeping items?
L. SCHECHTMAN: Household is good. Okay.
We’re — we don’t really have very many housekeeping items today. We — I just want to inform you that we are
going to have a working lunch so that we can get through this agenda and I will
be reading the conflict of interest statement before we — before we get
started. But I do want to point out that
I’d like everyone to be thinking about future meetings or meeting singular
before the closure of the — of this effort.
So we will get to that item toward the end of the meeting, but please be
thinking about it as we progress through today’s agenda.
Okay. The following announcement addresses the
issue of conflict of interest with respect to this meeting and is made a part
of the record to preclude even the appearance of such. Based on the agenda submitted for today’s
meeting, all such government employees have been screened for their financial
interests related to the topics at hand.
FDA has determined that all financial interests and firms regulated by
the Food and Drug Administration present no potential for a conflict of
interest at this meeting.
In the event that the discussions
involve any other products or firms not already on the agenda for which a
participant has a financial interest, the participants are aware of the need to
be excluded from further participation.
Such an action will be noted for the record. In the interest of fairness, all other guest
participants are asked to address any current or previous financial involvement
with any firm whose products upon which they wish to comment.
M. STOTO: Okay.
Thank you. Since we don’t have
any products, we’re talking about that’s not as relevant. But conflict of interest is an important
issue, so I’m glad that we had a chance to do that. I think we’re going to hold the approval of
the minutes until later so we can get right into the — into Dr. Butler’s
presentation. So David, if you’re ready?
D. BUTLER: Good morning, everyone, and thank you for
accommodating my schedule by allowing me to present first thing this
morning. There we go. I have prepared a set of slides; I also have
copies of these slides should anyone need to take a look at them. What I’m here to talk about is an interim
letter report from the committee on the disposition on the Air Force Health
Study, a National Academy of Sciences, Institute of Medicine study that was
tasked to address the disposition of the AFHS.
I will just run through these
quickly since everyone has seen these slides before. They’re our standard opening slides on what
the National Academies is, and specifically, what the Institute of Medicine
does. As I’ve mentioned before, we have
a process whereby we form volunteer committees of experts and write reports
based on their systematic review of available scientific evidence. Those reports are then subject to an external
peer review.
The Congress tasked us, and more
specifically, tasked the Department of Veterans Affairs to ask us to address
five separate topics related to the Ranch Hand Study: the scientific merit of maintaining the data
assets collected in the course of the study; whether or not any obstacles
existed to maintaining those data assets; the advisability of providing
independent oversight of those materials past the time of the scheduled end of
the study and the mechanism for doing so; whether there should be any further
study of these materials; the advisability of extending the study itself, and
if so, who or what should be responsible for extending the study; and finally,
the advisability of making the materials collected in the course of the study
available to outside researchers.
In order to accomplish this task,
the committee on the disposition of the Air Force Health Study was formed. You’ll see the names of the individuals
there. Their expertise ranges from
epidemiology through biostatistics, occupational environmental medicine and
bioethics.
The interim letter report that I’m
talking about today is a very small document and it’ll probably take longer for
me to talk about it than it would take for you to just sit down and read
it. It is a highly focused document that
addresses only one component of that charge that I just ran through a moment
ago, specifically whether or not any obstacles exist to retaining and
maintaining the electronic data, the paper records that have been rendered in
pdf form, and the laboratory specimens that have been collected in the course
of the study.
The committee — and you’ll note most
of what I have here is in quotes both because it makes sure that I get things across
clearly and also because it saves me trying to rewrite what’s already been
written anyway. The committee thought it
was important to offer its findings, conclusions and recommendations on this
topic alone because they were aware of the fact that the time clock on the
study is running very rapidly and very loudly at the moment. And they wanted to make sure that their
recommendations would have a chance to be considered while the individuals, the
investigators with the AFHS who are most familiar with those data assets and
the funding to support them were still available.
In gathering information for this
letter report — and I should add the final report that we’re working on — the
committee held meetings earlier this year, including workshops where we invited
parties, including Dr. Stoto, to come and talk with us and also conducted a
subcommittee visit to the research facilities in San Antonio in May. We also asked a number of questions to the
AFHS staff, which they were terribly patient about answering. And we really appreciate all of the efforts
that they have put forward to make sure that we got all of the information that
we requested.
Two working groups went to San
Antonio and they looked separately at the data and at the laboratory
specimens. I’ll start out with the
bottom line and then just drop down into a little bit of detail that’s in the
report. The bottom line was that the
committee found that the data and the specimens had been maintained at the level
typical of most epidemiological research.
Now because this is a study that’s
gone on for so very long, the state-of-the-art has changed over the time;
computer programs have changed; the ability to manage data electronically have
changed over time. And as a result, you
have a data set which was collected at various points in history and is in
various states of electronic documentation.
The subcommittee noted that study personnel weren’t tasked with
rendering all of those data assets accessible to outside research — researchers
over time and that their findings concerning it should not be viewed as a
criticism of the AFHS staff.
Instead, what they found were a set
of working principles for managing and analyzing the data and specimens that
works very well for the AFHS investigators.
Every — the committees, the subcommittees asked staff to go through a
series of exercises that were intended to elicit how easy it would be to access
certain data and to analyze that data.
To do so, what they did is they asked staff to write a small SAS program
to randomly draw case numbers from five of — five Ranch Hands and five
comparison subjects and use those case numbers as sort of the example
population for the particular exercises.
In the course of going through those
exercises, the committee found that data, especially data that had been
collected earlier in the studies wasn’t necessarily documented in electronic
forms. So if you were wondering what
particular code was associated with a variable that had been collected in an
early stage of the study, you might pull out a book where it had been — that
information had been hand-annotated.
Over the course of the study, the
collection of variables changed.
Variables that might have the same name over the course of the study
might’ve been coded in different ways over its history. That’s, again, researchers responding to the
incentives and the local conditions when they were doing the work rather than
looking to do some grand overarching data set.
With the laboratory specimens, a
system has evolved whereby the data were, at the time the committee was there
in May, where you had to go through a two-step look-up process to find a
particular sample: both consulting an
original database and then consulting exceptions to that database that had
occurred over time as specimens had changed because a particular freezer
might’ve failed or a specimen otherwise had to be moved.
The committee found, after its
review of the subcommittee information and its review of the reports and other
documentation that had been prepared over the course of the study, that the
medical records, other study data and laboratory specimens — you’ll notice I
echo those words because those are the words that are in the charge — collected
in the course of the study have been properly maintained. But they are not currently organized or
documented in a manner that would allow them to be easily understood,
evaluated, managed or analyzed by persons outside of the AFHS.
Therefore, the committee concluded,
again addressing a specific item in the charge, the present state of the
documentation and organization is an obstacle to retaining and maintaining
these materials once the currently scheduled termination date of the study
occurs. The committee therefore formed a
primary recommendation that actions should be taken prior to the scheduled
termination date to reorganize and document the data assets in a form and
format that does allow them to be easily understood, evaluated, managed or
analyzed by people outside of the AFHS staff.
It proposed a number of specific
recommendations separately addressing the electronic and paper records that
have been rendered in pdf form and the laboratory specimens. For the data, first of all, they proposed
that a comprehensive inventory of master data files be created. Let me explain what that means. Over the course of the study, a rather large
number of data files had been collected or generated. These include the raw data files that were
the original input for these data, the cleaned up version to those files that
were used for — as the basis of analyses, and files that were generated in the
course of particular analyses.
What the committee refers to as
“master data files” be the — that subset, that small subset of the rather large
number of files generated over the course of the study that staff consider to
be the definitive files, the files you would start every study with if you were
going to conduct a new study. And what
they’re asking here is that those files simply be identified in a clear manner
and inventoried. So if one wanted to
look at pulmonary assessment data from Cycle 3, you’d be able to pick up a book
and immediately say this is the definitive file you would use to study those
data.
Second, to create a comprehensive
inventory of the variables contained in the master data files. Now this is something that has been done over
the course of the study, but not necessarily consistently over the course of
the study. The committee wanted to make
it clear that a number of their recommendations stemmed from looking at
materials that had been prepared as part of one cycle’s examination or another,
but weren’t necessarily identifiable in easy — in an easy way in other cycles.
A comprehensive inventory of
variables — the data dictionary that was produced for the Cycle 6 examination
is an example of the document that they’re looking for there: something that tells you what the variable
names were, what the descriptions are, what the codes were used — the codes
that were used to summarize the data that were collected and the like.
A master data code book containing
the name of data variables — this is basically a document that would be
generated from the requests above it, simply having sort of a one-stop shop
where the files and the variables were associated in a single document so that
somebody looking to do a particular analysis over time or of a particular cycle
would be able to identify those pieces of information they needed in order to
do the work.
Finally, for the documents that have
been scanned into image pdf files, a document containing the content, format
and location of those materials. Again,
what we’re not talking about is something where you go through the — Joel liked
to talk about the “linear mile” of paper — not a document that goes through
every last piece of paper there, but simply the type of materials that the AFHS
investigators prepared in response to questions to us when we asked about how
those information — that information had been inventoried to begin with.
This is not an effort, the committee
wants to make clear, where it believes you would need to go back through every
one of the thousands of data files or the thousands of reports and rewrite them
or make every last piece of information consistent. Instead, this is a documentation effort: take those key pieces of information, the key
documents which have, in some cases, already been prepared, and identify them
clearly and say this is the definitive piece of information you want to pick up
if this is the thing you want to analyze in the future.
As I said, the committee noted that
some of these documents already exist in some form and the documentation for
the later examination cycles exhibits many of the features that the committee
suggests. The concern — their concern is
simply that information for all the cycles be compiled into easily identified,
definitive reference documents with uniform information content. And what I didn’t write down here, but which
the committee suggested, is that these be in an electronic form, searchable pdf
files, for example, that would facilitate someone finding a particular piece of
information that they were looking for.
The committee also had a few
suggestions regarding laboratory specimens.
Let me cut to the bottom line here before I go through these. These recommendations were based on the state
of the laboratory specimens at the time the committee visited. The committee understands that since then,
there has been a reorganization of those samples which addresses many of the
items that are spoken of in the report:
to do an inventory and create a single specimen database.
The committee also suggests to
compile the information regarding specimen history into a reference
database. If someone is going to — were
to want to analyze these specimens in the future, they would want to know what
the specimen history was to be confident that they could do good analysis work
and to compile all the protocols regarding the receipt, the maintenance, the
dispersal and the return of the specimens where specimens had been sent out to
other investigators and have not yet been destroyed into a single reference
document to document the status of those to performed — perform the planned
re-assay to aid in the evaluation of specimen stability and condition.
We’re looking forward to hearing the
report on the progress on that today.
And simply in the reorganization, to make sure that individual samples
are documented so that someone looking at them would know when they had been
collected if there are any vials that aren’t already marked with that
information.
As I noted, the committee said that
they were aware of the re-inventory and physical reorganization of the
specimens and said that it was in agreement in the effort since it achieved the
goals that it was looking for. The
committee drew one final recommendation concerning these. We understand that the recommendations that
are being offered are not costless and do involve time and staff commitments
when there’s very little time left.
The committee recommended that the
Air Force, as custodian of the research materials, take responsibility to
ensure their proper documentation and their organization for historic reasons,
and also to allow the possibility of future use, and indicted that if available
program funds weren’t sufficient to accomplish the actions that the committee
recommended, that supplemental funding be provided to carry out such work in a
complete and timely manner.
This is only one small part of the
committee’s final report on this topic, which is in advance stages of
completion and which I’m working at basically on any moments that I’m not
devoting to other things: eating,
sleeping and the like. We are working to
complete this report as quickly as possible and get it out so we can have
conclusions on all of the topics that the committee was charged with addressing
available for the consideration of the Congress and other parties.
The complete report is posted for
download at the National Academies Press web site. That address will take you to a page that
will allow you either to look at the report page-by-page on the web or allow
you to download your own pdf copy of it after you provide your e-mail address
or somebody’s e-mail address to NAP.
And as before, this was part of a
larger research effort regarding veterans’ health that the Institute of
Medicine has been conducting at the behest of the Department of Veterans
Affairs and others. And we have a
comprehensive web site which pulls all the information from these disparate
projects together so that people looking for information can, we hope, find it
easily. That’s the end of my
presentation. I’d be happy to take
questions.
M. STOTO: Okay.
Thank you very much, David. Are
there questions from the Committee? Ron?
R. TREWYN: It would appear based on the fact that the
recommendation at this early stage is to reformat both the, you know, some of
the aspects reorganized for both the data and the specimens that we can
probably anticipate at the end of this that there’s going to be a recommendation
that the materials be maintained and be available to other than Air Force
personnel. Otherwise, this initial stage
probably would not have been recommended early on. Is that roughly a fair assumption?
D. BUTLER: Well, the committee makes clear in the
interim letter report that it reaches no specific conclusion on that topic in
this report. It notes that whatever
recommendations it offers and whatever policy decisions are taken in response
to those recommendations, the committee believes there’s merit in doing a
complete documentation of this data set.
If nothing else, the committee understands that the electronic and paper
records are subject to review by the national archivists for p
