Department of Health and Human Services




October 19-20, 2000



Day Two


Conference Center

Hilton Palacio del Rio

San Antonio, Texas



Robert W. Harrison, M.D., University of Rochester, Chairman

Michael A. Stoto, Ph.D., George Washington University

Michael Gough, M.D.

Robert C. Stills, Ph.D., NIEHS

Paul R. Camacho, Ph.D., University of Massachusetts-Boston

Steve Selvin, Ph.D., University of California-Berkeley


Ronald F. Coene, P.E., Deputy Director, NCTR,

Exec Sec of the Committee

Barbara Jewell, NCTR, staff


Air Force:

COL Harry E. Marden, M.D., Brooks Air Force Base

LTC Karen A. Fox, M.D., Brooks Air Force Base

LTC Bruce Burnham, Chief of Population Research

Dr. Joel Michalek, Principal Investigator


Dr. Judson Miner, Program Management Support

Manuel A. Blanca, Program Management Support

Meghan Yeager, SAIC

William Grubbs, SAIC

MAJ Jack Spey (Ret.) Ranch Hands Vietnam Association





Review Statement of Work (Continued) 434



Future meeting dates 547


Public Comment 568


Adjourn 571


[8:12 a.m.]

DR. HARRISON: Good morning.


DR. MICHALEK: This is actually the --

MR. COENE: Oh, you've got a one-pager, Joel, that you've laid in front of us, is that correct, that we probably have lost in our papers?

DR. HARRISON: Starts off with review at work.

DR. MICHALEK: What we have here is the Statement of Work. The Statement of Work is a document that is written by the government that tells SAIC what to do. It says, the contractor shall do this, the contractor shall do that. It is the document which is the basis for the entire physical exam, the travel, the lodging, the tracking, the clinical activities, the reporting, the statistical analysis, the whole nine yards. The date of release.

It is a document that both parties sign up to; it is a contract between the government and this company, SAIC. And what you have there in that loose-leaf is --

DR. STOTO: Or maybe another company.

DR. MICHALEK: Yes. What you have in the loose-leaf is the contract from the last physical, 1997 physical. It is that contract that we will modify slightly for the new one, and so it provides a framework for discussion.

So what's in that contract in very cursory description is this: Now at this point, we've kind of reached the limit of what I can do with Powerpoint.

We are talking about kind of a hefty document there; we're talking about a couple of hundred different measure of how, we're talking about 67 or 58 different laboratory measurements; I'm not going to list those in Powerpoint.

What you need to do is look at it. The most document for you will probably be the Examiner's Handbook, which is at the very end. The Examiner's Handbook is what's given to the staff at Scripps. Just thumb through the Statement of Work -- eventually you'll come to something called an addendum.

DR. MINER: It's about page 61, or so.

DR. MICHALEK: It's called Addendum A. Air Force Health Study Examiner's Handbook. That was the document that, the details of what was done at the last physical in narrative form, stripped of all the contracting lingo.

It says very directly -- for example, on page 62, paragraph 2: The general physical exam shall conclude an assessment of -- and then it lists all these things. And then it goes on: The dermatology exam shall include these things -- so on.

This is the document that will probably be the one you will want to focus on, because this has the bulk of the scientific content of our activities at the next round.

All the special testing is on page 64; pulmonary testing, APG, do an exam for occult blood. It's all here.

At the bottom of page 64, Item 13 -- we're not going to do that next time. That's the adipose tissue sampling we did last cycle that we're not going to do next cycle; extracting 12 grams of fat from the abdomen.

So I have just given you a brief outline of what's in the Statement of Work, and I have outlined those portions of it that you want to focus on from a scientific point of view, which are all of course summarized in the Examiner's Handbook, but they're also given the contractual language throughout the statement of work.

The specifications for the last part of the testing is there, along with all the details on quality control, and which quality control charts they're going to use, what coefficients of variation they have to have, how the lab has to be kept certified, how the staff has to be maintained stable, how samples have to be kept and a certain number of watts, and all the detail are there about the nuts and bolts of conducting a big study at a research level of quality.

It's not just like the family clinic; this is -- we think -- top of the line activity.

So that's the document. And let's just look at the rest for a minute.

So the idea is that we will look at the document

in December; and the idea is to think about what we should do or not do at the next physical exam.

Now on those lines, we need to need these things. First of all, it's not a good idea to sign up to a contract and then later on change our minds about something. That will incur a large amount of cost, and will disrupt the process.

Our goal here is to make decisions up front and then follow through -- I hope delay, without any changes throughout he whole activity. And that's like, called contracting limitations. Because any change to the contract will incur extra cost, and that will disrupt our process. We only have a fixed amount of money, and we don't want to have to do that.

Then there's the protocol. We just can't decide "Well, we're not going to do neuro anymore." It's in the protocol that we're going to do neuro. We have a lot of other reasons to want to do neuro, but -- in other words, there's a framework here of activities that are specified by the protocol. There's a structure to this that goes back to 1980, which is in the protocol; and that's what that's about.

There are logistical constraints, too. You might say "Well, we're going to add a new test." It might happen that -- the caffeine breath test, for example, might take four hours. We don't have four hours. We're going to do so many other procedures and these men are going to see so many doctors and do so many blood draws that there isn't enough time to introduce a four hour test while they're at the clinic; it's infeasible.

So you have to worry about that, too; we call it logistical constraints. For example, a certain test might require that they be fasting for a full day or something beforehand. Well, that could be a logistical problem if you have elderly individuals that -- the ranges of ages here are --.

COL. MARDEN: One of the other logistical constraints is that when the guys come in they come in on a flight that the contractor is scheduled, and they're scheduled to depart on a fixed flight; it isn't an open ticket. So any kind of deviation from the schedule creates massive headaches for the contractor in trying to either reschedule a flight or get a procedure done within the time constraints of the flight that the guy has scheduled.

And there's our so-called technical legacies.

DR. HARRISON: Now wait a minute now.

So how much slack is there in the schedule?

DR. MICHALEK: There's not a lot.

DR. HARRISON: Instead of saying there's logistical constraints --


DR. MINER: Last cycle --

DR. HARRISON: In order for us to consider this, I need to know. Are you saying that anything that takes more than 15 minutes should not be considered?

DR. MICHALEK: No, no, I'm not saying that.

DR. MINER: Last year or last cycle--

DR. HARRISON: In order for us to consider this, I need to -- say that anything that takes more than 15 minutes should not be considered.

DR. MICHALEK: Wait a minute, now.

DR. MICHALEK: Go ahead.

DR. HARRISON: So how much slack is there in the schedule?

DR. MICHALEK: There's not a lot.

DR. MINER: Last cycle, based on looking at what happened in the previous cycle, we went from a 2-1/2 day exam to a 2-day exam. We decided that was not a good thing to do, and we're going back to a 2-1/2 day exam. So there is a little more time this time.

DR. HARRISON: So compared to the last time, you've got four hours.

DR. MINER: We do have four?

DR. MICHALEK: Four hours.

DR. MINER: We might have four hours, right, but not four hours twice.

DR. HARRISON: Understood.

DR. STOTO: Well, conceivably, if there was really important, and it would take an extra day, it could be changed.


DR. MINER: If we take an extra day, though, then that adds another --

DR. MICHALEK: Another night at the hotel.

DR. STOTO: Right, that's what I was going to ask. Is there a total constraint on the cost per --

COL. MARDEN: It's a relative constraint.

DR. MINER: We have a cost estimate already outlined for this cycle. I can't tell you what that is because I've got contractors in the room.

DR. STOTO: I understand.

MR. COENE: Is it based on 2-1/2 days, is it based on 2-1/2 days?


LTC BURNHAM: So adding a day makes significant --

DR. HARRISON: What you're saying is, you've got your money allocated into different slots and you're going to try to -- and you -- it's easy for you to shuffle stuff within those slots, but to change from one slot to the next is at least an accounting problem, if nothing else.

LTC BURNHAM: Or to go back and get more money --

DR. HARRISON: Well, that doesn't sound too practical.

In terms of -- I mean, what you're saying, though, is that you don't want to change the protocol; that makes sense. The logistical constraint, though, instead of just that being a category, when I look at this Statement of Work and the other proposals, there actually is a fair amount of slack in the system.

DR. MICHALEK: There is slack in the system.

DR. HARRISON: It's not real tight.

DR. MICHALEK: There's some slack, yes; and it's hard for me to tell you what that is, exactly.

Another example would be, you wouldn't want to do a debilitating test on the day they have to leave, because you have to get this guy to airport, and we don't want to have him stumbling out the door all weakened from a big procedure.

COL. MARDEN: Or on our 92 year old subject.

DR. MICHALEK: Yes. By the way, the earliest birth year in this study is 1910, and the latest birth year is 1956. So there's quite a range in ages in this group.

The oldest individual that showed up last time was in his 80s.

Technical legacies. We certainly would like to be able to compare one report with another. We like to be able to do longitudinalities; we like to see repeated measures over time. So it's disruptive to say "Okay, we're not going to measure this variable anymore. Because then we lose, we don't have a track anymore."

There are a lot of things we do because we're interested in time trends. So that's what that is. And we don't like to change our definition of cases very often. I know there's new information on diabetes, and we'll take that into account; the new ADA definitions.

But if we make a change to the case definition, such as heart disease for example, because we want to create a continuity of cross-reports over time, we'll do the old definition and then we'll do the new definition, both. So you'll have an overlap -- a continuity across reports.

And similarly with statistical analysis, which I'll give in more detail in a minute. We have over the years changed our models or added to our statistical modeling; but every time we do that, we always try to include the previously-used model so we track across time, the same way.

And finally, once again, you'd want to be able to take the year 2002 report and open it up and say "What happened last time?" So you open up the last report, and you don't want to have a big change in format, because that just makes it harder to communicate to yourself and to others what happened, you know, with this group.

So the longitudinal aspect is important, and that's why we're doing a longitudinal study. So report format is -- there are things that we can do with report format that won't disrupt our ability to do longitudinal comparisons; but we have to be careful there, too.

DR. STOTO: One thing that helped me with struggling witness this issue a year ago was to realize that in addition to these reports that come out based on each major exam, you also do papers in the literature, where you have more freedom to change methods and do something that may be more modern or appropriate and so on.

DR. MICHALEK: That's an excellent point. Go ahead.

DR. HARRISON: But the actual data from the first exam to the present is -- the actual data itself, what you got out of the Scripps study, is in electronic format?

DR. MICHALEK: Yes, and it's on the web page.

DR. HARRISON: Okay, but it's -- it's in that RAID array somewhere.

DR. MICHALEK: Right. That's it. It's all saved and it's all on line.

DR. MINER: Well, not quite all.

DR. MICHALEK: All the electronic data that we used in our reports is available.

DR. HARRISON: Now that's not my question. Every CBC, every differential, every urinalysis report, is in an electronic format.


DR. HARRISON: It may not be accessible to me through the web page, but it's there.

DR. MINER: Yes. Yes.

DR. MICHALEK: I'd like to amplify Mike Stoto for a second. Mike made a very good point.

We have a what's called a fixed price contract with SAIC to do these reports, these statistical analyses. They are not free to do exploration. If they were, the cost would go the roof.

They have a plan, they have a drill: "You will apply this Model 1 to this variable using those exclusions and these covariates, and this main effects model, and you will report such-and-such." That's a drill; that's the only way we can get this done.

So when we see things in the report later on that we find interesting, the way we explore those -- me and Billy Jackson, who isn't here today, and our in-house staff; or sometimes we can go back to Bill Grubbs, but we have to be real careful there about the constraints of the contract.

And it was done that way on purpose, by the way, way back at the beginning. That we would avoid what are called post hac fishing expedition-types of analyses. "Well, that's interesting, let's follow that PI. And go on that one. Wow, that was neat." Goes on and on.

We don't want to get into that game, you see. So there's a drill, there's a very important drill here, and it's captured in something called a Statistical Analysis Plan that the company writes for us as part of the requirements of the contract.

And all of the exploration and all of the research and detail and long periods of work are done by us in-house to write those research papers.

DR. CAMACHO: These guys go through a drill. They put thought out there.

DR. MICHALEK: They're going to write a 4,000 page report.

DR. CAMACHO: Then if you want to follow some path that's your business and you aren't here to die; you're on the Air Force dime.

DR. MICHALEK: That's it. Right.

They have to produce a 4,000 report by a certain date. And it has to be a firm date; got to be January of the year 2004, and it's got to be sent to the Surgeon General, and there's no question about it.

So with that time constraint, and the constraints that I've just listed, this is the situation we're in with those big, fat reports.

DR. STOTO: And the stuff that these guys do are the things that go in the peer review literature, that Joel's group does, that we heard about yesterday.

DR. MICHALEK: Now the report is peer reviewed, too, and I want to get to that. But first of all, here's what happens at Scripps; this is all shown in the Examiner's Handbook.

DR. MINER: Joel, could I interrupt just for a second? On the report format specifically, since we were talking about that. If you look in your Statement of Work, you'll see some blocks in there, but those are the contract data requirements list items. And that then refers back to a data item description which is part of the contract, which then sets the format, and how many copies and how gets it and so on and so forth; that's what they have to follow in their activities.

So as you go through here, you might see something that refers to the statistical analysis plan or to the science -- reports final. But that's a set, contractual, legal piece and definition that they have to do.

DR. MICHALEK: You'll see a block that says CDRL. That stands for Contract Data Requirements List. We didn't give you all those; those are a set of forms there, attached to the back of the contract, for every deliverable, which is contract language for a product that they hand over to the government.

There's a form in the back that says who they're going to give it to, exactly when and how many copies, and who's going to approve it and all that. We didn't give you all that stuff, we just gave you the contract itself.

MR. COENE: A clarification, Joel: in reviewing Round 5, didn't the group question some of the statistical plan, or at least -- so that they questioned what was contracted for. It says one of the early deliverables is a statistical plan again?

DR. MICHALEK: Correct.

MR. COENE: So it seems to somehow, if the committee had trouble with that last time, they would like to see that before --

DR. MICHALEK: Okay. We can give you a copy of last cycle's statistical analysis plan. Deliver that to you.

DR. MINER: But the Statistical Analysis Plan, if you look at your --

DR. MICHALEK: It reflects what's in the contract.

DR. MINER: -- table 3 6. Table of Contents 3.6, where it says: statistically analyze the data, and then go to that part in your Statement of Work, it describes exactly what has to be in the statistical analysis plan.

DR. HARRISON: Yes, but what we're saying is, we want to see the plan.

DR. MINER: The plan reflects what this says to do.

COL. MARDEN: Which is the chicken and which is the egg.

DR. MINER: And so we --

DR. STOTO: This is it. Starting on page -- bottom half of page 19, through 22 or so.

MR. COENE: The issue of nonsigificance. Yes, okay.

DR. STOTO: But we didn't even discuss it. But this is -- what's there to discuss.

DR. HARRISON: If Stoto and Camacho, the big-time statisticians in the group, think that this is enough to work with, that's --.

DR. MINER: Well, again, what their -- their plan should reflect what we asked them to put into it. So if you want to change what goes into that plan, change this first; and then you can review it to make sure that the plan reflects that.

DR. HARRISON: But remember, the devil is in the details, and the next meeting is in the first part of December.

DR. MINER: Absolutely.

DR. MICHALEK: And the details are right there.

DR. MINER: No, I wasn't saying don't review the stat plan, I'm saying start with this first and then --.

DR. STOTO: The other thing that would be helpful to me is to see the results that came out of it last time.

DR. MICHALEK: All right.

DR. STOTO: And there was a chapter on statistical methods, I recall; and then maybe one chapter, say the one dealing with diabetes as a sample; and then there was a summary chapter.

DR. MINER: Was that in the plan?

DR. HARRISON: What you're saying -- Wait a minute, though. What you're saying, to get me right, Ron, is that Mike thinks that certainly he would like to see and probably the other people in this area would like to see -- they already have this, they want the Statistical Plan, they want the statistical chapter from the last cycle, and a representative chapter from the last cycle, and it should be the same representative chapter for all three of you to review so that you are all talking about the same thing.

LTC BURNHAM: You can get that off the web site.

DR. STOTO: Well, I know.

LTC BURNHAM: I'm saying, I'm telling Ron, you can download that by chapter.

DR. HARRISON: That's Ron's --

MR. COENE: We agree what would be a good -- knowing their requirement now, what would be a good package for them.

DR. HARRISON: Yes. You all can work that out.

MR. COENE: A meaningful package that would allow them to compensate --

DR. STOTO: I would add, the summary chapter, too.

LTC BURNHAM: You have it on CD.

You said They.

DR. STOTO: I have access to the web, too.

MR. COENE: If we could make it easier by pulling opponents, us subset.

DR. HARRISON: I haven't seen a CD --

DR. MICHALEK: And we gave you the last report, the whole 4,000 pages on CD.

MR. COENE: Let's with your help pull together this subset that will allow them to focus on this issue.

DR. MINER: They want to focus the CD.

DR. MICHALEK: I would like to amplify something that Jay said. First, it is true that the Statistical Analysis Plan reflects what's in the contract, but not exactly. There are times when Grubbs would discover something that we wrote in the contract "Did you really mean it this way?" And I'll look at it and say "Oh, darn, missed that point."

So Bill would come back and tell us, "Oh, you really want this, don't you?" And I'd say "Yes, you're right, Bill." So that plan would be sometimes a tiny bit different than what's in the contract, because we can negotiate certain cut points. I'd say in there we're going to use a cut point of 3.5; "Darn, it was the wrong cut point" and Bill would know that, because he knows the details from Scripps.

DR. MINER: Nope, nope, nope.


DR. MICHALEK: It's not exactly the same as the contract.

DR. MINER: It has to be.

DR. MICHALEK: Then we amend the contract.



DR. HARRISON: So recorded.

DR. MICHALEK: There's a slight evolution here.

DR. HARRISON: Joel, don't get tied up in that, because that's something that we would all understand. That's not a problem.

DR. MICHALEK: All right.

DR. MINER: It matches. We mod the contract.

DR. MICHALEK: One option is to put the stat plan on the web page; you know, just get it by point and click. It's about a hundred pages; not a big document.

Part of the other activity besides getting the men out to California, physically examining them and sending them home and analyzing and writing their report, the SAIC will deliver data for public release, just like they're doing now for all the other cycles -- and by the way, cycle is jargon to me; the physical exams.

Cycle 5 is the 8/97 exam, Cycle 4 is the '92 exam and so on. Cycle 6 is the one we're talking about in the year 2002. They're going to produce SAS files and flat files of the data for the public. These are datasets that are identical to those used in the report, but have the case number replaced by a fake ID number so that no one could get that data and somehow get into our system and merge it with something. You can't because it's a private -- it's been fixed so they can't do that.

Otherwise they're the same datasets that are used in their analysis.

DR. HARRISON: So where's the relationship between the fake numbers and --

DR. MICHALEK: The key is held by only one person; her name is Lydia and she's on our staff.

DR. HARRISON: So it's not in the same computer?

DR. MICHALEK: It's in a special place that only one person can get to.


DR. MICHALEK: And they're going to release these flat files and SAS files, they're going to produce documentation so that everyone knows what's in the file. In SAS that's captured in the contents; that means labeling and first flat files, the documentation is --

They're on our web page, and they're going to produce that again for the next cycle.

DR. HARRISON: Joel, you do have Lydia backed up.

DR. MICHALEK: No one is backed up. If Lydia is killed or hurt at some point, we have her in a position with this study where all of us are one deep. In that if I disappear, there's no one with -- maybe Jay Miner who may be the closest person that would have my level of expertise.

DR. HARRISON: I was given a cup of coffee, and so I might not have followed this as closely as I should.

The SAS files and flat files that are delivered that you have somewhere are actually coded so that each individual can be identified.

DR. MICHALEK: Yes, yes. Only the public data.

DR. HARRISON: And those are kept in that RAID array somewhere?


DR. HARRISON: And are backed up somewhere.

DR. MICHALEK: Right. Everything's backed up every day.

DR. HARRISON: It's only the public stuff that is entirely dependent on Lydia?

DR. MICHALEK: That's it.

DR. HARRISON: Okay. Go ahead.

DR. MICHALEK: And by the way, let's talk about that for a second.

DR. HARRISON: No, that's fine because it means that Lydia is backed up.

DR. STOTO: In fact, why do we even need to keep that key?

COL MARDEN: That's true; it could very easily be destroyed once you put everything on the web.

DR. MICHALEK: Because someone may have a question some day. Someone may write a letter: This guy had peripheral neuropathy, what else did he have?

DR. STOTO: We don't know. We don't know the answer to that.

COL MARDEN: We had to sanitize the data.

DR. MICHALEK: We have the data, they could do a Freedom of Information Act request and they want to know, "This particular case, tell me about this case." Then I can link that file to our real file, and I can pull the record, and then I can answer the question.

DR. CAMACHO: But that's exactly what the archive business should be about.


DR. CAMACHO: Just I'm saying, if something happened down the road where all kinds of lights went off and they said "Can we go back and look at this? It might be pertinent to something really serious here."

DR. MICHALEK: Right. Right.

DR. CAMACHO: They should be able to go back --

DR. MICHALEK: They can.

DR. CAMACHO: Going through all the IRB stuff, some disaster strikes --

DR. MICHALEK: Right, and they could say --

[Simultaneous discussion]

DR. CAMACHO: You could.


DR. HARRISON: May I suggest that after Joel has finished going through the description, because he still has a few slides to do, let's agree that one of the expanded areas of discussion will be how the maintenance of this study past its planned death will impact on this present statement of work.

Okay? Because I think as a database person, you probably have some ideas about this.

DR. STOTO: I agree with that. I also think, as a discussion item either today or at the next meeting, we need to think about confidentiality issues.

DR. MICHALEK: Exactly. I want to emphasize one more thing, since we talked about Lydia. Lydia knows where everything is. She knows everything. Billy Jackson, Norma, Fatima, myself, we know where everything is. It took years to reach this point.

Well, we've been with this thing for 20 years.

COL MARDEN: You realize you're going to be executed and --


DR. MICHALEK: Yes, and there are no backups. We have five years left. We're all healthy, we're all enthusiastic, hard-working and I predict we'll all be around in the year 2006. But when this ends and we all walk out the door, it's all lost.

Because now what you've got is 120 gigabytes of data, 6 million documents -- now you take a strange person who walks in the door. You are lost. You may never understand this thing; you may never get there.

DR. STOTO: Well, that's a discussion that we asked for.

DR. CAMACHO: That's a big discussion.

COL. MARDEN: The caretaker issue.

DR. HARRISON: It sounds like we have another letter from the committee in the making here.

DR. MICHALEK: Right. That's a topic for discussion. Okay.

Here's what's in the staff plan. I actually have a slide on it.

By means of careful work on the part of Bill Grubbs and his crew, they read our contract and they meticulously go through every line, and they compare with what they did last time and the time before, and what they know about Scripps Clinic or they know about the lab, and they call up people and they have contacts at the Scripps lab and everywhere else; and they write a plan. They exercise their expertise. And they tell us what they think we asked for.

And in so doing sometimes we make some changes, because this is a very complicated process. And they tell us very carefully what they're going to do, and that is their blueprint for their 13 months of statistical activity involving five statisticians, and their report writing.

DR. STOTO: Before you go on, one of the issues that came up late in the process last time was this issue of how you report significant results in the summary tables.

Are we going to discuss that today?

DR. MICHALEK: It's on the table for discussion, yes. If you want to make a decision on that today, we could; or you could wait until December. Because I think it might be handy to have a copy of the report in front of everyone when we do that.

DR. HARRISON: The other thing that would be helpful today would not be to discuss it in great detail, but for Mike to take 30 seconds and make a statement about the problem as it was perceived by us, so that the new members of the committee will have that tucked away to think about over the intervening weeks.

DR. MICHALEK: Okay, then let's talk about it now.

DR. STOTO: But basically what it is, is that for each major outcome they would produce a summary table that said, there are six or eight variables that they looked at with four statistical models; then they would report whether or not there was a significant difference of some sort for each of those things; just whether it was significant or not.

DR. MINER: You mean plus-minus, sort of?

DR. STOTO: Well, --

DR. MINER: There were n's and ns's, capital NS's, and --

DR. STOTO: Well, either significant or not significant. Not positive or negative relationship, you know. Yes or no.

DR. GOUGH: No, but just dichotomous. DR. STOTO: Yes or no.

DR. GOUGH: With no other information?

Or are there no numbers.

DR. STOTO: There were no numbers. Maybe there was a p-value, but --

DR. GOUGH: That's the same thing, though.

DR. STOTO: Well, essentially, yes. But there was no -- I think the key thing is that there probably should be some information that says, you know, the difference was 3 millimeters of mercury or something or other. Or the relative risk was 1.7.

DR. CAMACHO: I'd like ask him -- what do you think?

DR. HARRISON: Just a second.

DR. MICHALEK: What we've got is that in the chapters, all the detail is there. At the end of the chapter are the tables you described. And the appendix is all the detail that you want. The appendix shows every single number and all the mean differences and all the standard deviations along with the p-value.

So what we talked about was material in the appendix was what you really wanted to see, and you want that moved forward to replace --

DR. STOTO: No. I don't want all the material in the appendix. The issue is that I know that it's all there, but I want to put the critical information in the summary so that I can look at the summary and it make sense all by itself. I can drill down further if necessary, but --

DR. HARRISON: What you're talking about is probably the trickiest part of science; and that is arranging the presentation of the data of the data not so that it suits you, but so that an outsider, coming in, will be able to follow something that feels natural to them to obtain the information that they need.

And what I'm hearing right now -- and just a second, Jay -- is that Joel, you and SAIC have organized it in a way that you think makes sense, and Mike -- I forget who else it was; but there were a couple of people that when they looked at it thought that those tables could be used more informatively.


DR. MINER: Actually, those tables were put in there at the request of the advisory committee in that format.

DR. HARRISON: And are now going to --

DR. MINER: You had asked that there be--

DR. STOTO: Science has advanced, statistical methodology has improved since then.

DR. MINER: And indeed, we are open to any way and any suggestions.

DR. HARRISON: Yes; what we're talking about is iterative change. You know, you put something in it's almost right, and you change it a little bit and it's a little better, and then that's the way it works.

DR. MINER: And we're not locked into any displays.

DR. STOTO: I think the critical issue is we want to be able to talk about clinical significance of these results in addition to statistical significance.

DR. MICHALEK: We agree.

COL MARDEN: And that's great.

DR. HARRISON: I don't know how -- you know, I find, though -- even though I was there, Mike, I find that I'm not getting ahold of this particular point as comfortably enough; and I suspect that anyone who wasn't there last night was completely --


DR. STOTO: No, I think that, you know, Steve is right, we have to look at the tables.

DR. HARRISON: But what will happen is that when you get the package that we've talked about that has one of the chapters in it, it'll have the summary -- it'll have the summary area with the tables in it, and maybe you all can exchange an e-mail message or two about what you -- pointing out the issues that you see, because they may miss it the first time, Paul and --.

DR. CAMACHO: I've got to get a list of the committee, too. But do we have the right to, is it acceptable for us to contact everybody here?

DR. MINER: By law, yes.

DR. CAMACHO: By law. Lawyers.

I may talk to this individual, that woman, this individual? I can freely call anybody about this study?

DR. MINER: Yes, you can.

MR. COENE: The contractors? He's saying the contractors, calling the contractors.

That's a little --

DR. CAMACHO: If I want to call him about stats, I can call this gentleman about stats and ask him questions?

DR. HARRISON: Dr. Grubbs?

DR. MINER: Actually, I don't see--.

DR. HARRISON: What I would suggest, though, whether the law says that you can or not is not really -- might be superseded by the question of which would be the best way to keep our queries organized and so on.

Now, I don't want to be perceived as being overly controlling, but what I would suggest is that if you have a phone call to make, I think the phone call should go to Joel. Joel is the Principal Investigator of the study. If he can't answer your question, it's Joel's obligation to send you on to the right person, and Joel should know the right person.


DR. HARRISON: I would suggest if you have e-mail to exchange, I think that -- I think it would be nice to make it a habit to copy Ron, who can then either decide just to hold on to it or it can decide to copy me or someone else.

Whatever the law says, I would say that from a procedural standpoint, it would make sense, very much sense, that everything should funnel in to Joel and Joel can dish it from there.

Is that acceptable?

DR. MICHALEK: Yes, of course.


DR. MINER: All technical issues should go to him. If you have a program management question, you can --

DR. MICHALEK: Or a contracting question.

DR. MINER: Contracting, you can come over this way.

But technical that way.

DR. HARRISON: That's not what I'm saying. Let Joel tell you that it's -- that's the way I would do it. I wouldn't try to distinguish anything. I would just say, "Joel, how much money is allocated for such-and-such?" And Joel would say "Hell, I don't know, Jay knows. Here's his number." Something like that. Or "I'll put Jay in touch with you," one of those things.

And I don't mean that so that you can keep -- I just think it's important from a -- this is too big a thing; it can get too unwieldy and disorganized.

DR. CAMACHO: Sometimes you just have a question or an idea pops into your head.

DR. HARRISON: Yes, I don't mean to be monolithic about it.

I'm not going to get worried whatever you do. This is just a suggestion that I would go through Joel. Joel is very open, he's not a problem.

DR. STOTO: We used basically that model when I worked at the Institute of Medicine, and for exactly those reasons. And I think it was a very important thing to do.

DR. MICHALEK: That's another good point; you may say "Why are we doing this? You know, how did we ever get into this particular thing?" That's the kind of question you should call me on. And I'll tell you, "Well, we decided 15 years ago that's the way we were going to do it, on the advice of the committee, for example.


DR. MICHALEK: So I'll give you the legacy, or I'll tell you -- I don't know. Maybe you have a better idea. So yes, those are important discussions.

DR. HARRISON: All right, what else, Joel?

DR. MICHALEK: Just what we got through talking about, the report format is in the contract. Literature review is an important point. Last cycle we had a single physician doing a literature review for every chapter. We'd like to change that next time, have a specialist, for example, doing immunology, another specialist for endocrinology.

So that's going to be a change from last time. We had Dr. Dave Williams at Scripps write all of it, and we think that we could do a better job with specialists. And a discussion of the results, too.

DR. STOTO: Will there be someone, an editor who can make sure they're in consistent form, and parallel?

DR. MICHALEK: Yes. Well, that's the last bullet here.

DR. MINER: That's their charge.

DR. MICHALEK: Report quality control.

DR. STOTO: No, that's not what I mean. I mean if you have ten different people writing literature reviews, they'll have ten different models for doing it.

DR. MICHALEK: Yes, and that's part of SAIC's job, to produce a readable report. And they send us draft chapters. So they have their own editors and their own writers, and they're going to smooth it out and make it uniform.

DR. MINER: The answer is yes.

DR. STOTO: I think that that should be explicit, as part of the --

DR. MICHALEK: It's in there.

DR. HARRISON: You might even consider that SAIC should produce almost a writing manual that, I'm sure -- you're already ahead of me. But it has a little description of how the chapter will be laid out, what the -- template.

All right.

DR. MICHALEK: So what we've got then at the end stage is a process where we write these chapters; SAIC writes them, and deliver them to us, we do a first proofread, and send comments back. We do one kind of cleanup, and then we send them to you. And then we get into, we're talking -- how many chapters in the last report, about --?

DR. GRUBBS: Twenty.

DR. MICHALEK: Twenty chapters, some of them are up to 300 pages per chapter, right? Some of them are quite lengthy. And we send them to you, and then you develop a process to do a peer review of those chapters.

Now as I recall from last cycle, you may want to send them out to your own specialist. Last cycle we had some specialists on the committee such as Irene Check, immunology. And of course we still have Dr. Harrison on endocrinology.

So this is a very serious end stage activity to the report, to look at every single word in the whole thing.

DR. HARRISON: Last time, too, as I recall, the schedule got a little truncated so we were actually reviewing material that had not yet been --

DR. MICHALEK: That's right, we had to skip the first step in order to stay on time.

DR. CAMACHO: They're no funding for people to do the peer review, is there? I mean -- you're talking asking me asking somebody, here's a 300-page chapter. "Hop on this, will you, and get it back me in a couple of weeks."

DR. HARRISON: Anything that we do is Ron's financial problem; it's not the Air Force's.

DR. CAMACHO: Okay. So Ron's Mr. Daddy Big Bucks.

Practically speaking --

MR. COENE: other than bringing in a couple of experts, because we had a couple of vacancies at the time on the committee, we have not used outside reviewers.

DR. HARRISON: We've had ad hoc -- you know, we --

MR. COENE: At one time we had, on some of --

DR. HARRISON: First time -- that was ad hoc.


DR. CAMACHO: All I'm saying is that depending on the time crunch, it may be really difficult to go shopping around to associates and colleagues to tell them, "take a look at 300 pages here in your -- the free time you've got."

DR. HARRISON: That's something that we need to discuss when Joel's finished his presentation. I'll make a note for a review, but my comment is just that --

DR. MICHALEK: Let me just emphasize the point differently: Your name will be on the report, because you are the peer reviewers.

DR. HARRISON: Understood. But what I'm getting at --

DR. MICHALEK: Drove that one home.


DR. HARRISON: What I'm getting at is that we need to make sure that that schedule has enough cushion in it that what we're reviewing is --

DR. CAMACHO: Is your final.

DR. HARRISON: -- reasonably clean stuff and not the rough draft stuff.

DR. CAMACHO: Otherwise, you know what happens? There's this bag.

DR. MICHALEK: Jay, can you recall what happened to our process last time and how it got disrupted?

DR. MINER: Umm --

COL. MARDEN: Freedom of Information Act.

DR. MINER: Actually, part of the trouble was that we couldn't engage the committee early enough. And that had to do with funding, I think, on y'all's side of the house, and not being able to get together.

MR. COENE: But then there was another part to it, because when we then started to move on it, we found out that we were looking at documents that hadn't received the Air Force's review, and we said "Hey, we don't want to do that."

DR. MICHALEK: And how did we get into that problem?

DR. MINER: Well, initially we were trying to, because of the truncated time, though, send them documents at the same time that we got them, first goaround.

DR. HARRISON: That's what I'm saying.

DR. MINER: They wanted to see our comments on these documents first, and incorporated; and so in our deliverable schedule, we didn't build in enough time to do that.

MR. COENE: And we need to see that, then. This time -- we now know we don't want it that way.

DR. HARRISON: That's what I'm saying, Jay; that we need that timing.

DR. MINER: But we had enough actually slop time in some of the earlier chapters that, had we engaged you, we could have met everything. But it wasn't built in contractually, either; and we will --

MR. COENE: Fix that.

DR. MINER: -- fix that next time.

MR. COENE: We should see, you know I guess in December a new timeline so we can look at those; and then -- so the committee understands their involvement, how much time that you have programmed in for us, if that meets their --

DR. CAMACHO: If we think that's adequate. You know, prior planning prevents future --.

DR. STOTO: Are we talking about something that's happening in 2004, or something like that?


DR. STOTO: We need to talk about it.

MR. COENE: Yes, because it's in the contract. So you have to --

DR. HARRISON: The other thing we can talk about doing later on is what's been mentioned before, and that is specifying either a three times yearly or four times yearly meeting, and at least specifying the months for those meetings so that we are on a schedule and not just kind of loosely around.

DR. CAMACHO: You're looking for the committee to help the project. Calling me at the last minute, you know any of that last-minute-Charlie stuff, it drives me crazy. Because especially if it's a project like this which I have an interest in, and an interest in helping in.

And these guys will call me, "Paul, we needed it yesterday." "J.C. Why the hell didn't you--?


"This thing has been in the -- call me a week, two weeks ago, three weeks ago, a month ago, six months ago." You know?


LTC BURNHAM: But see, you were a replacement.

MR. COENE: Yes, we need to be sure to examine the timeline. Because it's in that timeline --

DR. HARRISON: "You want it when?"

MR. COENE: I'm not going to be here, but there will --. Jay, how do you -- you got back into this and contracted for this? Maybe that's what HHS will do.

LTC BURNHAM: But someone will replace you, right?

MR. COENE: I-- hey, I sure can't speak for the Secretary.

DR. STOTO: Is that something that the committee should get involved with?

MR. COENE: Well, you can do anything you want.

DR. STOTO: Maybe you can't comment.

MR. COENE: Yes, obviously I think that probably -- I don't know. I'm very disappointed in the Department's -- I'll go on record -- and its response to this.

DR. HARRISON: Who is the person making the decision on your replacement?

DR. STOTO: That's the right question to ask.

DR. HARRISON: That can't be a secret.

COL MARDEN: Yes, but it could be unknown. I mean, look at the Department he works in.

DR. HARRISON: Ron knows everything.

Who's the new --

MR. COENE: In theory -- Director Cassiano is the director.

DR. HARRISON: Director Cassiano.

DR. CAMACHO: I'm glad this came up; then this is an issue that we're going to have to talk to you about and find out about.

DR. HARRISON: Well, what I would say is --

DR. CAMACHO: That's a big danger right there. If you dropped out of space tomorrow, you're saying there's no provision --

MR. COENE: Barbara will still be here.

DR. CAMACHO: But there's no real decision-making provisions, et cetera, so everything goes in limbo on our side.

DR. HARRISON: What I would say, Paul, is that, if I get the feel for this, first of all just about every one of us on the committee are experienced enough to have some idea of what the issues are and what's going on. So we don't really need to query Ron, and we especially don't need to query him on the public record about what's going on with his position.

DR. CAMACHO: No, but we should indicate for the record that this is a concern. Because.

DR. HARRISON: Well, what we can state for the record, if it was the committee's will, what we can state for the record is that I should make inquiries on behalf of the committee out of concern for continuity and planning for this next cycle.

And since it is a public issue, I would think that people such as representatives of the Ranch Hand organization would be very much concerned that through bureaucratic shilly-shallying around that the proper support was not forthcoming from the Food and Drug Administration to provide the Air Force with the advice that they need.

DR. CAMACHO: I can tell you this right now; the Veterans Affairs Committee is going to be very concerned about this. I'm not speaking for them, but I'll bet --

MR. COENE: Let me put this on the record. I will make every attempt to have identified for you the new exec sec at the December meeting.

DR. STOTO: That's critical, because you know, on January 20th next year, the whole Department of Health and Human Services is going to grind to a halt for six months at the Secretarial level.

DR. HARRISON: It's probably grinding to a halt already. That's what's happening, and we're in this very awkward timing.

MR. COENE: I will make every attempt to have Dr. Cassiano inform you who he is going to assign --

DR. HARRISON: I'm going to make every attempt to call Dr. Cassiano next week, just in an informal way, and just --

MR. COENE: Okay.

DR. CAMACHO: I'll work with you on the -- I'll just talk to you, because I know the committee's going to be very concerned. I just know it.

DR. HARRISON: Well, this is a public meeting, so I would expect that those people who are associated with other political groups will do what they're expected to do.

How's that?

COL. MARDEN: Ron, when are you scheduled to retire?

MR. COENE: December 31st.

DR. HARRISON: So, that's something that I hadn't thought about.

Okay, anything else, Jay?

DR. MINER: Yes. I think it would be helpful then for next time, they'll build us a Gantt chart on the actual review process of the chapters --

MS. YEAGER: A sample.

DR. MINER: A sample, and I'll present that next time, or have Joel present it next time.

MR. COENE: We'll get some idea of what you have anticipated the committee has, a time and where we have to fit the meetings in then.

DR. MICHALEK: Okay, just two more slides; just to remind you that the basic statistical analysis structure is in the contract, telling the 4 models that I talked about earlier, and the definition of dioxin categories are in there, too.

So that's it, and there isn't much to say except now it's up to you to ask questions and think it over.

DR. HARRISON: All right. Questions, Mike Gough?

DR. GOUGH: Would it be possible to have, in the new statement of work, to have a red lined version where things that are changed are highlighted or bolded or something so that we can -- when we read it, we don't have to read every word, but we can see what the changes are to see if they make sense to us?

DR. MICHALEK: We could do that, of course; but I think you'd be encumbered by that more than helped. Because there are so many, we call it wordsmithing. There will be changes in grammar, punctuation --

DR. STOTO: Well, how about if you try to just isolate the ones you think are important, somehow.

DR. GOUGH: Where there's a new paragraph --

DR. MICHALEK: So you might want to wait until like the second revision, and then start including all the -- red highlight.

DR. HARRISON: What he's saying is that in Word there's a thing called highlighting.


DR. HARRISON: And if you print it out in color it comes out as yellow; but if you print it out in black and white, it just comes out as a little grade strip across.


DR. HARRISON: So when you're reviewing contract you know the parts in there that are wordsmithing and the parts in there that are changes, so why don't you just highlight the parts that are changes?

DR. MICHALEK: All right, we can do that.

DR. GOUGH: And it would also facilitate the committee's discussion of it, when we go through it.

DR. HARRISON: Of course the problem with that is, that if he fails to highlight something, that you --

DR. GOUGH: We'll never see it.

DR. HARRISON: -- subsequently proceed as being an important change, then all hell breaks loose.

DR. GOUGH: But that's our responsibility.

The other thing is, because of the focus on diabetes, what specific additions do you intend to make -- are you thinking about making?

DR. MICHALEK: As far as diabetes goes, all I can recall is that we will introduce the latest ADA definition, and alongside that we'll use the definition we used in the previous report. We'll be measuring fasting glucose this time; that we had not measured before.

Other than that, I believe all the same measurements will be made again.

DR. GOUGH: Well, is that -- Bob, can you think of anything else that needs to be added? Or will you?

DR. HARRISON: I'll think about it. You know, I don't remember what's --

DR. STOTO: We do have the hemoglobin A1C. It's already in there, right?

DR. MICHALEK: A1C hemoglobin is there. You might think about the 1992 report. There we measured pro-insulin, glucagon -- but we dropped those on your advice.

DR. HARRISON: I don't really see that as being -- I'll think about it. But the way I would think about it would be, is there any doubt that these patients have diabetes? Is there any doubt that they all have diabetes that has the same fundamental cause? You know, we talked before about Type I and Type II; and as long as those things are tied down, I'm not -- I mean you can look for other signs that you know should be there; diabetes causes kidney damage and things -- that doesn't enhance the observation that there's a relationship between exposure and the subsequent development of diabetes.

DR. GOUGH: Then from that specific, it leads into a more general thing. I was surprised yesterday to learn that there was a short term memory deficit in 1982. Has there been attempt -- has that ever shown up again?

DR. MICHALEK: No, we never gave that Wexler memory scale at any other physical; we only gave it in '82. Well, certainly --

COL. MARDEN: But it wasn't analyzed until --

DR. MICHALEK: It wasn't analyzed until recently. Because why? Well, because prioritized.

DR. GOUGH: Well, there should be a follow-up on that, I would think.

DR. MICHALEK: We're going to do the Wechsler memory scale on the next physical. And by the way, we've already consulted with our psychologist experts that we need to give exactly the same version of the WMS that was given in '82. There are new versions out today, no good; you've got to do exactly that one. But we know how to do that, so.

DR. GOUGH: I thought about the composition of the advisory committee. When I stepped down from the advisory committee, it was because I thought there should be a physician as a chairman, because all of this is clinical science now.

DR. HARRISON: And you've changed your mind since then, right?

DR. GOUGH: No; but there are two physicians now on the panel? You and Dr. --



MR. COENE: Favata, Landrigen, and Osay.

DR. GOUGH: Okay, four then.

MR. COENE: Four.

DR. GOUGH: But nevertheless, we're really thin on the ground. And it seems to me that for these other specific endpoints that you identified yesterday with the pluses, that at a minimum, I think the Air Force should consult with SAIC who should consult with the people at Scripps in those departments about what would you -- if you wanted to follow up on this suggestion, what are the tests you would add or something. And let us know about those things. Because there's a wealth of information out there; it's not going to be around this table, as you well know.

DR. MICHALEK: Good point, yes.

DR. GOUGH: And, let's see. I'm including the neuropathy, the peripheral neuropathy.

And I think that we should have for us, the Advisory Committee and for our accountability, that if things slip, if things get -- on this time table, if things slip, if things get late to us, we don't make up that deficit; we get the time we were allotted. Because this is the last time we'll do it, and the review is going to have to be complete, I think.

DR. MICHALEK: That's right.

DR. GOUGH: And I can't imagine it's going to be a long period of time.

In reference to this idea of meeting more frequently, I have a monthly schedule. January we'll meet in Denver; February in California; March, Florida or Puerto Rico; May or April, D.C. or Tennessee; June, Pennsylvania; July, New Hampshire; August, Maine; September, Washington State; October, New York City; November, San Antonio -- and we won't meet in December because of the holidays.

DR. HARRISON: Do I hear a motion?


MR. COENE: I want to go on record, we did get -- through all of the machinations of this committee and Department's focus on it, the Secretary, through the acting director of the NIH and the Commissioner of the Food and Drug Administration agreed that NIH would put $48,000 up last year, and made available $72,000 in fiscal 2001, which we've just started.

So that there is that amount of money available to support the committee. And that's discretion; that's over and above Barbara and I's salaries.

LTC BURNHAM: How much -- is that enough for four meetings?

MR. COENE: This year -- yes, this year.

DR. HARRISON: How much do you figure one meeting costs?

MR. COENE: $20,000.

DR. CAMACHO: It's in discretionary budget?

MR. COENE: Yes, it's a reimbursement. The NIH transferred $48,000 in 2000 and $72,000 in 2001.

DR. HARRISON: So you have $58,000 left now?


DR. HARRISON: And -- okay.

DR. STOTO: The issue is not money so much as the staff. In other words, someone having responsibility for it.

DR. CAMACHO: Well, it's both. We can get to that later, right?

MR. COENE: Like I say, I'll talk to Cassiano.

DR. HARRISON: Okay. Mike and then Paul.

I'm sorry. Joel, are you finished?

DR. MICHALEK: I just have a couple more things to go through.

DR. GOUGH: Oh, I thought you were finished, too.

DR. HARRISON: Are there things that you all have relevant to Joel's presentation, or did you think he was finished?

DR. CAMACHO: I thought he was finished.

DR. STOTO: I had something about the last slide.


DR. STOTO: The last slide, about the one report that talks about dioxin levels being greater or less than 10 parts per trillion being kind of a magic number. And that shows up in the analysis at various times, where essentially it's assumed to be zero if it's less than 10. Or not quite that.

I think that's an issue that needs to be discussed, and I don't think we're prepared to discuss it now, but we can flag that for --

DR. GOUGH: What was the discussion?

There's got to be a cut point, I think, and it can't be zero, because zero doesn't exist with dioxin concentrations.

DR. STOTO: I don't know that it's as simple as that.

DR. GOUGH: Well, complicate it for me.

DR. STOTO: Well, you can use the number that was given, and in analysis and taken into account in the analysis that there's a background level. There's all sorts of things that can be done. Maybe this is the right thing to do, but I just think it needs to be discussed.

DR. MICHALEK: We have some data to show that, by the way. We have a few slides.

DR. HARRISON: It sounds to me like -- I think I mentioned this before. It sounds to me like the EPA's argument that any concentration is relevant. And from a biological standpoint, I have to strongly disagree. I think something that is -- something that is 100,000-fold below the concentration required to --

DR. STOTO: I'm not arguing about that. The issue is that --

DR. HARRISON: I think 10 is too low.

DR. STOTO: Well, the issue is that certain individuals are excluded from certain analyses or treated in one group versus another, if they have 9 parts per trillion. And I think that an alternative would be just to use the number 9 just as if you used the No. 11.

I think there are various ways of handling this statistically. It's not a question of saying whether it's safe or not or whether it's background or not, but just having you do the statistical analysis; and I think that there are issues that need to be discussed there.

DR. HARRISON: Okay, but I'm trying to insert the biological part in here. And what I'm saying form a biological standpoint is that the sensitivity of the analysis is far greater than the biological organism.

As far as the biological organism is concerned, I would contend that one part per billion is imperceptible to the organism. And if one part per billion is imperceptible, then all of these are zero.

DR. STOTO: Well, I think that biology is relevant to the statistical discussion, but there's more to it than that.

DR. HARRISON: Okay. All right.

DR. GOUGH: What's the air around, if you took samples, and I'm sure it get easy -- that you measure the first time, it's 10 parts per trillion, and you measure the same sample a dozen times. What's the plus or minus around 10.

DR. MICHALEK: On an individual?

COL. MARDEN: On a given sample.

DR. GOUGH: On the individual sample. Just what is the variation in the test itself?

DR. MICHALEK: It's a 9 percent c.v., so I'd have to figure that out.

DR. STOTO: Plus or minus one.

DR. MICHALEK: The mission depends on the means. I've got to work on that. I'll give you an answer next time.

DR. GOUGH: Because if the cloud is big enough, then the 10 is --.

DR. HARRISON: So 10 would be anywhere from 9 to 11.

DR. MICHALEK: All right, let's put it this way; if it were a 10 percent c.v. --

DR. STOTO: I think we should cut this off now and then have a full discussion on it when we can look at all the facts.

DR. GOUGH: Joel, that's an answer -- somebody knows that.

DR. MICHALEK: I can answer it.

DR. GOUGH: Okay.

DR. HARRISON: Let me ask Mike this.

Mike, what do you consider to be the correct process to -- how would you like to discuss this?

DR. STOTO: I think that we need to look at the statistical plan where there are points in the plan where you say, you know, treat everybody -- do this analysis only on people who have background greater than 10 or something like that; and look at those decisions that reflect this cut point, and think through what are the options, what are the alternatives, what are the pros and cons.

DR. HARRISON: So what you're saying is that you want to discuss the 10 cut point -- and this should be an action item, actually -- you wanted to make sure that we discussed the 10 cut point as a part of your analysis of the statistical section of the statement of work in the December meeting.

DR. STOTO: You know, what they do is in some ways now this exposure study of obesity to say, well anybody with a body mass index of less than 30, we're going to leave them out of the analysis. That leads to certain biases.


DR. HARRISON: And I would be opposed to that.

DR. STOTO: Well, that's kind of what they're doing.

DR. MICHALEK: But the reason -- but Mike, the reason I would be opposed to that is that even within the normal BMI range of 20 to 25, people with a BMI of 20 have less diabetes than people with a BMI of 25.

So I can show a biological significance within the normal range for a particular measurement. So from my perspective, I defend it because of -- from the standpoint of biological relevance.

DR. STOTO: I'm not trying to say these guys are wrong, I'm saying that it needs a discussion. This is the kind of discussion we would make.

DR. HARRISON: What I would suggest is that there should be no assay, that assays lower than the body can perceive.

DR. SELVIN: Let me support Mike in this. I don't think it's a biological question. The biological question follows after the analysis is complete, and it's statistical optimum to use the data as measured.

It's a waste of time and money, so to speak, to cut the data into two pieces and analyze it as a binary variable. You don't lose anything by analyzing, in its continuous form, understanding what's going on in the analysis, and then you can cut it where you want with biological plausibility to describe the phenomenon.

DR. HARRISON: Well, if we cut it where I want it --

DR. SELVIN: You can do that post-analysis.

DR. STOTO: That's the interpretation, But we're talking about how to do the analysis.

DR. SELVIN: If you take a normally-distributed variable, just to be a little technical. If you take a normally-distributed variable and cut it, it's akin to throwing away 30 percent of your data. It's 30 percent less efficient to deal with the binary variable than it is a continuance.

DR. MINER: We would like to respond.

Joel, please.

DR. MICHALEK: May I respond?

DR. HARRISON: Well, are you finished?


DR. HARRISON: Is Dr. Selvin finished, because that's --

DR. MICHALEK: Go ahead, finish.

DR. SELVIN: No, I'd like to hear what you have to say.

DR. MICHALEK: First of all, all continuous data is analyzed twice. It's analyzed continuous form and it's analyzed in binary form.

For example, blood glucose. We analyze it as a continuous variable using every single measurement--

DR. STOTO: This is a different issue, Joel.

DR. MICHALEK: I know, but I'm getting there, I'm getting there.

DR. HARRISON: Because this has to do with the statistical model, right?

DR. CAMACHO: Let's let him --.

DR. MICHALEK: Every single variable is analyzed at least twice. Now dioxin, we use that 10 parts per trillion cut point in two places; right here on the initial dose, because we don't like to extrapolate people to Vietnam that have 1 parts per trillion. We only like to extract people that are above background.

Now there are other ways to do that, and I agree. But that's what we did; I'm telling you what we did. The 10 parts per trillion cut point here, we used it down here on this dioxin category thing which is on the slide you saw, but we did not use it here. Because in this model we didn't use dioxin at all. We said all Ranch Handers versus all Controls. All Ranch Hander versus all officer controls.

We did not use it down here. Here we're using every single dioxin measurement at its absolute face value on every single subject, because we're regressing health on dioxin. Right down to zero, we're using all the data, without any truncation, without any explosions, everybody is in that model, that last model.

DR. HARRISON: So what's your explanation for truncating, though, in the two categories?

DR. MICHALEK: We're only truncating on that initial dose estimate, because we believe that the first order model does not hold at background levels. We believe that when you're at background levels, you're at steady state. That first order model doesn't hold. That's why we cut it at 10.

Now a way to modify that is to just let everybody below 10 have their current value, and then you include everyone in the model. And I agree with that modification; that's fine. I don't see an issue there.

DR. HARRISON: If you did a regression, if you thought that the less than 10 was background, you're saying then that the regression line was flat, and so when you regress back to Vietnam, you're going to be at the same number that you measured.

DR. MICHALEK: No. If you attempt to use the first order model with someone with 2 parts per trillion, you're telling me that you believe -- go ahead.

DR. STOTO: Please. I don't remember the details of this. We need to see the plan to see what's at stake here. We just can't have this discussion now.

DR. HARRISON: Well, I think we can, though, Mike. Because if I understand correctly, what you're saying is that if you start out with a very low level in Vietnam, and you have a regression -- and you have a certain half-life, then you may reach background within five years.

At that point, then, by the time the study starts, you're measuring background levels several times. And you have no way to go back and estimate the original value.

DR. STOTO: The issue here is how to do a statistical analysis. It's not the biology or the biology is relevant to that question, but the issue is how to do statistical analysis; and I don't remember the details of what they did, and I'm not prepared to discuss this without seeing the details on paper in front of me.

DR. HARRISON: Okay. All right, so we'll put that on the agenda for the next meeting.


DR. MINER: One of the purposes of having a December meeting was first to bring up sticky issues here, and not so much to get wrapped around the axle with them, but let you all go back and think about it and get stuff lined up and then come back in December and make some decisions, yes.

DR. HARRISON: Okay. all right.

DR. CAMACHO: My only concern is that the time line for this is short.


I'd like to raise another issue.

Oh, I'm sorry. Go ahead, Joel.

DR. MICHALEK: Just one more thing. I wanted to emphasize another point.

Another thing that you should watch for as you go through this is covariates.

One of the strengths of our study is the fact that we're able to adjust for smoking, current smoking and entire lifetime history of smoking as measured by pack-years; same thing with drinking. This is one of the few studies that has a complete set of covariates in every analysis.

And by the way, every one of these analyses are done twice; once is not adjusted and one is adjusted. So you're talking about four -- and there's one for each of these; that's three, five, six, seven times two, that's 14 analyses for each of 200 outcomes -- more than that, because the lab variables are done twice, continuously distributed and binary. We're talking hundreds of statistical analyses here.

DR. SELVIN: Why do you do the unadjusted?

DR. HARRISON: Because we want to see -- this goes back to Paula Maier, used to be on the committee, 1985, introduced the idea of showing both unadjusted and adjusted, and since then we've liked it, because that way we can see the effect of the covariates.

And by the way, many times the unadjusteds are not that much different from the adjusted, and that lends credence to the results.

The point on the covariates is that they may not be up to date; there may be a new covariate that you thought of that we didn't, and a new risk factor. In other words, a new confounder that we missed in the contract. So you need to think about that, and this is an important piece.

Another good example of that is personality type. It's a covariate for heart disease, but our measurement of it is pretty lousy. It was the Jenkins Activity Scale. These men rebelled, they don't want to look at it anymore; and the reason is they've taken it every time, that's one thing; secondly, the Jenkins Activity questionnaire is directed at a normal working individual, you know, who is employed; because there are questions in there about waiting in line at work or things happening at the office. Many of these guys are retired. They say, "What are you doing? Why are you bothering me with this?" You know, and they just shove it aside or they'll take their pencil and mark straight down the page, all Yes's.


Or they'll do little designs.


So if we're going to measure personality type, we need to find some other way to do it.

DR. STOTO: Why do you have to measure it again?

DR. MICHALEK: Well, if that's the decision, fine; we won't. You think about it.

DR. MINER: A logistical problem, perhaps raised here. If we meet in December, very short time line for lots of decisions; meet again in early March would still make it under the wire for statement of work changes.

So --

DR. CAMACHO: That's a good suggestion. How much time --

DR. MINER: But that's you'all's call.

DR. CAMACHO: How much time are we having in December? Is it one day, day and a half --?

MR. COENE: That's to be discussed.

DR. HARRISON: That's for us to discuss.

MR. COENE: This morning.


DR. MINER: I'm not saying slip the December meeting, because I think there's plenty; but maybe another meeting early, then.

DR. HARRISON: Joel, do you have any --?

DR. MICHALEK: No, I'm all done.

DR. CAMACHO: I had one thing about the data in the future.

Well, we were asked to come up with this sort of I guess very beginning of talking about archiving all the stuff in six years.

So we were asked to put a statement in for the record; so the suggestion is something along these lines, that:

The Advisory Committee is concerned about the termination of the Ranch Hand II Project in 2006. Consideration must be given for the development of an archive which will ensure the preservation of all pertinent data samples and other research materials associated with the project.

Given the high profile of the study, -- why don't you read that?

DR. SILLS: [reading]

Given the high profile of the study materials, and the fact that the Ranch Hand study is one of the most comprehensive and well-organized studies with consistent successive samples from the same controls and exposed veterans, the Committee emphasized urgent need to maintain the funding of the materials and samples which are too valuable to place at risk.

DR. CAMACHO: [reading]

So an initial estimation study should be developed which can assess the multiple factors criteria for future access and the accompanying costs involved for such an archive. A designee or subcommittee of the Advisory Committee will work the study team to develop such an initial estimate, RFP or whatever for this endeavor.

Something along those lines.

DR. STOTO: I think that that's very good. I would add one thing which I think is implicit that we all understand, but I don't think is explicit there; and that is that the value of the data in these samples will continue long after the data gathering stops, because of the possibility of new hypotheses, new ways of analyzing samples, and so on and so forth. I think that's implicit.

DR. HARRISON: You know, you can always think of things to add; but I think that the two of you have done a really nice job of putting together all of the thoughts that we've had about the importance of maintaining this.

Mike, you were going to say something?

DR. GOUGH: I think it's a brilliant paragraph because it has an action plan in it. I mean, it doesn't say "this is what we recommend"; it says "this is what should be done." So I think that's really good.

DR. HARRISON: So -- our executive director is --

MR. COENE: It seems to me that probably that the thought needs to be taken out of the minutes of this meeting and delivered, at least, to a couple of appropriate places; I think the Surgeon General of the Air Force and the Secretary of HHS.

It seemed to me that that at least needs to go on record and at least -- and we've committed to do something in here, we the committee, and the project team. But that we need to alert those powers --.

DR. HARRISON: Why don't I undertake to do this. First of all, does the committee agree that we would like to see this statement inserted in the minutes as one of our concerns? Is that general consensus?

Any objections?

Okay. So you all will deliver your statement to Barbara, and Barbara will make sure it gets into the record.

MR. COENE: Well, and it's been captured.

DR. HARRISON: All right.

MS. JEWELL: But you're going to change that just a little bit? I saw you writing.


DR. CAMACHO: I like that statement; you want to tell me about --

DR. SILLS: How about the --

MS. JEWELL: If you get it to me, I'll get it to Dan.

DR. CAMACHO: Okay. Value of the --

MR. COENE: We'll make sure it's in there.

DR. STOTO: It's the value of the data, the clinical data and samples continues far beyond the time needed to gather the data, or something like that.

DR. HARRISON: Now, what I can propose to do for the committee is something similar to what I did the last time. And that is, I'll compose a letter to the Secretary of the Air Force and to Secretary Shalala, saying that there was an issue that arose during the latest meeting of the Ranch Hand Advisory Committee concerning the preservation of these samples beyond the termination of the study in 2006.

And when I did that letter before, I e-mailed it around to the committee members, who e-mailed back suggestions and corrections. I incorporated those and they sent the letter out on behalf of the committee.

If that's acceptable, I'll do that again and we'll get the same results --no.

DR. CAMACHO: But see, that's what -- I have some knowledge base about this, okay? And we've got a good chunk of years to do this in. By the time we march to the end, if we let me have some input in this --

DR. HARRISON: If we play our cards right.

DR. CAMACHO: -- if we play the cards right, we're going to go to a couple different committees, walk right through from authorization and appropriations and find the dough and make sure it gets done; if we do our game plan right with no last-minute-Charlie stuff.

DR. HARRISON: Okay. I would also like to bring up something else, if I may.

DR. STOTO: I'd like to say one more thing about this. I think the concept of having made a big investment in this study so far is an idea to get in.

DR. HARRISON: I'll certainly incorporate it in the letter.

I want to bring up something else. I got to thinking about this night, and I think if I were Joel, if I were the principal investigator on this study, and I had to plan what I was going to do over the next few years. I would make the assumption, number one, that we'll be successful in obtaining some sort of support to maintain this archive of samples and data beyond the planned termination of the data. Because to plan otherwise is kind of futile.

And I know that I've got a certain amount of money to go to the end of the study. I would make sure that I included all of the longitudinal things that have been a part of the protocol all along, because that's important for completion of the longitudinal study. So I've set that money aside, and then I'd look at what I had left and I'd say, "What can I do with this money that would enhance the value of this archive for the studies that are going to be performed past the year 2006?

Now obviously it's kind of based on what I -- just triggered me yesterday with the Epstein-Barr transformation, saving live cells as an archive. And I don't really know if there's anything else that could be done, but for example -- and I guess the other part of it is, I don't think that you're going to solve -- I don't think you're going to discover why documentation increases the occurrence of database in a couple of studies.

On the other hand, I'm not sure what the SAS file format is. What I mean by that is, that I know that nowadays there are SQL relational databases --

DR. CAMACHO: That's a different--

DR. HARRISON: There's XML-accessible -- my question, don't get tied up on the specifics. My question is, for instance, is it worth considering yet a third format for the entire database that might make it more accessible, more searchable, more evaluable? And I'm only mentioning that as a suggestion, not something that I feel like there's something to be done. Yes, sir.

COL. MARDEN: We already know that there's lots of data that was electronically archived 15-20 years ago that is almost unusable because of the legacy systems and the legacy types of format that it was archived in.

DR. SELVIN: 8-inch floppy drives, something like that.

COL. MARDEN: Yes. So my 5-1/4 inch is getting pretty long in the tooth.

The point being, I think there's no substitute for the hard copy, no matter how many ways we digitize the data, I think we're going to have to keep that hard copy.

DR. HARRISON: For instance, what about a plan to bring all the legacy stuff up to one single format?

DR. CAMACHO: Wait. When you're storing data, there's a variety of ways to do it. SAS-- it's not going to happen. SAS goes out of business and three years later nobody has any of the software to do a SAS file. But the fact is a standard SDF, everything's in a row, you've got the data record one row or if you have multiple rows, they're linked relationally by the ID, et cetera, et cetera. That's it, I don't care what you've got. Whatever data program you've got in the future, it's going to pick it up. An SDF file is an SDF file; its standard columns, there they are, we go out this far; it doesn't make a damn.

I mean, you don't --

DR. MICHALEK: We put it out two ways: flat files and SAS files.

DR. CAMACHO: And the SAS file can be changed and modulated and --

DR. HARRISON: A flat file is not relational, though, right?

DR. CAMACHO: No, but it doesn't make a difference. If I have two flat files and they have something -- a column in common, that's it. It can be sucked up into anything. It's just a matter of --

DR. HARRISON: Well, I'm raising it as--

DR. SELVIN: Archiving the data isn't near the problem as it is the biological material. It's trivial to put the data away someplace where people can get at it. That's a small issue.

DR. CAMACHO: That shouldn't be a problem, that's right.

DR. HARRISON: Well, the data is already in a format that you can't get at.

DR. CAMACHO: So it can be changed, it can be fixed.

DR. SELVIN: Maybe you can't make it last for a hundred years, but you can surely make it last the next ten years.

DR. HARRISON: How much people use things depends on its accessibility.

DR. SELVIN: Right. I'm just saying that the data is not as -- is a small problem both cost-wise and effort-wise compared to keeping the biological samples alive and frozen and maintained and some knowledgeable --.

DR. HARRISON: We've already got the biological samples in the proper storage. And if we produce any other biological samples, they'll be put in the proper storage.

DR. CAMACHO: Well, let's have a subcommittee on that.

DR. SELVIN: Well, I know little about this, but just having it in storage doesn't really do it, because you have to have people who know what's in storage and how it's accessible and --

DR. CAMACHO: It's the cataloging; the whole nine yards on that.

DR. HARRISON: Well, all right. I don't care whether we're talking about file formats or catalogs; but even for a catalog --

DR. CAMACHO: Well, that's what we said we're going to do here, though. A subcommittee or advisory committee will work with a study team, a designee or whatever to develop such an initial estimate and even an RFP, if possible. For the endeavor; we need an RFP, we need a study done on it, but let's get the basic parameters.

DR. HARRISON: My point is that there is money within this contract to carry out some of this.

DR. CAMACHO: Well, some of it should be carried out, then.

DR. HARRISON: And I'm suggesting that that should be a consideration.

DR. CAMACHO: That was a budget, right? What's the budget bottom line here? What have you got, every dime spent all the way to 2006?


DR. CAMACHO: So there's a big chunk hanging around?

DR. MINER: No. Oh, no, no, no. We have talked with our money people at Air Force and said "We are going to need some shutdown money, presumably to help archive" -- and they have agreed to do that. What they are very much against, the Air Force is not in the health study business, and they hardly have enough money to keep planes flying right now; and I think that will only get worse.

So we get a lot of angry generals that say "Well, why is part of my budget going to ra-ra-ra" type of thing. So there's that attitude every time we go for money. Now Congress says "Okay, fine, but you're going to do this."

So in some of your achiving things, keep that in mind, that the Air Force higher-up, their mission is fire and steel on target. It's not doing health studies.

DR. CAMACHO: What do you think, in just a ballpark, what we could weasel out of the government?

Just for looking at this issue, because I think we can get money elsewhere. I just think it can be done.

DR. MINER: And that's great.

DR. CAMACHO: Maybe I'm wrong, but I think it can be done with a good game plan.

DR. MINER: I can't give you right now what I think it would cost to archive.

DR. CAMACHO: Just to put this system -- let's put it to you, we're taking up time. Let's put the committee together, if it's the three of us, Steve, myself --

DR. HARRISON: Wait a minute, who's chairing this thing?

DR. CAMACHO: Well, I was making my suggestion. I know the database end of this stuff, and he's saying he'll take a lead on some sample stuff, and we'll put a little thing together for December.

DR. HARRISON: Are you going to be responsible for the sample stuff?

DR. CAMACHO: If we don't do anything, then we don't have a ballpark. It doesn't make a difference if we're wrong.

DR. HARRISON: I'm not proposing --.

Hold on, we're getting a little disorganized here. I'm not proposing that we do nothing. I'm just saying that I haven't participated in this committee for what seems like forever without having some interest in seeing what happens to it. So I'd like to stay involved in some way.

COL. MARDEN: I've got a full colonel lab officer that can probably help us with the archiving of the biologicals.

DR. MINER: The other thing, I think you were addressing more ease of use in a relational database type of activity versus all of our flat files sitting out there where an individual kind of has to know flat files, or look up in a data dictionary and so forth.

We've kicked this around a lot and have --.

DR. STILLS: Can I ask a question; when you say flat files --

DR. CAMACHO: Just think of everything you'd call them --

DR. HARRISON: Think of a spreadsheet with all this information on it, and think of another spreadsheet with all information on it from another guy, and think about how you're going to figure out how to connect all those together. There's no easy way.

The way I see the progression of this, Paul, and I don't want to -- I'm not trying to stifle anything; but the way I see the progression of this, when I first came on the committee, the Air Force did the study, accumulated all this information, and stuck it in the National Technical-something archives or whatever, where no one knew where the hell it was, no one used it, no one did anything.

In spite of what Joel says, the only things that got published were things that had to do with the technical aspects of doing statistics; and then they started doing the publication on the biological aspects, you know? Which are the things that attract interest.

And then lastly now we have where the material is being put on the web site and can be accessed, publicly accessed. So I'm seeing a progression from a study that was really being done sort of yn-yn-yn in itself, on out to being more and more accessible to the scientific community.

My question is, what are the last pieces that could be put on, so that someone could say "Gee, I never heard of this thing before, I'm wondering if it's got any relevance to my interest," and be able to get into it without being so dedicated that they're willing to spend the next, you know, two weeks figuring out how things are.

DR. CAMACHO: That's what --

DR. HARRISON: I'm interested in making it more accessible.

DR. CAMACHO: Let's put a couple of ideas on the table and let them grow. It's not like we're going to come to a decision immediately.

DR. HARRISON: I understand.

DR. MINER: We do have file descriptions of the flat files out there with all the datasets. You can go to the web site and say Okay, the file neoplasia/85-dat, and here's what's in it and it describes every column and row, what's there.

DR. HARRISON: So what's happening with the raw data that's on these older formats? Are they already in the process of being transferred so that's what Joel is saying, by the end of the year all of that's going to be on the web site?



DR. HARRISON: So all that's already being done.


DR. HARRISON: All right. So it's the sense of this committee that we have a subcommittee to work on this?

That's fair enough.

And Paul, you want to work on it.

DR. CAMACHO: I know I do.

DR. STILLS: I probably --

DR. HARRISON: You say you don't?

DR. CAMACHO: Oh, I definitely do.

DR. STILLS: I think I would probably want to work more on the health issues than the format and --


DR. GOUGH: I don't have any expertise in this, really.

DR. HARRISON: But you're our legacy -- legacy.


DR. GOUGH: At last, I'm a legacy. This means I get special privileges.

DR. HARRISON: Yes, you do.

[Simultaneous discussion]

DR. HARRISON: I think that we've got a social scientist, a --

DR. GOUGH: I think Steve.

DR. HARRISON: You think rather?

Okay, then Steve.

DR. GOUGH: There could be three of us. Is that all right?

DR. HARRISON: Well, and then it will be four of us; because I want to do the biology part.

DR. GOUGH: Oh, all right.

DR. HARRISON: That will be four of us, that's fine.

Yes, Jay?

DR. MINER: Your question was, all of the raw data going to be out there on the web? The answer is no, only the data that were used in the analyses.

Now we have, and Joel can describe what we have that we're not using the analyses; that is not out there, and --

DR. CAMACHO: No, but it should be preserved.

DR. MINER: But it should be preserved.

DR. CAMACHO: Because down the road -- my concern is down-the-road concerns. We don't know what --

DR. HARRISON: I'm using the term preserved and accessible; I don't mean by that freely accessible, but I mean it shouldn't be on something where you have to go to a museum to find the drive to read the floppy off of.

DR. STOTO: I think an issue to be considered is, to what degree, what are the conditions under which people can get access to the other data.

DR. HARRISON: That's another issue.

Yes, Joel?

DR. MICHALEK: I need to give you some more information, because you're heading down a path there that -- I don't think you know where you're headed, and that's not your fault. This is because you don't understand the full level of complexity.

We have perhaps thousands of datasets, we have thousands of SAS programs, we have hundreds and thousands of Fortran programs. We have datasets that are what we call raw datasets. They are delivered by NORC or SAIC or Lou Harris.

Now Lou Harris is a good example. We get a raw questionnaire file from baseline. Now if you were to go into that building and download that file and bring it up on your machine and start running with that file, you have made a fatal error.

The reason being that much of that data is incorrect. How do I know that? Because we've checked it. And what appears in the report are, we take a certain streak of that data; we take a particular column, and we check every number against the hard copy. Then we produce an analysis dataset, a pristine, clean, 100 percent checked data, that's what goes in the report.

But the raw data sits there, and it's wrong. I know it's wrong, Lydia knows it's wrong, that particular version of that dataset created on such-and-such a day is wrong, and we know it. There are hundreds of such examples in this study. It is a huge collection of modified -- original, edited, modified, extracted, merged and massaged data that I know where everything is. So does Lydia.

In other words, when you say we're going to release everything to the public, you are creating a mistake.

DR. HARRISON: I'm not saying release.

DR. MICHALEK: I know that.

DR. HARRISON: We're talking about --

DR. MICHALEK: I want to introduce this level of complications so that you understand that what we release to the public is what we know is absolutely correct.

DR. HARRISON: Joel, what I'm saying is that we need to consider if there isn't a way that all of those datasets are maintained, and that there is some sort of a flag that says that that dataset is unchecked and probably wrong.

And I'm not saying that that dataset needs to be available; I'm not saying that that dataset needs to be accessible by anyone on the outside at this point. But if some time, five or ten years from now, a decision is made that that particular dataset needs to be evaluated again, I would like for it to be possible for that to be done. And I would like for there to be sufficient information about that dataset that a naive person would have a chance of knowing what the quality of the dataset was and what its position was in the progression from raw data to a report.

DR. MICHALEK: You have just laid out a task that if it were pursued would put a serious dent in our ability to write research papers.

DR. STOTO: Can I suggest an alternative?

DR. HARRISON: Of course.

DR. STOTO: I mean, the alternative may be that in 2006, they, after having cleaned all the data, they dump all these raw data files and not bother about preserving them; but just make sure they preserve something more than --

DR. CAMACHO: These are interesting ideas, but why come to this conclusion today? It's a whole study in itself.

DR. HARRISON: Well, this is what we're supposed to do.

DR. STOTO: Yes. But I'm just saying that preserving everything is not the only option, and probably isn't the best option.

DR. MICHALEK: That is -- anxiety.

DR. CAMACHO: Yes, but nobody's making a decision here.

DR. MICHALEK: No, we're just talking.

DR. CAMACHO: We're just talking; let's kick the ball around and come up with something in December, listen to it, come up with a better plan for March. Then there's 2001, there's 2002, 2003. By the time we roll down the road, we ought to know what we're doing by 2004. In the meantime, we're trying to get the money.

DR. HARRISON: We've been rolling for almost 20 years. We still don't know what we're doing.

DR. MICHALEK: Let me give you another example.

DR. HARRISON: Now wait a minute, now. You're planning to write papers after 2006.

DR. MICHALEK: That's another point, by the way. We are rapidly reaching a point -- you know it takes up to five years to get a paper published. It took us five years to get our chloracne paper published. How much time do we have left on the study? Five years.

In two years, when I submit an article to a journal, it's very likely I won't be around; there will be no staff, there will be no study to receive referee reports. So we're reaching a point here where we're going to have to change our minds about what we're going to publish in this study or how we're going to publish it.

DR. HARRISON: Well, that's possible.

DR. MICHALEK: It's another hot topic for discussion, and we're going to reach that point in about two years.

DR. HARRISON: You know, one of the things you might consider is whether you should will this dataset to the Scripps Institute.

DR. MICHALEK: The Scripps Institute is not such a firm rock in the United States. That place is on the edge of bankruptcy quite a lot.

DR. HARRISON: Everyplace in the United States is on the edge of bankruptcy.

COL MARDEN: Will it to the Institute of Medicine.

DR. MICHALEK: But the point is that if you willed it to somebody that had an interest in it, then what you'd see would be just those papers that you got started and someone else was coauthor on would then, in the natural course, continue to be published and so on.

DR. MICHALEK: Now there's a problem, you see. If you submit a paper say a year from now, and it goes to a journal, and we don't get a referee report back until June of 2000, we don't get good referee reports back until 2006 when we're shutting down.

We're going to shut down. Now how is that other coauthor, say Jim Albers, University of Michigan, what's he going to do? He has no access to anything, he has no patient folders he can look at, he has no one he can talk to--

COL. MARDEN: That's back to the archive issue.

DR. HARRISON: Well, that depends on whether we are successful with the effort that we've agreed we want to undertake to --

DR. MICHALEK: Let me just say, what you have to figure out a way of doing is capture what's in my head and what's in lydia's head, and Billy Jackson and Bill Grubbs. Somehow --

DR. HARRISON: When I was in Arkansas, up in Fort Smith where the Campbell Soup factory is, they had one guy there who was the only guy who knew how to make those kettles work. And one day they sat down and started asking him questions about what he did when different things happened.

And they constructed a set of if-then-else rules, and then they fired his ass.


DR. MICHALEK: It's not quite that simple. I wish it was that simple.

DR. STOTO: Joel, presumably you and the other coworkers will retire in 2006, but hopefully you're going to continue living.

DR. MICHALEK: I'll move on to another job.

DR. STOTO: Move on to another job, and you might move on to a job at a university, and the university might be an appropriate repository for this --

DR. CAMACHO: There's 50 states. Every state -- if you want to put it --

COL. MARDEN: There are ethics considerations here.

DR. CAMACHO: If you want to put it in state universities.

DR. STOTO: I know, I'm sensitive to that, and I'm wondering if there's anything we could do because -- I mean, Joel is a national resource as well, believe it or not.

DR. MICHALEK: And Lydia.

COL. MARDEN: So we're going to mummify you when you die.

DR. MICHALEK: We need to make my cells immortal.

DR. STOTO: But I think it's worth recognizing that, and seeing what can be done about that.

COL. MARDEN: That's an interesting point.

DR. HARRISON: So our action plan is that we have a subcommittee of Paul, Mike, Steve and myself who are going to work on this issue. Paul is going to chair the subcommittee.

We have action items for the next meeting that we've already discussed and are already a part of the minutes.

You've got your hand raised. I was going to ask, is there anything else that we need to discuss before we take a break and then have our public statement?

DR. CAMACHO: This is just a response to your -- the 12th we want to come up with some parameters of a ballpark that we're going to modify over the next months, several months, years, couple of years, and have a whole plan together.

You had said archives, and you mentioned the university. Every state in the Union has a state university, and every state in the union had troops go to the war. Every state in the Union has an obligation to have their universities take a piece of the action, regardless of whether the cyclops has a piece of the action, to maintain this archive, put it that way. There's a lot of alternatives. And that's our committee's job, is to look out throughout these alternatives.

DR. HARRISON: What we want to do is, we want to make sure that the plan is solid, and not speculative.

DR. CAMACHO: It's not going to be developed in six months; it's going to take a year or two.

DR. HARRISON: Before we take the break, I proposed yesterday that we meet on December 7 and 8th.

Is there some consensus that those are reasonable dates?

DR. GOUGH: Well, I have prior engagement on the 8th.

DR. SELVIN: And it's from 11 to 2.

DR. GOUGH: So if you have a really long lunch -- just work around me.


DR. SELVIN: The first two weeks in December are impossible. I have two large classes with final examinations.

DR. HARRISON: And that's the same problem you have, right?

DR. CAMACHO: Well, no; I can get proctors to give out the exam.

DR. HARRISON: So those are possible?

DR. CAMACHO: The dates are possible.

DR. HARRISON: My experience is that the committee never meets as a whole.

Is it possible that, during those days, that you could participate briefly by phone?

DR. STILLS: Something like that.

DR. HARRISON: Okay, let's see.

We don't have -- who's missing?

MR. COENE: We don't have Landrigen, Favata, and Osay.

DR. HARRISON: So that's a significant number of people.

You're okay, Mike?

DR. STOTO: I'm okay. I'm pretty much okay the next two weeks after that, too.

DR. HARRISON: And Bob, you're not okay, right?

DR. SILLS: The first two weeks -- the first week I'm going to be in a meeting --

DR. HARRISON: You're chairing a meeting yourself.

DR. SILLS: I chair a meeting, and then the next week I'm in Toronto.

MS. JEWELL: How about the last week in November?

MR. COENE: There's not much room for the subcommittee to work, although we could postpone that discussion if we just --

DR. HARRISON: Barbara's just brought up, what about the last week in November?

DR. HARRISON: Because Thanksgiving is on the 23rd.

[Simultaneous discussion]

DR. HARRISON: So that's not bad for you?

DR. SILLS: That should be fine with me.

DR. HARRISON: That would be fine with you. That's the week before your exams; that would be --

DR. CAMACHO: Fine with me. It's a Thursday and Friday.

DR. HARRISON: That would be fine with you. That's still bad with you?

DR. SELVIN: I can't take any more time away from class.

DR. HARRISON: I understand.

So that sounds like that's even a better time, would be November 30th and December 1st. So if you all will hold those dates.

MR. COENE: And we'll try to clear them on Monday.

DR. HARRISON: We'll check with the other three members of the committee. I suspect that will be okay with you all? You want it as soon as we can generate it?

MR. COENE: Hold it a second. That really crunches the time we have to get those outside reviewers reviewing those six proposals, seven proposals or whatever the number.

DR. HARRISON: Yes, it does, but we'll just have to do it the best we can.

DR. MICHALEK: Well, you know, that deadline isn't so important, either. I mean the deadline on those could be pushed up to March of next year.

DR. HARRISON: And we just work on the statement of work for this time?

DR. MICHALEK: Yes. Let's plan on that. You don't need to talk about those proposals until March of next year.

DR. HARRISON: Okay, now --

MR. COENE: That's fine. That would -- because we'll be hustling to put this committee back together.

LTC BURNHAM: You're going to be getting into some fairly detailed stuff on that statement of work. Is two days going to be enough?

DR. CAMACHO: Can we go to Saturday, too?

DR. HARRISON: No, no. Something that can't be done in a day and a half isn't worth doing.

Now right now we don't have a particular date or set of dates or proposed set of dates for the March meeting. Does anyone have any suggestions?

DR. GOUGH: Do you mind if I go get my calendar, which I've forgotten?

DR. HARRISON: No. Not at all.

In fact, why don't we do this --

MR. COENE: Pick that up after the break.

DR. HARRISON: We're tentatively okay for December, and after the break we're going to try to do March.

DR. GOUGH: Can I ask one technical question, about the results?

Yesterday you said that this increase in carotid arteries increases with dioxin body burden in the comparisons as well as in the Ranch Hands. Isn't that true of diabetes as well?


DR. GOUGH: Isn't that a puzzling -- it sounds like there's a marker here that we don't understand --

VOICE: What is the chicken, what is the egg?

DR. GOUGH: Yes. Are there other examples of that where there's --?

DR. MICHALEK: We haven't looked in the comparison group for those kinds of trends, in every variable.

DR. GOUGH: Yes, I think it would be silly to do it everywhere, but that's just such a puzzle.

DR. MICHALEK: We have not looked extensively, we've only looked at heart disease and diabetes.

LTC BURNHAM: That makes sense, though, because the comparison group does have levels. But in many instances, in other studies, your comparison group doesn't have any exposure; but in this one they all do have.

DR. GOUGH: Yes, but ten -- I mean, all of us are running around with 5 to 10 ppt. And if the slopes are the same, it's as though, if dioxin were the causative or associative agent and the slopes are the same, it's as though if you're exposed to just a little bit, the little bit's more potent than if you're exposed to a lot.

That doesn't make any sense, but it's a real puzzle to me as to why --

DR. MICHALEK: But what happens in the Ranch Hand group is that individuals with higher levels, above background, increases the risk even further. So you have an increased risk --

DR. GOUGH: Okay, I'm sorry. I didn't understand.

In the comparisons let's arbitrarily say it goes up to 10, and in the Ranch Hands it goes up to 600. So at 10, in the two groups --

DR. MICHALEK: The two groups are roughly parallel up to 10.

DR. GOUGH: Up to 10 --

DR. MICHALEK: And then beyond 10, the Ranch Hand risk keeps increasing with increased dioxin. There were controls with beyond that.

DR. GOUGH: Okay. That's what I understand.

DR. MICHALEK: That's how the diabetes works.

DR. HARRISON: Why don't we do this. Barbara has stuff for everybody to sign. Why don't we take a break and let's --

DR. STOTO: Can we ask whether Jack has stuff to say, or?

MS. JEWELL: That's after.

MR. COENE: At 11 o'clock he's on.

JACK: I've got to write my speech here.


DR. STOTO: How much do you think you're going to have?

MAJ SPEY: Two or three minutes.

DR. STOTO: Why don't we just do it now?

DR. HARRISON: No, let's take a break first.


DR. HARRISON: To get back to the issues of meeting times, we're already honed in on November 30th and December 1st. And now the March dates, does anyone --

DR. STOTO: I think that the very last, end of March is difficult for me.

DR. HARRISON: The very last what?

DR. STOTO: 29th and 30th.

DR. HARRISON: We want to do somewhere in the first 2 weeks of March.

DR. STOTO: Then I'm okay.

DR. HARRISON: It looks, according to my calendar, that March 1st and 2nd are again a Thursday - Friday time period, and obviously then the 8th and 9th are the Thursday - Friday of the second week.

I mean, there being no conflict with either of those dates, and March 1st isn't -- that's not Mother's Day, that's in May.

It's not Air Force Appreciation Day or anything like that.


MS. JEWELL: And where are we having this meeting?

DR. HARRISON: Well, that's something else to discuss; but I think that considering the financial constraints and considering that the Air Force has all the money in the world, that it would be cheaper for them to fly to where we're closer than for us to fly to them. So we might want to do this in D.C.

DR. STOTO: Where's the December meeting?


DR. HARRISON: That's my suggestion; I'm just making those observations. What Ron is saying is that it costs about $20,000 a meeting, and if we want to try and have three more meetings in this fiscal year, then --

DR. STOTO: I'm just asking, I'm not complaining. Or even offering an opinion.


DR. GOUGH: I want to go to California again.

DR. HARRISON: That's the next cycle, though.

MS. JEWELL: The Ranch Handers are out there.

DR. HARRISON: So that's 2002, right?

DR. GOUGH: We could take Jack with us.

DR. HARRISON: So if no one has any real objections, can I get you for the next week to hold both of those Thursdays-Fridays, 1st, 2nd, 8th, 9th, and Barbara and Ron can contact the other members and see if one gives us more yield than at the other, and we'll pick whichever one gives us the highest yield.

DR. GOUGH: You're talking about meeting Thursday morning.


DR. GOUGH: So it's really Wednesday, Thursday.

DR. HARRISON: No, what I'm talking about is --

DR. CAMACHO: He's coming from California --

DR. HARRISON: What I'm talking about is, you arrive Wednesday --

[Simultaneous discussion]

DR. HARRISON: So that being done -- where's Jack?

MS. JEWELL: Where's our public?

DR. HARRISON: Because if we don't have any -- yes?

DR. GOUGH: Well, this is sort of administrative, and it's talking to Ron at dinner last night, but evidently the Secretary's office --

COL. MARDEN: The public and I were talking.

DR. GOUGH: -- the Secretary's office did not make any real effort to inform veterans organizations that we were having this meeting.

MR. COENE: That's -- we were able to determine.

DR. GOUGH: Well, should we, should the committee send a letter to the Secretary about that? I mean, because we're going to be the ones -- the Congress says we're supposed to make every effort to do outreach, and we've failed.

MR. COENE: I don't -- it rains on me in the end, but it doesn't -- somebody needs to --

DR. CAMACHO: See -- I'm sorry.

DR. GOUGH: One of the criticisms has been, or one of the -- this committee's got to make more of an outreach to the veterans community. And because of all the lateness and slowness in getting the committee appointed, that wasn't done, but that's supposed to be done through the Secretary's office, of HHS.

And they didn't do anything, or the office didn't do anything. So I think that we should, as a committee, send a letter to the Secretary and say that we would like to have more veterans participation and her cooperation in getting the information out.

DR. CAMACHO: What do you mean by participation.

DR. GOUGH: They actually come and testify.

COL. MARDEN: Well, here's Jack.

DR. GOUGH: We have an hour for the public. Jack is here; oftentimes there's no one here.

MS. JEWELL: And he wasn't here because he was notified by the Office of the Secretary. Is that correct?

JACK: No, ma'am.

DR. STOTO: At the last meeting, the ones who did come found out about it at the last moment and gave us a lot of trouble for it.

MS. JEWELL: Well, no, actually the last two meetings, the Office of the Secretary did notify them and we had the first good turnout we've ever had from veterans groups.

DR. HARRISON: So why don't we do--

DR. GOUGH: I'll be happy to write the letter, and send it to Bob.

DR. HARRISON: Well, the other question that you haven't asked is, What is the mechanism for -- I mean, we have to tell the Office of the Secretary when we are going to meet. And then the Office of the Secretary has to provide the notification.


MR. COENE: And that's what we do.

MS. JEWELL: Yes, and we did all that. And we made telephone calls and we e-mailed the second time. The person that was taking care of it retired, and they've not obviously named anyone else.

DR. HARRISON: Okay, Mike, we're going to be a letter-generating committee here, that's good. So why don't you generate a letter, same format as what I'm doing, you generate the letter, you circulate it through e-mail, you're writing a letter on behalf of the committee, so there should be a reasonable consensus about its content. And when that is reached, then the letter will go out and --

DR. GOUGH: And it should be quick, because we've got a meeting in seven weeks.

DR. HARRISON: Yes, and in that letter it should contain -- obviously it has to contain the next meeting dates.


COL. MARDEN: You could do the time-honored military thing of, "Dear Madam Secretary, unless we hear different from you, we'll announce our meetings to the veterans organizations."

DR. GOUGH: You don't want to take that responsibility.

DR. HARRISON: I think -- the other thing that I would do, Joel, whoever your -- is this acceptable with your webmeister? If you look on your web page, in the actual HTML text, it should have somewhere in the key word section there, it should have Ranch Hand -- you know, it should have a number of keywords that search engines will use to categorize that page.

Can you include on your web page the next Ranch Hand Advisory Committee meeting date and the tentative time period for the March meeting, even? You know, you can say the March meeting will be held within the first two weeks of March, and just have that on your web page so it will come up on search engines.

DR. MICHALEK: The answer is yes, of course.

DR. STOTO: Is there some single veterans organization that we can work with to spread the word to others?

DR. CAMACHO: You're going to get into trouble.

MR. COENE: There's a list of 12 that we're aware of that have been contacted and were contacted for members.

DR. STOTO: Can you let them know about this, too? In addition to the Secretary.

MR. COENE: The point is yes, we can do all of this. The issue is to keep this in an elevated and within at least some radar screen within the Office of the Secretary. If we just --

DR. STOTO: Well, I think you should do it, and I don't think you should do --

MR. COENE: Yes, that's what I -- that's my -- if we just go along and just ignore it --

DR. CAMACHO: But we can go around them, like midnight requisition; but it doesn't solve the problem of staying on the Secretary's radar screen and making sure that this project is in the infrastructure of this agency.

DR. HARRISON: That's right.

MR. COENE: Yes. That letter will be another reminder.

DR. CAMACHO: And that takes some clout, and we'll have to beat them over the head if we have to, and have friends beat them over the head.

MR. COENE: Not too hard. I'm good at writing letters back for the Secretary.


COL. MARDEN: You mean, we should just address the letter 'Dear Ron' and then you write back, 'Dear Bob'?

MR. COENE: Yes, I'm going to see it -- but at least we do hit a few nerve endings as it comes up and then back down again.

DR. CAMACHO: What's the flow of the agenda? Here's my point --

DR. HARRISON: We're almost finished.

DR. CAMACHO: I know. So my task is to put this letter in format for you to look at and consider; is that true?

DR. HARRISON: What letter are you talking about?

DR. CAMACHO: I'm talking about the diabetes stuff, the archive. You are sending this letter, but you told me to sketch this out or put this in writing, and it will be for you to submit it?

DR. HARRISON: That's an insert that goes into the minutes, right?

DR. CAMACHO: Then I don't have to worry about that, it's being done.

DR. HARRISON: Well, you're going to give what you have written to Barbara so that she can make sure that --

DR. CAMACHO: Oh, I'm giving it to Barbara. Okay.

DR. HARRISON: And then I'm writing a letter; based on your text and based on your discussions, I'm writing a letter that's going to incorporate that issue and also incorporate the issue of the continuity of this study past 2006.

DR. CAMACHO: So you're writing that letter to Donna Shalala?


DR. CAMACHO: And I'm writing the archive piece to her, and you're going to pick that up -- or am I cc'ing you?

DR. HARRISON: I'm just going to pick that up. In fact, I think all you need to do is give your handwritten copy to Barbara. I don't think we need to generate anything else.

DR. STOTO: We need to go in a couple minutes.

DR. HARRISON: So there being no other business on our agenda, it's now time for the public statement.

DR. CAMACHO: Just this point. You're arguing about Ron and his replacement and the budget; are you writing that letter?

DR. HARRISON: Those are all mine.

DR. CAMACHO: Okay, and then the veteran public notification, what did we decide we were going to do with that?

DR. GOUGH: I'm going to write a letter, and I'll circulate it to the committee.

DR. CAMACHO: Okay. So here I've done my duty by passing this over there.

DR. HARRISON: And done it well.

So, anybody else before the public statement?

Let's go. Major?


MAJ SPEY: I'd just like to thank everyone for the opportunity just to be here.

One of the things that comes to mind about this study is that what has been found so far, particularly concerning the diabetes finding, is that it's generally stated that those in contact with herbicides have an increased risk of diabetes, and that's not the case.

We're seeing, those of us that carry a relative, comparatively large volume of diabetes in our blood, showing an increased risk of type II diabetes, type II being, as you all know, generally controlled by weight, diet, some medication possibly. We're not talking insulin-dependent.

We're seeing some variation in body chemistry which is unexplained; but that variation is not showing in terms of mortality or general morbidity; it's just a change in chemistry, none of which is understood yet, and may never be understood as it applies to causation.

My fear is that many, many veterans unnecessarily are concerned about their health, long-term health, simply because they went to Vietnam and served in Vietnam. And this study compares the Ranch Hand cohort and the comparison group cohort who were Vietnam veterans, and it was done so out of good science and it was done so intentionally so that we weren't mixing apples with oranges.

And what we're finding is that all of the hoopla that we've heard sine 1975 concerning "Agent Orange" has upset, in my mind and in the minds of many of us, unnecessarily; and it's a very minor, I consider it a very minor deviation in health of those of us who have an elevation of dioxin in our blood over and above somewhere around 30 parts per trillion.

You are looking for a needle in a haystack, and there's some that will argue that the haystack doesn't even exist, but that's just my own personal view. But I just want to emphasize the importance of this study and sticking with the protocol right through to the end so that there can be no criticism of this study as a result of some sort of modification towards the end. It's the finest study that's ever been done on the human population, bar none. It's the hallmark epidemiology study that's ever been conducted by this nation's scientific community, and I think that's well-recognized by the scientific community. And the members of Operation Ranch Hand are damn proud to be a part of it.

Thank you very much.

DR. HARRISON: Thank you very much, Major.

Well, there being nothing else for us to fiddle-faddle about, I declare this meeting adjourned.

Whereupon, at 10:43 a.m., the meeting adjourned.)

- - -