Animal & Veterinary
Presentation: Mathematical Validity of CVM Risk Assessment
Slide 1
Mathematical Validity of CVM Risk Assessment
Tony Cox
Slide 2
Is the CVM RA sound and useful?

Sound:
 Predicted risks are calculated correctly by the model, given its assumptions
 Model can be calibrated by simulation

Useful:
 Model assumptions approximate reality well enough to support effective risk management decisions
Slide 3
Explicit assumptions
 A strength of the model is its listing and discussion of formulas, assumptions, and their limitations (e.g., Appendix C)
 Examples: Attribution of fluoroquinolone resistance to chicken, stability of risk estimates over time, careseeking behavior.
 Implicit assumptions are also important.
Slide 4
Other Key Modeling Assumptions
 Independence of uncertain quantities
 Extrapolation between populations
 Aggregation of event sequences
 Modeling of input uncertainties and formula uncertainties (e.g., beta and gamma distributions, productsand ratios of uncertain quantities)
Slide 5
Independence Assumptions

Example: Should inputs be modeled as statistically independent (e.g., pbm and pnm)?
 Alternative: Estimate conditional distribution (pnm  pbm). Then joint distribution: Pr(pnm, pnm) = Pr(pbm )Pr(pnm  pbm)

Generalization: Condition each uncertain quantity on its relevant causal predecessors.
 pm*(pbc  pm)*(pt  pm, pbc)*(p+  pm, pbc, pt)
 Expected impact: Small when independence is a reasonable approximation.
Slide 6
Extrapolation
 Example: Scaling estimates up by nUS/nEN.
 "Expected observed cases" is a different (simulationoriented) conceptual approach from typical Bayesian conditioning of a prior on observations.
 As stated in the report, different populations may have different distributions of relevant factors.
Slide 7
Aggregation of Events
 Example: p = pmpbcptp+ = (pmpbc)(ptp+)
 Estimating all components and combining allows detailed information in outputs

Estimating p directly and by different "factorings" might slightly increase precision of estimates.
 Exact PDF of product of beta distributions is known
Slide 8
Modeling Input Uncertainties
 Specific parametric distributions (gamma and beta) are reasonable, but strongest conclusions come from sensitivity analysis.
 Uncertainties about joint distributions and dependencies among uncertainties could be analyzed further  Minor refinements
 Trying other technical options for estimating joint distribution of inputs could add to confidence from current sensitivity analyses.
Slide 9
Model Formula Uncertainties
 Most of the model's formulas are intended as simple logical identities based on sums, products, and ratios.
 Even simple nonlinear formulas (ratios) can introduce some biases (but often < 1%).
 Potential bias for whole model should ideally be estimated (e.g., via simulation calibration.)
Slide 10
Potential Extensions

Calibrate by simulation.
 "True" values  simulated sample values  model estimates of values

Additional sensitivity analyses
 Initial sensitivity analyses are encouraging, i.e., main results not very sensitive to input or model uncertainties
 Sensitivity to population heterogeneity
Slide 11
Conclusions
 Model structure and calculations are welldocumented and logical. ("Face validity")

Modelbased risk predictions are credible.
 Calibration and extensions could quantify credibility

Uncertainties in input quantities are explicitly and, in general, appropriately modeled.
 Could model dependencies among variables
 Could do further sensitivity analyses to assumed distributions of inputs and/or estimate them differently
 High value from going beyond point estimates
Slide 12
Potential Workshop Issues

Discussion of model
 What do you see are limitations of model?
 Do you feel there are significant data gaps?
 What are positive aspects of model?
 What aspects would you consider changing?
 How can this model be used to help industry reduce the level of risk?
 Mathematics of model
 Use of the model for other antimicrobialfoodborne pathogen combinations