-

# Presentation: Mathematical Validity of CVM Risk Assessment

Slide 1

Mathematical Validity of CVM Risk Assessment

Tony Cox

Slide 2

Is the CVM RA sound and useful?

• Sound:

• Predicted risks are calculated correctly by the model, given its assumptions
• Model can be calibrated by simulation
• Useful:

• Model assumptions approximate reality well enough to support effective risk management decisions

Slide 3

Explicit assumptions

• A strength of the model is its listing and discussion of formulas, assumptions, and their limitations (e.g., Appendix C)
• Examples: Attribution of fluoroquinolone resistance to chicken, stability of risk estimates over time, care-seeking behavior.
• Implicit assumptions are also important.

Slide 4

Other Key Modeling Assumptions

• Independence of uncertain quantities
• Extrapolation between populations
• Aggregation of event sequences
• Modeling of input uncertainties and formula uncertainties (e.g., beta and gamma distributions, productsand ratios of uncertain quantities)

Slide 5

Independence Assumptions

• Example: Should inputs be modeled as statistically independent (e.g., pbm and pnm)?

• Alternative: Estimate conditional distribution (pnm | pbm). Then joint distribution: Pr(pnm, pnm) = Pr(pbm )Pr(pnm | pbm)
• Generalization: Condition each uncertain quantity on its relevant causal predecessors.

• pm*(pbc | pm)*(pt | pm, pbc)*(p+ | pm, pbc, pt)
• Expected impact: Small when independence is a reasonable approximation.

Slide 6

Extrapolation

• Example: Scaling estimates up by nUS/nEN.
• "Expected observed cases" is a different (simulation-oriented) conceptual approach from typical Bayesian conditioning of a prior on observations.
• As stated in the report, different populations may have different distributions of relevant factors.

Slide 7

Aggregation of Events

• Example: p = pmpbcptp+ = (pmpbc)(ptp+)
• Estimating all components and combining allows detailed information in outputs
• Estimating p directly and by different "factorings" might slightly increase precision of estimates.

• Exact PDF of product of beta distributions is known

Slide 8

Modeling Input Uncertainties

• Specific parametric distributions (gamma and beta) are reasonable, but strongest conclusions come from sensitivity analysis.
• Uncertainties about joint distributions and dependencies among uncertainties could be analyzed further -- Minor refinements
• Trying other technical options for estimating joint distribution of inputs could add to confidence from current sensitivity analyses.

Slide 9

Model Formula Uncertainties

• Most of the model's formulas are intended as simple logical identities based on sums, products, and ratios.
• Even simple non-linear formulas (ratios) can introduce some biases (but often < 1%).
• Potential bias for whole model should ideally be estimated (e.g., via simulation calibration.)

Slide 10

Potential Extensions

• Calibrate by simulation.

• "True" values -- simulated sample values -- model estimates of values

• Initial sensitivity analyses are encouraging, i.e., main results not very sensitive to input or model uncertainties
• Sensitivity to population heterogeneity

Slide 11

Conclusions

• Model structure and calculations are well-documented and logical. ("Face validity")
• Model-based risk predictions are credible.

• Calibration and extensions could quantify credibility
• Uncertainties in input quantities are explicitly and, in general, appropriately modeled.

• Could model dependencies among variables
• Could do further sensitivity analyses to assumed distributions of inputs and/or estimate them differently
• High value from going beyond point estimates

Slide 12

Potential Workshop Issues

• Discussion of model

• What do you see are limitations of model?
• Do you feel there are significant data gaps?
• What are positive aspects of model?
• What aspects would you consider changing?
• How can this model be used to help industry reduce the level of risk?
• Mathematics of model
• Use of the model for other antimicrobial-foodborne pathogen combinations
-
-