(P13-078-24) Grilling the Data: Interpreting the Results of Nutritional Epidemiology Studies in the Context of Variation Expected Due to Analytic Flexibility

Objectives: When investigators analyze data from nutritional epidemiology studies, they make numerous potentially justifiable, but still subjective, analytic decisions on which the direction, magnitude, and statistical significance of findings may be contingent.

We present a novel approach to account for analytic flexibility in nutritional epidemiology studies called multiverse analysis. This method involves defining and implementing all plausible and justifiable analytic approaches for addressing a research question. Investigators can subsequently consider the range of all plausible results and express more confidence in results that are consistent across all or most justifiable analytic specifications. We apply this new method to the effect of unprocessed red meat on all-cause mortality.

Methods: We reviewed all nutritional epidemiology studies addressing the effect of red meat and all-cause mortality from a recently published systematic review and documented variations in analytic choices made across studies. We used data from the 2007 to 2014 National Health and Nutrition Examination Survey (NHANES) and applied multiverse analysis to investigate the effect of unprocessed red meat on all-cause mortality. Our choice of analytic methods was informed by analytic methods used in published studies addressing the same or similar questions.

Results: We applied multiverse analysis to NHANES, with 10,661 participants. We performed 1,208 unique analyses investigating the effect of red meat on all-cause mortality, of which 48 (3.97%) produced statistically significant results. We found considerable variability in results across analytic specifications, with hazard ratios ranging between 0.51 to 1.75 (median: 0.94 [IQR: 0.83 to 1.05]).

Conclusions: While this study may provide insights on the health effects of red meat, the primary objective is to demonstrate the application of multiverse analysis to nutritional epidemiology. We show that the application of multiverse analysis to nutritional epidemiology is feasible and presents an innovative solution to the problem of analytic flexibility. The broader application of multiverse analysis may enhance confidence in nutrition as a field by encouraging investigators to acknowledge an additional source of uncertainty.

Funding Sources: None