Study Design Considerations for Observational Comparative Effectiveness Research - PowerPoint Presentation

Slide1

Study Design Considerations for Observational Comparative Effectiveness Research

Prepared for:

Agency for Healthcare Research and Quality (AHRQ)

www.ahrq.gov

T

his presentation will

:Provide a rationale for study design choice and describe key design featuresDefine start of followup Define inclusion and exclusion criteria at start of followupDefine exposures of interest at start of followup.Define outcome(s) of interestDefine potential confounders

Outline of Material

Conventional designs

Cohort

Case-controlCase-cohortSelf-controlled designsCase-crossoverCase-time-controlledSelf-controlled case series

Study Design Overview

Exposures or treatments are not assigned, a situation which leads to challenges ensuring internal validity, that is, the absence of bias.

To ensure internal validity, treatment groups compared must have the same underlying risk for outcome within subgroups definable by measured covariates (e.g., no unmeasured confounding).

Confounding by indication leads to higher propensity for/more intensive treatment in those with the most severe disease.With confounding by frailty, frail patients (close to death) are less likely to be treated with preventive treatments.Ensuring a study’s internal validity is a prerequisite for its external validity or generalizability.Issues of Bias in Observational

Comparative Effectiveness Research

Cohorts are defined by their exposure at a certain point in time (i.e., baseline date) and are followed over time for the occurrence of the outcome.

Advantages:

Has a clear timeline separating potential confounders from the exposure and the exposure from the outcome Allows estimation of actual incidence (risk or rate)Can assess multiple outcomes Is easy to conceptualize Limitations:Is inefficient for ad hoc studies when the incidence of the outcome is lowStudy Design: Cohort Study

Identifies all incident cases that develop an outcome and compares exposure history to controls

Samples controls at random from cohort members at risk for developing an outcome

Advantages:Oversampling cases increases computational efficiency of ad hoc studies when compared with a cohort study Can assess multiple exposuresLimitations:Is difficult to conceptualizeHas potential for recall bias in ad hoc studies

Study Design: Case-Control Study

Cohorts defined as in a cohort study

Cohort members followed for incidence of outcomes

Additional information required for analysis collected for a random sample of the cohort and all casesIncreased efficiency, when compared with a full-cohort design, if additional information needs to be collectedDecreased efficiency, when compared with a nested case-control design, unless studying multiple outcomes or estimating riskStudy Design: Case-Cohort Study

Prior exposure history of cases used as the control

Removes confounding effect of measured and

unmeasured characteristics that are stable over time (e.g., genetics)Appropriate for studying acute effects of transient exposures Advantages:Self-controlledAbility to assess short-term reversible effectsAbility to inform about the time window for these effectsLimitations:

Assumes constant prevalence of treatments over time

Does not allow estimation of treatment effect in a population

Study Design: Case-Crossover Study

Adjusts for calendar time trends in the prevalence of treatments, which can bias the case-crossover design

Divides the case-crossover odds ratio by the equivalent odds ratio estimated in controls

Advantages:Not dependent on assumption of no temporal changes in the prevalence of treatmentLimitations:Need for controls adds complexityControl for time trend can introduce confoundingStudy Design: Case-Time-Controlled Study

Estimates the immediate effect of treatment in those treated at least once

Dependent on cases that have changes in treatment during a defined observation period

Advantages:Controls for factors that are stable over timeCohort design has the potential to increase efficiency Well suited for rare adverse events in vaccine safety studiesLimitations:Limited applicability in many comparative effectiveness research studiesStudy Design: Self-Controlled Case-Series

Study setting

Consideration of the study population and data source(s)

Inclusion and exclusion criteriaShould be clearly defined Include details about the study time periodChoice of comparators Reduces potential for confounding by comparing treatment of interest with a different treatment for the same indication or an indication with the same potential for confoundingStudy Design Features

New-User Design

The conventional

prevalent user design is prone to confounding and selection bias as a result of changes in treatment effects over time.Including only new users reduces bias and confounding associated with inclusion of prevalent users.There must be a clear starting point for followup under similar conditions of medicalization.Immortal Time BiasOccurs as a result of defining the exposure during the followup time rather than before followup New-user design and use of comparator treatments reduce potential for this bias

Other Study Design Considerations

Knowledge of study design options is essential to increase internal and external validity of observational comparative effectiveness research.

Biases introduced by suboptimal study design cannot usually be removed by statistical analysis.

Cohort design is preferred when data have already been collected; the validity of a nested case-control study is equivalent, given proper control selection and timing of exposures and covariates. It is important to define the start of followup, inclusion and exclusion criteria, outcome of interest, and potential confounders at the outset. Conclusions

Summary Checklist (1 of 3)

Guidance

Key Considerations

Provide

rationale for study design choice and describe key design features

Cohort study proposals should clearly

define the cohort entry date (baseline date), employ a new-user design (or provide a rationale for including prevalent users), and plans for reporting losses to followup.

Case-control study proposals should clearly describe the control sampling method, employ a new-user design (or provide a rationale for assessing confounders at the index date), and assess potential for recall bias (if applicable).

Case-cohort study proposals should include how the sampling scheme will be accounted for during analysis.

Case-crossover study proposals should discuss the potential for confounding by time-varying factors and clearly state how the resulting effect estimate can be interpreted.

Case-time-controlled study proposals should clearly weigh the pros and cons of accounting for calendar time trends in the prevalence of exposure.

Summary Checklist (2 of 3)

Guidance

Key Considerations

Define start of followup (baseline)

The time point

for start of followup should be clearly defined, meaningful, and ideally anchored to the time of a medical intervention (e.g., initiation of drug use).

If alternative approaches are proposed, the rationale should be provided and implications discussed.

Define

inclusion and exclusion criteria at start of followup

Exclusion and inclusion criteria should be defined at the start of

followup (baseline) and solely based on information available at this point in time (i.e., ignoring potentially known events after baseline).

The definition should include the time window for assessment (usually the same for all cohort members).

Summary Checklist (3 of 3)

Guidance

Key Considerations

Define exposure (treatments) of interest at start of followup

Use an active comparator (indicated as alternative treatment at same stage of disease progression) when possible.

Define outcome(s) of interest

Provide information on measures of accuracy if possible.

Define potential confounders

Potential confounders known to be associated with treatment and outcome should be prespecified when possible.

Confounders should be assessed before exposure or treatment initiation to ensure they are not affected by the exposure.

Approaches to empirical identification of confounders should be described if planned.