Getting - PowerPoint Presentation

Slide1

Getting the statistics right for integrative research involving Ayurveda

Ashwini Mathur(Novartis Healthcare Pvt. Ltd, Hyderabad)August 1, 2013, Samyukti 2013

1Slide2

Disclaimer/AcknowledgementsAll views expressed are authors’ and do not reflect the views of Novartis.

Images taken from the internet are freely available and are not copyright protected.Vinay Mahajan (Novartis)Vivek Sanker, Sriranjini Jaideep, Ashwini VK (Institute of Ayurveda and Integrative Medicine, Bengaluru)Girish Tillu (Symbiosis School of Biomedical Sciences, Pune)2Slide3

GOOD SCIENCE – Ethical, scientifically unbiased, transparent

FRAUD

Opaque, Secretive

X

LUCK

Opaque, Secretive

“BAD” SCIENCE

BIASED“BAD” SCIENCE

DECISIONS

WHAT DECISION WAS TAKEN ?

CORRECT INCORRECT

HOW DECISION WAS TAKEN ?

KNOWINGLY

UN-KNOWINGLY

INTRODUCTION

Apr 19, 2012

Statistical Designs in Clinical Trials

3Slide4

BackgroundScientific Questions is the “driver”

Statistics as a scientific endeavor is one of the tools that can help answer the questionStatistical science is the “driver”Statistics as a scientific endeavor is one of the tools that can help generate the question that needs to be answered.4Slide5

BackgroundBasic issues related to statistical science that can lead to biased results

Design of trialSample sizeMultiple Decisions from one experiment5Slide6

ExampleTeam visits IHST Campus to evaluate the research facilities available for conducting

Ayurvedic clinical researchAim is to evaluate academic credentials of staffInfrastructure to conduct Ayurvedic treatmentsQuality control of Ayurvedic formulationsWhile visiting IHST, a team of 4 evaluators, carry out their research and prepare their reportOne of the evaluators finds a Rs. 1000 note at the entrance of IHSTWhat do you do?6Slide7

ExamplePublish report saying:-

IHST is suitable for clinical research andEveryone who goes their will find Rs. 1000Rs 1000 was found but is in-consequentialRs 1000 was found and it will be found again and reason is that it is an area of high people mobilityRs 1000 was found but we cannot say anything as this experiment was not designed to look for Rs 1000.To prove that Rs. 1000 is a real finding, a designed experiment needs to be done.How ?7Slide8

Apr 19, 2012

Statistical Designs in Clinical Trials 8CLINICAL RESEARCH ....Slide9

Statistics

Lies, Damned Lies, Statistics – Mark TwainLies, Damned Lies and THEREFORE StatisticsCorrect Answer to Incorrect QuestionApproximate Answer to Correct Question9Slide10

Apr 19, 2012

Statistical Designs in Clinical Trials 10CLINICAL RESEARCH AND STATISTICAL PRINCIPLESSlide11

General Problems in Clinical Trials

PublicationLack of transparency Publication biasScientific issues with trial design Ethical issues in reportingPublications generate many more questions than provide answers!11Slide12

Methodology, Quality and Scope

Transparency issuesTrials published in many journals not indexed by the databases Trials conducted could have been reported in language other than English. Tendency of publishing more positive studies vs. negative studies. This publication bias would result in building biased scientific literature.

Scientific issues

Studies

were of short duration with lesser patients. The duration may not reflective of the true clinical setting, giving rise to meaningless results. Smaller studies tend to overestimate the treatment

effects.

Methodological

quality of the trials was suboptimal. Randomization, single arm studiesEthical issuesSmaller sample sizesNegative unpublished workApplication of western endpoints to traditional methods served12Slide13

Statistical Issues

Design and sample sizeAddress issues related to Complex InterventionsUnknown effect sizeAnalysisAddress issues related to bias due to design13Slide14

Sample Size

Sample SizeDepends on effect size, “noise”, ErrorSample Size if not specifiedUnethicalLarge study or Small studyInterpretation of results is problematicBiological significance vs. statistical significanceNegative results – due to ineffective treatment or due to more than planned “noise”14Slide15

Sample Size

Sample size could be estimated to do Hypothesis testing provided a clear hypothesis is statedSample size could also be calculated to “estimate” the effect size15Slide16

Design

16Whole system Ayurvedic Interventions are complexmultiple component intervention and adjustment of the components depending on the individualWestern biomedicine to a large extent, the interventions have been “simple” which have allowed double blind randomized clinical trials. Many situations, even in the western biomedicine where these “ideal” trials are infeasible and in these cases non randomized un-blinded trials, observational studies, case studies and case series have been used. Slide17

Design

17Some examplesEvaluation of public health interventionsTrials in therapeutic areas such as oncology and psychiatryMedical device trialsTrials which involve invasive interventions like surgery.Trials in these areas have biases associated with them and a goal for these trials is as much about understanding the intervention as it is about understanding the limitations and biases associated with the trial itself. Slide18

Design

18In my opinion it is much better to get an approximate answer (biased results) to the exact question (for e.g whole system intervention for aging as a multi-center observational study at Ayurveda Hospitals) compared to an exact answer (unbiased results) to an approximate question (simplified intervention, for e.g. only using a capsule made of the traditional herbs and doing a multi-center double blind randomized study).Evidence from non-randomized designs is more convincing

when

confounders

are well-understood and

measured

there

is historical evidence which has a theoretical basiseffect sizes are large. Ayurvedic interventions lend them into this category where non randomized trials should be okay.Slide19

Design

19Slide20

Design

20Slide21

Design

21Slide22

Design

22Slide23

Analyses

Propensity scores method is useful where many confounders need to be controlled for but the data are limited. The principle is based on the fact that propensity scores capture the information about the relationship between confounders and treatment allocation (not the outcomes as is the case in stratification and regression techniques), so that selection bias is removed when comparisons are made between groups with similar propensity scores. In many Ayurveda Trials, selection bias could be a major component of the overall bias due to non-randomized nature of allocating the interventions23Slide24

Analyses

If confounding variables or characteristics which determine the allocation are captured correctly, then the bias associated with the selection bias could be removed using the propensity score method The method involves calculation for each subject their chance of receiving the experimental intervention from their baseline characteristics or in other words estimates a subject’s propensity of receiving the experimental intervention based on his or her characteristic24Slide25

Analyses

In a randomized trial with two equal sized treatment groups, the propensity will be the 0.5 for each subject and will not depend on his or her characteristic. In non-randomized trials, for example for an Ayurveda intervention where two treatment groups are Ayurveda whole system intervention and normal western biomedicine intervention, it is likely that treatment assignment depend on baseline characteristics. 25Slide26

Analyses

It might be that patients with diagnoses of the disease which is closer to how it is described in traditional Ayurveda texts may be more likely to receive Ayurveda intervention. In this case the average propensity score in the Ayurveda intervention group will differ from the average in the western biomedicine group. In this case selection bias is a problem that needs to be addressed and propensity score method can be used to do that.26Slide27

Discussion

RCT is the preferred method of assessing intervention effects but not at the cost of diluting the intended interventionStatistical methods exist which allow for designing and analyzing pragmatic whole system trialsGuidelines for designing, analyzing and reporting Ayurvedic whole system trials need to be developed.27Slide28

28

Development of Reporting Standards

Guidelines should have 3 basic attributes such that the reporting reflects

Transparency: reporting must be honest and accurate

Science: reporting must be scientifically unbiased

Ethics: reporting should be based on trials which are ethically conductedSlide29

Thank You !

Contact details:ashwini.mathur@novartis.com29