/
SAS Implementation of the Graphical Multiple Testing Procedure for Single and Families SAS Implementation of the Graphical Multiple Testing Procedure for Single and Families

SAS Implementation of the Graphical Multiple Testing Procedure for Single and Families - PowerPoint Presentation

mitsue-stanley
mitsue-stanley . @mitsue-stanley
Follow
342 views
Uploaded On 2020-01-18

SAS Implementation of the Graphical Multiple Testing Procedure for Single and Families - PPT Presentation

SAS Implementation of the Graphical Multiple Testing Procedure for Single and Families of Hypotheses Xianwei Sherry Bu 2019 NICASA Midwest Chapter Meeting Support for this presentation was provided by AbbVie AbbVie participated in the review and approval of the content ID: 773181

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "SAS Implementation of the Graphical Mult..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

SAS Implementation of the Graphical Multiple Testing Procedure for Single and Families of HypothesesXianwei (Sherry) Bu 2019 NIC-ASA Midwest Chapter Meeting

Support for this presentation was provided by AbbVie. AbbVie participated in the review and approval of the content.Xianwei (Sherry) Bu is an employee of AbbVie Inc. Disclosure SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 2

Multiplicity issues are often encountered in clinical trials: multiple doses, multiple endpoints or subgroups, etc. Testing multiple hypotheses may increase the familywise error rate(FWER)Multiple testing procedures For non-hierarchical hypotheses: Bonferroni procedure, Simes procedure, Holm step-down procedure, Hochberg step-up procedure, Hommel procedure, Dunnett procedure. For hierarchical hypotheses: Fixed-sequence procedure, Fallback procedure, Gatekeeping procedures. Graphical approaches: can be used to represent most existing sequential testing procedures. Overview of multiple testing methodology SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 3

Strongly controls FWERα propagation with re-testing strategy Flexible to support almost any hypothesis testing strategy for any set of clinical objectives Intuitive to visualize and communicate to the team Graphical Procedure to Control FWER SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 4

Illustrative ExampleH – High DoseL – Low DoseEndpoints: Endpoint 1Endpoint 2Endpoint 3 Total alpha=0.05 Graphical procedure SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 5

R package available (gMCP)Stepwise rejective algorithm available from literature for rejection and adjusted p-value Need SAS macro to integrate with programming for submission purposeCompute rejection and adjusted p-valuesIntermediate weights/transition matrices Implementation of the Graphical Procedure SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 6

A general definition of an adjusted p-value: The adjusted p-value for a hypothesis is the smallest significance level at which one would reject the hypothesis using the given multiple testing procedure.If the adjusted p-value for a hypothesis is less than or equal to α , the hypothesis is rejected. An advantage of adjusted p -values is that they can be used for quite complex decision rule. Adjusted p-values SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 7

Bonferroni procedure:Holm procedure: Fixed-sequence procedure: Hochberg procedure: Adjusted p-values in some commonly used procedures SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 8

The above formulas can provide locally adjusted p-values for specific procedures. The above formulas can’t provide overall adjusted p-values for the whole graphical procedure. Our goal: Develop a general method to compute overall adjusted p-values for different graphical procedure.Two examples for adjusted p-values Adjusted p-values(continued) SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 9

Example 1 (single hypothesis endpoints)   H1 H2 H3 H4 L1 L2 L3 L4 H1 0 1 0 0 0 0 0 0 H2 0 0 1 0 0 0 0 0 H3 0 0 0 1/2 1/2 0 0 0 H4 0 0 0 0 1 0 0 0L100000100L200000010L31/20000001/2L410000000  H1H2H3H4L1L2L3L4weight1/20001/2000 Initial weights: Initial transition matrix: Input from the graphical procedure: SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 10

SAS Macros for Adjusted P-Values (single endpoints) Define a SAS module gmcp . p, w and g are inputs Input: raw p-values, original weight vector w and transition matrix g Call gmcp SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 11

Step by step rejection process Raw p = { 0.01 0.02 0.003 0.018 0.03 0.026 0.02 0.06 } Initial weights and transition matrices a t 1 st step: P- adj (H1 ) = 0.01/0.5=0.02 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 12

Step by step rejection process Raw p = { 0.01 0.02 0.003 0.018 0.03 0.026 0.02 0.06} Intermediate weights and transition matrices after 1st step: P- adj (H2) = max(P- adj (H1), 0.02/0.5)=0.04 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 13

Algorithm of updating graph After the jth hypothesis is rejected, then SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 14

Step by step rejection process Raw p = { 0.01 0.02 0.003 0.018 0.03 0.026 0.02 0.06 }Intermediate weights and transition matricesAfter 2nd step: P- adj (H3) = max(P- adj (H2), 0.003/0.5)=0.04 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 15

Step by step rejection process Raw p = { 0.01 0.02 0.003 0.018 0.03 0.026 0.02 0.06 } Intermediate weights and transition matrices after 3nd step: P- adj (L1) = max(P- adj (H3), 0.03/0.75)=0.04 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 16

Step by step rejection process Raw p = { 0.01 0.02 0.003 0.018 0.03 0.026 0.02 0.06 }Intermediate weights and transition matrices after 4th step: P- adj (L2) = max(P- adj (L1), 0.026/0.75)=0.04 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 17

Step by step rejection process Raw p = { 0.01 0.02 0.003 0.018 0.03 0.026 0.02 0.06 }Intermediate weights and transition matrices after 5th step: P- adj (L3) = max(P- adj (L2), 0.02/0.75)=0.04 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 18

Step by step rejection process Raw p = { 0.01 0.02 0.003 0.018 0.03 0.026 0.02 0.06 }Intermediate weights and transition matrices after 6th step: P- adj (H4) = max(P- adj (L3), 0.018/0.75)=0.04 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 19

Step by step rejection process Raw p = { 0.01 0.02 0.003 0.018 0.03 0.026 0.02 0.06 }Intermediate weights and transition matrices after 7th step: P- adj (L4) = max(P- adj (H4), 0.06/1)=0.06 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 20

Hypo: H1 H2 H3 H4 L1 L2 L3 L4Raw p = {0.01 0.02 0.003 0.018 0.03 0.026 0.02 0.06} Final output The adjusted p-value 0.04 is from H2, other hypotheses’ rejections are based on the rejection of H2, even if their calculated adjusted p-values are smaller than 0.04, we still use 0.04 as their adjusted p-values. SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 21

Example 2 (families of hypotheses)   H1 H2 H3 H4F H5F weight 1/2 1/2 0 0 0   H1 H2 H3 H4F H5F H1 0 3/4 0 1/4 0 H2 0 0 3/4 0 1/4 H3 1 0 0 0 0 H4F 0 1 0 0 0 H5F 1 0000Multiple endpoints in these two nodesInitial weights:Initial transition matrix:Input from the graphical procedure:SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 201922

Raw P-values(H4F)=(0.0015 0.02)Use Bonferroni procedure within H4F, thenLocally adjusted P-values (H4F)=(0.0015 0.02)*2=(0.003 0.04)Raw P-values(H5F)=(0.0031 0.001)Use Hochberg procedure within H5F, thenP(2) =0.0031P(1) =min(0.0031, 0.001*2)=0.002Locally adjusted P-values (H5F)=(0.0031 0.002) Locally adjusted p-values for H4F and H5F SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 23

SAS Macros for Adjusted P-Values (families of hypotheses) Define a SAS module graph. p, w, ind and g are inputs Input locally adjusted p-values Call graph SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 24

Step by step rejection process (* means locally adjusted p-values)input p = {0.1 0.007 0.05 0.003* 0.04* 0.0031* 0.002*} Initial weights and transition matrices a t 1 st step: P- adj (H2) = 0.007/0.5=0.014 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 25

Intermediate weights and transition matrices after 1st step:Step by step rejection process ( * means locally adjusted p-values)input p = {0.1 0.007 0.05 0.003* 0.04* 0.0031* 0.002*} P- adj (H5F-2) = max(P- adj (H2), 0.002/0.125)=0.016 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 26

Intermediate weights and transition matrices after 2nd step:Step by step rejection process ( * means locally adjusted p-values)input p = {0.1 0.007 0.05 0.003* 0.04* 0.0031* 0.002*} P- adj (H5F-1) = max(P- adj (H5F-2), 0.0031/0.125)=0.0248 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 27

Intermediate weights and transition matrices after 3rd step:Step by step rejection process ( * means locally adjusted p-values)input p = {0.1 0.007 0.05 0.003* 0.04* 0.0031* 0.002*} P- adj (H3) = max(P- adj (H5F-1), 0.05/0.375)=0.133 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 28

Intermediate weights and transition matrices after 4th step:Step by step rejection process ( * means locally adjusted p-values)input p = {0.1 0.007 0.05 0.003* 0.04* 0.0031* 0.002*} P- adj (H1) = max(P- adj (H3), 0.1/1)=0.133 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 29

Intermediate weights and transition matrices after 5th and 6th step: Step by step rejection process (* means locally adjusted p-values)input p = {0.1 0.007 0.05 0.003* 0.04* 0.0031* 0.002*} P- adj (H4F-1) = max(P- adj (H1), 0.003/1)=0.133 P- adj (H4F-2) = max(P- adj (H1), 0.04/1)=0.133 SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 30

Hypo: H1 H2 H3 H4F H4F H5F H5F Input p = {0.1 0.007 0.05 0.003* 0.04* 0.0031* 0.002*} Final output SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 31

These SAS macros are implementation of graphical multiple testing procedure for both single hypothesis endpoints and families of Hypotheses endpoints. These SAS macros are applicable to any user specified graphs via inputting initial weight vector and initial transition matrix, since the updating graph algorithm is fixed. Note: In these SAS macros, inputting p-values for single hypothesis endpoints are raw p-valves, inputting p-values for families of Hypotheses endpoints are locally adjusted p-valves. Concluding remarks SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 32

Backup slides

Algorithm for Computing Adjusted P-Values (single hypothesis endpoints) SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 34

Algorithm for Computing Adjusted P-Values (families of hypotheses) SAS Implementation of the Graphical Multiple Testing Procedure | NIC-ASA | Oct. 25, 2019 35