Development Equivalance initiative 31 Mar 2016 Company Confidential 2016 1 Yuanyuan Duan Mark D Johnson Yanbing Zheng Lanju Zhang NonClinical Statistics AbbVie Inc May 2016 Disclosure ID: 679129
Download Presentation The PPT/PDF document "Statistical Methods for Comparability A..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Statistical Methods for Comparability Assessment in Drug Development
Equivalance initiative| 31 Mar 2016| Company Confidential © 2016
1
Yuanyuan Duan,
Mark D Johnson, Yanbing Zheng, Lanju Zhang Non-Clinical Statistics, AbbVie Inc
May 2016Slide2
Disclosure
The presentation was sponsored by AbbVie. AbbVie contributed to the design, research, and interpretation of data, writing, reviewing, and approving the presentation
All authors are employees of AbbVie, Inc.
05 May 2016Process Comparison| May 2016 | MBSW Meeting
2Slide3
Agenda
IntroductionCurrent Regulatory Perspective
Comparability assessment
Real-World ExamplesUnivariate: Scale Down ModelMultivariate: Dissolution Profile Comparison
Linear Model: HDX MSNonlinear Model: Parallelism testing of logistic model05 May 2016
Process Comparison| May 2016 | MBSW Meeting3Slide4
Process Comparison| May 2016 | MBSW Meeting
4
Introduction
Change is the law of life.
John F KennedySlide5
Process Comparison| May 2016 | MBSW Meeting
5
Introduction
Change is throughout
the lifecycle
of
drug research
, development and manufacturing.Slide6
Process Comparison| May 2016 | MBSW Meeting
6
Introduction
Examples
Analytical method
Method
change
Critical
reagent change
Critical method parameter change
Manufacturing process
Scale up &scale down
Critical parameter change, temperature,
PH,
pressure
Process component change,
eg
, cell line, equipment, buffer
Transfer
Method transfer to another lab
Process to another siteSlide7
Process Comparison| May 2016 | MBSW Meeting
7
Current Regulatory Perspective
Regulations and
Guidance
for Manufacturing Changes
FDCA
Section 506A and FDAMA Section 116
21
CFR 601.12 Changes to an approved biologics application
ICH
Q5E : Comparability of biotechnological/biological Products subject to changes in their manufacturing
process
ICH
Q8: Pharmaceutical development
ICH
Q9: Quality risk management
ICH
Q10 Pharmaceutical quality
systemSlide8
Process Comparison| May 2016 | MBSW Meeting
8
Comparability
assessment- Purpose
Demonstrate
the comparability and quality consistency of pre- and post- change products
Demonstrate the change does not have an adverse effect on the quality, safety and efficacy of the drug products
Different from
biosimilarity
/bioequivalence
Comparability is to demonstrate the consistency of the same product before and after some change to its manufacturing process
Biosimilarity
/bioequivalence is to demonstrate the consistency between a
biosimilar
/bioequivalent drug product and its reference originator drug productSlide9
Comparability assessment- Equivalence Test
For evaluating process or method changes, the Equivalence approach is recommendedThe hypotheses for this method are:
H0: μ
1-μ2> θ or μ
1-μ2< -θ (θ : Equivalence Margin) H1: Not H
0Recommended approaches for this analysis are:Confidence Interval (CI) approachTwo one-sided test (TOST)Challenge to this approach is accurately identifying θ.05 May 2016Process Comparison| May 2016 | MBSW Meeting9Slide10
Comparability assessment- Equivalence Margin
The equivalence margin may be established using the following two approaches:
Fixed margin:
Defined based on experts’ knowledge about the process or method being evaluated.
Different margin criteria are required for different assays (i.e., 1% might be suitable for HPLC; 2.5% might be suitable for Bioassay)Challenge is to identify a specific number for equivalence marginNon-fixed margin:
Function of assay variability , e.g. θ = cσConsistent rule across many assay methodsBased on statistical power of rejecting equivalence between the methods/processes using a limited observation total05 May 2016Process Comparison| May 2016 | MBSW Meeting10Slide11
Real-world Examples
Univariate: Scale Down Model
Multivariate: Dissolution Profile ComparisonLinear Model: HDX MS
Nonlinear Model: parallelism testing of logistic model
05 May 2016Process Comparison| May 2016 | MBSW Meeting11Slide12
Background: Scale Down Model
Process A: 3L, scale down cell culture processProcess B: 3000L, manufacturing processThe scale-down model mimics the
manufacturing process from thaw to harvest following the same split ratiosPerformance of the 3 L scale-down model
is compared to 3000 L manufacturing scale performance Purpose: show comparability, indicating that the scale-down model, is sufficiently predictive of the 3000 L production scale process performance and acceptable for use in supporting upstream process characterization studies.
05 May 2016Process Comparison| May 2016 | MBSW Meeting
12Slide13
Scale Down Model Work Flow
Define approach used to qualify
model.Identify representative scale-down (i.e., 3 L) and manufacturing scale (i.e., 3000 L) data sets
Determine performance and operating parameters for assessment and provide rationale for
selection.Determine critical quality attributes (CQA) for assessment and provide rationale for selection.Perform qualitative and statistical analyses
05 May 2016Process Comparison| May 2016 | MBSW Meeting13Slide14
Scale Down Model Study Example: Results
Process Comparison| May 2016 | MBSW Meeting
14
Runs
1
2
3
4
5
6
7
8
9
10
Product
Concentration (g/L)
3000L
12.10
11.60
11.68
11.69
12.30
11.83
11.53
11.99
11.76
11.88
3L
12.19
12.03
11.97
12.31
12.66
11.61
12.01
11.60
12.23
11.57
Null hypothesis
|
μ
1
-
μ
2
|
≥
θ
Alternative hypothesis
|
μ
1
-
μ
2
|<
θ
θ
is goal post, indifference zone, equivalence margin
-0.72
0.72
Equivalence test
-0.42
0.05Slide15
Limitation of sample size, especially for manufacturing scale (i.e., 3000 L) data sets With
small sample sizes of 10 or fewer observations it’s unlikely the normality test will detect non-normality. Non-parametric confidence interval is sorted-data based, it can only achieve claimed confidence level with large sample size
------What is the power performance of using normal confidence interval for non-normal data?
Equal-variance test is also not suitable for small sample size------Skip the equal variance test and assume non-equal
varianceGruman, Jamie A.; Cribbie, Robert A.; and Arpin-Cribbie, Chantal A. (2007) "The Effects of Heteroscedasticity
on Tests of Equivalence," Journal of Modern Applied Statistical Methods: Vol. 6: Iss. 1, Article 13.Problem Faced in Scale Down/Up Comparison
15
Process Comparison| May 2016 | MBSW MeetingSlide16
Real-world Examples
Univariate: Scale Down ModelMultivariate: Dissolution Profile Comparison
Linear Model: HDX MSNonlinear Model: parallelism testing of logistic model
05 May 2016
Process Comparison| May 2016 | MBSW Meeting16Slide17
Background: Dissolution Profile Comparison
Dissolution profiles measure “%
Dissolved vs. Time”Goal: Test whether the
profiles for the test lot was biologically similar to the reference lots.
A change in production, formulation, lot-to-lot or brand-to-brand variation, etc. often requires proof of similarity between the dissolution profiles of the new and old production process.
17
Process Comparison| May 2016 | MBSW MeetingSlide18
Dissolution data are analyzed by similarity factor and multivariate statistical method between ref and test product Test Whether the variance meet the f2, f1 requirement
Model-dependent approachUses time as a covariate.
Model-independent approachFit a multivariate normal distribution across the time points.
Some approaches frame this problem as a hypothesis test for the mean of a multivariate normal distribution. Concludes similarity when failing to reject the null hypothesis.Rewards highly variable data. More reasonable approaches frame the problem as an equivalence test
.Rewards good data.Establishes a meaningful difference between profiles.Work Flow
18Process Comparison| May 2016 | MBSW MeetingSlide19
19
Comparing
Profiles--Methods
SK method: Translates a multivariate normal distribution to a univariate measure of the mean difference of the profiles as proposed in
Saranadasa and Krishnamoorthy (2005).Uses an equivalence hypothesis to determine “similarity.” Assumes constant difference among time points, estimates
a constant difference between profiles using a weighted average of the mean differences at each time point. δ is a measure of the mean difference for all time points.H0: - d0 > d or d > d0 vs. HA: - d0 < d < d0.
d
0
= 10% is recommended by the authors.
Mean difference of 10 at all time
points.
Intersection-Union
Test (IUT
):
H
0
:
|
δ
k
|
>
δ
0k
for some
k
vs. H
A
:
|
δ
k
|
<
δ
0k
for all
k
, where
δ
k
is an element of
δ
(
p
dimensional
mean profile difference
).
Can use non-constant threshold,
δ
0k
.
Uses a univariate
t
distribution
by each time point.
IUT
is very conservative and has very low powerSaranadasa, H., Krishnamoorthy, K. (2005). A Multivariate Test for Similarity of Two Dissolution Profiles. Journal of Biopharmaceutical Statistics. 15(2), 265-278.Berger, R. L. and Hsu, J. C. (1996). Bioequivalence Trials, Intersection-Union Tests and Equivalence Confidence Sets. Statistical Science, 11(4), 283-319. 19Process Comparison| May 2016 | MBSW MeetingSlide20
Example & Result
Method
Conclusion (based on the
first four time points)f
272.0SKPvalue < 0.0001
and claim equivalenceIUTReject H0 and claim equivalence (δ0k=10)
20
Process Comparison| May 2016 | MBSW Meeting
Each point represents the mean of 12 tabletsSlide21
Example & Result
Method
Conclusion (based on the
first four time points)f
262.0SKPvalue < 0.0001
and claim equivalenceIUTDo not reject H0 and can not claim equivalence (δ0k=10)
21
Process Comparison| May 2016 | MBSW Meeting
Each point represents the mean of 12 tabletsSlide22
Real-world Examples
Univariate: Scale Down ModelMultivariate: Dissolution Profile Comparison
Linear Model: HDX MSNonlinear Model: parallelism testing of logistic model
05 May 2016
Process Comparison| May 2016 | MBSW Meeting22Slide23
The function, efficacy, and safety of protein biopharmaceuticals are tied to their three-dimensional structure.
Production of protein drugs are sensitive to the use of cells, expression systems, and/or growth conditions (in vivo chemical modifications)In addition, PTM may occur through changes of purification strategies as well as filling, vialing and storage steps
(in vitro chemical modifications)NMR or X-ray are either too complex or time consuming, or not applicable for routine biopharmaceutical analysis.
Hydrogen/deuterium exchange (HDX) mass spectrometry (MS) has become a key technique to understand protein structure and dynamics.
Background: HDX MSHoude, Damian, Steven A. Berkowitz, and John R. Engen. "The utility of hydrogen/deuterium exchange mass spectrometry in biopharmaceutical comparability studies."
Journal of pharmaceutical sciences 100.6 (2011): 2071-2086.23Process Comparison| May 2016 | MBSW MeetingSlide24
HDX Work Flow
HDX-MS:
Requires
only
small sample amount compared
to other techniquesProteins can be studied under physiological conditionsImportant to understand the relationship
of
structure
and
its
functions
Houde
, Damian, Steven A. Berkowitz, and John R. Engen. "The utility of hydrogen/deuterium exchange mass spectrometry in biopharmaceutical comparability studies."
Journal of pharmaceutical sciences
100.6 (2011): 2071-2086.
24
Process Comparison| May 2016 | MBSW MeetingSlide25
Example Data
Houde, Damian, Steven A. Berkowitz, and John R. Engen. "The utility of hydrogen/deuterium exchange mass spectrometry in biopharmaceutical comparability studies."
Journal of pharmaceutical sciences 100.6 (2011): 2071-2086.
25
Process Comparison| May 2016 | MBSW MeetingSlide26
HDX Study Example: Results
Process Comparison| May 2016 | MBSW Meeting
26
HDX Parameters
1
2
3
4
5
6
7
8
9
10
slope
Process
A
1.77192
1.779029
1.763656
1.608017
1.615043
1.725259
1.362013
1.491681
1.717594
1.763892
Process
B
1.642158
1.478013
1.524606
1.569309
1.3576829
1.523655
1.50949
1.377245
1.495579
1.471715
Null hypothesis
|
μ
1
-
μ
2
|
≥
θ
Alternative hypothesis
|
μ
1
-
μ
2
|<
θ
θ
is goal post, indifference zone, equivalence margin
-0.12
0.12
Equivalence test
0.07
0.25
Because of large variance in protein drug measurement, usually we collect data from more than 10 lots for each process
After model fitting, the estimated slope and intercept are used for equivalence test. Slide27
One protein has multiple peptides, to prove the comparability we need to confirm all the peptides being comparable, multiplicity need to be considered
.ANCOVA can also be used for process comparison (as Q1E), lot can be treated as nested in process. The choice of p-value cut-off need further discussion. We only compare the slope and intercept, so the linear model assumption need to be checked.
If we only have 1 curve in each group, we can also use Parallelism Test.How to incorporate variability associated with slope and intercept estimation?
Problem Faced in Linear Example
27Process Comparison| May 2016 | MBSW MeetingSlide28
Real-world Examples
Univariate: Scale Down ModelMultivariate: Dissolution Profile Comparison
Linear Model: HDX MSNonlinear Model: parallelism testing of logistic model
05 May 2016
Process Comparison| May 2016 | MBSW Meeting28Slide29
Background: Parallelism Testing
Parallelism between sets of dose-response data is a prerequisite for determination of
relative biological activity or binding capacity
For example, relative
potency assay methods for CMC quality control of new biological entities require statistical evaluation to demonstrate similarity between reference standard and
sample.F-statistics were applied to demonstrate curve similarity. Due to the mathematical calculation of F-statistics false negative statistical results are generated for very precise assays.The revised USP Chapters 〈1032〉 and 〈1034〉 suggest testing parallelism using an equivalence method.
29
Process Comparison| May 2016 | MBSW MeetingSlide30
Parallelism Test Work Flow
Measure of similarity
ratio (sample vs. reference) of upper asymptotic, slope, and range (lower – upper asymptotic)Construct confidence intervals on ratio
Feiller’s method Compute extreme values
Goalpost based on Tolerance interval of historical data 05 May 2016Process Comparison| May 2016 | MBSW Meeting
30Slide31
Why Ratio? ---To Avoid
Different Scale
31
Process Comparison| May 2016 | MBSW MeetingSlide32
Construct Confidence Interval on Ratio—Feiller Theorem
05/05/2016
Process Comparison| May 2016 | MBSW Meeting
32
Mean: (
,
); variance:
,
;
Covariance :
=0 ;
=
+
For any parameter
(d, b, d-a) :
Slide33
Setting Goalpost
05/05/2016
Process Comparison| May 2016 | MBSW Meeting
33
based
on
historical data
Upper asymptote
0.89
-
1.13
Range
0.86
-
1.16
Slope
0.70 - 1.43
Upper asymptote
0.89
-
1.13
Range
0.86
-
1.16
Slope
0.70 - 1.43
Result : Harmonized generic goalpostsSlide34
Parallelism Test Study Example: Results
Process Comparison| May 2016 | MBSW Meeting
34
Parameters
a
b
c
d
Standard
Estimate
-1.571
SE
0.047
Null hypothesis
|
μ
1
/
μ
2
|
≥
θ
Alternative hypothesis
|
μ
1
/
μ
2
|<
θ
θ
is goal post, indifference zone, equivalence margin
-0.70
1.43
Equivalence test
0.97
1.16
Parameters
a
b
c
d
Sample
Estimate
-1.668
SE
0.051
Slide35
Reviewed Current Regulatory Perspective
Equivalence TestReal-world Examples shown how equivalence test is used in univariate, multivariate, linear and non-linear model.
Summary
35
Process Comparison| May 2016 | MBSW MeetingSlide36
Thank you for your attention!Slide37
The power for an equivalence test is the probability that we will correctly conclude that the means are equivalent, when in fact they actually are equivalent. When truth is equivalence
: p-value<0.05 (reject the null hypothesis and conclude equivalence)
Simulate from different distributions with different means and same variance
( normal, uniform, lognormal, exponential)Normal(mean, sigma); Uniform(Mean-sigma, Mean+sigma
), Lognormal( log(mean),sigma); exponential(1/mean)Simulate 10000 tests of x1, x2, with true mean difference=delta, goalpost=3*sd(x1). For simplicity, we assume equal variance
Backup Slides: Definition of powerSlide38
power
normal
uniform
lognormalexponentialDelta=0,N=4
68%73%59%65%Delta=0,N=10
99%99%74%94%Delta=sd,N=450%28%37%19%Delta=sd,N=1090%70%46%27%Backup Slides: Performance of small sample size/non-normal data