Clinical Trials: BIO Submits Comments on the Draft Guidance for Industry on Adaptive Design Clinical Trials for Drugs and Biologics

Re: FDA–2010–D–0090: Adaptive Design Clinical Trials for Drugs and Biologics

Dear Sir/Madam:

The Biotechnology Industry Organization (BIO) thanks the Food and Drug Administration (FDA) for the opportunity to submit comments on the draft guidance on Adaptive Design Clinical Trials for Drugs and Biologics.

BIO represents more than 1,200 biotechnology companies, academic institutions, state biotechnology centers and related organizations across the United States and in more than 30 other nations. BIO members are involved in the research and development of innovative healthcare, agricultural, industrial and environmental biotechnology products, thereby expanding the boundaries of science to benefit humanity by providing better healthcare, enhanced agriculture, and a cleaner and safer environment.

GENERAL COMMENTS

BIO recommends that the guidance be structured in the way that adaptation methods, and statistical and logistical considerations for using adaptive designs in exploratory studies, are described separately from those for adequate and well-controlled (A&WC) studies. As currently drafted, the guidance mostly focuses on A&WC studies. In our view, early phase programs can much benefit from using adaptive design (AD) to gain efficiency and effectiveness. We are pleased to read that FDA does encourage “sponsors to gain experience with these adaptive design methods in this setting” (Section IV.D.) by applying methods described in the “less well-understood” section. However, the section of Statistical Considerations for Less Well-understood Adaptive Design Methods” provides guidance for “more complex approaches in section IV and that is intended to be an A&WC trial”. We propose that the guidance be revised so that there is a stand alone section that provides guidance on methods, trial logistics, documentations, interactions with the FDA, etc., for exploratory studies.

For a complete definition of “bias”, BIO proposes that a Type II error be included in this definition. For instance, a common consequence of operational bias in symptom trials (e.g., major depressive disorder studies) is the higher than real placebo response. In that occasion, there is no statistical method that can correct the bias (as pointed in IV.A.3), and it might result in a false negative conclusion that test drug has no effect. We feel that both Type I and Type II errors need to be included when defining “bias” in clinical trials.

BIO also believes that the guidance could better influence the use of adaptive designs in drugs and biologics development by providing case studies to illustrate the points the guidance wishes to address.

Lastly, BIO notes that citations are not directly linked to the texts in the guidance document. It would be very helpful to make explicit references to the literature provided at the end of this guidance. Please see the table below for detailed comments.

*Priority: 1= High, 2= Medium, 3= Low
SPECIFIC COMMENTS GUIDELINE SECTION TITLE: III. DESCRIPTION OF AND MOTIVATION FOR ADAPTIVE DESIGNS Line Number Comment and Rationale Proposed change (if applicable)
Priority (1, 2, 3)
73-74
We believe that blinded data review also need up-front planning, but this sentence appears to imply that up-front planning is only needed for unblinded looks at the interim data. Please modify the wording to remove that implication.
1
94-97
These appear to give more latitude to sample size re-estimation based on blinded data review, and allow sample size to be increased without a prospectively defined plan. We disagree with this “ad-hoc” approach. If such cases exist in reality, they should be considered outside the AD category and thus should not need to be described as one of the AD cases.
1
152
We would appreciate it if a specific example of "particular data involved" is provided for clarity.
3
256-259
This sentence seems to describe the adaptive dropping-arm design or response-adaptive randomization design. However, in those designs, data collections for those subjects who have already been randomly assigned to the sub-optimal dose groups will continue as these subjects continue to participate in planned study visits, unless the entire study is terminated. Instead, enrollment of subjects into the sub-optimal dose group(s) is curtailed.
1
271-274
This sentence needs editorial modification. The meaning is unclear.
3
GUIDELINE SECTION TITLE: IV. GENERAL CONCERNS ASSOCIATED WITH USING ADAPTIVE DESIGN IN DRUG DEVELOPMENT Line Number Comment and Rationale Proposed change (if applicable)
Priority (1, 2, 3)
345-348
This sentence is confusing. We are not clear what the key differences are between two kinds of “bias”, one from a conventionally designed clinical trial and the other from an AD study, with respect to decision making. Both “biases” would have negative impact on making appropriate statistical inferences.
1
GUIDELINE SECTION TITLE: V. GENERALLY WELL-UNDERSTOOD ADAPTIVE DESIGNS WITH VALID APPROACHES TO IMPLEMENTATION Line Number Comment and Rationale Proposed change (if applicable)
Priority (1, 2, 3)
611-638:
Using baseline data to modify study eligibility criteria does not belong to the scope of adaptive design. This section is more relevant to Section 4.E..
2
636-638
These lines state that changes to the eligibility criteria can be made without risking the integrity of the study. This could be interpreted as allowing restriction of the analysis set to subjects meeting the amended criteria. It is unlikely that this interpretation is intended, as this enrichment of the first stage data leads to bias.
Please clarify that the final analysis must still be based on the full analysis set, or move discussion of this type of adaptation to the "less well understood" section of the guidance (Section VI).
640-708
This section seems to suggest that it is acceptable to alter study sample size/duration based on incorrect initial assumptions on variance or event rate in a non-prospectively defined manner (i.e., not stated as an approach in the initial protocol) provided the blind is maintained
We suggest that all AD studies that are intended to be covered by this guidance must be “adapted by design”. Please alter the wording to clarify this.
1
676-684
It is unclear from this section whether or not changing the timepoint for the primary analysis is an acceptable practice; e.g., changing from a 12-week assessment to a 24-week assessment.
Please clarify whether this section applies to simply extending the duration of a study or if it is intended to allow changing the timepoint at which the primary variable is assessed.
648-650
Theoretically, an incorrect assumption could also lead to an overpowering of a study. We are not clear as to why it is implied that underpowering is the only concern.
1
752-760
This section seems to suggest that a steering committee can modify a study based on endpoints unrelated to efficacy (such as safety). Yet, this suggestion would contradict lines 392-405 that says the role of the DMC (Data Monitoring Committee, which reviews unblinded data) is safety monitoring.
2
971-973
This text seems to suggest that control of the Type 1 error rate will address the bias issue potentially introduced by using biomarker interim data to monitor a clinical trial. We have a different view on this subject.
We suggest adding statements such as: “The appropriate statistical analyses should be conducted to fully assess the relationships between biomarker change and ultimate clinical outcome measure improvement before creating an interim efficacy evaluation algorithm where only biomarker data are used”.
1
GUIDELINE SECTION TITLE: VI. ADAPTIVE STUDY DESIGNS WHOSE PROPERTIES ARE LESS WELL UNDERSTOOD Line Number Comment and Rationale Proposed change (if applicable)
Priority (1, 2, 3)
1047-1048
The normative statement that sample size re-estimation methods "should be used only for increases in the sample size" should be clarified. Specifically, it is not clear whether this sentence
1
suggests that the methods in question are more problematic when they seek to decrease the sample size than when they increase the sample size, or it suggests that because there exists a well understood methodology for reducing the sample size (i.e., group sequential designs), the sample size re-estimation procedures are less attractive in that situation?
1053-1068.
This paragraph would be improved by the addition of a reference to the Chen, DeMets, and Lan (Stat Med 2004) article (which is already included in the guidance references). Specifically, the paragraph should note that in some circumstances, the original (unweighted) statistics can be used, thus avoiding the problems associated with determining appropriate point and interval estimates. Adding the article by Gao, Ware, and Mehta (JBS 2008 18: 1184–1196), which extends the ideas of Chen, et al., to the references list of this guidance would also be useful.
2
1186-1187
One could argue that both the clinical endpoint and the non-inferiority margin are the primary features of a non-inferiority trial that are not suitable for adaptation
Please revise this text to read, “Chief among these features is the non-inferiority margin and the clinical endpoint.”
2
1117-1119
This passage raises the possibility of changing the order of testing of endpoints based on interim results. We know of no statistical methodology to avoid the bias associated with such an adaptation.
Please clarify whether changing the order of testing of endpoints is an acceptable practice.

GUIDELINE SECTION TITLE: VII. STATISTEICAL CONSIDERATIONS FOR LESS WELL-UNDERSTOOD ADAPTIVE DESIGN METHODS Line Number Comment and Rationale Proposed change (if applicable)
Priority (1, 2, 3)
1240-1246
Please provide more information on why sample size increases may increase the bias should.
Please elaborate in the Guidance on the basis for the statement that sample size increases may increase bias, for example, through a literature reference.
2
1341-1342
It would add clarity if the guidance would add that the controversial nature of this use of simulations is driven by the difficulty in fully characterizing the null space for simulations.
3
1346-1350
The “requirement” that adaptive A&WC trials have an SAP by the time that the study is finalized represents a change from the accepted norm that the SAP just needs to be completed and signed-off prior to any analysis – either interim or final. We think that to ensure all statistical methods and adaptation algorithm are finalized and described in the protocol is a more practical and reasonable suggestion than to finalize SAP before the study starts. Perhaps the FDA uses “SAP” in the different concept than we do. SAP in our practice describes the details for how to implement methods (at the calculation level) presented in the protocol, and focus on the final data analyses. Interim data evaluation and adaptation should be presented in the study protocol or its appendix, or in an independent document entitled “Simulation Report”.
Please change “SAP” to “Protocol or SAP” in this context.
1
GUIDELINE SECTION TITLE: IX. CONTENT OF AN ADAPTIVE DESIGN PROTOCOL Line Number Comment and Rationale Proposed change (if applicable)
Priority (1, 2, 3)
1485
Please change “protocol, SAP, and supportive information” to “protocol, SAP, or other supportive documentation”
3
1484-1491
The division of content between protocol and SAP is unclear. Regardless, there is a clear implication that the protocol and SAP should be written, and most likely finalized, in a parallel and not sequential setting. The information provided in this entire section addresses the documentation that would be necessary for an adaptive A&WC trial, we would like to see something similar (albeit it could possibly be more brief) regarding what documentation is expected for adaptive exploratory trials.
Please modify the Guidance to include information about how this section applies to exploratory studies. Also, the guidance should clearly describe the expected content of the protocol, relative to the SAP.
1
1493-1497
We are not clear why it is necessary to justify the use of an adaptive design. This reads as if the default must absolutely be a traditional design and that we would need to have /provide substantial justification before employing an adaptive approach.
Please change the language from justifying the use of an AD to justifying the choice of the AD design (vs. other type of AD, conventional parallel-group design, etc)
1
1506
A summary of “each adaptation and its impact upon critical issues” is a non-trivial requirement. Please provide more information on how to identify the adaptations to be characterized (it will likely not be possible to completely characterize each and every possible adaptation that might happen).
2
1524
We would like to have clarity around the requirement that “computer programs used in the simulations should be included in the documentation”. Does this require companies to use the software that FDA has capacity to run?
1
BIO Comments on FDA-2010-D-0090: Draft Guidance for Industry on Adaptive Design Clinical Trials for Drugs
and Biologics.
Page 9 of 9
GUIDELINE SECTION TITLE: XI. DOCUMENTATION AND PRACTICES TO PROTECT STUDY BLINDING AND INFORMATION SHARING FOR ADAPTIVE DESIGNS Line Number Comment and Rationale Proposed change (if applicable)
Priority (1, 2, 3)
1684-1699, and 1727
We are perplexed as to why the FDA would want or suggest that there be a SOP. Most of the information “suggested” for the SOP would fall into a DMC charter. It is unclear whether the guidance is requesting that companies have a general SOP for “all adaptive trials”
Elaborate on the requested SOP and explain how it differs from a DMC charter.
1

Conclusion

Thank you for this opportunity to comment on the draft guidance on Adaptive Design Clinical Trials for Drugs and Biologics. We would be pleased to provide further input or clarification of our comments, as needed.

Sincerely,

/s/
Katie McCarthy
Director, Science & Regulatory Affairs
The Biotechnology Industry Organization (BIO)