Multiple Testing Problems in Pharmaceutical Statistics

From MultXpert

Revision as of 17:36, 4 November 2010 by Wikiadmin (Talk | contribs)
(diff) ← Older revision | Current revision (diff) | Newer revision → (diff)
Jump to: navigation, search
Book cover
This book was edited by Dr. Alex Dmitrienko (Eli Lilly and Company, U.S.A.), Prof. Ajit C. Tamhane (Northwestern University, U.S.A.) and Dr. Frank Bretz (Novartis, Switzerland; Hannover Medical School, Germany) and is published in the Chapman and Hall/CRC Biostatistics Series. The book is available on Amazon's web site.


The main goal of this book is to review and summarize the fast growing area of multiple comparison research with emphasis on pharmaceutical applications. The book consists of seven chapters that describe important multiplicity problems encountered in pre-clinical and clinical trial settings. Each chapter provides a detailed overview of methodological issues in multiple testing with emphasis on recently developed approaches not covered in other books. Case studies based on pre-clinical experiments and clinical trials are used to help the reader quickly learn popular multiple testing methods and apply them to real-life problems. The book offers advice from subject matter experts and reviews relevant regulatory guidelines. It also provides useful information for practitioners by emphasizing technical details and implementation of the statistical methods using popular statistical software, including SAS and R.

This book is aimed mainly at biostatisticians involved in pre-clinical and clinical trial research. However, the individual chapters include high-level introductory material to make them accessible to a broad audience of pharmaceutical researchers, including drug discovery scientists, medical scientists and regulatory scientists.

The book includes seven chapters:

  • Chapter 1, "Multiplicity Problems in Clinical Trials: A Regulatory Perspective".
  • Chapter 2, "Multiple Testing Methodology".
  • Chapter 3, "Multiple Testing in Dose-Response Problems".
  • Chapter 4, "Analysis of Multiple Endpoints in Clinical Trials".
  • Chapter 5, "Gatekeeping Procedures in Clinical Trials".
  • Chapter 6, "Adaptive Designs and Confirmatory Hypothesis Testing".
  • Chapter 7, "Design and Analysis of Microarray Experiments for Pharmacogenomics".


Biometrics: The first part of the book is a cohesive exploration of the uses of methods of multiple hypothesis testing. These chapters are understandable for a reader who wants to study multiple testing methods beyond the post-hoc tests presented in a course on design of experiments... The final chapter provides an accessible description of problems in testing data from microarrays.

Journal of Biopharmaceutical Statistics: If you are a statistician in the pharmaceutical industry looking for a comprehensive description of multiple testing in a clinical trial, this book is for you. Each of the main sources of multiplicity in clinical trials, such as several doses, endpoints, interim analyses etc., are discussed in detail. On the way, you will encounter dose-finding, adaptive designs and even microarray experiments. The book can undoubtedly be of value to wider groups-in fact we would rank it as the best book on multiplicity given its up-to-date material... the book certainly attains its objective of being a modern summary of the approaches to multiplicity issues primarily in clinical trials, serving as an excellent guide for those who find themselves face to face with simultaneous testing and would like to have an overview over possible ways of tackling the problem.

Chapter summaries

Chapter 1, "Multiplicity Problems in Clinical Trials: A Regulatory Perspective"

Mohammad Huque (U.S. Food and Drug Administration), Joachim Röhmel (Bremen Institute for Prevention Research and Social Medicine)

This chapter gives a broad introduction to different types of multiplicity problems that commonly arise in confirmatory controlled clinical trials. It focuses on multiplicity induced by multiple endpoints as well as other multiple comparison problems, including problems encountered in trials with multiple dose-control comparisons, trials with multiple subgroups, trials with an active control, etc. The chapter also discusses multiplicity considerations for safety endpoints and multiplicity concerns for several special situations.

Chapter 2, "Multiple Testing Methodology"

Alex Dmitrienko (Eli Lilly and Company), Frank Bretz (Novartis), Peter H. Westfall (Texas Tech University), James Troendle (National Institutes of Health), Brian L. Wiens (Alcon Laboratories), Ajit C. Tamhane (Northwestern University), Jason C. Hsu (Ohio State University)

This chapter gives an overview of concepts and principles that play a central role in multiple testing. This includes definitions of error rates and popular testing principles (closure and partitioning principles). The chapter also introduces multiple testing procedures widely used in pre-clinical and clinical studies, including procedures based on univariate p-values, parametric procedures and resampling-based procedures. These topics provide a foundation for the pharmaceutical applications considered in the subsequent chapters.

Chapter 3, "Multiple Testing in Dose-Response Problems"

Frank Bretz (Novartis), Ajit C. Tamhane (Northwestern University), José Pinheiro (Novartis)

This chapter provides an overview of statistical methods for analyzing clinical dose response studies comparing several dose levels with a control. The emphasis is on efficacy evaluation but most methods can be directly applied to safety or combined efficacy/safety evaluation. Three distinct classes of methods are discussed. This chapter begins with a review of trend tests to detect an overall dose response effect. Next, problems of finding the minimum effective dose and the maximum safe dose using multiple hypotheses testing methods are considered. Finally, the chapter discusses fitting models to dose response curves, and combining the modeling information with hypotheses testing approaches to obtain more powerful hybrid multiple comparison procedures. Illustrative numerical examples are given and available software is mentioned.

Chapter 4, "Analysis of Multiple Endpoints in Clinical Trials"

Ajit C. Tamhane (Northwestern University), Alex Dmitrienko (Eli Lilly and Company)

This chapter provides an overview of statistical methods for analyzing multiple endpoints in clinical trials for comparing a treatment with a control (placebo). Four classes of methods are discussed in the chapter:

  • Union-intersection procedures and other multiple testing procedures for demonstrating the treatment’s superiority on at least one endpoint.
  • Global procedures for demonstrating the combined effect of the treatment on all endpoints.
  • Intersection-union procedures for demonstrating the treatment's superiority on all endpoints.
  • Hybrid superiority-noninferiority procedures for establishing the treatment's superiority on at least one endpoint and noninferiority on all other endpoints.

Procedures based on p-values, normal theory and resampling are discussed. Illustrative examples are given from recent clinical trials.

Chapter 5, "Gatekeeping Procedures in Clinical Trials"

Alex Dmitrienko (Eli Lilly and Company), Ajit C. Tamhane (Northwestern University)

This chapter describes a class of procedures, called gatekeeping procedures, for testing hierarchically ordered hypotheses. Such hypotheses commonly arise in clinical trials when dealing with multiple endpoints, dose-control comparisons and subgroup analyses. The chapter reviews three main classes of gatekeeping procedures (serial, parallel and tree-structured gatekeeping procedures) that control the overall Type I error rate and efficiently account for the hierarchical structure of multiple objectives.

Chapter 6, "Adaptive Designs and Confirmatory Hypothesis Testing"

Willi Maurer (Novartis), Michael Branson (Novartis), Martin Posch (Medical University of Vienna)

This chapter provides an overview of statistical methods for the design and analysis of adaptive designs and related confirmatory hypotheses testing problems. The chapter starts with a discussion of several causes of multiplicity and bias in adaptive designs, followed by a brief review of repeated hypothesis testing at interim analyses leading to group-sequential designs as well as common blinded and unblinded sample size adjustment methods. The major focus of the chapter is on multiple hypothesis selection and testing in adaptive designs based on the closure principle applied to combination tests or conditional error rate functions. Applications of these methods include adaptive treatment or subgroup selection at an interim analysis, which are illustrated by two real case studies.

Chapter 7, "Design and Analysis of Microarray Experiments for Pharmacogenomics"

Jason C. Hsu (The Ohio State University), Youlan Rao (The Ohio State University), Yoonkyung Lee (The Ohio State University), Jane Chang (Bowling Green State University), Kristin Bergsteinsdottir (Univeristy of Iceland), Magnus Karl Magnússon (Landspitali-University Hospital), Tao Wang (Pfizer), Eirikur Steingrímsson (Univeristy of Iceland)

Pharmacogenomics is the co-development of a drug that targets a subgroup of patients and a device that predicts whether a patient is in the subgroup of responders to the drug. Such a development involves a training study, followed by a validation study if warranted. This chapter discusses the design of pharmacogenomic studies based on established statistical principles and describes the analysis of data collected in these studies in a way that takes the multitude of multiplicity issues into account. Both aspects are critical to the success of pharmacogenomic development. A proof of concept experiment is used to show how proper design and analysis can smooth the path from discovery to clinical use.

Code and data sets

The SAS code and data sets used in the book can be downloaded in a single ZIP archive. After dowloading the archive, please extract the programs to the Book folder on the C drive, e.g., 'c:\book\'.

The following programs and data sets are included in the archive.

Chapter 2, "Multiple Testing Methodology"

  • Program 2.1 computes adjusted p-values for the Bonferroni, Holm, fixed-sequence, fallback, Hommel and Hochberg procedures in the dose-finding trial example (Section 2.6.10).
  • Program 2.2 calculates lower limits of one-sided simultaneous confidence intervals for the Bonferroni, Holm, fixed-sequence and fallback procedures in the dose-finding trial example (Section 2.6.11).
  • Program 2.3 computes adjusted p-values for the single-step and step-down Dunnett procedures in the dose-finding trial example (Section 2.7.4).
  • Program 2.4 derives lower limits of one-sided simultaneous confidence intervals for the single-step and step-down Dunnett procedures in the dose-finding trial example (Section 2.7.4).
  • Program 2.5 implements the resampling-based procedures discussed in Section 2.8.
  • Scenario1, Scenario2 and Scenario3 data sets include the data from the dose-finding trial example.

Chapter 4, "Analysis of Multiple Endpoints in Clinical Trials"

  • Program 4.1 implements the OLS and GLS procedures in the osteoarthritis trial example introduced in Section 4.4.2. Program 4.2 performs sample size calculations for the OLS and GLS procedures in this clinical trial example.
  • Program 4.3 implements the Tamhane-Logan superiority-noninferiority procedure in the Alzheimer's disease trial example (Section 4.6).

Chapter 5, "Gatekeeping Procedures in Clinical Trials"

  • Program 5.1 implements the direct-calculation algorithm for serial gatekeeping procedures defined in Section 5.4.2 to compute adjusted p-values for the three-branch serial gatekeeping procedure in the Type II diabetes clinical trial example (Section 5.3.3).
  • Program 5.2 computes adjusted p-values for the two-stage parallel gatekeeping procedure based on the truncated and regular Holm tests in the cardiovascular clinical trial example (Section 5.4.3). This program also utilizes the direct-calculation algorithm.
  • Program 5.3 calculates adjusted p-values for the Bonferroni tree gatekeeping procedure in the combination-therapy clinical trial example (Section 5.5.2).

For more information about SAS and R implementation of multiple testing procedures and gatekeeping procedures, see also