Multiple Testing Problems

Multiple Testing Problems in Clinical Trials 2013
Two-day course taught by Alex Dmitrienko at the IBS German Region meeting in Berlin on September 19-20, 2013.

Back to Short Courses.

Dmitrienko A, Tamhane A, Bretz F (editors). Multiple Testing Problems in Pharmaceutical Statistics. Chapman and Hall/CRC, 2009.

SAS and R software implementation
Download ZIP archive with SAS and R programs used in this course. After downloading the ZIP files, please extract the programs to the Course folder on the C drive (c:\course).

For more information on software implementation of commonly used multiple testing procedures and gatekeeping procedures, see Implementation of Multiple Testing Procedures and Implementation of Gatekeeping Procedures.

Regulatory guidelines
To download applicable regulatory guidelines, see Regulatory Guidance.


  1. Brechenmacher, T., Xu, J., Dmitrienko, A., Tamhane, A.C. (2011). A mixture gatekeeping procedure based on the Hommel test for clinical trial applications. Journal of Biopharmaceutical Statistics. 21, 748-767. See Brechenmacher Xu Dmitrienko Tamhane 2011.
  2. Bretz, F., Maurer, W., Brannath, W., Posch, M. (2009). A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine. 28, 586-604.
  3. Dmitrienko, A., Offen, W.W., Westfall, P.H. (2003). Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Statistics in Medicine. 22, 2387-2400. See Dmitrienko Offen Westfall 2003.
  4. Dmitrienko, A., Tamhane, A.C. (2007). Gatekeeping procedures with clinical trial applications. Pharmaceutical Statistics. 6, 171-180. See Dmitrienko Tamhane 2007.
  5. Dmitrienko, A., Wiens, B.L., Tamhane, A.C., Wang, X. (2007). Tree-structured gatekeeping tests in clinical trials with hierarchically ordered multiple objectives. Statistics in Medicine. 26, 2465-2478. See Dmitrienko Wiens Tamhane Wang 2007.
  6. Dmitrienko, A., Tamhane, A.C., Wiens, B.L. (2008). General multistage gatekeeping procedures. Biometrical Journal. 50, 667-677. See Dmitrienko Tamhane Wiens 2008.
  7. Dmitrienko, A., Tamhane, A.C. (2009). Gatekeeping procedures in clinical trials. Multiple Testing Problems in Pharmaceutical Statistics. Dmitrienko, A., Tamhane, A.C., Bretz, F. (editors). Chapman and Hall/CRC Press, New York. See Dmitrienko Tamhane 2009.
  8. Dmitrienko, A., Kordzakhia, G., Tamhane, A.C. (2011). Multistage and mixture parallel gatekeeping procedures in clinical trials. Journal of Biopharmaceutical Statistics. 21, 726-747. See Dmitrienko Kordzakhia Tamhane 2011.
  9. Dmitrienko, A., Tamhane, A.C. (2011). Mixtures of multiple testing procedures for gatekeeping applications in clinical trials. Statistics in Medicine. 30, 1473-1488. See Dmitrienko Tamhane 2011.
  10. Dmitrienko, A., D'Agostino, R.B., Huque, M.F. (2013). Key multiplicity issues in clinical drug development. Statistics in Medicine. 32, 1079-1111.
  11. Dmitrienko, A., Tamhane, A.C. (2013). General theory of mixture procedures for gatekeeping. Biometrical Journal. 55, 402-419. See Dmitrienko Tamhane 2013.
  12. Dunnett, C.W. (1955). A multiple comparison procedure for comparing several treatments with a control. Journal of the American Statistical Association. 50, 1096-1121.
  13. Dunnett, C.W., Tamhane, A.C. (1992). A step-up multiple test procedure. Journal of the American Statistical Association. 87, 162-170.
  14. Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple significance testing. Biometrika. 75, 800-802.
  15. Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics. 6, 65-70.
  16. Hommel, G. (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika. 75, 383-386.
  17. Hothorn, T., Bretz, F., Westfall, P. (2008). Simultaneous inference in general parametric models. Biometrical Journal. 50, 346-363.
  18. Huque, M.F., Alosh, M. (2008). A flexible fixed-sequence testing method for hierarchically ordered correlated multiple endpoints in clinical trials. Journal of Statistical Planning and Inference. 138, 321-335.
  19. Kordzakhia, G., Dmitrienko, A. (2013). Superchain procedures in clinical trials with multiple objectives. Statistics in Medicine. 32, 486-508. See Kordzakhia Dmitrienko 2013.
  20. Marcus, R., Peritz, E., Gabriel, K.R. (1976). On closed testing procedures with special reference to ordered analysis of variance. Biometrika. 63, 655-660.
  21. Maurer, W., Hothorn, L. A., Lehmacher, W. (1995). Multiple comparisons in drug clinical trials and preclinical assays: a priori ordered hypotheses. Biometrie in der Chemisch-in-Pharmazeutischen Industrie. 6. Vollman, J. (editor). Fischer-Verlag, Stuttgart, 3-18.
  22. Meltzer, H.Y. et al. (2011). Lurasidone in the treatment of schizophrenia: a randomized, double-blind, placebo- and olanzapine-controlled study. American Journal of Psychiatry. 168, 957-967.
  23. Millen, B., Dmitrienko, A. (2011). Chain procedures: A class of flexible closed testing procedures with clinical trial applications. Statistics in Biopharmaceutical Research. 3, 14-30. See Millen Dmitrienko 2011.
  24. Sarkar, S.K. (2008). On the Simes inequality and its generalization. Beyond Parametrics in Interdisciplinary Research: Festschrift in Honor of Professor Pranab K. Sen. Balakrishnan N, Pena EA, Silvapulle MJ (editors). Institute of Mathematical Statistics, Beachwood, Ohio, 231-242.
  25. Simes, R.J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika. 63, 655-660.
  26. Wiens, B. (2003). A fixed-sequence Bonferroni procedure for testing multiple endpoints. Pharmaceutical Statistics. 2, 211-215.
  27. Wiens, B., Dmitrienko, A. (2005). The fallback procedure for evaluating a single family of hypotheses. Journal of Biopharmaceutical Statistics. 15, 929-942.

There are no products listed under this category.