ParProc macro

From MultXpert

Jump to: navigation, search

Return to Implementation of Multiple Testing Procedures.


The ParProc macro computes multiplicity-adjusted p-values for the following parametric multiple testing procedures in one-sided hypothesis testing problems with a balanced one-way layout and equally weighted null hypotheses:

  • Single-step Dunnett procedure.
  • Step-down Dunnett procedure.

General reference

Dmitrienko, A., Bretz, F., Westfall, P.H., Troendle, J., Wiens, B.L., Tamhane, A.C., Hsu, J.C. (2009). Multiple testing methodology. Multiple Testing Problems in Pharmaceutical Statistics. Dmitrienko, A., Tamhane, A.C., Bretz, F. (editors). Chapman and Hall/CRC Press, New York.


Download ParProc macro.


Using the clinical trial used to illustrate the PvalProc macro and PvalCI macro, consider a problem of testing three null hypotheses of no treatment effect corresponding to three dose-placebo comparisons in a confirmatory trial in the major depressive disorder population. The null hypotheses of interest are defined as follows:

  • Null hypothesis H1: No difference between Dose H and Placebo.
  • Null hypothesis H2: No difference between Dose M and Placebo.
  • Null hypothesis H3: No difference between Dose L and Placebo.

The the three null hypotheses are equally weighted.

Suppose that the comparisons are performed based on a simple ANOVA model and the t test statistics are given by

  • Null hypothesis H1 (Dose H versus Placebo): t=2.297.
  • Null hypothesis H2 (Dose M versus Placebo): t=2.497.
  • Null hypothesis H3 (Dose L versus Placebo): t=1.897.

Assuming a balanced design with n=180 patients per arm, each test statistics follows a t distribution with 2*(n-1)=358 degrees of freedom. The single-step or step-down Dunnett procedures can be used to perform a parametric multiplicity adjustment in this trial, which accounts for the fact that the test statistics are positively correlated. Given the balanced design, the correlation between each pair of test statistics is 0.5.

To carry out the single-step or step-down Dunnett procedures, the test statistics are included in a data set (T variable) and the data set is then passed to the ParProc macro:

   data example;
   input t;

The ordering of p-values in the EXAMPLE data set is not important for the single-step or step-down Dunnett procedures. The former is a single-step procedure and thus each null hypothesis is tested independently of the others and the latter is a stepwise procedure which uses a data-driven testing sequence, i.e., the test statistics are arranged from largest to smallest before the multiplicity adjustment is performed.

Note also that the corresponding one-sided p-values are easily computed from the null distribution of the test statistics:

   data pvalue;
   set example;

The resulting p-values are the p-values used in the major depressive disorder trial example introduced in PvalProc macro:

  • Null hypothesis H1 (Dose H versus Placebo): p=0.0111.
  • Null hypothesis H2 (Dose M versus Placebo): p=0.0065.
  • Null hypothesis H3 (Dose L versus Placebo): p=0.0293.

The ParProc macro is called as follows:


As in the PvalProc macro, the IN argument specifies the name of the data set with the outcomes of the individual tests (test statistics in this case) and the OUT argument defines the name of the data sets with multiplicity-adjusted p-values. Further, the N argument specifies the common sample size per arm.

The ADJP data set generated by the ParProc macro contains the adjusted p-values for the single-step and step-down Dunnett procedures:

   proc print data=adjp noobs label;
   format raw dunnett stepdunnett 6.4;
   var test raw dunnett stepdunnett;

The adjusted p-values are displayed below:

                     Single-     Step-
                      step       down
   Test       Raw    Dunnett    Dunnett
     1     0.0111    0.0289     0.0204
     2     0.0065    0.0172     0.0172
     3     0.0293    0.0722     0.0291

Using a one-sided α level of 0.025 (corresponding to a two-sided α=0.05), the single-step Dunnett procedure detects only one significant difference (Dose M versus Placebo, adjusted p=0.0172), whereas step-down Dunnett procedure finds evidence of a significant treatment effect for two doses (Dose H versus Placebo, adjusted p=0.0204; Dose M versus Placebo, adjusted p=0.0172).

Other procedures

To find out more about software implementation of other multiple testing procedures and gatekeeping procedures, see Software.