Navigation
SEARCH

PvalCI macro

From MultXpert

Jump to: navigation, search

Return to Implementation of Multiple Testing Procedures.

Description

The PvalCI macro computes one-sided multiplicity-adjusted confidence intervals (simultaneous confidence intervals) for the following p-valued-based multiple testing procedures in general hypothesis testing problems with equally or unequally weighted null hypotheses:

  • Bonferroni Procedure.
  • Holm Procedure.
  • Fixed-sequence Procedure.
  • Fallback Procedure.

Note that the macro does not support the semiparametric Hommel or Hochberg procedures since simultaneous confidence intervals for these procedures have not been explicitly defined.

General reference

Dmitrienko, A., Bretz, F., Westfall, P.H., Troendle, J., Wiens, B.L., Tamhane, A.C., Hsu, J.C. (2009). Multiple testing methodology. Multiple Testing Problems in Pharmaceutical Statistics. Dmitrienko, A., Tamhane, A.C., Bretz, F. (editors). Chapman and Hall/CRC Press, New York.

Download

Download PvalCI macro.

Example

To illustrate the PvalCI macro, a confirmatory trial in patients with major depressive disorder will be used. The trial's objective is to evaluate the efficacy of three doses of a novel treatment (labeled L, M and H) versus a placebo. The primary efficacy endpoint is continuous (Montgomery-Asberg Depression Rating Scale total score) and can be assumed to be normally distributed. The associated null hypotheses of no treatment effect are defined as follows:

  • Null hypothesis H1: No difference between Dose H and Placebo.
  • Null hypothesis H2: No difference between Dose M and Placebo.
  • Null hypothesis H3: No difference between Dose L and Placebo.

These null hypotheses are equally weighted. The mean treatment differences for the three dose-placebo comparisons are given by

  • Null hypothesis H1 (Dose H versus Placebo): mean=2.3.
  • Null hypothesis H2 (Dose M versus Placebo): mean=2.5.
  • Null hypothesis H3 (Dose L versus Placebo): mean=1.9.

The pooled standard deviation is sd=9.5 and the common sample size per arm is n=180. The p-values for the three dose-placebo tests are easily computed using the PROBT function. The one-sided p-values are given by:

  • Null hypothesis H1 (Dose H versus Placebo): p=0.0111.
  • Null hypothesis H2 (Dose M versus Placebo): p=0.0065.
  • Null hypothesis H3 (Dose L versus Placebo): p=0.0293.

To compute one-sided simultaneous confidence intervals for the true mean treatment differences with a joint coverage probability of 97.5%, the raw one-sided p-values (RAW_P variable), sample mean treatment differences (EST variable), pooled standard error (SE variable) and hypothesis weights (WEIGHT variable) are defined in a data set which is then passed to the PvalCI macro:

   data example;
   input raw_p est sd weight;
   se=sd*sqrt(2/180);
   datalines;
   0.0111 2.3 9.5 0.3333
   0.0065 2.5 9.5 0.3333
   0.0293 1.9 9.5 0.3334    
   ;

Note that the pooled standard error is computed based on the pooled standard deviation and common sample size. The ordering of p-values is important for the fixed-sequence and fallback procedures since these multiple testing procedures assume a pre-specified testing sequence. In this example, the two multiple testing procedures are applied starting with the first p-value in the data set, i.e., with the comparison between Dose H and Placebo.

To call the PvalCI macro, the following parameters need to be specified:

  • Name of the data set with the trial parameters (IN argument).
  • Joint coverage probability of one-sided simultaneous confidence intervals (COVPROB argument).
  • Name of the data set with one-sided simultaneous confidence intervals (OUT argument).

The PvalCI macro is called as follows to obtain one-sided simultaneous confidence intervals for the true mean treatment differences in the major depressive disorder trial:

   %pvalci(in=example,covprob=0.975,out=adjci);

The PvalCI macro computes lower limits of one-sided simultaneous confidence intervals for the four p-value-based multiple testing procedures and the user can select simultaneous confidence intervals for one or more multiple testing procedures as well as univariate confidence intervals:

   proc print data=adjci noobs label;
   format est se univariate bonferroni holm 6.2;
   var test est se univariate bonferroni holm;
   run;

The lower limits of one-sided simultaneous confidence intervals for the Bonferroni and Holm procedures are listed below:

                       Standard
   Test    Estimate     error      Univariate    Bonferroni      Holm
   
     1        2.30        1.00         0.34         -0.10        0.00
     2        2.50        1.00         0.54          0.10        0.00
     3        1.90        1.00        -0.06         -0.50       -0.06

It is important to note that these simultaneous confidence intervals are consistent with the adjusted p-values computed by the PvalProc macro, i.e., a lower limit is greater or equal to 0 if and only if the corresponding adjusted p-value is less than or equal to 0.025.

Other procedures

To find out more about software implementation of other multiple testing procedures and gatekeeping procedures, see Software.