Department of Industrial Engineering and Management Sciences

Northwestern University, Evanston, Illinois 60208-3119, U.S.A.

*Working Paper No. 08-04*

Mixtures of Multiple Testing Procedures with Gatekeeping Applications

Alex Dmitrienko

Eli Lilly and Company

Ajit C. Tamhane

Northwestern University

Lingyun Liu

Northwestern University

December 2008

Abstract

This paper introduces a general framework for constructing gatekeeping procedures

for multiple testing problems arising in clinical trials with hierarchical objectives.

These problems frequently exhibit a complex structure, including multiple families of

hypotheses and logical restrictions. The framework is based on combining multiple

tests across families and enables clinical trial sponsors to set up powerful and flexible

multiple testing procedures (e.g., gatekeeping procedures based on Dunnett tests

that account for logical restrictions among the hypotheses of interest). A clinical trial

example is used to illustrate the general approach.

Keywords and Phrases: Multiple comparisons; Closure principle; Gatekeeping proce-

dures; Bonferroni test; Dunnett test.

1. Introduction

Gatekeeping procedures are commonly used in multiple testing problems with a

hierarchical structure, including problems arising in clinical trials with multiple ob-

jectives. These objectives may represent primary endpoints, secondary endpoints and

subgroup analyses, etc. To account for the hierarchical structure of these objectives,

null hypotheses associated with the objectives are grouped into families. Consider,

for example, a multiple testing problem involving

*n*null hypotheses*H*1*,...,Hn*thatare grouped into

*m*families:*Fk*=

*{Hi, i ∈ Nk}, k*= 1

*,...,m, m ≥*2

*,*

where

*N*1 =*{*1*,...,n*1*}*,*Nk*=*{n*1 +*...*+*nk−*1 + 1*,...,n*1 +*...*+*nk}*,*k*= 2*,...,m*,and

*n*1 +*...*+*nm*=*n*.Dmitrienko, Tamhane and Wiens (2008) introduced a framework for constructing

multistage parallel gatekeeping procedures. A parallel gatekeeping procedure tests

hypotheses in Family

*Fk*,*k*= 2*,...,m*, only if one or more hypotheses are rejected in*Fk−*1. Dmitrienko, Tamhane and Wiens proposed a general algorithm for setting up

parallel gatekeeping procedures with an attractive stepwise form based on tests from

a broad class of multiple testing procedures (known as separable procedures).

One of the limitations of the framework proposed by these authors is that it

cannot be used in problems with logical restrictions, i.e., when the acceptance or

rejection of hypotheses in

*Fk*,*k*= 2*,...,m*, depends on the outcomes of signifi-cance tests in

*F*1*,...,Fk−*1. Multiple testing problems with logical restrictions arefrequently encountered in clinical trials. Examples are given in Chen, Luo and

Capizzi (2005), Quan, Luo and Capizzi (2005), Dmitrienko, Offen, Wang and Xiao

(2006), Dmitrienko, Wiens, Tamhane and Wang (2007), Dmitrienko, Tamhane, Liu

and Wiens (2008).

This paper describes a framework that enables clinical trial sponsors to set up

flexible multiple testing procedures for problems with a very general class of logical

restrictions (monotone logical restrictions). The framework is based on combining

multiple tests across families using the concept of a mixture of multiple testing pro-

cedures. This term is used here to make an analogy with mixtures of distributions

(Everitt and Hand, 1981). To specify a mixture distribution, one needs to specify

component distributions and a mixing distribution. Similarly, in the case of mix-

ture procedures, one needs to select component procedures and a mixing function.

The mixing function is selected to take into account the logical relationships among

multiple families and provide strong control of the familywise error rate (FWER)

(Hochberg and Tamhane, 1987). The mixture-based framework uses the closure prin-

ciple (Marcus, Peritz and Gabriel, 1976) to achieve FWER control.

The paper is organized as follows. Section 2 introduces the mixture-based frame-

work for an arbitrary number of families. Section 3 defines a class of monotone logical

restrictions. Section 4 describes mixing functions that can be used to construct mix-

tures of multiple testing procedures. Properties of mixture procedures are described

in Section 5. Lastly, Section 6 gives examples of mixture procedures (including mix-

tures of Bonferroni and Dunnett procedures) and a clinical trial example to illustrate

the mixture-based framework.

2. Mixture procedures

Consider the multiple testing problem defined in the Introduction. Let

*Hk*denotethe closed family associated with

*Fk*, i.e.,*Hk*=

*{HIk , Ik ⊆ Nk},*where

*HIk*=

*∩i∈Ik Hi.*

Further, consider multiple testing procedures, known as component procedures,

*T*1*,...,Tm*.The procedure

*Tk*,*k*= 1*,...,m*, is assumed to be a closed testing procedure thatcontrols the FWER in the strong sense within

*Fk*. This means that there exists aset of

*α*-level tests for each intersection hypothesis in*Hk*such that*Tk*rejects*Hi*,*i ∈ Nk*, if and only if (iff) all intersection hypotheses including

*Hi*are rejected by the

intersection hypothesis tests. For example, if

*Tk*is the Holm procedure, each inter-section hypothesis is tested using the Bonferroni test at

*α*. Let*pk*(*Ik*),*Ik ⊆ Nk*, bethe

*p*-value for the intersection hypothesis test associated with*HIk*. The intersectionhypothesis

*HIk*is rejected iff*pk*(*Ik*)*≤ α*.A mixture of the component procedures, denoted by

*T*, is a procedure for testingall hypotheses in

*F*=*F*1*∪ ... ∪ Fm*. Let*N*=*{*1*,...,n}*and let*H*denote the closedfamily associated with

*F*, i.e.,*H*=*{HI,I ⊆ N}*. For each index set*I ⊆ N*, let*Ik*=

*I ∩ Nk*,

*k*= 1

*,...,m*. To define a mixture procedure, one needs to define

*α*-

level tests for all intersection hypotheses in

*H*. Consider any non-empty intersectionhypothesis

*HI*,*I ⊆ N*. The test for this intersection hypothesis is defined as follows:Case 1.

*HI*contains hypotheses only from*Fk*,*k*= 1*,...,m*, i.e.,*I*=*Ik*. The*p*-valuefor

*HI*is given by*p*(*I*) =*pk*(*Ik*).Case 2.

*HI*contains hypotheses from*Fi*1*,...,Fis*for*s ≥*2, i.e.,*I*=*Ii*1*∪ ... ∪ Iis*.The

*p*-value for*HI*is given by*p*(

*I*) =

*mI*(

*pi*1 (

*Ii*1 )

*,...,pis*(

*Iis*))

*,*

where

*mI*(*xi*1*,...,xis*) is a mixing function.Mixing functions have the following properties:

*•*0

*≤ mI*(

*xi*1

*,...,xis*)

*≤*1, 0

*≤ xik ≤*1,

*k*= 1

*,...,s*.

*• mI*(

*xi*1

*,...,xis*)

*≤ α*if

*xi*1

*≤ α*.

*•*The test for

*HI*is an

*α*-level test, i.e.,

*P*(

*p*(

*I*)

*≤ α*)

*≤ α*.

Examples of mixing functions are given in Section 4.

Given the

*p*-values for each intersection in the closed family, the*p*-value for ahypothesis in

*F*is computed using the closure principle. For the hypothesis*Hi*,*i ∈ N*, the adjusted

*p*-value is defined as the maximum over the

*p*-values for the

intersections containing this hypothesis, i.e.,

˜

*pi*= max*I*:

*i∈I*

*p*(

*I*)

*.*

Since the mixture procedure is constructed using

*α*-level tests for all intersectionhypotheses in

*H*, the procedure controls the FWER in the strong sense at an*α*level.3. Logical restrictions

Mixtures of multiple testing procedures are constructed to account for logical

relationships among the hypotheses in

*F*1*,...,Fm*. Dmitrienko, Wiens, Tamhane andWang (2007) and Dmitrienko, Tamhane, Liu and Wiens (2008) proposed to formulate

logical relationships in terms of serial and parallel gatekeeping sets. In this case a

hypothesis in

*Fk*+1,*k*= 1*,...,m−*1, is tested iff all hypotheses are rejected in a certainsubset of

*F*1*,...,Fk*(known as the serial gatekeeping set) and at least one hypothesisis rejected in another subset of

*F*1*,...,Fk*(known as the parallel gatekeeping set).A more general family of monotone logical restrictions is introduced below. The

restrictions are defined using restriction functions. Consider a hypothesis in

*Fs*+1,*s*= 1

*,...,m−*1, say,

*Hi*,

*i ∈ Ns*+1. The restriction function

*Li*(

*I*),

*I ⊆ N*1

*∪...∪Ns*,

assumes two values,

*Li*(*I*) = 0 or 1. Here*Li*(*I*) = 0 means that*Hi*is not testable, i.e.,it is accepted without test if the hypotheses

*Hj*,*j ∈ I*, are accepted and*Li*(*I*)=1means that

*Hi*is testable. The function*Li*(*I*) meets the following conditions:*•*Monotonicity condition: If

*Li*(

*I*) = 0 and

*I ⊆ I*then

*Li*(

*I*)=0.

*•*Parallel gatekeeping condition: If

*Nk ⊆ I*then

*Li*(

*I*) = 0 for all

*i ∈ Ns*for

*s*=

*k*+ 1

*,...,m*.

Note that, by the monotonicity condition, if a hypothesis in

*Fs*+1 is not testablegiven a set of accepted hypotheses in

*F*1*,...,Fs*, it will remain non-testable if morehypotheses are accepted in

*F*1*,...,Fs*. Further, it follows from the parallel gatekeepingcondition that all hypotheses are non-testable (and are automatically accepted) in

*Fs*+1 if all hypotheses are accepted in

*Fk*,

*k*= 1

*,...,s*.

In order to account for logical restrictions, the definition of a mixture of two

multiple testing procedures needs to be modified as follows:

Case 1.

*HI*contains hypotheses only from*Fk*,*k*= 1*,...,m*, i.e.,*I*=*Ik*. The*p*-valuefor

*HI*is given by*p*(*I*) =*pk*(*Ik*).Case 2.

*HI*contains hypotheses from*Fi*1*,...,Fis*for*s ≥*2, i.e.,*I*=*Ii*1*∪ ... ∪ Iis*.For any

*k*= 2*,...,s*, let*I∗**ik*

be the subset of

*Iik*that includes the indices of

hypotheses that are logically consistent, i.e., testable, with the hypotheses from

*Fi*1

*,...,Fik−*1 . In other words,

*I∗*

*ik*

=

*{i*:*i ∈ Iik*and*Li*(*Ii*1*∪ ... ∪ Iik−*1 )=1*}.*Assume first that

*I∗**is*

is not empty. In this case the

*p*-value for*HI*is given by*p*(

*I*) =

*mI*(

*pi*1 (

*Ii*1 )

*,pi*2 (

*I∗*

*i*2

)

*,...,pis*(*I∗**is*

))

*,*where

*pik*(*I∗**ik*

)=1if

*I∗**ik*

is empty,

*k*= 2*,...,s −*1. Further, if*I∗**ir*+1

*,...,I∗*

*is*

are

empty for some

*r*= 1*,...,s −*1 then*p*(

*I*) =

*mJ*(

*pi*1 (

*Ii*1 )

*,pi*2 (

*I∗*

*i*2

)

*,...,pir*(*I∗**ir*

))

*,*where

*J*=*Ii*1*∪ ... ∪ Iir*and*pik*(*I∗**ik*

)=1if

*I∗**ik*

is empty.

4. Mixing functions

This section defines mixing functions based on the Bonferroni and Dunnett global

tests. Both these mixing functions satisfy the properties listed in Section 2 and have

the same general form:

*mI*(

*xi*1

*,...,xis*) = min

(

*xi*1

*ci*1

*,...,*

*xis*

*cis*

)

*,*

where

*I*=*Ii*1*∪ ... ∪ Iis*as before and*ci*1*,...,cis*is a non-increasing sequence of co-efficients with 1 =

*ci*1*≥ ... ≥ cis ≥*0. This sequence is non-increasing to account forthe hierarchical structure of the problem, i.e., families placed earlier in the sequence

are more important (and receive greater weights) than those later in the sequence.

The Bonferroni and Dunnett mixing functions differ in terms of the choice of these

coefficients. For the Bonferroni mixing function, the coefficients are denoted by

*b*’sand for the Dunnett mixing function by

*d*’s.4.1 Bonferroni mixing function

To define this function, consider the error rate function of the procedure

*Tk*,*k*=1

*,...,m −*1, introduced in Dmitrienko, Tamhane and Wiens (2008). Since an exactexpression for the error rate function is, in general, difficult to derive, we will focus

on an upper bound,

*ek*(*Ik*), for the true error rate function, i.e.,*P*(

*pk*(

*Ik*)

*≤ α*)

*≤ ek*(

*Ik*)

for fixed

*α*. As in Dmitrienko, Tamhane and Wiens (2008), we will treat*ek*(*Ik*) asthe actual error rate function. Error rate functions have the following properties:

*ek*(

*∅*)=0

*, ek*(

*I*)

*≤ ek*(

*I*) if

*I ⊆ I , ek*(

*Nk*) =

*α.*

Also, let

*fk*(*Ik*) =*ek*(*Ik*)*/α*.Assume that

*T*1*,...,Tm−*1 are separable, i.e.,*fk*(*Ik*)*<*1 for all*α*if*Ik*is a propersubset of

*Nk*,*k*= 1*,...,m −*1. The Bonferroni mixing function is given by*mI*(

*xi*1

*,...,xis*) = min

(

*xi*1

*bi*1

*,...,*

*xis*

*bis*

)

*,*

where

*bi*1 = 1 and*bik*=*bik−*1 (1*− fik−*1 (*Iik−*1 )),*k*= 2*,...,s*. It is clear that0

*≤ mI*(*xi*1*,...,xis*)*≤*1 if 0*≤ xik ≤*1and

*mI*(

*xi*1

*,...,xis*)

*≤ α*if

*xi*1

*≤ α.*

Since

*T*1*,...,Tm−*1 are separable,*bik >*0 if*Iir−*1 is a proper subset of*Nir−*1 for all*r*= 2

*,...,k*. On the other hand,

*bik*=

*...*=

*bis*= 0 if

*Iik−*1 =

*Nik−*1 and thus

*mI*(

*xi*1

*,...,xis*) = min

(

*xi*1

*bi*1

*,...,*

*xik−*1

*bik−*1

)

*.*

It is easy to verify that the resulting test for

*HI*is an*α*-level test. By the Bonferroniinequality,

*P*(

*p*(

*I*)

*≤ α*)

*≤*

*s*

∑

*k*=1

*P*(

*pik*(

*Iik*)

*≤ αbik*)

*≤*

*s−*1

∑

*k*=1

*αbik fik*(

*Iik*) +

*αbis*

since

*P*(*pik*(*Iik*)*≤ x*)*≤ xfik*(*Iik*),*k*= 1*,...,s −*1, and*P*(*pis*(*Iis*)*≤ x*)*≤ x*. Further,it is easy to see that

*bis−*1*fis−*1 (*Iis−*1 ) +*bis*=*bis−*1 since*bis*=*bis−*1 (1*− fis−*1 (*Iis−*1 )).Doing this recursively, we have

*s−*1

∑

*k*=1

*bik fik*(

*Iik*) +

*bis*=

*bi*1 = 1

and thus

*P*(*p*(*I*)*≤ α*)*≤ α*.4.2 Dunnett mixing function

The Bonferroni mixing function defined above is based on the Bonferroni inequal-

ity and thus does not account for the correlation among

*pi*1 (*Ii*1 )*,...,pis*(*Iis*). Bycontrast, the Dunnett mixing function explicitly utilizes the joint distribution of the

*p*-values.

Assume again that

*Tk*is separable,*k*= 1*,...,m−*1. The Dunnett mixing functionis given by

*mI*(

*xi*1

*,...,xis*) = min

(

*xi*1

*di*1

*,...,*

*xis*

*dis*

)

*,*

where

*di*1 = 1 and*dik*,*k*= 2*,...,s*, are defined sequentially as follows:*P*(

*pi*1 (

*Ii*1 )

*≤ αdi*1 or

*pi*2 (

*Ni*2 )

*≤ αdi*2 ) =

*α,*

*P*(

*pi*1 (

*Ii*1 )

*≤ αdi*1 or

*pi*2 (

*Ii*2 )

*≤ αdi*2 or

*pi*3 (

*Ni*3 )

*≤ αdi*3 ) =

*α,*

*...*

*P*(

*pi*1 (

*Ii*1 )

*≤ αdi*1 or

*pi*2 (

*Ii*2 )

*≤ αdi*2 or

*...*or

*pis−*2 (

*Iis−*2 )

*≤ αdis−*2 or

*pis−*1 (

*Nis−*1 )

*≤ αdis−*1 ) =

*α,*

*P*(

*pi*1 (

*Ii*1 )

*≤ αdi*1 or

*pi*2 (

*Ii*2 )

*≤ αdi*2 or

*...*or

*pis−*1 (

*Iis−*1 )

*≤ αdis−*1 or

*pis*(

*Iis*)

*≤ αdis*) =

*α.*

It follows from the equations that

*dik**>*0 if

*Iir−*1 is a proper subset of

*Nir−*1 ,

*r*=

2

*,...,k*, and*dik*=*...*=*dis*= 0 if*Iik−*1 =*Nik−*1 .As in Section 4.1, it is easy to see that 0

*≤ mI*(*xi*1*,...,xis*)*≤*1 if 0*≤ xik ≤*1 and*mI*(

*xi*1

*,...,xis*)

*≤ α*if

*xi*1

*≤ α*. Further, by the definition of

*dik*,

*k*= 1

*,...,s*,

*P*(

*p*(

*I*)

*≤ α*) =

*P*(

*pi*1 (

*Ii*1 )

*≤ αdi*1 or

*...*or

*pis*(

*Iis*)

*≤ αdis*)

=

*α*and thus the resulting test for

*HI*is an*α*-level test.Since the Dunnett mixing function takes into account the joint distribution of test

statistics, mixture procedures based on this function are more powerful than those

based on the Bonferroni mixing function.

5. Properties of mixture procedures

This section summarizes key properties of mixture procedures.

5.1 General properties

We will begin with a discussion of general properties, including consistency with

logical restrictions and independence (inferences in

*F*1 are independent of inferencesin

*F*2).Proposition 1

*Assume that T is consonant in F*1*,...,Fk, k*= 1*,...,m −*1*, then**the mixture procedure T is consistent with the logical restrictions in Fk*+1

*. In other*

*words, T accepts Hi, i ∈ Nk*+1

*, at the α level if Li*(

*A*1

*∪ ... ∪ Ak*)=0

*, where Ar is*

*the index set of accepted hypotheses in Fr, r*= 1

*,...,k.*

Note that, if

*T*is not consonant in*F*1*,...,Fk*, the logical restrictions may beviolated in

*Fk*+1 in the sense that*Hi*,*i ∈ Nk*+1, may be rejected even though*Li*(*A*1*∪**... ∪ Ak*) = 0. However, the logical restrictions can always be enforced by modifying

multiplicity-adjusted

*p*-values in*Fk*+1. This can be done using an algorithm similarto that proposed in Kordzakhia et al. (2008).

Proposition 2

*The mixture procedure T is equivalent to the procedure T*1*within the**first family. In other words, T rejects a hypothesis in F*1

*at the α level iff T*1

*rejects*

*this hypothesis at the α level.*

Proposition 3

*The mixture procedure T is equivalent to the procedure Tk, k*=2

*,...,m, within Fk if T rejects all hypotheses in F*1*,...,Fk−*1*. In other words, T**rejects a hypothesis in Fk at the α level iff Tk rejects this hypothesis at the α level*

*provided all hypotheses in F*1

*,...,Fk−*1

*are rejected by T.*

The proofs of Propositions 1, 2 and 3 are given in the Appendix.

5.2 Stepwise mixture procedures with parallel gatekeeping restrictions

When parallel gatekeeping restrictions are considered, mixture procedures based

on the Bonferroni mixing function admit a stepwise representation. This means that

the mixture procedure is, in fact, identical to a stepwise application of the com-

ponent procedures with an adjustment of the significance level in the last

*m −*1families. This result is equivalent to the main result in Dmitrienko, Tamhane and

Wiens (2008) and shows that the mixture framework is an extension of the framework

of multistage gatekeeping procedures introduced in that paper. In particular, mul-

tistage gatekeeping procedures considered by Dmitrienko, Tamhane and Wiens are

mixtures of component procedures used at individual stages based on the Bonferroni

mixing function.

To demonstrate that mixture procedures based on the Bonferroni mixing func-

tion are equivalent to multistage gatekeeping procedures proposed by Dmitrienko,

Tamhane and Wiens, we will consider a two-family problem. The proof can be ex-

tended to the general case of

*m*families by recursion.Proposition 4

*Assume that**• Only parallel gatekeeping restrictions are imposed, i.e., Li*(

*N*1)=0

*, i ∈ N*2

*, and*

*Li*(

*I*1)=1

*, i ∈ N*2

*, I*1

*⊂ N*1

*.*

*• The procedure T*1

*is separable and consonant.*

*• The Bonferroni mixing function is used.*

*The mixture procedure T has the following two-stage structure:*

*• The hypotheses in F*1

*are tested at the familywise level α*1 =

*α using T*1

*.*

*• The hypotheses in F*2

*are tested at the level α*2 =

*α−e*1(

*A*1)

*using T*2

*, where e*1(

*I*)

*is the error rate function of T*1

*and A*1

*is the index set of accepted hypotheses*

*in F*1

*.*

The proof of Proposition 4 is given in the Appendix.

5.3 Mixture procedures with general logical restrictions

As shown in Proposition 4, mixture procedures in problems with parallel gate-

keeping restrictions have an attractive stepwise form. The following counterexample

shows that mixtures of testing procedures with general logical restrictions may not

have a stepwise form.

Consider a two-family problem with

*N*1 =*{*1*,*2*}*and*N*2 =*{*3*,...,n}*. Assumethat the hypotheses within each family are equally weighted. Further, consider a

mixture of the Bonferroni procedure in

*F*1 and Holm procedure in*F*2 based on theBonferroni mixing function. The following logical restrictions are assumed:

*• H*3

*,...,Hn−*1 are testable iff

*H*2 is rejected.

*• Hn*is testable iff at least one hypothesis in

*F*1 is rejected.

In other words,

*•*If

*I*1 =

*∅*or

*I*1 =

*{*1

*}*, then

*L*3(

*I*1) =

*L*4(

*I*1) =

*...*=

*Ln*(

*I*1) = 1.

*•*If

*I*1 =

*{*2

*}*, then

*L*3(

*I*1) =

*L*4(

*I*1) =

*...*=

*Ln−*1(

*I*1) = 0 and

*Ln*(

*I*1) = 1.

*•*If

*I*1 =

*{*1

*,*2

*}*, then

*L*3(

*I*1) =

*L*4(

*I*1) =

*...*=

*Ln*(

*I*1) = 0.

To demonstrate that the mixture of the two procedures does not have a stepwise

form, it is sufficient to focus on the case when

*T*1 (Bonferroni procedure) rejects*H*1but accepts

*H*2. By the logical restrictions, only one hypothesis is testable in*F*2 inthis case (namely, the hypothesis

*Hn*). If the mixture procedure had a stepwise form,this hypothesis would have been tested by

*T*2 (Holm procedure), i.e., its decisionrule would have been expressed in terms of

*pn*compared to an appropriately chosensignificance level. However, as shown in Proposition 5, this is not the case.

Proposition 5

*Let q*(1)*≤ ... ≤ q*(*n−*2)*denote the ordered p-values in F*2*and assume**that pn is the kth ordered p-value, i.e., pn*=

*q*(

*k*)

*, k*= 1

*,...,n−*2

*. Then the hypothesis*

*Hn is rejected iff all of the following conditions are met*

*pn ≤ α/*2

*, q*(

*i*)

*≤ α/*(

*n − i −*1)

*, i*= 1

*,...,k.*

The proof of Proposition 5 is given in the Appendix.

6. Examples

In this section we will give examples of mixture procedures that help illustrate

the general method introduced in Section 2.

6.1 Mixtures of Bonferroni procedures

Consider a problem of testing

*n*hypotheses and let*w*1*,...,wn*denote the weightsassigned to the hypotheses in the

*m*families. The weights are non-negative and sumto 1 within each family, i.e.,

*wi ≥*0

*, i*= 1

*,...,n,*

∑

*i∈Nk*

*wi*= 1

*, k*= 1

*,...,m.*

The

*n*hypotheses are grouped into*m*families. Assume that the first*m−*1 familiesare tested using a weighted version of the Bonferroni procedure and the last family

is tested using a weighted version of the Holm procedure. In other words,

*pk*(

*Ik*) = min

*i∈Ik*

(

*pi/wi*) if*Ik ⊆ Nk, k*= 1*,...,m −*1*,**pm*(

*Im*) =

∑

*k∈Im*

*wk*

min

*i∈Im*

(

*pi/wi*) if*Im ⊆ Nm.*We will assume first that parallel gatekeeping restrictions are imposed, i.e.,

*Li*(

*Nk−*1) = 0

*, i ∈ Nk, k*= 2

*,...,m,*

*Li*(

*Ik−*1) = 1

*, i ∈ Nk, Ik−*1

*⊂ Nk−*1

*, k*= 2

*,...,m.*

Noting that the error rate function for the weighted Bonferroni procedure is given by

*ek*(

*Ik*) =

*α*

∑

*i∈Ik*

*wi, k*= 1

*,...,m −*1

*,*

it can be shown that the mixture of the

*m*procedures based on the Bonferroni mixingfunction is defined as follows. Let

*HI*,*I ⊆ N*, be a non-empty intersection hypothesis.If

*I ⊆ Nk*,*k*= 1*,...,m*, then*p*(*I*) =*pk*(*Ik*), where*Ik*=*I ∩ Nk*. If*HI*containshypotheses from

*Fi*1*,...,Fis*for*s ≥*2, the*p*-value for*HI*is given by*p*(

*I*) = min

*i∈I*

*pi*

*vi*(

*I*)

*,*

where

*vi*(

*I*) =

*v∗*

*k*(

*I*)

*wi, i ∈ Iik , k*= 1

*,...,s −*1

*,*

*vi*(

*I*) =

*v∗*

*s*(

*I*)

*wi, i ∈ Iis*and

*is*=

*m,*

*vi*(

*I*) =

*v∗*

*s*(

*I*)

*wi/*

∑

*k∈Iis*

*wk, i ∈ Iis*and

*is*=

*m,*

*v∗*

1(

*I*) = 1*, v∗**k*+1(

*I*) =

*v∗*

*k*(

*I*)

1

*−*∑

*i∈Iik*

*wi*

*, k*= 1*,...,s −*1*.*The resulting procedure is equivalent to the Bonferroni-based parallel gatekeeping

procedure (Dmitrienko, Offen and Westfall, 2003).

Further, we will consider the general case of monotone logical restrictions. The

mixture procedure based on the Bonferroni mixing function has a structure similar

to that of the parallel gatekeeping procedure. First

*p*(*I*) =*pk*(*Ik*) if*I ⊆ Nk*,*k*=1

*,...,m*, where*Ik*=*I ∩ Nk*. Further, if*HI*contains hypotheses from*Fi*1*,...,Fis*for*s ≥*2, then the

*p*-value for

*HI*is given by

*p*(

*I*) = min

*i∈I∗*

*pi*

*vi*(

*I*)

*,*

where

*I∗*=*Ii*1*∪ I∗**i*2

*∪ ... ∪ I∗*

*is*

and

*vi*(

*I*) =

*v∗*

*k*(

*I*)

*wi, i ∈ I∗*

*ik*

*, k*= 1

*,...,s −*1

*,*

*vi*(

*I*) =

*v∗*

*s*(

*I*)

*wi, i ∈ I∗*

*is*

and

*is*=*m,**vi*(

*I*) =

*v∗*

*s*(

*I*)

*wi/*

∑

*k∈Iis*

*wk, i ∈ I∗*

*is*

and

*is*=*m,**v∗*

1(

*I*) = 1*, v∗**k*+1(

*I*) =

*v∗*

*k*(

*I*)

1

*−*∑

*i∈Iik*

*wi*

*, k*= 1*,...,s −*1*.*Note that the presence of logical restrictions has an impact only on the index sets used

in the decision rule in the sense that a hypothesis is removed from the decision rule

if is not consistent with the logical restrictions. The process of combining component

procedures is not affected by logical restrictions and therefore

*v∗*1(

*I*)*,...,v∗**s*(

*I*) remain

the same. This mixture procedure is equivalent to the tree gatekeeping procedure

based on Algorithm III (Kordzakhia et al., 2008).

It is also important to note that the weighting scheme used in this mixture pro-

cedure satisfies the monotonicity condition (Condition 3) formulated in Dmitrienko,

Tamhane, Liu and Wiens (2008). Weighting schemes proposed in other papers, in-

cluding Algorithm 2 in Dmitrienko, Tamhane, Liu and Wiens (2008), do not always

satisfy the monotonicity condition and gatekeeping procedures based on those schemes

can be inconsistent with the prespecified logical restrictions. In this case, the logical

restrictions need to be enforced as explained in Section 5.1.

6.2 Mixtures of Dunnett procedures

The algorithm given in Section 6.1 can be easily extended to construct more power-

ful mixture procedures, e.g., mixtures of Dunnett procedures based on the Bonferroni

mixing function. Considering a general problem of testing

*n*hypotheses grouped into*m*families, let

*ti*,

*i ∈ N*, denote the test statistic associated with

*Hi*and assume that

*ti*,

*i ∈ Nk*, follow a multivariate

*t*distribution for any

*k*= 1

*,...,m*. Suppose that the

hypotheses in

*Fk*,*k*= 1*,...,m*, are tested using the Dunnett procedure. In this case,the

*p*-value for the intersection hypothesis*HIk*,*Ik ⊆ Nk*,*k*= 1*,...,m*, is given by*pk*(

*Ik*)=1

*− G|Ik|*

(

max

*i∈Ik*

*ti*

)

*,*

where

*Gn*(*x*) is the cumulative distribution function of the*n*-variate one-sided Dun-nett distribution, i.e.,

*G|Ik|*(

*x*) =

*P*

(

max

*i∈Ik*

*t∗*

*i ≤ x*

)

*,*

and

*t∗**i*,

*i ∈ Nk*, have the same joint distribution as

*ti*,

*i ∈ Nk*, under the global null

hypothesis. A mixture of the Dunnett procedures based on the Bonferroni mixing

function can now be defined using the steps described in Section 6.1.

6.3 Clinical trial example

The mixture procedures introduced in Sections 6.1 and 6.2 will be illustrated here

using a clinical trial example from Dmitrienko, Offen, Wang and Xiao (2006) and

Dmitrienko, Wiens, Tamhane and Wang (2007, Section 6). Consider a clinical trial

in patients with Type II diabetes conducted to test three doses of an experimental

treatment versus placebo. The three doses are labeled L, M and H and the placebo is

Table 1. Test statistics and raw

*p*-values in the Type II diabetes clinicaltrial example.

Family

Null

Test

*P*-value

hypothesis statistic

*F*1

*H*1

2

*.*810

*.*005*H*2

2

*.*560

*.*011*H*3

2

*.*390

*.*018*F*2

*H*4

2

*.*610

*.*009*H*5

2

*.*240

*.*026*H*6

2

*.*500

*.*013*F*3

*H*7

2

*.*600

*.*010*H*8

2

*.*780

*.*006*H*9

1

*.*960

*.*051labeled Plac. The dose-placebo comparisons are performed with respect to three or-

dered endpoints, Endpoint P (Hemoglobin A1c), Endpoint S1 (Fasting serum glucose)

and Endpoint S2 (HDL cholesterol). The sample size per arm is 87 patients.

The resulting nine hypotheses of no treatment effect (three dose-placebo compar-

isons times three endpoints) are grouped into three families:

*•*Family

*F*1: H-Plac (

*H*1), M-Plac (

*H*2) and L-Plac (

*H*3) comparisons for End-

point P.

*•*Family

*F*2: H-Plac (

*H*4), M-Plac (

*H*5) and L-Plac (

*H*6) comparisons for End-

point S1.

*•*Family

*F*3: H-Plac (

*H*7), M-Plac (

*H*8) and L-Plac (

*H*9) comparisons for End-

point S2.

The three doses are assumed to be equally important and thus the hypotheses are

equally weighted within each family, i.e.,

*wi*= 1*/*3,*i*= 1*,...,*9. The two-sample*t*statistics and associated

*p*-values for the nine hypotheses are listed in Table 1.The null hypotheses in this clinical trial example will be tested using three multiple

testing procedures:

*•*Procedure 1 (Mixture of Bonferroni and Holm procedures with parallel gate-

keeping restrictions). The hypotheses in

*F*1 and*F*2 are tested using the Bonfer-roni procedure and the hypotheses in

*F*3 are tested using the Holm procedure.The mixture procedure is based on the Bonferroni mixing function with the

parallel gatekeeping restrictions defined in Section 6.1.

*•*Procedure 2 (Mixture of Bonferroni and Holm procedures with multiple-sequence

restrictions). This procedure is similar to Procedure 1 in the sense that it is also

a mixture of the Bonferroni procedures in

*F*1 and*F*2 and Holm procedure in*F*3 based on the Bonferroni mixing function. However, unlike Procedure 1, this

procedure uses a more general type of logical restrictions known as multiple-

sequence restrictions. A hypothesis in

*Fk*,*k*= 2*,*3, is tested if higher-levelhypotheses associated with the same dose are rejected, e.g.,

*H*7 is testable iff*H*1 and

*H*4 are rejected. More formally,

–

*Li*(*I*1)=0if*I*1 contains*i −*3 and*Li*(*I*1) = 1 otherwise,*i*= 4*,*5*,*6.–

*Li*(*I*1*∪I*2)=0if*I*1*∪I*2 contains*i−*3 or*i−*6 and*Li*(*I*1*∪I*2) = 1 otherwise,*i*= 7

*,*8

*,*9.

*•*Procedure 3 (Mixture of Dunnett procedures with multiple-sequence restric-

tions). This procedure is a mixture of the Dunnett procedures in

*F*1,*F*2 and*F*3 based on the Bonferroni mixing function and imposes multiple-sequence re-

strictions defined above.

Beginning with Procedures 1 and 2, adjusted

*p*-values can be computed using thealgorithm given in Section 6.1. This algorithm is based on a complete enumeration

of all non-empty intersections of the original nine hypotheses. A

*p*-value is computedfor each intersection and then the

*p*-values for the original hypotheses are found usingthe closure principle (see Section 2 for more details). As an illustration, consider the

intersection hypothesis corresponding to the index set

*I*=*{*1*,*3*,*5*,*6*,*7*,*8*,*9*}*, i.e.,*HI*=

*H*1

*∩ H*3

*∩ H*5

*∩ H*6

*∩ H*7

*∩ H*8

*∩ H*9

*.*

Assuming parallel gatekeeping restrictions (Procedure 1), one first needs to define

*p*-values for

*HI*1 ,

*HI*2 and

*HI*3 , where

*I*1 =

*{*1

*,*3

*}*,

*I*2 =

*{*5

*,*6

*}*and

*I*3 =

*{*7

*,*8

*,*9

*}*.

Using the raw

*p*-values displayed in Table 1, the*p*-values are computed based onthe Bonferroni and Holm procedures as shown below

*p*1(

*I*1) =

*n*1 min(

*p*1

*,p*2)=0

*.*015

*,*

*p*2(

*I*2) =

*n*2 min(

*p*5

*,p*6)=0

*.*039

*,*

*p*3(

*I*3) =

*|I*3

*|*min(

*p*7

*,p*8

*,p*9)=0

*.*018

*,*

where

*n*1 = 3,*n*2 = 3 and*|I*3*|*= 3. Using the Bonferroni mixing function, the*p*-valuefor

*HI*is given by*p*(

*I*) = min

(

*p*1(

*I*1)

*b*1

*,*

*p*2(

*I*2)

*b*2

*,*

*p*3(

*I*3)

*b*3

)

*,*

Table 2. Mixtures of three procedures with parallel gatekeeping restric-

tions (Procedure 1) and multiple-sequence restrictions (Procedure 2) in

the Type II diabetes clinical trial example. The asterisk identifies the

adjusted

*p*-values that are significant at the 0.05 level.Family

Null

Adjusted

*p*-valuehypothesis Procedure 1 Procedure 2

*F*1

*H*1

0

*.*015*∗*0

*.*015*∗**H*2

0

*.*033*∗*0

*.*033*∗**H*3

0

*.*0540

*.*054*F*2

*H*4

0

*.*041*∗*0

*.*041*∗**H*5

0

*.*0780

*.*078*H*6

0

*.*0540

*.*054*F*3

*H*7

0

*.*0540

*.*045*∗**H*8

0

*.*0540

*.*078*H*9

0

*.*0770

*.*077where

*b*1 = 1 and, to compute*bk*,*k*= 2*,*3, one needs to utilize the error ratefunction of the Bonferroni procedure. As shown in Section 6.1,

*ek*(*Ik*) =*α|Ik|/nk*or,equivalently,

*fk*(*Ik*) =*|Ik|/nk*,*k*= 1*,*2, and thus*b*2

=

*b*1(1*− f*1(*I*1)) = 1*−**|I*1

*|*

*n*1

=

1

3

*,*

*b*3

=

*b*2(1*− f*2(*I*2)) =1

3

(

1

*−**|I*2

*|*

*n*2

)

=

1

9

*.*

This immediately implies that

*p*(*I*)=0*.*015.Now consider the case of multiple-sequence restrictions (Procedure 2). The index

sets

*I*2 and*I*3 need to be modified to account for the logical restrictions. Note that*H*6 depends on

*H*3,

*H*7 depends on

*H*1,

*H*8 depends on

*H*5,

*H*9 depends on

*H*3 and

*H*6. Thus the modified index sets are given by

*I∗*

2 =

*{*5*}*and*I∗*3 =

*∅*. The next stepis to compute the

*p*-values for*HI*1 and*HI∗*2

,

*p*1(

*I*1) =

*n*1 min(

*p*1

*,p*2)=0

*.*015

*,*

*p*2(

*I∗*

2 ) =

*n*2*p*5 = 0*.*078*.*Lastly, the

*p*-value for*HI*is given by*p*(

*I*) = min

(

*p*1(

*I*1)

*b*1

*,*

*p*2(

*I∗*

2 )

*b*2

)

*,*

where

*bk*,*k*= 1*,*2, are defined above, i.e.,*b*1 = 1 and*b*2 = 1*/*3, and therefore*p*(

*I*)=0

*.*015.

Table 2 displays the raw

*p*-values for the nine hypotheses of interest along withthe adjusted

*p*-values produced by the two procedures. Procedure 1 rejects threehypotheses in this problem (

*H*1,*H*2 and*H*4) and Procedure 2 one more hypothesis(

*H*7). It is easy to verify that, as shown in Proposition 1, both procedures are con-sistent with the logical restrictions (note that the Bonferroni procedures in

*F*1 and*F*2 are consonant and thus there is no need to enforce the logical restrictions). Fur-

ther, as stated in Proposition 2, Procedures 1 and 2 are equivalent to the Bonferroni

procedure in

*F*1. Indeed, the adjusted*p*-values for the hypotheses in*F*1 are equal toBonferroni-adjusted

*p*-values (each raw*p*-value is multiplied by 3).Further, it is worth noting that Procedure 1 is based on parallel gatekeeping

restrictions and thus, as shown in Proposition 4, it has a stepwise representation.

This procedure is identical to a stepwise application of the Bonferroni procedures in

*F*1 and

*F*2 and Holm procedure in

*F*3 with appropriate adjustments of the significance

levels in

*F*2 and*F*3. For more information, see Dmitrienko, Tamhane and Wiens (2008,Section 6).

The calculation of adjusted

*p*-values for Procedure 3 is based on an algorithmsimilar to the one used in Section 6.1. The only change that needs to be made is that

the Bonferroni and Holm

*p*-values for intersection hypotheses need to be replaced bythe Dunnett

*p*-values defined in Section 6.2. To illustrate the process, select the sameintersection as above, i.e.,

*HI*=

*H*1

*∩ H*3

*∩ H*5

*∩ H*6

*∩ H*7

*∩ H*8

*∩ H*9

*.*

and consider the multiple-sequence restrictions. The modified index sets are

*I∗*2 =

*{*5*}*and

*I∗*3 =

*∅*. Given the sample size per arm (87 patients) and number of doses(3 doses), the Dunnett

*p*-values for*HI*1 and*HI∗*2

are computed using the one-sided

Dunnett distribution with 3 and 344 degrees of freedom. These

*p*-values are given by*p*1(

*I*1) = 1

*− G*2(max(

*t*1

*,t*3)) = 0

*.*0073

*,*

*p*2(

*I∗*

2 ) = 1

*− G*1(*t*5)=0*.*0336*,*where

*F*(*x*) is the cumulative distribution function of the Dunnett distribution. Fur-ther, this mixture is also based on the Bonferroni mixing function and thus

*b*1 = 1and

*b*2 = 1*/*3. Therefore,*p*(

*I*) = min

(

*p*1(

*I*1)

*b*1

*,*

*p*2(

*I∗*

2 )

*b*2

)

= 0

*.*0073*.*The adjusted

*p*-values produced by Procedure 3 are shown in Table 3. One cansee from this table that the mixture of Dunnett procedures rejects more hypotheses

Table 3. Mixture of three procedures with multiple-sequence restric-

tions (Procedure 3) in the Type II diabetes clinical trial example. The

asterisk identifies the adjusted

*p*-values that are significant at the 0.05level.

Family

Null

Adjusted

hypothesis

*p*-value

*F*1

*H*1

0

*.*007*∗**H*2

0

*.*015*∗**H*3

0

*.*023*∗**F*2

*H*4

0

*.*019*∗**H*5

0

*.*034*∗**H*6

0

*.*023*∗**F*3

*H*7

0

*.*023*∗**H*8

0

*.*034*∗**H*9

0

*.*064than a similar procedure based on the Bonferroni and Holm procedures (Procedure

2). Specifically, Procedure 3 rejects eight hypotheses whereas Procedure 2 rejects only

four hypotheses. This is a direct consequence of the fact that the Dunnett procedure

is uniformly more powerful than the Bonferroni procedure.

It is worth noting that the mixture of Dunnett procedures defined above can serve

as a computationally attractive alternative to the Dunnett-based parallel gatekeeping

procedure with logical restrictions introduced in Dmitrienko, Offen, Wang and Xiao

(2006). The parallel gatekeeping procedure requires the computation of a vector

of critical values for each intersection hypothesis in the closed family based on the

multivariate distribution of the associated test statistics. Even in the case of nine

hypotheses, the algorithm is computationally intensive (it involves the evaluation

of multivariate probabilities for up to six dimensions). By contrast, the mixture

procedure is based on regular Dunnett-adjusted

*p*-values that are combined acrossthe three families. This approach considerably simplifies the calculation of adjusted

*p*-values and leads to a relatively small reduction in the overall power compared to

the Dunnett-based parallel gatekeeping procedure.

References

Chen, X., Luo, X., Capizzi, T. (2005). The application of enhanced parallel gate-

keeping strategies.

*Statistics in Medicine*. 24, 1385–1397.Dmitrienko, A., Offen, W.W., Westfall, P.H. (2003). Gatekeeping strategies for

clinical trials that do not require all primary effects to be significant.

*Statistics**in Medicine*. 22, 2387–2400.

Dmitrienko, A., Offen, W., Wang, O., Xiao, D. (2006). Gatekeeping procedures in

dose-response clinical trials based on the Dunnett test.

*Pharmaceutical Statis-**tics*. 5, 19–28.

Dmitrienko, A., Wiens, B.L., Tamhane, A.C., Wang, X. (2007). Tree-structured

gatekeeping tests in clinical trials with hierarchically ordered multiple objec-

tives.

*Statistics in Medicine*. 26, 2465–2478.Dmitrienko, A., Tamhane, A., Liu, L., Wiens, B. (2008). A note on tree gatekeeping

procedures in clinical trials.

*Statistics in Medicine*. 27, 3446–3451.Dmitrienko, A., Tamhane, A., Wiens, B. (2008). General multistage gatekeeping

procedures.

*Biometrical Journal*. 50, 667–677.Everitt, B.S., Hand, D.J. (1981).

*Finite Mixture Distributions*. Chapman and Hall,London, New York.

Hochberg, Y., Tamhane, A.C. (1987).

*Multiple Comparison Procedures*. New York:John Wiley and Sons.

Kordzakhia, G., Dinh, P., Bai, S., Lawrence, J., Yang, P. (2008). Bonferroni-based

tree-structured gatekeeping testing procedures. Unpublished manuscript.

Marcus, R. Peritz, E., Gabriel, K.R. (1976). On closed testing procedures with

special reference to ordered analysis of variance.

*Biometrika*. 63, 655–660.Quan, H., Luo, X., Capizzi, T. (2005). Multiplicity adjustment for multiple end-

points in clinical trials with multiple doses of an active treatment.

*Statistics in**Medicine*. 24, 2151–2170.

Appendix

Proof of Proposition 1. Consider a hypothesis in

*Fk*+1,*k*= 1*,...,m −*1, say,*Hi*,

*i ∈ Nk*+1, and assume that

*Li*(

*A*1

*∪ ... ∪ Ak*) = 0. Let

*Is*=

*As*,

*s*= 1

*,...,k*,

*Ik*+1 =

*{i}*. Further, let

*J*=

*I*1

*∪ ... ∪ Ik*and

*I*=

*I*1

*∪ ... ∪ Ik*+1. Considering the

intersection hypothesis

*HI*, note that*Li*(*I*1*∪ ... ∪ Ik*) = 0 and thus*I∗*

*k*+1 =

*{i*:

*i ∈ Ik*+1 and

*Li*(

*I*1

*∪ ... ∪ Ik*)=1

*}*

is empty. Therefore,

*p*(

*I*) =

*mJ*(

*p*1(

*I*1)

*,p*2(

*I∗*

2 )

*,...,pk*(*I∗**k*))

*.*

Note that the mixture procedure

*T*accepts all hypotheses*Hj*,*j ∈ J*and*T*is con-sonant in

*F*1*,...,Fk*. Therefore,*p*(*J*)*> α*(if*p*(*J*) was less than or equal to*α*, then*T*would reject at least one hypothesis

*Hj*with

*i ∈ J*; however, all hypotheses

*Hj*,

*j ∈ J*, are rejected, which implies that

*p*(

*J*)

*> α*). Further,

*p*(

*I*) =

*p*(

*J*)

*> α*and the

index set

*I*contains*i*. Thus,*T*accepts*Hi*. The proof is complete.Proof of Proposition 2. Assume first that

*T*1 rejects*Hi*,*i ∈ N*1. This means that*p*1(

*I*1)

*≤ α*for any

*I*1

*⊆ N*1 if

*i ∈ I*1. Now consider any index set

*I ⊆ N*that contains

*i*. In general,

*I*=

*Ii*1

*∪...∪Iis*, where

*Iir ⊆ Nir*,

*r*= 1

*,...,s*,

*i*1 = 1 and

*s*= 1

*,...,m*,

and

*p*(

*I*) =

*mI*(

*pi*1 (

*Ii*1 )

*,...,pis*(

*Iis*))

*.*

By the definition of a mixture function,

*mI*(*xi*1*,...,xis*)*≤ α*if*xi*1*≤ α*and, since*xi*1 =

*p*1(

*I*1)

*≤ α*, we conclude that

*p*(

*I*) is no greater than

*α*and thus

*T*rejects

*Hi*.

Now assume that

*T*rejects*Hi*,*i ∈ N*1. In this case,*p*(*I*)*≤ α*for any*I ⊆ N*that contains

*i*, which immediately implies that*p*1(*I*1)*≤ α*for any*I*1*⊆ N*1 if*i ∈ I*1.Therefore,

*T*1 rejects*Hi*. The proof is complete.Proof of Proposition 3. Assume that all hypotheses in

*F*1*,...,Fk−*1 are rejected by*T*and

*Tk*rejects

*Hi*,

*i ∈ Nk*, i.e.,

*pk*(

*Ik*)

*≤ α*for any

*Ik ⊆ Nk*if

*i ∈ Ik*. Consider any

index set

*I ⊆ N*that contains*i*. If this set includes any indices from*N*1*∪...∪Nk−*1,*p*(

*I*) is no greater than

*α*since

*T*rejects all hypotheses in

*F*1

*,...,Fk−*1. If this set

does not include any indices from

*N*1*∪ ... ∪ Nk−*1, the*p*-value for*HI*is given by*p*(

*I*) =

*mI*(

*pi*1 (

*Ii*1 )

*,...,pis*(

*Iis*))

*,*

where

*I*=*Ii*1*∪ ... ∪ Iis*,*Iir ⊆ Nir*,*r*= 1*,...,s*,*i*1 =*k*and*s*= 1*,...,m − k*+ 1.As in Proposition 2, recall that

*mI*(*xi*1*,...,xis*)*≤ α*if*xi*1*≤ α*and*xi*1 =*pk*(*Ik*)*≤ α*.Thus

*p*(*I*)*≤ α*, which implies that*T*rejects*Hi*.On the other hand, if

*T*rejects*Hi*,*i ∈ Nk*, the arguments used in the proof ofProposition 2 can be applied to show that

*Tk*rejects*Hi*. The proof is complete.Proof of Proposition 4. The first statement follows from Proposition 2. Consider

the second statement and assume that

*T*1 rejects*Hk*,*k ∈ R*1, at the*α*level and*T*2rejects

*Hj*,*j ∈ N*2, at the level*α*2 =*α − e*1(*A*1). Here*R*1*⊆ N*1 is the index set ofhypotheses rejected in

*F*1. Considering any*I ⊆ N*with*j ∈ I*, let*I*1 =*I ∩ N*1 and*I*2 =

*I ∩ N*2. If

*I*1

*∩ R*1 =

*∅*, then

*p*1(

*I*1)

*≤ α*and thus

*p*(

*I*) = min

(

*p*1(

*I*1)

*,*

*p*2(

*I*2)

1

*− f*1(*I*1))

*≤ p*1(

*I*1)

*≤ α.*

Further, if

*I*1*∩ R*1 =*∅*, then*I*1*⊆ A*1 and, by the monotonicity of the error ratefunction,

*f*1(*I*1)*≤ f*1(*A*1). Since*T*2 rejects*Hj*at the level*α*2 =*α − e*1(*A*1),*p*2(

*I*2)

*≤ α − e*1(

*A*1) =

*α*(1

*− f*1(

*A*1))

*≤ α*(1

*− f*1(

*I*1))

and

*p*(

*I*) = min

(

*p*1(

*I*1)

*,*

*p*2(

*I*2)

1

*− f*1(*I*1))

*≤*

*p*2(

*I*2)

1

*− f*1(*I*1)*≤ α.*

This means that

*T*rejects*Hj*at the*α*level.Assume now that

*T*rejects*Hk*,*k ∈ R*1, and*Hj*,*j ∈ N*2, at the*α*level. Considerany

*I*2*⊆ N*2 such that*j ∈ I*2 and let*I*=*I*1*∪ I*2, where*I*1 =*A*1. Recall that*T*isequivalent to

*T*1 in*F*1 and thus*T*1 also rejects*Hk*,*k ∈ R*1. Since*T*1 is consonant, weconclude that

*p*1(*I*1)*> α*. On the other hand,*T*rejects*Hj*and thus*p*(

*I*) = min

(

*p*1(

*I*1)

*,*

*p*2(

*I*2)

1

*− f*1(*I*1))

*≤ α.*

This implies that

*p*2(

*I*2)

*≤ α*(1

*− f*1(

*I*1)) =

*α − e*1(

*I*1)

if

*I*2*⊆ N*2 and*j ∈ I*2. Therefore,*T*2 rejects*Hj*,*j ∈ N*2, at the level*α*2 =*α − e*1(*A*1).The proof is complete.

Proof of Proposition 5. Note first that the

*p*-values for intersection hypotheses in*H*1 and

*H*2 are given by

*p*1(

*I*1) = 2min

*i∈I*1

*pi, I*1

*⊆ N*1

*,*

*p*2(

*I*2) =

*|I*2

*|*min

*i∈I*2

*pi, I*2

*⊆ N*2

*,*

where

*pi*is the raw*p*-value for testing*Hi*,*i ∈ N*. Also, the error rate function forthe Bonferroni procedure is

*e*1(*I*1) =*|I*1*|α/*2, where*|I*1*|*is the cardinality of the set*I*1 (Dmitrienko, Tamhane and Wiens, 2008). Therefore, the

*p*-values for intersection

hypotheses in

*H*are given byCase 1. If

*I*1 =*{*1*,*2*}*,*p*(

*I*) = 2 min

*i∈I*1

*pi.*

Case 2. If

*I*1 =*{*1*}*,*p*(

*I*) = 2 min

(

*p*1

*,|I*2

*|*min

*i∈I*2

*pi*

)

*.*

Case 3. If

*I*1 =*{*2*}*,*p*(

*I*) = 2 min(

*p*2

*,pn*)

*.*

Case 4. If

*I*1 =*∅*,*p*(

*I*) =

*|I*2

*|*min

*i∈I*2

*pi.*

Recall now that

*H*1 is rejected and*H*2 is accepted. This means that all intersectionhypotheses in

*H*that include*H*1 are rejected. Therefore, to determine the conditionsunder which the mixture procedure rejects

*Hn*, it is sufficient to concentrate on theintersection hypotheses that include

*Hn*but exclude*H*1. It follows from Cases 3 and4 that

*Hn*is rejected iff*pn ≤ α/*2 (note that*p*2*> α/*2 since*H*2 is accepted) and the*k*smallest

*p*-values are significant at the Holm-adjusted significance levels in

*F*2, i.e.,

*q*(

*i*)

*≤ α/*(

*n − i −*1),

*i*= 1

*,...,k*. The proof is complete.