GROUP SEQUENTIAL EXPERIMENT PLANS A.M. POCOCK: AN
ADAPTATION FOR MEDICINE, WITH POWER TRIALS USING
COMPUTER SIMULATION.
E H Lehmann GP Haugesund, Norway
#
SUMMARY
Experiment plans of the sequential type are
attractive for ethical and economical reasons.
A simple way to design two-sample group sequential
experiments of the type advocated by Pocock is
described. The procedure is based on Student's t-test
and Wilcoxon's two-sample rank test and is validated
with regard to realised power, level of significance
and average realised sample size by computer simulation.
The purpose of the present text is to further more
frequent use of sequential experiment plans in
medicine by aiding users with limited statistical
training.
#
Dedication: To Knut Westlund, mentor and friend.
#
(150107)
PREFACE
During the 1960s, the Faculty of Medicine in Bergen suffered
from a lack of statisticians. Having received my M.D. licence
in 1965, and trying to do research, I had to act as my own
statistician, found statistics interesting, and picked up as
much reading and formal education as circumstances permitted.
Rather soon, I got the opportunity of teaching medical
statistics in Bergen to students as well as to colleagues.
Later, I did the same in Tromsoe and Trondheim. In this period,
it was necessary to encourage the idea of being one's own
statistician. However, my goal was to be a clinical physician,
and in 1983, I left the field of statistics to start my own
private general practice.
Attempts at practice research soon brought statistics to the
foreground again. The need to economize with respect to patients
called for the relatively new type of experiment design termed
'group sequential'. I felt that this type of design had much to
give to clinical research. I wanted to present it to fellow
physicians, in order to facilitate cooperation between physician
and statistician, and to aid those still having to manage their
statistics alone. Together with Hogne Sandvik in Bergen, I wrote
a computer program for this purpose.
A brief presentation group sequential plans, aimed at
clinicians, has been given by Pocock in his textbook about
clinical trials. However, the plan he recommends is prepeared
for bell-formed distributions, based as it is on the normal
distribution and Student's t-test. - In medical statistics, we
have to deal with all imaginable types of distribution. Hence,
for fixed sample experimentation, the rank test named after
Wilcoxon, Mann and Whitney (W-test) is a standard tool,
presented in every textbook of medical statistics.
How would group sequential experiments, performed with
non-normal distributions and using both those tests, stand with
regard to power and realised sample sizes? To satisfy my
curiosity, I performed computer simulations, using constructed
example distributions. The present text shows the results. It
text is written in the non-mathematical language of elementary
textbooks.
Curiosity satisfied, then why issue the present work?
Presumably, there are still clinicians wanting to do research on
their own terms (and not on the terms of the drug companies).
Perhaps this work could inspire somebody like that to try group
sequential design.
I want to thank my teachers, no one named, no one forgotten.
I am also grateful to my wife and best friend, Torunn Moksheim
Lehmann, dr. philos., for her patient assistance.
Haugesund, Norway, 2007
Egil Henrik Lehmann, G.P.
Cand. med. (Bergen, Norway)
Postgrad. Cert. Appl. Stat. (Sheffield Hallam Univ., U.K.)
Page 1
INTRODUCTION
Medicine cannot advance without experimenting with human beings
and research animals. This places an ethical burden on those
responsible: As a rule, one should make determined efforts not
to include more individuals than strictly necessary, especially
when the procedures are risky or painful.
One way to diminish the number of participants is by using
so-called sequential experiment plans. The basic idea is to
terminate the experiment as soon as a result which is
statistically significaant has been reached. By this method,
statistically significant results may be attained using an
average 30 to 40 per cent fewer individuals in the long run,
than necessary by ordinary fixed sample experimentations. Even
larger savings are possible in situation where advance
information is sparse.
The history and basic information needed to plan group
sequential experiments in the two-sample situation has been
summarised by Pocock ((1) and references). His tables were
derived from the normal distribution with known standard
deviation (SD). The first statistician to apply Student's
t-test to group sequential plans was Armitage (2). Use of the
two-sample rank test named after Wilcoxon, Mann and Whitley
('W-test') for this purpose was suggested by Pocock (3).
Therefore, the adaptation described below is based largely on
known information. The author merely wants to point out a safe
road that fellow experimenters can take if necessary, i.e. if a
sequential study must be planned without the assistence of a
statistician. Even if statistical assistance is available, being
able to present a preliminary plan may improve cooperation.
In general, sequential plans may be classified as group
sequential plans (Repeated test plans, RST) versus fully
sequential plans. In RST plans, the individuals (animals or
persons) are brought into the experiment a certain number at a
time, and when the total amount of data collected up to this
point has been tested, the next group is entered. As soon as a
statistically significant result emerges, the study is
terminated.
A maximal number of individuals that can be entered is
stipulated beforehand. If this number is reached, the study is
terminated regardless of result. How to plan this maximal
sample size will be shown below. - However, it goes without
saying that any medical experiment may be terminated at any time
for ethical og other good reasons.
Since the RST design constitutes an extension of methods already
well known by experimenters, namely the t-test and the
W-test, it is not demanding with regard to theory and analysis.
RST plans may be subdivided according to spacing of statistical
tests and levels of significance used. The simplest type is a
two-sample plan where samples are kept equally large. Likewise,
groups are (as far as possible) equally large, and statistical
tests are performed at a constant nominal level of significance
(Pnom). This is the type of plan advocated by Pocock (1).
Page 2
The pros and cons of sequential plans will not be discussed
here. Suffice it to say that such plans are useful only in
cases where the timespan for patient treatment and observation
is short compared to total study time. Moreover, sequential
plans may seem a logical choice in cases when individuals
present themselves 'naturally' spread over time, as patients
do in clinical work.
LIMITATIONS. TERMINILOGY.
The present text is limited to the simple two-sample type of
experiment, to continuous (measurement) data, and to Student's
t-test and Wilcoxon's two sample rank test (below, the
'W-test').
A 'group' contains an equal number of individuals for each of
the two samples. The test performed when data from one group has
been joined with the data previously collected is termed a
'look'. All the tests, except for the last one, are termed
'interim' tests. The actual number of tests is unknown a priori,
since the experiment is to be terminated if one of the interim
tests leads to the verdict of statistical significance. The
maximal number of tests is of course equal to the maximal number
of groups.
When an experiment (fixed sample or sequential) is planned, the
first step must be to define in numbers the main hypotesis to be
investigated. Thereafter, a decision must be made regarding the
Power and nominal level of significance (LOS, Pnom) of the
statistical test. (A brief reminder: Power is the probability of
correctly concluding 'significance'. Level of significance is
the risk of INCORRECTLY concluding 'significance'.
To decide sample and study size, a calculation which takes
targeted power and level of significance into account is
indispensable. Table 1 and Table 2 show the results of such
calculations, so that the user need not be burdened with them.
Sample size also depends on the expected difference between the
two samples with regard to mean values. This difference is
conventionally expressed in standard deviation units.
Nominal P values for test based group sequential plans of the
Pocock type are reproduced in Table 3 below. Armed with tables
1, 2 and 3, it should be easy for an experimenter with a
rudimentary statistical training (such as is given in medical
school) to plan group sequential experiments ad modum Pocock.
Page 3
However, the user needs to be certain that this procedure is
reliable with regard to realised (attained) power and realised
level of significance.
To find out how far this ideal can be achieved, validation
experiments have been carried out with example distributions.
The resulting tables are numbered 4 to 7 and have been labelled
according to distribution and wanted power and total level of
significance (LOS, Ptot).
MORE ABOUT DETERMINATION OF MAXIMAL SAMPLE SIZES
When planning fixed sample experiments, we want sample
sizes that fulfil our power demands. When it comes to sequential
plans, the whole idea is that sample size is a variable that
take a value dictated by the results. Still, we must feel
confident that targeted power is reached.
We see that there must be put an upper limit on total sample
size, or else one would run the risk of having to continue the
experiment forever. Our task is then to stipulate this upper
limit (maximal sample size) at a level that takes care of power.
Consequently, we need to establish maximal sample sizes for
conventionally desired total power levels of 80, 90 and 95 per
cent, and total LOS/Ptot (one- and two-sided tests) of 5 per
cent (0.05).
These sample sizes can be calculated from the sample sizes for
ordinary fixed group experiments. Recommended sample sizes for
two-group fixed sample experiments are shown by Table 1. All we
need to do is to increase the sample size found in Table 1 by a
certain percentage taken from Table 2. Table 2 was established
by the present author by simulations, i.e. trial and error.
The ordinary method of calculating sample sizes for fixed sample
experiments is based on the so-called 'standardised difference',
or 'shift'. The shift may be described as the a hypothetical a
priori stipulated difference between population mean values that
the experiment should reveal, divided on a hypothetical a priori
assessed distribution SD.
Consequently, at this point, if the distribution SD is poorly
known, it must be estimated. If experience indicates an
expected sample range, a reasonable thumb rule is to assume
distribution S.D. equal to 1/4 of the sample range (11). If the
distribution must be expected to be rectangular or U-shaped, the
SD should then be set equal to 1/3 of the range.
Sample size calculations are sensitive to assumed distribution
SD. If the latter is set too low, the calculated sample size
will be too small, which may ruin the experiment. Hence, if the
SD estimates at hand are vague, a high estimate must be used.
This might perhaps lead to larger sample sizes than necessary.
However, sample sizes that turn out to be larger than necessary
are not a problem in group sequential experimentation. In fact,
the very purpose of sequential plans is to let the data
determine realised sample size.
Page 4
Table 1 shows sample sizes for fixed size experiments. The
ordinary method of calculating sample sizes is texbook material
(see e.g. ref 4, pp 137-146). By simulation trials, I found that in
general, adding three units to sample sizes calculated by the
ordinary method produce sample sizes which take care of power in
small-sample non-normal experiments. This 'adjustment' has been
used in Table 1.
Table 2 shows percentage units to add to sample sizes for fixed
sample experiments in order to establish maximal sample sizes
for group sequential experiments. The table was established by
extensive simulation trials, i.e., by trial and error.
NOMINAL LEVELS OF SIGNIFICANCE
When planning a group sequential experiment, a series of
statistical tests (t-tests or W-tests) is envisaged. Each
test must then be performed on a nominal level of significance
(LOS, Pnom) which is stricter than wanted total LOS (Ptot).
Moreover, Pnom must be set to a value which ensures that
total, realised LOS, Ptot is close to the target value, usually
0.05.
Column 3 in Table 3 shows recommended nominal levels of
significance for group sequential tests (Pnom), as tabulated by
McPherson (5). In the fourth column are given Pnom values for
Ptot 0.025 (one sided), equivalent to 0.05 (two sided).
A two-sided test is just two one-sided tests combined into a
single calculation, each performed with half of the wanted LOS.
For the sake of simplicity, in this text we shall discuss only
one-sided tests. Whether to use one-sided or two-sided tests
sometimes lead to unnecessary discussion. The answer is
completely dependent on the hypothesis being tested. Using a
two-sided test when investigating a one-sided hypothesis is
wasteful, and thus against the basic idea of sequential plans.
A SUMMARY OF THE PROCEDURE
The planning of a group sequential experiment ad modum
Pocock may then be summarized in eight steps:
1) Stipulate preliminary null hypothesis (H0) and alternative
hypothesis (H1). For this, consider the range of possible
differences between means which hopefully will emerge from the
experiment. Proceed with the smallest mean difference that is of
practical importance.
2) Stipulate target: Power and LOS wanted.
3) Stipulate maximal number of groups (the same as maximal number of
tests, also termed 'looks').
4) Find in Table 1 the sample size needed for a fixed sample experiment
with the required shift, LOS and power.
Page 5
5) Calculate maximal sample size by adding relevant percentage of
units indicated by Table 2.
6) Calculate group size by dividing maximal sample size by
number of groups. The remainder of the division is included in
the last group.
7) Find in Table 3 the nominal LOS.
8) Select test: Student's two-sample t-test or the Wilcoxon's
two-sample rank test. As we know, the W-test is regarded as
superior when the distribution of observations is not
bell-shaped. Using both tests as a control of each other is
a good idea.
The experiment plan is now ready, as far as the statistical
design is concerned. Of course, there is much more to study
design than the statistical part, but that need not concern us
here.
For an example, envisage a situation where we want verify the
existence of a difference at least as large as 0.5 SD units.
In statistical terms, we want to test the null hypothesis (H0)
'standardised difference is zero' against the alternative
hypothesis (H1) 'standardised difference is 0.5 or more'. In
case H1 should be true, want the chance of a 'significant'
result (power) to be at least 0.9.
In case the real standardised difference should not be 0.5, but
rather zero, we want the risk of a falsely significant result to
be at most 0.05. (At present, the possibility of a negative
standardised difference or a standardised difference taking a
value between zero and 0.5 is ignored.)
A one-sided test is indicated here, since we have a one-sided
hypothesis. (If it were just as possible that the standardised
difference could take the value minus 0.5 as plus 0.5, we would
have needed a two sided test.)
Table 1 shows that a fixed sample experiment for this purpose
should have a sample size of 72 and hence a total study size of
144.
Let number of looks be e.g. five. Table 2 shows that a suitable
maximal sample size can be found by increasing 72 with 28 per
cent, which leads to a maximal sample size of 92. Maximal total
study size is then 184.
However, if H1 is true, the study will most likely be terminated
before maximum sample size is reached. The realised study size
might turn out to be in the region of 100 - 110 individuals.
This compares well with the study size of a similar fixed shape
experiment, which would be 72 x 2, or 144. -
Page 6
Now, if H0 should be true, the total number of individuals, 184,
might be entered before termination. In this case the group
sequential design would be uneconomical in comparison with the
fixed group design. There are ways to manage the latter
situation, ethically as well as economically, but that need not
concern us here.
Since 92 divided by 5 is not an integer, we cannot space the
looks so that all groups have exactly the same size. However,
if groups are set to 18 (per sample) and the last group to 20,
we have a satisfactory distribution of looks.
VALIDATION STUDY: STOCHASTIC SIMULATIONS
The purpose of the present validation study is to verify by
computer simulation that group sequential plans, constructed
according to the above principles, work well in practice.
That is, with realistic simulated data they produce results that
reach or pass the targets with respect to realised total power
and realised total LOS. They should also lead to a marked
reduction (on the average) of number of individuals involved, in
comparison with fixed sample experiments.
The data (the observations) are supposed to emerge from
empirical example distributions, constructed on basis of
experience to represent distribution types frequently
encountered in practice.
In addition to assuming H1: Shift = 0.5, we shall also test a
plan assuming H1: Shift = 1.0 Moreover, in addition to aiming at a
total LOS of 0.05, we shall study the result of aiming at a
total LOS of 0.025. Our power requirement (the chance we want to
have of revealing the alternative hypothesis if it is true) will
be 0.8, 0.9, or 0.95 (80, 90 or 95 per cent).
EXAMPLE DISTRIBUTIONS
The following empirical frequency distributions were used for the
stochastic simulations.
Class limits were 1,2,...M with M up to 13. Thus, there were M-1
intervals, the lower limit of the lowest interval was 1 and the upper
limit of the highest interval was M. The number of intervals was
kept low in order to save computer time. This implies approximation,
but experience indicated that using as few as 6 to 8 class intervals
should not lead to critical errors in the present context.
Below, interval numbers are given as (n) and frequencies as
proportions. The parameter values have been calculated from the
frequencies. Skewness was calculated from 3rd moment and kurtosis
(peakedness) from 4th moment using textbook formulas (5).
Page 7
Example A - Bell-shaped (truncated normal) distribution.
(1):.00298 (2):.00934 (3):.02784 (4):.06559 (5):.12098 (6):.17566
(7):.19742 (8):.17566 (9):.12098 (10):.06559 (11):.02784 (12):.00934
(13):.00298
Range: 13 Mean: 7.52 SD: 1.02 Skewness: 0.00 Kurtosis: 2.92
Example B - Unimodal, positive skew.
(1):.05 (2):.55 (3):.21 (4):.11 (5):.05 (6):.02 (7):.01
Range: 7 Mean: 3.16 SD: 1.13 Skewness: 2.06 Kurtosis: 5.08
Example C - Rectangular (uniform).
(1):.1667 (2):.1667 (3):.1667 (4):.1667 (5):.1667 (6):.1667
Range: 6 Mean: 7.41 SD: 1.71 Skewness: 0.00 Kurtosis: 1.73
Example D: J-shaped, positive skew.
(1):.5 (2):.25 (3):.1 (4):.07 (5):.04 (6):.03 (7):.01
Range: 7 Mean: 2.53 SD: 1.40 Skewness: 2.35 Kurtosis: 4.73
Example E: Peaked and long-tailed, symmetric.
(1):.01 (2):.01 (3):.01 (4):.02 (5):.05 (6):.15 (7):.5 (8):.15
(9):.05 (10):.02 (11):.01 (12):.01 (13):.01
Range: 13 Mean: 7.50 SD: 1.61 Skewness: 0.00 Kurtosis: 7.2
Example F: Low dome, symmetric.
(1):.11 (2):.14 (3):.16 (4):.18 (5):.16 (6):.14 (7):.11
Range: 7 Mean: 4.50 SD: 1.85 Skewness: 0.00 Kurtosis: 1.93
Example G: U-shaped, symmetric
(1):.25 (2):.15 (3):.1 (4):.1 (5):.15 (6):.25
Range: 6 Mean: 4.0 SD: 1.96 Skewness: 0.00 Kurtosis: 1.42
Example H: Bimodal, symmetric
(1):.02 (2):.03 (3):.05 (4):.3 (5):.04 (6):.04 (7):.04 (8):.04 (9):.04
(10):.3 (11):.05 (12):.03 (13):.02
Range: 13 Mean: 7.5 SD: 3.22 Skewness: 0.00 Kurtosis: 1.54
Example I: Dome-shaped with outliers
(1):.05 (2):.20 (3):.30 (4):.20 (5):.05 (6):.00 (7):.05 (8):.10
(9):.05
Range: 9 Mean: 4.5 SD: 2.21 Skewness: 0.99 Kurtosis: 2.85
The 'freeware' simulation programs SEQY and SEQZ were used (6).
These programs simulate group sequential plans with statistical
testing at equidistant intervals, i.e. the groups are increased
at each test by a constant number of individuals. The groups are
made equally large, except that the last group is enlarged to
achieve the total sample size stipulated. As recommended by
Pocock, the same Pnom value is used for all tests.
Page 8
Data are supposed to emerge by random sampling from two
populations having congruent frequency distributions, only
separated by a difference in location on the number line (a
theoretical shift). The hypotheses tested is of the same kind as
in the fixed sample case: The null hypothesis: Shift = zero.
Tables for H1: Shift = 0.5 or H1: Shift = 1.0 has been produced.
The ordinary textbook version of the t-test is used, assuming
normal distributions with equal variances. Wilcoxon's test is
performed in the form of a t-test on ranks, which is known to
render results very close to those obtained when using tables.
The simulation method used here can be classified technically as
a composition method, where the distributions are simulated as a
compound of uniform distributions (7).
The reasons for using this composition method and a set of
empirical distributions instead of mathematically defined
distributions are the following: 1) A frequency distribution is
easier for most experimenters to understand and to compare with
actual data than a mathematically defined distribution. 2) Using
the composition method, any distribution type encountered in
practice can be simulated without being mathematically defined.
3) I could find no standard set of mathematically defined
distributions suitable for the present purpose.
The simulations were performed on personal computers using the
operating system MS-DOS 5. The pseudo-random number generator
of Microsoft Basic (Compiler version 4.5) was used. The random
number generator was re-seeded for every experiment simulated,
using the timer (clock) of Microsoft Basic. The risk of
repetition of random number sequences is therefore supposed to
be small. No sign of malfunction of the random number generator
was noticed.
VALIDATION TABLES
For all the validation simulations, the null hypothesis (H0) is
'Shift (delta) = 0.0'. The alternative hypotesis (H1) is
'Shift = 0.5', alternatively 'Shift = 1.0', see table headings.
For all validation data, number of experiments simulated
(replications) was 10000 (ten thousand).
Power is proportion of experiments terminated by a test concluding
'significant'. The 'beta error' of statistical theory is equal to
1 minus power.
Rightmost column ('Saving'): Difference between mean realised
sample size and sampe size needed for a comparable fixed sample
experiment. The difference is presented as a percentage of the
latter.
Page 9
DESIGNATION OF TABLES
The validation tables are numbered 4,5,6 or 7.
The tables are numbered with respect to H1 and target
total LOS, as shown below.
++++++++++++++++++++++++++++++++
Tables Shift Target
num- according total LOS
bered to H1 (one sided)
--------------------------------
4 - 0.5 0.05
5 - 1.0 0.05
6 - 0.5 0.025
7 - 1.0 0.025
++++++++++++++++++++++++++++++++
The table number is followed by a letter showing the type of
distribution used, A to I, see list of distributions above.
The table designations end with the number 1 or 2, according to
statistical test used: Student's t-test 1, Wilcoxon's two-sample
test 2.
For example, Table 6C1 shows the results of assuming a shift of
0.5 SD units, a J-shaped distribution with right skew, and
using Student's t-test.
RESULTS (VALIDATI0N): REALISED POWER AND LOS
We see that when using the t-test, the target values of realised
power and LOS were comfortably reached for the bell-shaped
distribution (Example A). As expected, for this distribution,
the W-test was inferior with regard to power.
The t-test also performed well for the rectangular and the
dome-shaped distributions (Examples C and F) except that when
group sizes were small (number of looks high), there was a
tendency towards unconservative LOS (values on the high side).
The W-test was inferior with regard to power. Here too in some
cases, for small groups, unconservative LOS values could be seen.
Consequently, when working with rectangular or dome-shaped
distributions, the t-test should be preferred and group sizes
kept to five or more (for each sample).
For skewed distributions (Examples B and D) the W-test was far
more powerful than the t-test while targeted LOS was attained
reasonably well. Also for the peaked, longtailed distribution,
this was the case (Example E). However, with the latter
distribution, when using the W-test, in a few instances when
groups were small, a tendency to unconservative LOS values was
observed.
Page 10
Consequently, when working with skewed and with peaked, longtailed
distributions, the W-test is reliable. With the latter type of
distribution, group sizes should be set to five or more (from
each sample).
The bimodal symmetric and the U-shaped symmetric distributions
(examples G and H) showed different properties, depending on the
magnitude of the shift. With shift = 0.5 SD, the W-test was more
powerful than the t-test, but when shift was 1.0, the inverse
was the case. This difference seems large enough to demand
attention. A tendency to unconservative LOS could be seen in
some cases with both distributions, even when groups were
relatively large.
Therefore, when working with data having U-shaped or bimodal
distributions, shift must be taken into consideration when
selecting the best test of the two ones in question. The problem
whether unconservative LOS values can be tolerated must also be
addressed, and if not, how to avoid or adjust for this danger.
Dome-shaped distributions with outlying values often emerge in
medicine (example I). The W-test showed clearly better power as
well as better LOS characteristics than the t-test in this case.
10.
ABOUT MAXIMAL SAMPLE SIZES
In some of the present trials, far higher power was attained
than originally planned (Examples B, D, E, with the W-test).
Hence, in hindsight, in the latter cases, maximal sample sizes
could have been markedly reduced without inappropriate loss of
power. However, we notice that in those cases where maximal
sample sizes could have been reduced, the design led to
relatively early termination of the experiments, so that the
loss of economy stemming from unnecessary large maximal sample
sizes was compensated for.
It has been stated that there is little to gain by increasing
the number of groups (looks, maximal number of tests) beyond
five (1). This was verified by several of the present cases
(Examples A, C, F). However, in those cases where the maximal
sample size could have been reduced, a considerable decrease in
average sample size was seen when using a maximum of ten groups,
compared to using a maximum of five groups (Examples B, D, E).
Page 11
SAVING IN TERMS OF SAMPLE SIZES
The main argument for using sequential methods is that in the
long run, they lead to a reduction of realised sample sizes.
Clearly, the apparent number 'saved' is influenced by the way
baseline fixed sample sizes are calculated.
The percentage gained, in terms of baseline fixed sample sizes,
are given in the rightmost columns of tables A to I. These
savings are seen to range from 11 to 66 per cent, and were
larger when targeted power was high (.95 and .90) than when it
was low (.80).
In the present paper, the baseline fixed sample sizes were
calculated by the standard texbook method, with the modification
of adding three units. The purpose of this 'adjustment' is to
conserve power when experiments are small. Without this
adjustment, sample size savings would perhaps have turned out
to be a little lower.
FINAL REMARKS
From time to time, the opinion is voiced that for ethical and
economical reasons, sequential experiments should be used more
often. That can hardly be expected to happen before these
methods have been popularised to a higher degree than today, and
have been made better known among clinical physicians and
experimenters. The present text is an attempt in this
direction.
Experimenters need not only to know that sequential methods
exist, but also, to 'see how they work'. Computer simulation can
demonstrate their strengths, weaknesses and oddities in an
unsurpassed manner. Hopefully, the present text is convincing
in this respect.
Attaining satisfactory power and LOS is the 'sine qua non' of
any study design. Thus, the present text has been concentrated
on these questions. Once power and LOS have been ascertained,
other problems may be addressed.
Evaluation of attained power and LOS is not made easier by the
fact that there appears to be no general rule by which to judge
realised values. Clearly, a certain variation around and/or
systematic deviation from target values could in many cases be
tolerated. Tentatively, realised power might be regarded as
satisfactory even if realised value must be expected to be one
or two percentage units below target value. Likewise, realised
LOS values might be regarded as satisfactory even if they turn
out to be one half to one percentage units above target value.
Page 12
It has been pointed out that the Pocock design is rigid, and
that exact group sizes may be difficult to attain in practice.
One may ask how critical this is, or whether modest deviations
from the plan with regard to group sizes can be tolerated.
Simulation trials have shown that delays, leading to enlargement
of one of the groups with up to 1/4, with compensatory reduction
of the following group, does not critically disturb the
experiment. The same holds for reduction of a group with
subsequent compensatory enlargement of the next group. The
latter may even slightly improve the performance of the design
with respect to average sample size. (Detailed information
may be given by the author on demand).
Many workers prefer confidence intervals to statistical tests.
Confidence intervals and statistical tests are two ways of
performing the same procedure. One of these methods does not
exclude the other. Supplementing the tests with confidence
intervals appear to be especially useful in cases where,
during the experiment, the data suggest that the experiment
should be terminated prematurely.
For further practical and theoretical information, see the
article by Armitage (8).
CONCLUSION
The outlined approach to planning group sequential experiments
may be termed safe and robust with regard to P value (alpha
error) as well as to Power (1 minus beta error), regardless of
distribution type. The trials showed that, in comparison with
fixed sample experimentation, in many cases, a saving if terms
of participants, of 50 per cent or more, can be achieved.
It is the opinion of the author that using the present approach,
group sequential design ad modum Pocock could be understood and
realised by clinicians with limited statistical training. This
could lead to improvement in clinical research with regard to
time, economy and ethics.
XXXXX
Page 13
REFERENCES
1) Pocock SJ. Clinical trials, a practical approach.
Chichester: John Wiley and Sons, 1995: 147-155.
2) Armitage P. Sequential medical trials, 2nd edition. Oxford:
Blackwell, 1975.
3) Pocock SG. Group sequential methods in the design and
analysis of clinical trials. Biometrika 1977; 64, 2: 191-199.
4) Armitage P. Statistical Methods in Medical Research. Malden,
Massachusetts, Blackwell Science 2002.
5) Hartung J, Elpelt B, Kl"sener K-H. Statistik. Lehr- und
Handbuch der angewandten Statistik. Mnchen, R. Oldenbourg
Verlag 1995: 47-49.
6) Lehmann EH, Sandvik H. MEDSEQ - dataprogram for sekvensielle
forsk. Tidsskrift for Den norske l'geforening 1995; 115: 2304.
7) Ripley BD. Stochastic simulation. New York: John Wiley &
Sons, 1987: 63.
8) Armitage P. Interim analysis in clinical trials. Statistics
in medicine 1991, 10:925-937.
XXXX
XXXXXXX
*** TABLES ***
Table 1
SAMPLE SIZES NEEDED FOR FIXED SAMPLE EXPERIMENTS WITH
STIPULATED STANDARDISED DIFFERENCE, LEVEL OF SIGNIFICANCE
('LOS') AND POWER.
Levels of significance: .05 and .025 (one sided tests).*
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Differ- ----------------- Power ----------------
ence,SD 0.8 0.9 0.95
units .05 .025 .05 .025 .05 .025
---------------------------------------------------------------
0.1 1244 1581 1723 2115 2178 2616
0.2 313 397 433 531 547 656
0.25 202 255 278 341 351 421
0.3 141 178 194 238 245 293
0.4 81 102 110 135 139 166
0.5 53 66 72 87 90 108
0.6 37 47 51 62 63 76
0.7 27 35 38 46 47 56
0.75 25 31 34 41 42 49
0.8 22 28 30 36 37 44
0.9 18 22 24 29 30 35
1 15 19 20 24 25 29
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
* Sample sizes for LOS .05 (two sided test) are the same as for
LOS .025 (one sided test).
Table 2
GROUP SEQUENTIAL EXPERIMENT PLANS: KEY PERCENTAGES FOR
CALCULATING MAXIMAL SAMPLE SIZES.
Levels of significance: .05 and .025 (one sided tests).*
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Wan- Group sequential plans
ted ------------------------------------
power LOS Looks=2** Looks=3 Looks=5 Looks=10
----------------------------------------------------------
0.80 0.05 16 23 31 42
0.90 0.05 15 21 28 38
0.95 0.05 14 20 25 35
0.80 0.025 14 20 25 35
0.90 0.025 13 19 23 32
0.95 0.025 12 18 21 28
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
* Sample sizes for LOS .05 (two sided test) are the same as for
LOS .025 (one sided test).
** 'Looks' is maximal number of groups, here equal to
maximal number of tests (interim tests + final test)
Table 3
TWO-SAMPLE, GROUP SEQUENTIAL EXPERIMENT PLANS: NOMINAL
LEVELS OF SIGNIFICANCE (Pnom), SET EQUAL FOR INTERIM* TESTS
AND FINAL TEST, AIMING AT A REALISED TOTAL LEVEL OF
SIGNIFICANCE OF 0.05 (ONE SIDED OR TWO SIDED). After
Armitage and Klim Mc Pherson (6).
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
'Looks' Interim One sided Two sided
** tests tests tests
----------------------------------------------------------------
1 0 0.0500 0.025 One group, i.e. fixed
2 1 0.0296 0.0148 - \sample experiments
3 2 0.0221 0.01105 -
4 3 0.0183 0.00915 -
5 4 0.0159 0.00795 - Group
6 5 0.0142 0.0071 -- sequential
7 6 0.0130 0.0065 - experiments.
8 7 0.0120 0.0060 -
9 8 0.0113 0.00565 -
10 9 0.0107 0.00535 -
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
* The expression 'interim tests' is used for all tests on the
plan except for the last one.
** The term 'looks' is used for maximal number of test sessions,
interim tests plus final test, which is also the meximal number
of groups.
Table 4 - A - 1
VALIDITY TRIAL
Distribution: Example A - Bell-shaped (truncated normal).
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
----------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .808 53 53.0 .044 -
2 .0296 61 .810 61 45.9 .038 13
3 .0221 65 .810 42 44.1 .040 17
5 .0159 69 .811 39 43.3 .040 18
10 .0107 75 .810 42 43.7 .041 18
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .900 72 72.0 .043 -
2 .0296 83 .908 41 57.1 .040 21
3 .0221 87 .906 58 52.9 .039 27
5 .0159 92 .900 54 50.3 .036 30
10 .0107 99 .905 45 49.2 .040 32
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .954 90 90.0 .043 -
2 .0296 103 .957 51 65.9 .037 27
3 .0221 108 .956 36 59.3 .037 34
5 .0159 113 .951 44 55.0 .034 39
10 .0107 122 .958 48 53.0 .038 41
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - A - 2
VALIDITY TRIAL
Distribution: Example A - Bell-shaped (truncated normal).
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
----------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .782 53 53.0 .040 -
2 .0296 61 .787 61 46.2 .038 13
3 .0221 65 .787 42 45.0 .035 15
5 .0159 69 .789 39 44.0 .042 17
10 .0107 75 .793 42 44.1 .042 17
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .891 72 72.0 .040 -
2 .0296 83 .889 41 58.1 .038 19
3 .0221 87 .888 58 54.2 .040 25
5 .0159 92 .883 54 51.7 .040 28
10 .0107 99 .884 45 50.8 .042 29
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .939 90 90.0 .040 -
2 .0296 103 .947 51 67.8 .035 25
3 .0221 108 .950 72 61.0 .035 32
5 .0159 113 .939 44 56.6 .035 37
10 .0107 122 .942 48 55.1 .039 39
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - B - 1
VALIDITY TRIAL
Distribution: Example B - Unimodal, positive skew.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .797 53 53.0 .048 -
2 .0296 61 .804 61 45.8 .045 14
3 .0221 65 .810 42 44.1 .043 17
5 .0159 69 .803 39 43.0 .045 19
10 .0107 75 .815 42 42.8 .047 19
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .897 72 72.0 .049 -
2 .0296 83 .907 41 57.1 .049 21
3 .0221 87 .905 58 52.8 .045 27
5 .0159 92 .910 54 49.8 .049 31
10 .0107 99 .915 45 47.7 .046 34
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .950 90 90.0 .051 -
2 .0296 103 .954 51 66.2 .046 26
3 .0221 108 .957 36 59.4 .045 34
5 .0159 113 .949 44 55.1 .046 39
10 .0107 122 .956 48 52.0 .045 42
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - B - 2
VALIDITY TRIAL
Distribution: Example B - Unimodal, positive skew.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .939 53 53.0 .047 -
2 .0296 61 .943 30 39.6 .046 25
3 .0221 65 .951 42 36.0 .043 32
5 .0159 69 .950 26 33.3 .045 37
10 .0107 75 .955 28 31.6 .048 40
Target: Power >= .9, Ptot <= .05 one sided (<= .1 two sided)
1 .05 72 .981 72 72.0 .047 -
2 .0296 83 .984 41 48.6 .046 33
3 .0221 87 .985 29 42.1 .044 42
5 .0159 92 .987 36 37.5 .047 48
10 .0107 99 .988 27 34.5 .050 52
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .994 90 90.0 .048 -
2 .0296 103 .995 51 56.3 .049 37
3 .0221 108 .995 36 47.0 .047 48
5 .0159 113 .995 44 40.3 .048 51
10 .0107 122 .996 36 36.3 .047 60
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - C - 1
VALIDITY TRIAL
Distribution: Example C - Rectangular (uniform)
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .813 53 53.0 .051 -
2 .0296 61 .818 61 45.8 .051 14
3 .0221 65 .815 42 44.2 .050 17
5 .0159 69 .809 39 43.2 .052 18
10 .0107 75 .818 42 43.5 .045 18
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .904 72 72.0 .050 -
2 .0296 83 .909 41 56.9 .051 21
3 .0221 87 .912 58 52.6 .048 27
5 .0159 92 .908 54 50.1 .050 30
10 .0107 99 .905 45 49.0 .050 32
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .954 90 90.0 .052 -
2 .0296 103 .956 51 65.2 .051 28
3 .0221 108 .957 36 58.9 .047 36
5 .0159 113 .954 44 53.8 .049 40
10 .0107 122 .959 48 52.2 .047 42
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - C - 2
VALIDITY TRIAL
Distribution: Example C - Rectangular (uniform).
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .772 53 53.0 .047 -
2 .0296 61 .786 61 46.6 .047 12
3 .0221 65 .782 42 45.2 .053 15
5 .0159 69 .776 39 44.7 .050 16
10 .0107 75 .782 42 45.3 .054 15
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .877 72 72.0 .052 -
2 .0296 83 .881 41 58.2 .051 19
3 .0221 87 .878 58 54.6 .051 24
5 .0159 92 .875 54 52.4 .049 27
10 .0107 99 .885 45 50.9 .055 29
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .929 90 90.0 .051 -
2 .0296 103 .937 51 67.3 .047 25
3 .0221 108 .940 72 61.2 .046 32
5 .0159 113 .936 44 56.9 .045 37
10 .0107 122 .935 48 55.8 .045 38
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - D - 1
VALIDITY TRIAL
Distribution: Example D - J-shaped, positive skew.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .811 53 53.0 .049 -
2 .0296 61 .924 30 45.4 .050 14
3 .0221 65 .812 42 44.0 .042 17
5 .0159 69 .815 39 42.6 .048 20
10 .0107 75 .826 42 42.3 .046 20
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .907 72 72.0 .047 -
2 .0296 83 .911 41 57.2 .045 21
3 .0221 87 .906 58 52.6 .050 27
5 .0159 92 .911 54 49.8 .046 31
10 .0107 99 .911 46 47.5 .041 34
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .954 90 90.0 .047 -
2 .0296 103 .955 51 65.8 .044 27
3 .0221 108 .953 36 58.9 .045 35
5 .0159 113 .958 44 53.6 .045 40
10 .0107 122 .960 48 51.8 .044 42
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - D - 2
VALIDITY TRIAL
Distribution: Example D - J-shaped, positive skew.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .964 53 53.0 .050 -
2 .0296 61 .970 30 37.3 .049 30
3 .0221 65 .973 21 33.1 .047 38
5 .0159 69 .971 26 30.2 .047 43
10 .0107 75 .978 21 28.0 .048 47
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .994 72 72.0 .047 -
2 .0296 83 .993 41 46.3 .048 36
3 .0221 87 .994 29 39.0 .044 46
5 .0159 92 .994 36 33.7 .045 53
10 .0107 99 .995 27 30.4 .047 58
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .998 90 90.0 .045 -
2 .0296 103 .998 51 54.5 .045 36
3 .0221 108 .998 36 43.7 .046 46
5 .0159 113 .998 22 36.4 .044 60
10 .0107 122 .999 24 32.5 .048 64
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - E - 1
VALIDITY TRIAL
Distribution: Example E - Peaked and long-tailed, symmetric.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .818 53 53.0 .048 -
2 .0296 61 .819 61 45.6 .047 14
3 .0221 65 .827 42 43.3 .045 18
5 .0159 69 .814 39 41.7 .045 21
10 .0107 75 .822 35 41.3 .040 22
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .904 72 72.0 .048 -
2 .0296 83 .909 41 56.9 .046 21
3 .0221 87 .910 58 51.8 .043 28
5 .0159 92 .914 36 48.4 .040 33
10 .0107 99 .916 36 46.3 .040 36
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .952 90 90.0 .046 -
2 .0296 103 .954 51 65.5 .043 27
3 .0221 108 .953 36 58.7 .042 35
5 .0159 113 .955 44 52.9 .045 41
10 .0107 122 .956 48 50.8 .040 44
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - E - 2
VALIDITY TRIAL
Distribution: Example E - Peaked and long-tailed, symmetric.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .973 53 53.0 .048 -
2 .0296 61 .982 30 36.3 .048 32
3 .0221 65 .983 21 31.3 .047 41
5 .0159 69 .986 26 28.4 .051 46
10 .0107 75 .985 21 26.1 .050 51
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .994 72 72.0 .056 -
2 .0296 83 .995 41 44.9 .050 38
3 .0221 87 .996 29 37.2 .046 48
5 .0159 92 .997 36 31.9 .043 56
10 .0107 99 .998 27 28.2 .045 61
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .999 90 90.0 .048 -
2 .0296 103 .999 51 53.3 .047 41
3 .0221 108 .999 36 41.9 .046 53
5 .0159 113 .999 22 34.5 .049 62
10 .0107 122 1.000 24 30.3 .043 66
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - F - 1
VALIDITY TRIAL
Distribution: Example F - Low dome, symmetric.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .805 53 53.0 .049 -
2 .0296 61 .809 61 45.7 .051 14
3 .0221 65 .812 42 44.1 .044 17
5 .0159 69 .799 39 43.3 .050 18
10 .0107 75 .808 42 44.1 .050 17
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .898 72 72.0 .050 -
2 .0296 83 .908 41 57.2 .049 21
3 .0221 87 .904 58 52.5 .049 27
5 .0159 92 .908 54 50.1 .051 30
10 .0107 99 .905 45 49.0 .048 32
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .944 90 90.0 .058 -
2 .0296 103 .953 51 65.6 .052 27
3 .0221 108 .951 36 59.7 .049 34
5 .0159 113 .950 44 54.8 .052 39
10 .0107 122 .952 48 53.4 .047 41
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - F - 2
VALIDITY TRIAL
Distribution: Example F - Low dome, symmetric
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .754 53 53.0 .051 -
2 .0296 61 .765 61 47.0 .048 11
3 .0221 65 .755 42 46.0 .051 13
5 .0159 69 .755 52 45.4 .053 14
10 .0107 75 .768 42 45.8 .050 14
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .861 72 72.0 .051 -
2 .0296 83 .868 41 59.2 .049 18
3 .0221 87 .865 58 55.7 .050 23
5 .0159 92 .858 54 53.5 .049 26
10 .0107 99 .863 45 52.7 .048 27
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .921 90 90.0 .053 -
2 .0296 103 .926 51 68.6 .047 24
3 .0221 108 .922 72 63.2 .052 30
5 .0159 113 .922 44 58.6 .050 35
10 .0107 122 .936 48 56.6 .049 37
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - G - 1
VALIDITY TRIAL
Distribution: Example G U-shaped, symmetric
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .812 53 53.0 .048 -
2 .0296 61 .817 61 46.0 .053 13
3 .0221 65 .811 42 44.5 .053 16
5 .0159 69 .811 39 43.0 .051 19
10 .0107 75 .814 42 44.1 .055 17
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .903 72 72.0 .048 -
2 .0296 83 .909 41 57.1 .051 21
3 .0221 87 .910 58 52.8 .049 27
5 .0159 92 .907 54 50.0 .045 31
10 .0107 99 .910 45 49.4 .053 31
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .950 90 90.0 .052 -
2 .0296 103 .956 51 66.0 .049 27
3 .0221 108 .958 36 59.1 .045 34
5 .0159 113 .954 44 54.5 .049 39
10 .0107 122 .961 48 52.4 .049 42
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - G - 2
VALIDITY TRIAL
Distribution: Example G U-shaped, symmetric
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .855 53 53.0 .050 -
2 .0296 61 .874 30 43.4 .048 18
3 .0221 65 .870 42 40.9 .052 23
5 .0159 69 .868 39 39.8 .048 25
10 .0107 75 .872 35 39.5 .049 25
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .934 72 72.0 .053 -
2 .0296 83 .939 41 53.8 .049 25
3 .0221 87 .943 58 49.0 .051 32
5 .0159 92 .944 36 45.8 .050 36
10 .0107 99 .950 36 43.1 .053 40
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .972 90 90.0 .047 -
2 .0296 103 .977 51 62.4 .052 31
3 .0221 108 .977 36 54.6 .048 39
5 .0159 113 .978 44 49.5 .049 45
10 .0107 122 .980 36 46.7 .049 48
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - H - 1
VALIDITY TRIAL
Distribution: Example H - Bimodal symmetric
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .821 53 53.0 .057 -
2 .0296 61 .830 30 45.2 .051 15
3 .0221 65 .834 42 43.4 .055 18
5 .0159 69 .829 39 42.1 .062 21
10 .0107 75 .832 42 43.2 .058 18
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .917 72 72.0 .065 -
2 .0296 83 .923 41 56.5 .061 22
3 .0221 87 .925 58 51.7 .053 28
5 .0159 92 .921 54 49.4 .054 31
10 .0107 99 .920 45 47.7 .060 34
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .962 90 90.0 .055 -
2 .0296 103 .967 51 64.5 .055 28
3 .0221 108 .966 36 58.0 .056 36
5 .0159 113 .964 44 53.3 .055 41
10 .0107 122 .965 48 51.5 .054 43
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - H - 2
VALIDITY TRIAL
Distribution: Example H - Bimodal symmetric
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .911 53 53.0 .061 -
2 .0296 61 .926 30 41.0 .059 23
3 .0221 65 .928 42 37.2 .058 30
5 .0159 69 .926 26 35.0 .065 34
10 .0107 75 .937 28 33.7 .064 36
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .974 72 72.0 .060 -
2 .0296 83 .978 41 50.1 .058 30
3 .0221 87 .977 29 43.9 .058 39
5 .0159 92 .978 36 39.4 .055 45
10 .0107 99 .982 36 37.0 .062 49
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .992 90 90.0 .054 -
2 .0296 103 .994 51 58.5 .061 35
3 .0221 108 .993 36 49.0 .059 46
5 .0159 113 .995 44 42.3 .059 53
10 .0107 122 .994 36 38.7 .059 57
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - I - 1
VALIDITY TRIAL
Distribution: Example I - Dome-shaped with outliers
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .810 53 53.0 .058 -
2 .0296 61 .815 30 45.4 .057 14
3 .0221 65 .818 42 43.6 .055 18
5 .0159 69 .816 39 42.2 .059 20
10 .0107 75 .818 42 42.6 .053 20
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .906 72 72.0 .057 -
2 .0296 83 .906 41 57.1 .055 21
3 .0221 87 .909 58 52.2 .059 27
5 .0159 92 .912 36 49.8 .058 31
10 .0107 99 .907 45 48.1 .060 33
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .956 90 90.0 .059 -
2 .0296 103 .977 61 73.5 .060 18
3 .0221 108 .955 36 58.7 .060 35
5 .0159 113 .952 44 54.4 .057 40
10 .0107 122 .954 48 52.1 .053 42
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 4 - I - 2
VALIDITY TRIAL
Distribution: Example I - Dome-shaped with outliers
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 53 .960 53 53.0 .052 -
2 .0296 61 .964 30 38.1 .049 28
3 .0221 65 .970 21 33.7 .051 36
5 .0159 69 .969 26 31.0 .052 42
10 .0107 75 .972 28 29.0 .055 45
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 72 .991 72 72.0 .054 -
2 .0296 83 .993 41 46.6 .049 35
3 .0221 87 .994 29 40.1 .049 44
5 .0159 92 .995 36 34.9 .055 52
10 .0107 99 .997 27 31.4 .052 56
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 90 .998 90 90.0 .052 -
2 .0296 103 .999 51 55.0 .055 39
3 .0221 108 .998 36 44.8 .052 50
5 .0159 113 .998 22 37.7 .049 61
10 .0107 122 .999 24 33.4 .054 63
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - A - 1
VALIDITY TRIAL
Distribution: Example A - Bell-shaped (truncated normal)
Test: Student's t-test (one sided)
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .839 15 15.0 .044 -
2 .0296 17 .835 17 12.7 .047 15
3 .0221 18 .840 12 12.2 .042 19
5 .0159 20 .854 12 12.3 .042 18
10 .0107 21 .842 12 12.3 .047 18
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .927 20 20.0 .045 -
2 .0296 23 .933 11 15.5 .049 23
3 .0221 24 .929 16 14.4 .041 28
5 .0159 26 .934 15 13.9 .043 31
10 .0107 28 .928 12 13.9 .044 31
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .969 25 25.0 .040 -
2 .0296 29 .974 14 18.0 .044 28
3 .0221 30 .969 10 16.1 .042 34
5 .0159 31 .970 12 14.7 .041 41
10 .0107 34 .970 12 14.3 .043 43
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - A - 2
VALIDITY TRIAL
Distribution: Example A - Bell-shaped (truncated normal)
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .808 15 15.0 .041 -
2 .0296 17 .817 17 12.7 .045 15
3 .0221 18 .809 12 12.6 .042 16
5 .0159 20 .832 12 12.0 .053 20
10 .0107 21 .821 12 12.4 .042 17
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .907 20 20.0 .043 -
2 .0296 23 .920 11 15.5 .044 23
3 .0221 24 .914 16 14.6 .045 27
5 .0159 26 .921 15 14.1 .047 30
10 .0107 28 .922 12 13.9 .046 31
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .950 25 25.0 .040 -
2 .0296 29 .958 14 18.2 .046 27
3 .0221 30 .961 20 16.6 .042 34
5 .0159 31 .959 12 14.9 .048 40
10 .0107 34 .966 12 13.1 .049 48
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - B - 1
VALIDITY TRIAL
Distribution: Example B - Unimodal, positive skew
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .823 15 15.0 .049 -
2 .0296 17 .830 8 12.4 .048 17
3 .0221 18 .835 12 11.9 .043 21
5 .0159 20 .850 12 11.8 .045 21
10 .0107 21 .845 10 11.6 .046 23
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .914 20 20.0 .049 -
2 .0296 23 .924 11 15.4 .047 23
3 .0221 24 .915 16 14.2 .046 29
5 .0159 26 .926 10 13.3 .043 34
10 .0107 28 .924 10 13.1 .048 35
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .957 25 25.0 .048 -
2 .0296 29 .963 14 18.2 .048 27
3 .0221 30 .964 10 16.0 .049 36
5 .0159 31 .960 12 14.4 .045 42
10 .0107 34 .965 12 13.6 .045 46
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - B - 2
VALIDITY TRIAL
Distribution: Example B - Unimodal, positive skew
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .897 15 15.0 .046 -
2 .0296 17 .902 8 11.6 .057 23
3 .0221 18 .906 12 11.3 .045 25
5 .0159 20 .905 8 10.2 .061 32
10 .0107 21 .916 10 10.5 .048 30
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .959 20 20.0 .051 -
2 .0296 23 .965 11 14.2 .051 29
3 .0221 24 .966 8 12.7 .050 37
5 .0159 26 .974 10 11.9 .048 41
10 .0107 28 .971 8 11.3 .051 43
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .985 25 25.0 .048 -
2 .0296 29 .988 14 16.6 .050 34
3 .0221 30 .987 10 14.5 .047 42
5 .0159 31 .988 12 12.4 .050 50
10 .0107 34 .990 9 10.3 .081 59
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - C - 1
VALIDITY TRIAL
Distribution: Example C - Rectangular (uniform)
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .849 15 15.0 .049 -
2 .0296 17 .844 17 12.8 .055 15
3 .0221 18 .837 12 12.4 .052 17
5 .0159 20 .847 12 12.6 .054 16
10 .0107 21 .842 12 12.5 .059 17
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .925 20 20.0 .051 -
2 .0296 23 .929 11 15.6 .052 22
3 .0221 24 .925 16 14.6 .054 27
5 .0159 26 .930 15 14.2 .054 29
10 .0107 28 .930 12 14.2 .069 29
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .969 25 25.0 .051 -
2 .0296 29 .973 14 17.9 .045 28
3 .0221 30 .970 10 16.3 .049 35
5 .0159 31 .970 12 15.0 .055 40
10 .0107 34 .969 12 14.6 .057 42
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - C - 2
VALIDITY TRIAL
Distribution: Example 4 - Rectangular
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .783 15 15.0 .045 -
2 .0296 17 .777 17 13.0 .052 13
3 .0221 18 .774 12 12.9 .047 14
5 .0159 20 .797 12 12.7 .059 15
10 .0107 21 .797 12 13.0 .051 13
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .884 20 20.0 .050 -
2 .0296 23 .889 11 16.0 .046 20
3 .0221 24 .890 16 15.0 .053 25
5 .0159 26 .890 15 14.9 .050 25
10 .0107 28 .882 12 15.2 .048 24
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .940 25 25.0 .050 -
2 .0296 29 .944 14 18.8 .052 25
3 .0221 30 .940 20 17.4 .050 30
5 .0159 31 .937 12 15.8 .058 37
10 .0107 34 .946 12 14.2 .091 43
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - D - 1
VALIDITY TRIAL
Distribution: Example D - J-shaped right skew
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .829 15 15.0 .048 -
2 .0296 17 .841 8 12.5 .048 17
3 .0221 18 .849 12 11.7 .044 22
5 .0159 20 .858 12 11.6 .045 23
10 .0107 21 .859 10 11.2 .044 25
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .919 20 20.0 .049 -
2 .0296 23 .929 11 15.3 .049 23
3 .0221 24 .925 16 14.0 .044 30
5 .0159 26 .933 10 13.0 .042 35
10 .0107 28 .931 10 12.6 .044 37
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .963 25 25.0 .049 -
2 .0296 29 .965 14 17.9 .048 28
3 .0221 30 .966 10 15.9 .046 36
5 .0159 31 .963 12 14.2 .046 43
10 .0107 34 .969 12 13.3 .044 47
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - D - 2
VALIDITY TRIAL
Distribution: Example D - J-shaped right skew
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .920 15 15.0 .044 -
2 .0296 17 .925 8 11.2 .048 25
3 .0221 18 .928 12 10.8 .049 28
5 .0159 20 .941 8 9.7 .065 35
10 .0107 21 .942 8 9.8 .050 35
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .972 20 20.0 .053 -
2 .0296 23 .974 11 13.7 .053 32
3 .0221 24 .978 8 12.2 .050 39
5 .0159 26 .980 10 11.2 .050 44
10 .0107 28 .983 8 10.5 .051 47
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .990 25 25.0 .050 -
2 .0296 29 .994 14 16.1 .053 36
3 .0221 30 .993 10 13.7 .048 45
5 .0159 31 .993 12 11.8 .054 53
10 .0107 34 .995 9 9.4 .055 62
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - E - 1
VALIDITY TRIAL
Distribution: Example E - Peaked and long-tailed, symmetric.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .842 15 15.0 .051 -
2 .0296 17 .851 8 12.0 .043 20
3 .0221 18 .856 12 11.3 .035 25
5 .0159 20 .870 8 11.0 .035 27
10 .0107 21 .872 8 10.6 .037 29
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .913 20 20.0 .051 -
2 .0296 23 .925 11 15.2 .046 24
3 .0221 24 .928 8 13.6 .039 32
5 .0159 26 .935 10 12.5 .040 38
10 .0107 28 .936 8 11.9 .035 41
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .956 25 25.0 .054 -
2 .0296 29 .967 14 17.8 .044 29
3 .0221 30 .964 10 15.6 .042 38
5 .0159 31 .963 12 13.7 .038 45
10 .0107 34 .967 9 12.6 .037 50
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - E - 2
VALIDITY TRIAL
Distribution: Example E - Peaked and longtailed, symmetric.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .946 15 15.0 .047 -
2 .0296 17 .952 8 10.9 .047 27
3 .0221 18 .950 12 10.4 .042 31
5 .0159 20 .963 8 8.9 .057 41
10 .0107 21 .962 8 9.1 .041 39
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .985 20 20.0 .050 -
2 .0296 23 .988 11 13.1 .049 35
3 .0221 24 .986 8 11.5 .048 43
5 .0159 26 .989 10 10.4 .049 48
10 .0107 28 .990 8 9.5 .052 53
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .995 25 25.0 .053 -
2 .0296 29 .997 14 15.6 .047 38
3 .0221 30 .996 10 13.2 .044 47
5 .0159 31 .996 12 10.9 .049 56
10 .0107 34 .998 6 8.5 .082 66
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - F - 1
VALIDITY TRIAL
Distribution: Example F - Low dome, symmetric.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .836 15 15.0 .049 -
2 .0296 17 .838 17 12.8 .055 15
3 .0221 18 .830 12 12.5 .052 17
5 .0159 20 .844 12 12.6 .050 16
10 .0107 21 .830 12 12.6 .056 16
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .919 20 20.0 .048 -
2 .0296 23 .928 11 15.6 .049 22
3 .0221 24 .921 16 14.6 .051 27
5 .0159 26 .929 15 14.1 .050 29
10 .0107 28 .921 12 14.3 .064 29
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .964 25 25.0 .052 -
2 .0296 29 .971 14 18.1 .053 28
3 .0221 30 .970 10 16.3 .053 35
5 .0159 31 .965 12 15.2 .054 39
10 .0107 34 .967 12 14.8 .053 41
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - F - 2
VALIDITY TRIAL
Distribution: Example F - Low dome, symmetric
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delcta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .784 15 15.0 .050 -
2 .0296 17 .776 17 13.0 .054 13
3 .0221 18 .773 12 12.9 .046 14
5 .0159 20 .803 12 12.5 .058 17
10 .0107 21 .790 12 13.0 .050 13
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .879 20 20.0 .054 -
2 .0296 23 .894 11 16.0 .049 20
3 .0221 24 .881 16 15.1 .050 25
5 .0159 26 .893 15 14.8 .051 26
10 .0107 28 .886 12 15.0 .053 25
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .931 25 25.0 .052 -
2 .0296 29 .948 14 18.7 .045 25
3 .0221 30 .942 20 17.3 .050 31
5 .0159 31 .941 12 15.6 .056 38
10 .0107 34 .945 12 14.3 .083 43
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - G - 1
VALIDITY TRIAL
Distribution: Example G - U-shaped symmetric.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .838 15 15.0 .052 -
2 .0296 17 .835 17 12.9 .052 14
3 .0221 18 .833 12 12.6 .051 16
5 .0159 20 .856 12 12.6 .055 16
10 .0107 21 .841 12 12.4 .066 17
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .929 20 20.0 .047 -
2 .0296 23 .931 11 15.6 .053 22
3 .0221 24 .923 16 14.7 .045 27
5 .0159 26 .931 15 14.2 .056 29
10 .0107 28 .925 12 14.3 .069 29
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .968 25 25.0 .050 -
2 .0296 29 .974 14 18.0 .048 28
3 .0221 30 .972 10 16.2 .051 35
5 .0159 31 .966 12 15.2 .054 39
10 .0107 34 .969 12 14.7 .056 41
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - G - 2
VALIDITY TRIAL
Distribution: Example G - U-shaped symmetric.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .785 15 15.0 .044 -
2 .0296 17 .793 17 12.9 .052 14
3 .0221 18 .783 12 12.9 .048 14
5 .0159 20 .814 12 12.4 .060 17
10 .0107 21 .800 12 12.9 .049 14
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .897 20 20.0 .049 -
2 .0296 23 .896 11 15.8 .050 21
3 .0221 24 .893 16 14.7 .051 27
5 .0159 26 .896 15 14.7 .049 27
10 .0107 28 .893 12 14.8 .053 26
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .941 25 25.0 .052 -
2 .0296 29 .947 14 18.6 .052 26
3 .0221 30 .946 20 17.0 .047 32
5 .0159 31 .941 12 15.5 .053 38
10 .0107 34 .948 12 14.2 .082 43
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - H - 1
VALIDITY TRIAL
Distribution: Example H - Bimodal symmetric
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .851 15 15.0 .048 -
2 .0296 17 .845 17 12.8 .050 15
3 .0221 18 .844 12 12.4 .051 17
5 .0159 20 .851 12 12.6 .052 16
10 .0107 21 .844 12 12.3 .069 18
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .930 20 20.0 .050 -
2 .0296 23 .930 11 15.6 .049 22
3 .0221 24 .927 16 14.6 .051 27
5 .0159 26 .935 15 14.1 .051 29
10 .0107 28 .935 12 13.9 .076 31
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .967 25 25.0 .047 -
2 .0296 29 .974 14 18.0 .049 28
3 .0221 30 .971 10 16.3 .049 35
5 .0159 31 .967 12 15.2 .051 39
10 .0107 34 .968 12 14.6 .057 42
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - H - 2
VALIDITY TRIAL
Distribution: Example H - Bimodal symmetric
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .782 15 15.0 .046 -
2 .0296 17 .782 17 13.0 .048 13
3 .0221 18 .781 12 12.8 .048 15
5 .0159 20 .802 12 12.5 .062 17
10 .0107 21 .792 12 12.9 .045 14
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .884 20 20.0 .050 -
2 .0296 23 .891 11 15.9 .050 21
3 .0221 24 .894 16 14.8 .047 26
5 .0159 26 .893 15 14.8 .047 26
10 .0107 28 .891 12 14.6 .052 27
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .938 25 25.0 .051 -
2 .0296 29 .950 14 18.6 .051 26
3 .0221 30 .940 20 17.1 .050 32
5 .0159 31 .940 12 15.7 .054 21
10 .0107 34 .949 12 14.0 .083 44
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - I - 1
VALIDITY TRIAL
Distribution: Example I - Dome-shaped with outliers
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .847 15 15.0 .051 -
2 .0296 17 .849 17 12.6 .053 16
3 .0221 18 .838 12 12.1 .045 19
5 .0159 20 .860 12 11.9 .047 21
10 .0107 21 .853 10 11.7 .051 22
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .924 20 20.0 .056 -
2 .0296 23 .929 11 15.4 .051 23
3 .0221 24 .925 16 14.3 .047 29
5 .0159 26 .935 10 13.3 .051 33
10 .0107 28 .933 19 13.0 .057 35
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .965 25 25.0 .052 -
2 .0296 29 .969 14 17.9 .048 28
3 .0221 30 .968 10 16.1 .048 36
5 .0159 31 .965 12 14.5 .048 42
10 .0107 34 .970 12 13.7 .047 45
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 5 - I - 2
VALIDITY TRIAL
Distribution: Example I - Dome-shaped with outliers
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .05 one sided (.1 two sided)
1 .05 15 .907 15 15.0 .047 -
2 .0296 17 .909 8 11.4 .056 24
3 .0221 18 .919 12 11.0 .049 27
5 .0159 20 .935 8 9.9 .065 34
10 .0107 21 .926 8 10.1 .051 33
Target: Power >= .9, Ptot <= .05 one sided (.1 two sided)
1 .05 20 .961 20 20.0 .053 -
2 .0296 23 .970 11 13.9 .050 31
3 .0221 24 .969 8 12.5 .056 37
5 .0159 26 .975 10 11.6 .051 42
10 .0107 28 .975 8 10.9 .050 45
Target: Power >= .95, Ptot <= .05 one sided (.1 two sided)
1 .05 25 .986 25 25.0 .050 -
2 .0296 29 .990 14 16.4 .052 34
3 .0221 30 .990 10 14.2 .049 43
5 .0159 31 .989 12 12.1 .052 52
10 .0107 34 .993 9 9.8 .082 61
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - A - 1
VALIDITY TRIAL
Distribution: Example A - Bell-shaped (truncated normal).
Test: Student's t-test (one sided)
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .795 66 66.0 .023 -
2 .0148 75 .808 75 57.6 .021 13
3 .01105 79 .807 52 55.6 .017 16
5 .00795 83 .806 48 54.2 .021 18
10 .00535 89 .798 56 54.7 .021 17
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .899 87 87.0 .019 -
2 .0148 98 .896 49 69.6 .019 20
3 .01105 104 .909 68 65.1 .019 25
5 .00795 107 .900 63 61.6 .018 29
10 .00535 115 .907 55 60.2 .020 31
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .955 108 108.0 .016 -
2 .0148 121 .957 60 79.2 .015 27
3 .01105 127 .952 84 72.6 .016 33
5 .00795 131 .953 52 67.7 .018 37
10 .00535 138 .955 52 64.1 .018 41
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - A - 2
VALIDITY TRIAL
Distribution: Example A - Bell-shaped (truncated normal).
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .795 66 66.0 .023 -
2 .0148 75 .794 75 58.4 .026 12
3 .01105 79 .798 52 55.8 .027 15
5 .00795 83 .798 48 54.2 .025 18
10 .00535 89 .810 48 53.6 .031 19
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .893 87 87.0 .025 -
2 .0148 98 .891 49 69.9 .025 20
3 .01105 104 .898 68 65.8 .026 24
5 .00795 107 .896 63 61.6 .028 29
10 .00535 115 .895 55 60.8 .025 30
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .943 108 108.0 .026 -
2 .0148 121 .950 60 79.9 .026 26
3 .01105 127 .953 84 72.2 .025 33
5 .00795 131 .949 52 66.9 .027 38
10 .00535 138 .950 52 64.4 .026 40
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - B - 1
VALIDITY TRIAL
Distribution: Example B - Peaked, right skew
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .782 66 66.0 .026 -
2 .0148 75 .796 75 57.5 .026 13
3 .01105 79 .797 52 55.4 .024 16
5 .00795 83 .799 48 53.7 .023 19
10 .00535 89 .801 48 53.6 .027 19
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .882 87 87.0 .027 -
2 .0148 98 .893 49 69.8 .028 20
3 .01105 104 .897 69 65.4 .026 25
5 .00795 107 .887 63 61.3 .024 30
10 .00535 115 .897 55 59.6 .026 31
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .941 108 108.0 .023 -
2 .0148 121 .947 60 79.5 .026 26
3 .01105 127 .949 84 72.3 .030 33
5 .00795 131 .944 52 67.0 .027 38
10 .00535 138 .941 52 63.7 .026 41
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - B - 2
VALIDITY TRIAL
Distribution: Example B - Peaked, right skew.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .945 66 66.0 .025 -
2 .0148 75 .956 37 48.9 .025 26
3 .01105 79 .954 52 44.3 .022 33
5 .00795 83 .957 32 41.2 .029 38
10 .00535 89 .963 32 38.5 .029 42
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .983 87 87.0 .028 -
2 .0148 98 .986 49 57.9 .025 33
3 .01105 104 .988 34 50.4 .025 42
5 .00795 107 .989 42 44.5 .025 49
10 .00535 115 .991 33 41.7 .027 52
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .995 108 108.0 .024 -
2 .0148 121 .996 60 66.8 .029 38
3 .01105 127 .997 42 55.3 .023 49
5 .00795 131 .996 52 47.8 .026 56
10 .00535 138 .996 39 43.0 .028 60
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - C - 1
VALIDITY TRIAL
Distribution: Example C - Rectangular.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .804 66 66.0 .026 -
2 .0148 75 .818 75 57.5 .026 13
3 .01105 79 .814 52 55.2 .026 16
5 .00795 83 .816 48 53.6 .028 19
10 .00535 89 .806 56 54.4 .027 18
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .902 87 87.0 .025 -
2 .0148 98 .907 49 68.7 .025 21
3 .01105 104 .912 68 64.2 .027 26
5 .00795 107 .906 63 60.8 .023 30
10 .00535 115 .915 55 59.4 .026 32
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .954 108 108.0 .023 -
2 .0148 121 .956 60 78.4 .024 27
3 .01105 127 .960 84 70.7 .028 35
5 .00795 131 .956 52 65.8 .024 39
10 .00535 138 .954 52 63.6 .026 41
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - C - 2
VALIDITY TRIAL
Distribution: Example C - Rectangular
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .771 66 66.0 .022 -
2 .0148 75 .778 75 58.9 .024 11
3 .01105 79 .776 52 56.8 .022 14
5 .00795 83 .778 48 55.5 .026 16
10 .00535 89 .777 56 55.3 .026 18
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .875 87 87.0 .026 -
2 .0148 98 .885 49 70.7 .026 19
3 .01105 104 .884 68 67.1 .025 23
5 .00795 107 .875 63 63.8 .027 27
10 .00535 115 .884 55 62.6 .023 28
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .931 108 108.0 .025 -
2 .0148 121 .936 60 81.5 .023 25
3 .01105 127 .937 84 74.3 .024 31
5 .00795 131 .939 52 69.0 .022 36
10 .00535 138 .936 65 66.7 .025 38
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - D - 1
VALIDITY TRIAL
Distribution: Example D - J-shaped right skew.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .801 66 66.0 .025 -
2 .0148 75 .813 75 57.0 .024 14
3 .01105 79 .817 52 54.6 .024 17
5 .00795 83 .814 48 53.4 .023 19
10 .00535 89 .815 48 52.8 .023 20
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .898 87 87.0 .025 -
2 .0148 98 .902 49 68.9 .023 21
3 .01105 104 .902 68 64.6 .024 26
5 .00795 107 .903 63 60.4 .022 31
10 .00535 115 .910 55 58.3 .023 33
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .946 108 108.0 .023 -
2 .0148 121 .952 60 78.8 .026 27
3 .01105 127 .953 84 72.1 .021 33
5 .00795 131 .950 52 65.6 .024 39
10 .00535 138 .955 52 62.1 .021 46
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - D - 2
VALIDITY TRIAL
Distribution: Example D - J-shaped right skew.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .975 66 66.0 .022 -
2 .0148 75 .977 37 46.1 .025 30
3 .01105 79 .979 26 40.7 .023 38
5 .00795 83 .979 32 36.9 .026 44
10 .00535 89 .980 32 34.4 .025 48
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .993 87 87.0 .022 -
2 .0148 98 .994 49 55.1 .023 37
3 .01105 104 .995 34 46.4 .022 47
5 .00795 107 .996 42 40.3 .026 54
10 .00535 115 .996 33 36.8 .025 58
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .998 108 108.0 .022 -
2 .0148 121 .999 60 64.1 .023 41
3 .01105 127 .998 42 51.8 .024 52
5 .00795 131 .999 26 43.5 .023 60
10 .00535 138 .998 39 38.2 .025 65
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - E - 1
VALIDITY TRIAL
Distribution: Example E - Peaked and long-tailed, symmetric.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .796 66 66.0 .027 -
2 .0148 75 .808 75 56.8 .026 16
3 .01105 79 .812 52 54.3 .024 18
5 .00795 83 .811 48 52.4 .023 21
10 .00535 89 .817 48 51.8 .023 22
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .887 87 87.0 .022 -
2 .0148 98 .899 49 68.7 .024 21
3 .01105 104 .904 68 63.8 .027 27
5 .00795 107 .901 63 59.6 .023 31
10 .00535 115 .899 55 58.2 .022 33
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .947 108 108.0 .027 -
2 .0148 121 .951 60 79.0 .025 27
3 .01105 127 .953 84 71.1 .024 34
5 .00795 131 .952 52 65.3 .024 40
10 .00535 138 .958 52 60.8 .025 44
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - E - 2
VALIDITY TRIAL
Distribution: Example E - Peaked long-tailed symmetric.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .984 66 66.0 .023 -
2 .0148 75 .987 37 44.5 .025 33
3 .01105 79 .987 26 39.0 .023 41
5 .00795 83 .988 32 35.0 .025 47
10 .00535 89 .991 24 31.6 .025 52
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .995 87 87.0 .023 -
2 .0148 98 .997 49 53.2 .027 39
3 .01105 104 .997 34 44.6 .021 49
5 .00795 107 .998 42 37.9 .024 56
10 .00535 115 .998 33 34.0 .023 61
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .999 108 108.0 .021 -
2 .0148 121 1.000 60 62.8 .026 42
3 .01105 127 1.000 42 49.7 .024 54
5 .00795 131 1.000 26 41.1 .025 62
10 .00535 138 1.000 26 35.7 .027 67
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - F - 1
VALIDITY TRIAL
Distribution: Example F - Low dome.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .802 66 66.0 .029 -
2 .0148 75 .811 75 57.6 .032 13
3 .01105 79 .803 52 55.5 .026 16
5 .00795 83 .802 48 54.3 .029 18
10 .00535 89 .802 56 54.5 .028 17
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .896 87 87.0 .027 -
2 .0148 98 .905 49 69.3 .027 20
3 .01105 104 .906 68 65.0 .026 25
5 .00795 107 .896 63 61.3 .026 30
10 .00535 115 .902 55 60.7 .025 30
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .943 108 108.0 .028 -
2 .0148 121 .950 60 79.1 .026 27
3 .01105 127 .953 84 71.8 .022 34
5 .00795 131 .950 52 67.2 .030 38
10 .00535 138 .951 52 64.0 .028 41
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - F - 2
VALIDITY TRIAL
Distribution: Example F - Low dome.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .744 66 66.0 .025 -
2 .0148 75 .749 75 59.4 .026 10
3 .01105 79 .754 52 57.8 .026 13
5 .00795 83 .748 64 56.6 .029 14
10 .00535 89 .752 56 57.1 .030 13
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .848 87 87.0 .029 -
2 .0148 98 .858 49 72.1 .025 17
3 .01105 104 .865 68 68.9 .025 21
5 .00795 107 .856 63 65.4 .023 22
10 .00535 115 .858 66 64.7 .029 26
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .917 108 108.0 .028 -
2 .0148 121 .925 60 82.6 .027 24
3 .01105 127 .920 84 77.0 .024 29
5 .00795 131 .921 78 72.1 .024 33
10 .00535 138 .920 65 68.9 .027 36
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - G - 1
VALIDITY TRIAL
Distribution: Example G - U-shaped, symmetric
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .809 66 66.0 .026 -
2 .0148 75 .823 75 57.1 .027 13
3 .01105 79 .822 52 54.9 .027 17
5 .00795 83 .807 48 54.1 .023 18
10 .00535 89 .820 56 54.0 .027 18
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .904 87 87.0 .026 -
2 .0148 98 .913 49 68.4 .025 21
3 .01105 104 .920 68 64.3 .029 26
5 .00795 107 .914 63 60.7 .026 30
10 .00535 115 .916 55 59.6 .027 31
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .953 108 108.0 .025 -
2 .0148 121 .959 60 78.1 .026 28
3 .01105 127 .957 84 71.3 .026 34
5 .00795 131 .962 52 65.4 .028 39
10 .00535 138 .957 52 62.6 .033 42
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - G - 2
VALIDITY TRIAL
Distribution: Example G - U-shaped, symmetric
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .859 66 66.0 .027 -
2 .0148 75 .876 37 54.0 .028 18
3 .01105 79 .880 52 51.1 .028 23
5 .00795 83 .871 48 49.2 .024 25
10 .00535 89 .881 48 48.1 .026 27
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .940 87 87.0 .027 -
2 .0148 98 .942 49 68.6 .027 21
3 .01105 104 .951 68 59.0 .027 32
5 .00795 107 .947 42 54.3 .028 38
10 .00535 115 .953 44 52.6 .028 40
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .972 108 108.0 .026 -
2 .0148 121 .978 60 73.8 .025 32
3 .01105 127 .978 42 64.9 .026 40
5 .00795 131 .977 52 58.6 .026 46
10 .00535 138 .979 52 55.0 .026 49
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - H - 1
VALIDITY TRIAL
Distribution: Example H - Bimodal symmetric
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .745 66 66.0 .028 -
2 .0148 75 .752 75 59.7 .025 10
3 .01105 79 .755 52 57.7 .027 13
5 .00795 83 .754 64 56.6 .028 14
10 .00535 89 .755 56 57.0 .027 14
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .853 87 87.0 .027 -
2 .0148 98 .860 49 72.3 .025 15
3 .01105 104 .865 68 68.4 .026 21
5 .00795 107 .860 63 64.9 .024 25
10 .00535 115 .857 66 65.0 .026 25
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .920 108 108.0 .025 -
2 .0148 121 .926 60 82.7 .025 25
3 .01105 127 .927 84 76.9 .025 29
5 .00795 131 .923 78 71.5 .026 34
10 .00535 138 .922 65 69.1 .029 36
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - H - 2
VALIDITY TRIAL
Distribution: Example H - Bimodal symmetric
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .921 66 66.0 .028 -
2 .0148 75 .931 37 50.9 .027 23
3 .01105 79 .934 52 46.6 .026 29
5 .00795 83 .938 32 43.7 .026 34
10 .00535 89 .934 40 41.8 .027 37
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .974 87 87.0 .028 -
2 .0148 98 .973 49 60.6 .027 30
3 .01105 104 .979 34 53.6 .028 38
5 .00795 107 .976 42 48.4 .025 44
10 .00535 115 .980 44 45.3 .029 48
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .990 108 108.0 .028 -
2 .0148 121 .992 60 68.7 .028 36
3 .01105 127 .993 42 59.3 .027 45
5 .00795 131 .993 52 51.5 .029 52
10 .00535 138 .993 39 47.1 .027 56
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - I - 1
VALIDITY TRIAL
Distribution: Example I - Dome-shaped with outliers
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .805 66 66.0 .025 -
2 .0148 75 .818 75 57.3 .026 13
3 .01105 79 .819 52 54.9 .027 17
5 .00795 83 .814 48 53.2 .026 19
10 .00535 89 .822 48 53.0 .029 20
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .899 87 87.0 .026 -
2 .0148 98 .908 49 68.3 .029 21
3 .01105 104 .910 68 64.1 .029 26
5 .00795 107 .911 63 60.5 .029 30
10 .00535 115 .912 55 58.7 .027 33
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .952 108 108.0 .026 -
2 .0148 121 .954 60 78.6 .029 27
3 .01105 127 .957 84 70.8 .026 34
5 .00795 131 .960 52 64.9 .029 40
10 .00535 138 .958 52 62.1 .028 43
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 6 - I - 2
VALIDITY TRIAL
Distribution: Example I - Dome-shaped with outliers
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 0.5 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 66 .965 66 66.0 .027 -
2 .0148 75 .968 37 46.8 .025 29
3 .01105 79 .970 26 42.0 .024 36
5 .00795 83 .972 32 37.7 .027 43
10 .00535 89 .976 32 35.5 .030 46
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 87 .992 87 87.0 .030 -
2 .0148 98 .996 49 55.9 .024 36
3 .01105 104 .993 34 47.7 .027 45
5 .00795 107 .994 42 41.4 .027 52
10 .00535 115 .996 33 37.6 .028 57
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 108 .999 108 108.0 .028 -
2 .0148 121 .999 60 64.6 .025 40
3 .01105 127 .999 42 52.7 .026 51
5 .00795 131 .999 52 44.7 .027 59
10 .00535 138 .998 39 39.6 .025 63
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - A - 1
VALIDITY TRIAL
Distribution: Example A - Bell-shaped (truncated normal).
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .843 19 19.0 .022 -
2 .0148 22 .860 22 16.6 .023 13
3 .01105 23 .863 14 15.7 .021 17
5 .00795 24 .852 16 15.3 .025 19
10 .00535 26 .856 14 15.4 .027 19
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .918 24 24.0 .021 -
2 .0148 27 .930 13 18.8 .022 22
3 .01105 29 .927 18 17.7 .023 26
5 .00795 30 .924 18 17.0 .020 29
10 .00535 32 .932 15 14.6 .026 39
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .964 29 29.0 .021 -
2 .0148 32 .961 16 20.9 .021 28
3 .01105 34 .966 22 19.3 .023 33
5 .00795 35 .962 14 17.8 .021 39
10 .00535 37 .962 15 17.2 .023 41
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - A - 2
VALIDITY TRIAL
Distribution: Example A - Bell-shaped (truncated normal).
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .822 19 19.0 .026 -
2 .0148 22 .850 11 16.5 .027 13
3 .01105 23 .844 14 15.6 .026 18
5 .00795 24 .843 16 15.0 .038 21
10 .00535 26 .844 14 15.1 .033 21
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .902 24 24.0 .026 -
2 .0148 27 .911 13 19.2 .025 20
3 .01105 29 .917 18 17.7 .027 26
5 .00795 30 .920 18 16.4 .029 32
10 .00535 32 .919 15 16.6 .026 31
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .951 29 29.0 .024 -
2 .0148 32 .953 16 21.1 .024 27
3 .01105 34 .957 22 19.3 .026 33
5 .00795 35 .955 14 18.1 .027 38
10 .00535 37 .951 15 17.3 .028 40
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - B - 1
VALIDITY TRIAL
Distribution: Example B - Unimodal, positive skew.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
----------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .823 19 19.0 .023 -
2 .0148 22 .851 11 16.4 .026 14
3 .01105 23 .849 14 15.4 .024 19
5 .00795 24 .843 12 14.7 .023 23
10 .00535 26 .839 14 14.5 .024 24
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .906 24 24.0 .027 -
2 .0148 27 .913 13 18.7 .025 22
3 .01105 29 .916 18 17.3 .025 29
5 .00795 30 .921 18 16.5 .024 31
10 .00535 32 .924 15 15.8 .023 34
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .950 29 29.0 .025 -
2 .0148 32 .954 16 21.0 .025 28
3 .01105 34 .955 22 19.0 .025 35
5 .00795 35 .951 14 17.6 .023 39
10 .00535 37 .957 15 16.7 .023 42
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - B - 2
VALIDITY TRIAL
Distribution: Example B - Unimodal, positive skew.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .908 19 19.0 .022 -
2 .0148 22 .925 11 15.1 .026 21
3 .01105 23 .925 14 13.8 .031 27
5 .00795 24 .923 12 12.8 .034 33
10 .00535 26 .933 10 12.5 .033 34
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .960 24 24.0 .023 -
2 .0148 27 .967 13 17.3 .026 28
3 .01105 29 .972 18 15.5 .026 35
5 .00795 30 .970 12 14.1 .031 41
10 .00535 32 .971 12 13.9 .028 42
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .983 29 29.0 .023 -
2 .0148 32 .986 16 19.2 .027 34
3 .01105 34 .986 11 16.9 .025 42
5 .00795 35 .985 14 15.5 .028 47
10 .00535 37 .988 12 14.1 .026 51
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - C - 1
VALIDITY TRIAL
Distribution: Example C - Rectangular (uniform).
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .841 19 19.0 .026 -
2 .0148 22 .865 22 16.5 .026 13
3 .01105 23 .852 14 16.1 .030 15
5 .00795 24 .844 16 15.8 .030 17
10 .00535 26 .846 14 16.0 .036 16
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .919 24 24.0 .025 -
2 .0148 27 .926 13 18.9 .026 21
3 .01105 29 .930 18 18.1 .029 25
5 .00795 30 .920 18 17.4 .026 27
10 .00535 32 .931 15 17.0 .031 29
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .964 29 29.0 .026 -
2 .0148 32 .963 16 21.1 .026 27
3 .01105 34 .969 22 19.3 .027 33
5 .00795 35 .962 14 18.3 .027 37
10 .00535 37 .960 15 17.5 .031 40
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - C - 2
VALIDITY TRIAL
Distribution: Example C - Rectangular (uniform).
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .782 19 19.0 .023 -
2 .0148 22 .804 22 17.0 .027 11
3 .01105 23 .795 14 16.5 .029 13
5 .00795 24 .791 16 16.0 .035 16
10 .00535 26 .792 14 16.3 .034 14
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .870 24 24.0 .024 -
2 .0148 27 .876 13 19.8 .026 17
3 .01105 29 .878 18 19.1 .028 20
5 .00795 30 .880 18 17.7 .029 26
10 .00535 32 .888 18 17.8 .027 26
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .927 29 29.0 .026 -
2 .0148 32 .928 16 21.8 .024 25
3 .01105 34 .936 22 20.4 .025 30
5 .00795 35 .928 21 19.5 .027 33
10 .00535 37 .930 18 18.8 .029 35
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - D - 1
VALIDITY TRIAL
Distribution: Example D - J-shaped, positive skew.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .833 19 19.0 .023 -
2 .0148 22 .853 11 16.3 .022 14
3 .01105 23 .851 14 15.2 .021 20
5 .00795 24 .856 12 14.4 .023 24
10 .00535 26 .853 12 14.5 .024 24
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .912 24 24.0 .023 -
2 .0148 27 .915 13 18.6 .023 23
3 .01105 29 .923 18 17.3 .021 28
5 .00795 30 .923 12 16.1 .022 33
10 .00535 32 .927 15 15.5 .022 35
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .947 29 29.0 .023 -
2 .0148 32 .950 16 21.0 .024 28
3 .01105 34 .956 22 19.0 .021 34
5 .00795 35 .956 14 17.4 .019 40
10 .00535 37 .957 15 16.1 .021 44
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - D - 2
VALIDITY TRIAL
Distribution: Example D - J-shaped, positive skew.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .931 19 19.0 .022 -
2 .0148 22 .949 11 14.5 .026 24
3 .01105 23 .949 14 12.9 .027 32
5 .00795 24 .949 12 11.8 .034 38
10 .00535 26 .951 10 11.6 .031 39
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .974 24 24.0 .026 -
2 .0148 27 .979 13 16.5 .026 31
3 .01105 29 .983 9 14.6 .026 39
5 .00795 30 .981 12 13.3 .025 45
10 .00535 32 .986 12 12.9 .022 46
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .991 29 29.0 .024 -
2 .0148 32 .993 16 18.5 .025 36
3 .01105 34 .995 11 16.0 .026 45
5 .00795 35 .992 14 14.4 .027 50
10 .00535 37 .993 12 13.0 .025 55
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - E - 1
VALIDITY TRIAL
Distribution: Example E - Peaked and long-tailed, symmetric.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .828 19 19.0 .024 -
2 .0148 22 .860 11 16.0 .023 16
3 .01105 23 .859 14 14.6 .018 23
5 .00795 24 .862 12 13.6 .018 28
10 .00535 26 .853 12 13.9 .019 27
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .909 24 24.0 .026 -
2 .0148 27 .915 13 18.4 .022 23
3 .01105 29 .930 18 16.7 .021 30
5 .00795 30 .922 12 15.5 .019 35
10 .00535 32 .930 12 14.6 .018 39
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .949 29 29.0 .026 -
2 .0148 32 .952 16 20.9 .021 28
3 .01105 34 .953 11 18.6 .020 36
5 .00795 35 .959 14 16.3 .016 44
10 .00535 37 .954 12 15.3 .019 47
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - E - 2
VALIDITY TRIAL
Distribution: Example E - Peaked and long-tailed, symmetric.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .956 19 19.0 .025 -
2 .0148 22 .970 11 13.9 .026 27
3 .01105 23 .969 14 12.3 .025 35
5 .00795 24 .967 8 10.7 .037 44
10 .00535 26 .974 8 10.5 .033 45
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .984 24 24.0 .025 -
2 .0148 27 .987 13 15.9 .025 34
3 .01105 29 .991 9 13.9 .027 42
5 .00795 30 .990 12 12.4 .029 48
10 .00535 32 .992 9 11.9 .025 50
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .995 29 29.0 .024 -
2 .0148 32 .997 16 17.9 .025 38
3 .01105 34 .996 11 15.0 .025 48
5 .00795 35 .997 14 13.5 .027 53
10 .00535 37 .998 9 12.0 .024 59
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - F - 1
VALIDITY TRIAL
Distribution: Example F - Low dome, symmetric.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (0.05 two sided)
1 .025 19 .845 19 19.0 .027 -
2 .0148 22 .865 22 16.7 .031 12
3 .01105 23 .850 14 16.0 .030 16
5 .00795 24 .841 16 15.6 .031 18
10 .00535 26 .842 14 16.1 .037 15
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .920 24 24.0 .024 -
2 .0148 27 .921 13 19.1 .027 20
3 .01105 29 .931 18 18.0 .030 25
5 .00795 30 .921 18 17.2 .028 28
10 .00535 32 .925 15 17.0 .030 29
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .960 29 29.0 .025 -
2 .0148 32 .960 16 21.0 .027 28
3 .01105 34 .965 22 19.5 .027 33
5 .00795 35 .961 14 18.4 .029 37
10 .00535 37 .954 15 17.5 .032 40
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - F - 2
VALIDITY TRIAL
Distribution: Example F - Low dome, symmetric
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (0.05 two sided)
1 .025 19 .778 19 19.0 .025 -
2 .0148 22 .804 22 16.9 .027 11
3 .01105 23 .797 14 16.3 .029 14
5 .00795 24 .796 16 15.8 .036 17
10 .00535 26 .792 16 16.4 .031 14
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .872 24 24.0 .024 -
2 .0148 27 .881 13 19.7 .026 18
3 .01105 29 .883 18 18.8 .025 22
5 .00795 30 .881 18 17.6 .035 27
10 .00535 32 .885 15 17.9 .027 25
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .927 29 29.0 .027 -
2 .0148 32 .933 16 21.9 .027 24
3 .01105 34 .932 22 20.6 .025 29
5 .00795 35 .929 21 19.3 .029 33
10 .00535 37 .921 18 18.8 .028 35
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - G - 1
VALIDITY TRIAL
Distribution: Example G - U-shaped symmetric.
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .844 19 19.0 .023 -
2 .0148 22 .863 22 16.7 .029 12
3 .01105 23 .850 14 16.2 .034 15
5 .00795 24 .845 16 16.0 .032 16
10 .00535 26 .846 14 16.2 .041 15
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .929 24 24.0 .026 -
2 .0148 27 .933 13 18.9 .027 21
3 .01105 29 .930 18 18.1 .028 25
5 .00795 30 .927 18 17.4 .029 28
10 .00535 32 .931 15 16.9 .037 30
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .961 29 29.0 .026 -
2 .0148 32 .966 16 21.0 .027 28
3 .01105 34 .968 22 19.5 .026 33
5 .00795 35 .961 14 18.3 .031 37
10 .00535 37 .962 15 17.5 .035 40
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - G - 2
VALIDITY TRIAL
Distribution: Example G - U-shaped symmetric.
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .785 19 19.0 .025 -
2 .0148 22 .814 22 16.8 .025 12
3 .01105 23 .807 14 16.2 .026 15
5 .00795 24 .795 16 15.9 .036 16
10 .00535 26 .806 14 16.1 .037 15
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .879 24 24.0 .026 -
2 .0148 27 .882 13 19.5 .023 19
3 .01105 29 .886 18 18.5 .024 23
5 .00795 30 .883 18 17.5 .028 27
10 .00535 32 .896 15 17.6 .024 27
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .933 29 29.0 .025 -
2 .0148 32 .933 16 21.7 .026 25
3 .01105 34 .933 22 20.1 .028 31
5 .00795 35 .930 21 19.1 .026 34
10 .00535 37 .934 15 18.3 .028 37
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - H - 1
VALIDITY TRIAL
Distribution: Example H - Bimodal symmetric
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
---------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .853 19 19.0 .024 -
2 .0148 22 .864 22 16.6 .024 13
3 .01105 23 .859 14 16.2 .029 15
5 .00795 24 .846 16 15.7 .032 17
10 .00535 26 .856 14 15.6 .044 18
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .929 24 24.0 .025 -
2 .0148 27 .929 13 18.9 .025 21
3 .01105 29 .934 18 18.0 .031 25
5 .00795 30 .932 18 17.3 .028 28
10 .00535 32 .935 15 16.6 .035 31
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .961 29 29.0 .026 -
2 .0148 32 .963 16 20.9 .028 28
3 .01105 34 .968 22 19.3 .029 33
5 .00795 35 .966 14 18.1 .025 38
10 .00535 37 .962 15 17.4 .036 40
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - H - 2
VALIDITY TRIAL
Distribution: Example H - Bimodal symmetric
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
----------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .796 19 19.0 .023 -
2 .0148 22 .811 22 16.8 .027 12
3 .01105 23 .812 14 16.2 .028 15
5 .00795 24 .801 16 15.6 .035 18
10 .00535 26 .799 14 16.1 .035 15
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .882 24 24.0 .025 -
2 .0148 27 .881 13 19.4 .025 19
3 .01105 29 .885 18 18.7 .027 22
5 .00795 30 .886 18 17.3 .030 28
10 .00535 32 .889 15 17.7 .027 26
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .929 29 29.0 .024 -
2 .0148 32 .937 16 21.5 .026 26
3 .01105 34 .933 22 20.4 .027 30
5 .00795 35 .931 21 19.1 .026 34
10 .00535 37 .933 15 18.3 .027 37
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - I - 1
VALIDITY TRIAL
Distribution: Example I - Dome-shaped with outliers
Test: Student's t-test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
----------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .852 19 19.0 .025 -
2 .0148 22 .859 11 16.5 .027 13
3 .01105 23 .852 14 15.7 .030 17
5 .00795 24 .856 12 14.8 .026 22
10 .00535 26 .862 14 14.9 .030 22
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .923 24 24.0 .031 -
2 .0148 27 .925 13 18.9 .028 21
3 .01105 29 .935 18 17.6 .028 27
5 .00795 30 .930 18 16.5 .027 31
10 .00535 32 .931 15 15.8 .026 34
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .960 29 29.0 .027 -
2 .0148 32 .961 16 21.0 .025 28
3 .01105 34 .961 22 19.0 .025 34
5 .00795 35 .959 14 17.7 .026 39
10 .00535 37 .960 15 16.6 .029 43
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Table 7 - I - 2
VALIDITY TRIAL
Distribution: Example I - Dome-shaped with outliers
Test: Wilcoxon's two-sample rank test (one sided).
Alternative hypothesis: Shift (delta) = 1.0 SD.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looks Nominal Sample -------------Realised------------- Saving,
LOS, size, Power Sample sizes, Total LOS, per
Pnom max. Median Mean Ptot cent
----------------------------------------------------------------------
Target: Power >= .8, Ptot <= .025 one sided (.05 two sided)
1 .025 19 .774 19 19.0 .024 -
2 .0148 22 .800 22 17.0 .025 11
3 .01105 23 .794 14 16.3 .025 14
5 .00795 24 .783 16 16.0 .033 16
10 .00535 26 .783 16 16.4 .034 16
Target: Power >= .9, Ptot <= .025 one sided (.05 two sided)
1 .025 24 .873 24 24.0 .023 -
2 .0148 27 .874 13 19.8 .025 17
3 .01105 29 .881 18 19.0 .025 21
5 .00795 30 .882 18 17.7 .033 26
10 .00535 32 .880 18 18.0 .026 25
Target: Power >= .95, Ptot <= .025 one sided (.05 two sided)
1 .025 29 .987 29 29.0 .026 -
2 .0148 32 .990 16 18.7 .027 36
3 .01105 34 .989 11 16.3 .026 44
5 .00795 35 .991 14 14.9 .030 49
10 .00535 37 .989 12 13.7 .028 53
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
END OF MANUSCRIPT
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Home page Department
Home page University
Department of Public Health and Primary Health Care, last updated 28.01.07
Hogne.Sandvik@isf.uib.no