Numerical Comparisons

We compared the different methods for the two situations discussed in Section 2. For the all-pairwise comparisons, we let $I=4$ ($q=6$) and ${\bf V}=$diag$(n_i^{-1})$. The specific values for $n_i$ are given in Table 1. The table further presents the estimates $\hat{t}_{1-\alpha}$ for each method and the associated probability $T_k(-\hat{{\bf t}}_{1-\alpha}, \hat{{\bf t}}_{1-\alpha}; {\bf R}, \nu)$ are given in italics below. Critical values and probabilities were calculated with three significant digits of accuracy using the methods of Genz and Bretz (2002). For the $I=4$ cases $t^{(2,3)}_l$ and $t^{(2,3)}_u$ seem to be good low-cost improvements over $t^{(2)}_l$ (Dawson-Sankoff) and $t^{(2)}_u$ (Hunter-Worsley). Further on, $t^{(2,3)}_l$ appears to always be better than $t^{(3)}_l$. The TK should not be used if the sample sizes differ by a large amount. Similar results apply to the other approximate correlation methods.

Table 1: Numerical results for all-pairwise comparisons with one-way layout ($I=4$).

$t^{(1)}_l$	$t^{(2)}_l$	$t^{(3)}_l$	$t^{(2,3)}_l$	$t_{1-\alpha}$	$t^{(2,3)}_u$	$t^{(3)}_u$	$t^{(2)}_u$	$t^{(1)}_u$	Šidák	TK	GT2	$\bar{\rho}$	$F$-test	$n_1$	$n_2$	$n_3$	$n_4$
2.028	2.672	2.683	2.686	2.693	2.725	2.695	2.751	2.792	2.784	2.693	2.775	2.699	2.932	10	10	10	10
0.803	0.948	0.949	0.950	0.950	0.954	0.951	0.956	0.960	0.959	0.950	0.959	0.951	0.972
2.011	2.641	2.652	2.655	2.660	2.679	2.660	2.705	2.752	2.744	2.661	2.738	2.664	2.897	10	12	14	16
0.802	0.948	0.950	0.950	0.950	0.953	0.950	0.955	0.960	0.959	0.950	0.959	0.951	0.972
2.000	2.619	2.628	2.634	2.638	2.654	2.638	2.677	2.729	2.721	2.643	2.716	2.644	2.876	10	14	18	22
0.802	0.948	0.949	0.950	0.950	0.952	0.951	0.955	0.960	0.959	0.951	0.959	0.951	0.973
1.993	2.602	2.612	2.618	2.622	2.636	2.622	2.658	2.713	2.705	2.630	2.702	2.630	2.863	10	16	22	28
0.803	0.948	0.949	0.950	0.950	0.951	0.951	0.954	0.960	0.959	0.951	0.959	0.951	0.973
1.989	2.589	2.601	2.607	2.610	2.622	2.611	2.643	2.702	2.694	2.621	2.691	2.620	2.853	10	18	26	34
0.804	0.947	0.949	0.950	0.950	0.951	0.951	0.954	0.960	0.960	0.951	0.959	0.951	0.973
1.985	2.578	2.590	2.597	2.600	2.611	2.601	2.631	2.694	2.686	2.615	2.684	2.613	2.846	10	20	30	40
0.805	0.947	0.949	0.950	0.950	0.951	0.951	0.954	0.961	0.960	0.952	0.960	0.952	0.974
1.971	2.519	2.538	2.552	2.554	2.560	2.555	2.578	2.663	2.655	2.589	2.654	2.584	2.818	10	40	70	100
0.811	0.945	0.948	0.950	0.950	0.951	0.950	0.953	0.962	0.962	0.954	0.962	0.954	0.976
1.967	2.490	2.515	2.532	2.535	2.536	2.540	2.557	2.654	2.647	2.582	2.646	2.575	2.810	10	60	110	160
0.814	0.944	0.948	0.950	0.950	0.951	0.951	0.953	0.964	0.963	0.956	0.963	0.955	0.976
1.965	2.472	2.501	2.521	2.524	2.529	2.526	2.545	2.650	2.643	2.579	2.642	2.571	2.806	10	80	150	220
0.817	0.943	0.947	0.949	0.950	0.951	0.951	0.953	0.964	0.964	0.957	0.964	0.956	0.977
1.964	2.458	2.490	2.514	2.517	2.521	2.519	2.537	2.647	2.640	2.577	2.640	2.568	2.804	10	100	190	280
0.819	0.942	0.946	0.949	0.950	0.950	0.951	0.953	0.965	0.964	0.957	0.964	0.956	0.977
1.976	2.546	2.560	2.570	2.571	2.581	2.573	2.596	2.675	2.667	2.599	2.666	2.597	2.828	80	40	20	10
0.808	0.947	0.948	0.950	0.950	0.952	0.950	0.953	0.962	0.961	0.953	0.961	0.953	0.975
1.966	2.488	2.511	2.527	2.528	2.534	2.530	2.546	2.652	2.644	2.580	2.644	2.576	2.808	270	90	30	10
0.816	0.945	0.948	0.950	0.950	0.951	0.950	0.952	0.964	0.963	0.956	0.963	0.956	0.977
1.963	2.454	2.484	2.505	2.506	2.511	2.509	2.521	2.644	2.637	2.574	2.637	2.569	2.801	640	160	40	10
0.822	0.943	0.947	0.950	0.950	0.951	0.950	0.952	0.965	0.965	0.958	0.965	0.958	0.978
1.961	2.431	2.466	2.491	2.494	2.497	2.495	2.506	2.642	2.634	2.572	2.634	2.566	2.799	1250	250	50	10
0.825	0.941	0.946	0.950	0.950	0.951	0.950	0.953	0.967	0.966	0.959	0.966	0.958	0.978
1.961	2.414	2.454	2.482	2.484	2.487	2.487	2.496	2.640	2.633	2.571	2.633	2.564	2.797	2160	360	60	10
0.828	0.940	0.947	0.950	0.950	0.951	0.950	0.952	0.968	0.967	0.961	0.967	0.959	0.979
1.961	2.400	2.445	2.474	2.477	2.480	2.480	2.488	2.640	2.632	2.570	2.632	2.563	2.797	3430	490	70	10
0.830	0.939	0.945	0.950	0.950	0.950	0.950	0.951	0.969	0.967	0.961	0.967	0.960	0.980
1.960	2.393	2.437	2.468	2.471	2.474	2.475	2.481	2.639	2.632	2.570	2.632	2.562	2.796	5120	640	80	10
0.832	0.940	0.945	0.950	0.950	0.950	0.951	0.952	0.968	0.968	0.962	0.968	0.961	0.980
1.960	2.387	2.423	2.459	2.463	2.466	2.466	2.472	2.639	2.632	2.569	2.631	2.561	2.796	10000	1000	100	10
0.834	0.939	0.945	0.950	0.950	0.950	0.950	0.951	0.970	0.969	0.963	0.968	0.962	0.980

For the many-to-one comparisons we looked at $J=2$ and $I=5$ (if $I \leq 4$, then $q=I-1 \leq 3$ and (2) holds for any ${\bf V}$ if the $\lambda_i$'s are defined appropriately). Table 2 specifies the values for $n_{i.}=\sum_j n_{ij}$. In all cases we set $n_{i2}=2$, $n_{i1}=n_{i.}-n_{i2}, i=1,2$ and $n_{i1}=2,$ $n_{i2}=n_{i.}-n_{i1}, i=3,4,5$ and . This ensures that the proportionality rule of Section 2 is violated. Similar results as for the all pairwise comparisons hold here. The hybrid bounds are found again to be good approximations to $t_{1-\alpha}$. The Hsu method is usually accurate to three significant digits and is a good competitor to $t^{(2,3)}_l$ and $t^{(2,3)}_u$. The Solow method is easily implemented and performs good for low $I$, but its performance deteriorates rapidly with increasing $I$.

Table 2: Numerical results for many-to-one comparisons with two-way layout ($I=5$).

$t^{(1)}_l$	$t^{(2)}_l$	$t^{(3)}_l$	$t^{(2,3)}_l$	$t_{1-\alpha}$	$t^{(2,3)}_u$	$t^{(3)}_u$	$t^{(2)}_u$	$t^{(1)}_u$	Šidák	Hsu	Solow	$\bar{\rho}$	$n_{1.}$	$n_{2.}$	$n_{3.}$	$n_{4.}$	$n_{5.}$
1.680	2.176	2.209	2.211	2.215	2.224	2.219	2.254	2.321	2.313	2.214	2.212	2.217	10	10	10	10	10
0.860	0.946	0.949	0.950	0.950	0.951	0.950	0.954	0.960	0.960	0.950	0.950	0.950
1.669	2.094	2.152	2.158	2.161	2.170	2.170	2.202	2.295	2.287	2.161	2.159	2.169	10	12	14	16	18
0.870	0.942	0.949	0.950	0.950	0.951	0.951	0.954	0.963	0.962	0.950	0.950	0.951
1.663	2.034	2.117	2.125	2.128	2.137	2.141	2.170	2.282	2.275	2.129	2.126	2.141	10	14	18	22	26
0.875	0.939	0.949	0.950	0.950	0.951	0.951	0.954	0.964	0.964	0.950	0.950	0.951
1.660	1.985	2.094	2.102	2.105	2.113	2.119	2.147	2.274	2.267	2.106	2.104	2.123	10	16	22	28	34
0.880	0.935	0.948	0.950	0.950	0.951	0.951	0.954	0.966	0.965	0.950	0.950	0.952
1.657	1.969	2.074	2.084	2.088	2.096	2.104	2.129	2.269	2.261	2.088	2.087	2.109	10	18	26	34	42
0.883	0.936	0.948	0.950	0.950	0.951	0.952	0.954	0.967	0.966	0.950	0.950	0.952
1.656	1.958	2.059	2.070	2.074	2.082	2.090	2.115	2.265	2.257	2.075	2.074	2.099	10	20	30	40	50
0.885	0.936	0.948	0.949	0.950	0.951	0.951	0.954	0.967	0.967	0.950	0.950	0.953
1.649	1.903	1.984	2.006	2.010	2.016	2.023	2.045	2.251	2.244	2.010	2.012	2.054	10	40	70	100	130
0.897	0.937	0.947	0.950	0.950	0.951	0.951	0.954	0.971	0.971	0.950	0.950	0.955
1.648	1.880	1.957	1.982	1.985	1.990	1.997	2.017	2.248	2.240	1.985	1.987	2.040	10	60	110	160	210
0.901	0.938	0.947	0.951	0.950	0.951	0.952	0.953	0.973	0.972	0.950	0.950	0.956
1.647	1.867	1.939	1.969	1.972	1.977	1.982	2.000	2.246	2.239	1.972	1.973	2.032	10	80	150	220	290
0.904	0.938	0.946	0.950	0.950	0.951	0.951	0.953	0.973	0.973	0.950	0.950	0.956
1.646	1.859	1.929	1.959	1.963	1.967	1.973	1.989	2.245	2.238	1.962	1.963	2.028	10	100	190	280	370
0.905	0.938	0.946	0.950	0.950	0.950	0.951	0.953	0.974	0.973	0.950	0.950	0.957
1.650	2.164	2.176	2.180	2.179	2.182	2.180	2.194	2.253	2.245	2.180	2.178	2.200	160	80	40	20	10
0.850	0.948	0.950	0.950	0.950	0.950	0.950	0.952	0.958	0.957	0.950	0.950	0.952
1.646	2.143	2.157	2.161	2.162	2.164	2.162	2.173	2.244	2.237	2.161	2.158	2.196	810	270	90	30	10
0.853	0.947	0.950	0.950	0.951	0.950	0.950	0.951	0.959	0.958	0.950	0.950	0.954
1.645	2.129	2.146	2.150	2.150	2.152	2.152	2.161	2.242	2.235	2.151	2.148	2.194	2560	640	160	40	10
0.855	0.948	0.950	0.950	0.950	0.950	0.950	0.951	0.960	0.959	0.950	0.949	0.954
1.645	2.120	2.139	2.145	2.145	2.146	2.145	2.154	2.242	2.234	2.144	2.141	2.194	6250	1250	250	50	10
0.856	0.947	0.949	0.950	0.950	0.951	0.950	0.952	0.961	0.960	0.950	0.950	0.955
1.645	2.115	2.135	2.141	2.140	2.143	2.141	2.148	2.242	2.234	2.139	2.136	2.193	12960	2160	360	60	10
0.858	0.946	0.949	0.950	0.950	0.950	0.950	0.951	0.961	0.960	0.950	0.950	0.956
1.645	2.108	2.129	2.135	2.135	2.137	2.136	2.144	2.242	2.234	2.135	2.134	2.193	24010	3430	490	70	10
0.859	0.947	0.949	0.950	0.950	0.950	0.950	0.951	0.961	0.960	0.950	0.950	0.956
1.645	2.104	2.125	2.133	2.132	2.135	2.133	2.141	2.241	2.234	2.132	2.130	2.193	40960	5120	640	80	10
0.859	0.946	0.949	0.950	0.950	0.950	0.949	0.951	0.962	0.961	0.950	0.950	0.956
1.645	2.097	2.121	2.127	2.128	2.129	2.128	2.135	2.241	2.234	2.128	2.125	2.192	100000	10000	1000	100	10
0.861	0.946	0.949	0.950	0.950	0.950	0.950	0.951	0.962	0.961	0.950	0.949	0.958