A nuisance parameter is removed by conditioning on a statistic that is sufficient for it. What remains carries no nuisance, so a reference distribution drawn from it is exact at every sample size: no asymptotic approximation and no bootstrap. The package supplies two such conditionings, each sampled by a pure-C back-end in which every replicate draws from its own random stream, so a result does not depend on the number of threads.
In an \(r \times c\) table the row and column totals are sufficient for the marginal parameters. Given both, the table follows the Fisher-Yates (multivariate hypergeometric) law, which is sampled exactly by permuting the column labels of the individual observations.
For nominal rows against an ordinal column, the default statistic is the dispersion of the row mean scores. Its components are the group contrasts themselves, so a significant result points directly at the rows that differ.
tab <- matrix(c(3, 228, 72,
10, 548, 67,
7, 276, 131), nrow = 3, byrow = TRUE)
ec_rc_test(tab, v = c(-1, 0, 1), B = 5000, seed = 1)
#> Exact conditional test in a two-way table, both margins fixed
#> statistic (dispersion) = 11.4791
#> p < 0.0002 (resolution floor); largest of 5,000 null draws = 4.73759
#> row mean scores: 0.2277, 0.0912, 0.2995When both classifications are ordered, stat = "linlin"
gives Goodman’s (1979) linear-by-linear association, and
"pearson" and "lr" give the familiar unordered
statistics referred to the exact conditional law rather than to the
chi-square.
The reference distribution itself is available for any margins.
null <- ec_rc_null(rows = c(30, 40), cols = c(35, 35), stat = "pearson", B = 2000, seed = 2)
quantile(null, c(0.5, 0.95))
#> 50% 95%
#> 0.2333333 3.7333333ec_pvalue() turns an observed value and a sample from
the null into a p-value, a mid-p value and a Monte Carlo interval for
the p-value.
In a linear model under a null in which the tested structure is absent, any statistic that is invariant to the coefficients and to the error scale depends on the data only through the residual direction, and that direction is uniform on the unit sphere of the residual space. So the reference stays exact when the statistic is the maximum over several candidate structures, for instance several spatial weight matrices, one of which was chosen because it fitted best.
ec_sphere_maxquad() samples the maximum over candidate
matrices of the normalised quadratic form in the residual direction.
Two permutation tests use the same back-end. They refer a statistic to the law obtained by permuting group labels, across individuals or across clusters (exchangeability blocks in the sense of Winkler et al., 2014).
ec_multi_test() tests a battery of categorical items
across groups with one p-value: the sum or the maximum over items of a
per-item statistic, in the spirit of Pesarin and Salmaso’s (2010)
nonparametric combination. Per-item p-values come from the same
permutations, so they can be used for a closed follow-up.
set.seed(2)
g <- rep(c("a", "b", "c"), c(60, 80, 60))
x <- cbind(rbinom(200, 1, 0.9), rbinom(200, 1, 0.95),
rbinom(200, 1, ifelse(g == "c", 0.7, 0.9))) # only item 3 differs
r <- ec_multi_test(x, g, B = 2000, seed = 1)
r
#> One exact test for 3 items across 3 groups, respondents exchangeable, pearson statistic
#> sum 20.9726 p = 0.0009995 mid-p = 0.0009995 Monte Carlo 95% interval [8.827e-05, 0.002827]
#> max 16.8473 p < 0.0005 (resolution floor); largest null draw 15.9338
r$items
#> item statistic p.value
#> 1 V1 1.318267 0.5357321339
#> 2 V2 2.807018 0.2368815592
#> 3 V3 16.847335 0.0004997501ec_diffnet_test() compares correlation networks across
groups without a Gaussian assumption. The edge statistic is the
heterogeneity of the Fisher-transformed correlations. The network
statistic is their sum or maximum over edges, as in the Network
Comparison Test of van Borkulo et al. (2023). With
rank = TRUE the procedure is distribution-free.
set.seed(1)
n <- 120; p <- 5
g <- rep(c("a", "b"), each = n / 2)
z <- matrix(rnorm(n * p), n)
z[g == "b", 2] <- 0.8 * z[g == "b", 1] + 0.6 * z[g == "b", 2]
ec_diffnet_test(z, g, B = 999, seed = 1)
#> Differential network over 10 edges, 2 groups, respondents exchangeable, ranks
#> sum 41.8385 p < 0.001 (resolution floor); largest null draw 33.5233
#> max 29.0095 p < 0.001 (resolution floor); largest null draw 15.3051
#> strongest edges:
#> V1 -- V2 statistic 29.01 p = 0.001
#> V1 -- V5 statistic 3.546 p = 0.065
#> V2 -- V5 statistic 3.47 p = 0.075
#> V3 -- V5 statistic 2.214 p = 0.137
#> V2 -- V4 statistic 1.464 p = 0.214Goodman, L. A. (1979). Simple models for the analysis of association in cross-classifications having ordered categories. Journal of the American Statistical Association, 74, 537-552. doi:10.1080/01621459.1979.10481650
Pesarin, F. and Salmaso, L. (2010). Permutation Tests for Complex Data. Wiley. doi:10.1002/9780470689516
van Borkulo, C. D., van Bork, R., Boschloo, L., Kossakowski, J. J., Tio, P., Schoevers, R. A., Borsboom, D. and Waldorp, L. J. (2023). Comparing network structures on three aspects: a permutation test. Psychological Methods, 28, 1273-1285. doi:10.1037/met0000476
Winkler, A. M., Ridgway, G. R., Webster, M. A., Smith, S. M. and Nichols, T. E. (2014). Permutation inference for the general linear model. NeuroImage, 92, 381-397. doi:10.1016/j.neuroimage.2014.01.060