New post based on an interesting recent conversation ‘Test-based confidence intervals or confidence-interval-based tests?’ open.substack.com/pub/tpmorris...

Test-based confidence intervals or confidence-interval-based tests?

Confidence intervals and tests often correspond. That is, if a one-sided test returns p=0.05 then the bound of a one-sided confidence interval should touch the null H0; if a one-sided test returns p>0...

There is really no reason that two different tests of the same null should agree though? I can't see why that would indicate that one of them does not preserve type 1 under some nulls.

Test based confidence intervals are known to be invalid when they use the variance under the null. They were historically used for Mantel Haenzel analyses. This was the motivation for Robins, Breslow and Greenlands work to derive a valid variance estimator (pubmed.ncbi.nlm.nih.gov/3741973/)

Estimators of the Mantel-Haenszel variance consistent in both sparse data and large-strata limiting models - PubMed

This paper proposes a new estimator of the variance of the Mantel-Haenszel estimator of common odds ratio that is easily computed and consistent in both sparse data and large-strata limiting models. M...