Unexpectedly, the theoretically best reject-region of T-test is bounded.

f_{t}\left(x;\mu,df\right)\equiv C\left(df\right)\left(1+\frac{\left(x-\mu\right)^{2}}{df}\right)^{-\frac{df+1}{2}} \lambda\left(x;\mu_{0},\mu_{1},df\right)\equiv\frac{f_{t}\left(x;\mu_{1},df\right)}{f_{t}\left(x;\mu_{0},df\right)}=\left(\frac{v+\left(x-\mu_{1}\right)^{2}}{v+\left(x-\mu_{0}\right)^{2}}\right)^{-\frac{df+1}{2}}{\longrightarrow\atop x\rightarrow\infty}1

For NHST H_{0}:T\sim t_{df} vs H_{1}:T-1\sim t_{df}, theoretically, p\left(t\right)=\int_{\left\{ x:\lambda\left(x\right)\ge\lambda\left(t\right)\right\} }f_{t}\left(x,\mu_{0},df\right)dx is s.t. \lim_{t\rightarrow\infty}p\left(t\right)=\frac{1}{2} , rather than zero. Nevertheless, pratically a large t, rejecting both H_{0} and H_{1}, should not be counted as any evidence to retain or reject H_{0}.

To verify the shape of \lambda\left(x\right) --



Classic Neyman-Pearson approach demo

It notes here that N-P approach does not utilize the information in the accurate p value. Actually, at the time N-P approach was firstly devised, the accurate p value was not available usually. Now almost all statistic softwares provide accurate p values and the N-P approach becomes obsolete. Wilkinson & APA TFSI (1999) recommended to report the accurate p value rather than just significance/insignificance, unless p is smaller than any meaningful precision.




--

Wilkinson, L. & APA TFSI (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.

习题:一类错误的注水

一个研究者每次都先看一下计算出的统计量再决定对零假设 \mu=0做单尾检验还是双尾检验。如果统计量 \bar{X}>0,就设对立假设为 \mu>0;如果统计量 \bar{X}<0,就设对立假设为 \mu<0。假如他的 \alpha=0.05请问他真实的一类错误率是多少?具体说,有许多次的实验,真实情形都是 \mu=0,他能检验出显著拒绝的比例会趋近于多少?

The tail(s) of p value


For any given H_0 vs H_1, the p value of any given point x is \underset{\theta\in H_{0}}{sup}P\left(\left\{ z|L\left(z\right)\ge L\left(x\right)\right\} |\theta\right), Where L\left(x\right)\equiv\frac{\underset{\theta\in H_{1}}{sup}\left[f\left(x|\theta\right)\right]}{\underset{\theta\in H_{0}}{sup}\left[f\left(x|\theta\right)\right]}

-- See R. Weber's Statistics Note (Chap 6.2 & 7.1)

I made some wrong comment on the pdf Null Ritual (Gigerenzer, Krauss, & Vitouch, 2004) Where three types of significance level (rather than p value) were discussed. I had written the comment to note that the chapter had ignored the role of H_1 in definition of p value. In almost every textbook, the two-tail p vs single-tail p are differentiated. Usually, the two-tail p is defined by H_1 like \mu\neq0.

Here I demonstrate a three-tail p value case on R platform.


z=(-1000:1000)*0.02;
f=0.5 * dchisq(abs(z),df=5);
h=dchisq(10,df=5)*.5;
plot(z,f,type="h",col=c("black","grey")[1+(f>h)]);
lines(c(-20,20),c(h,h));
## H_0 is \chi^2(5) * binomial(-1 vs 1) ##

Do you agree the region nearby zero under the "V" curve (which is below the horizontal line) should be the 3rd tail? I think so, if only H_1 includes all other possible distributions in the same shape.

You'll also agree there will be two asymmetrical tails if H_1 includes just two asymmetrical curves, for example, \mu=-2 and \mu=1 ( \sigma^2\equiv1) while H_0 is the standardized normal distribution.