Same problem in others: Mann Whitney: u_test Re: Kolmogorov smirnoff test for two samples: bug

[Kurt Hornik]

>>>>> Dave Goel writes:

> Dave Goel <deego3 at gmail.com> writes:

>> Consider a=[1 2 3 4], and b=[5 6 7 8].
>> 
>> The null is a = b.  The alt. is !=.
>> 
>> We get 
>> 
>> octave:3> kolmogorov_smirnov_test_2 (a,b,"!=")
>> pval: 0.0366311
>> 
>> 
>> 
>> Now consider an even more ridiculous null of a>b.  In this case, the
>> alt. is "<", So, p should get even lower. But, 
>> octave:5> kolmogorov_smirnov_test_2 (a,b,"<")
>> pval: 1
>> 
>> 
>> Now, consider a very plausible null of a<b. The alt. is ">".  P should
>> now be about 1, but we get:
>> 
>> octave:6> kolmogorov_smirnov_test_2 (a,b,">")
>> pval: 0.0183156
>> 
>> 
>> It seems that the meaning of < and > are flipped in the test.  Note
>> that the third argument refers to ALT, and not to NULL, as for
>> example, seen for the case of !=.
>> 
>> 
>> 
>> I have tested this with the latest octave as well. 


> I can now report that the exact same problem exists in u_test of
> octave, and probably more generally, in others as well.


> (FWIW, I checked matlab's ttest2, and there, the meaning of alt there
> is correctly interpreted - its results match its documentation for
> "both", "right" and "left").

Looks like code and docs are not in sync.

What the correct interpretation of the one-sided alternatives and thence
the correct fix is is not trivial: e.g., in R "less" (<) indicates that
F_y < F_x (in contrast to the meaning for the t and wilcoxon tests).

-k