[英]Kolmogorov-Smirnov 2 Sample Test Java give 0 p-value
I am using the Apache Commons Math Kolmogorov-Smirnov test in order to determine if the sample my RNG is producing is a Uniform Distribution.我正在使用 Apache Commons Math Kolmogorov-Smirnov 测试来确定我的 RNG 生成的样本是否是均匀分布。
I am using UniformIntegerDistribution
to produce a uniform distribution and I get a sample of 2000000 integers.我正在使用
UniformIntegerDistribution
来产生均匀分布,我得到了 2000000 个整数的样本。 Then I put them into a double[]
然后我把它们放到一个
double[]
I produce from my RNG 2000000 numbers as well and put them in a double[]
.我也从我的 RNG 2000000 数字中生成并将它们放在一个
double[]
中。
I have plotted the sample and I see it is uniform but the KS test gives me a p-value of 0.0 which would indicate that the null hypothesis of the two being drawn from the same distribution (ie Uniform) is invalid.我已经绘制了样本,我看到它是均匀的,但 KS 检验给我的 p 值为 0.0,这表明 null 假设两者来自同一分布(即均匀)是无效的。 Meaning that my RNG sample is not conforming to a uniform distribution.
这意味着我的 RNG 样本不符合均匀分布。
double alpha = test.kolmogorovSmirnovTest(a, b);
give me alpha = 0.0
给我
alpha = 0.0
And the method's Javadoc reads:该方法的 Javadoc 内容如下:
Computes the p-value, or observed significance level, of a two-sample Kolmogorov-Smirnov test evaluating the null hypothesis that x and y are samples drawn from the same probability distribution.
计算两样本 Kolmogorov-Smirnov 检验的 p 值或观察到的显着性水平,该检验评估 null 假设,即 x 和 y 是从相同概率分布中抽取的样本。
So I would expect the p-value to be high given that the I see the plot to be clearly uniform.所以我预计 p 值会很高,因为我看到 plot 显然是均匀的。
IntegerDistribution uniform = new UniformIntegerDistribution(1, 81);
ArrayList<Integer> lis = new ArrayList<>();
int i = 0;
while (i < 100000) {
//Creates a list of 20 numbers ε [1,80]
List<Integer> l = ls.createRandomNumbersInclusive(80, 20);
lis.addAll(l);
Assertions.assertFalse(l.stream().anyMatch(it -> it > 80));
Assertions.assertFalse(l.stream().anyMatch(it -> it < 1));
i++;
}
KolmogorovSmirnovTest test = new KolmogorovSmirnovTest();
var sample = uniform.sample(2000000);
List<Integer> ll = new ArrayList<>();
double[] a = new double[2000000];
for(var j = 0; j<2000000; j++) {
a[j] = sample[j];
}
double[] b = lis.stream().map(it -> Double.valueOf(it)).mapToDouble(Double::doubleValue).toArray();
var alpha = test.kolmogorovSmirnovTest(a, b);
System.out.println("Alpha "+ alpha); //This gives me 0.0
/** I am doing the below to get the count per numbers [1,80] and plot them.
* I see them being uniform
* 1 ===
* 2 ===
* ...
* 80 ===
*/
Map<Integer, Long> result = lis.stream().collect(Collectors.groupingBy(it -> it, Collectors.counting()));
What worries me is that if I create a new UniformIntegerDistribution
and get a sample2
and then put this in the test.kolmogorovSmirnovTest(a, b);
让我担心的是,如果我创建一个新的
UniformIntegerDistribution
并获得一个sample2
然后将其放入test.kolmogorovSmirnovTest(a, b);
, I indeed get a p-value close to 1 which is what I expect. ,我确实得到了一个接近 1 的 p 值,这是我所期望的。
I am either doing something wrong with Java or there is something in the numbers produced by the RNG that I am not getting.我要么对 Java 做错了什么,要么在 RNG 产生的数字中有一些我没有得到的东西。
The code for the createRandomNumbersInclusive
is createRandomNumbersInclusive
的代码是
public List<Integer> fetchNumberList(final int drawNumberMin, final int drawNumberMax, final int drawNumberCount) {
final List<Integer> range = new ArrayList<Integer>();
for (int i = drawNumberMin; i <= drawNumberMax; i++) {
range.add(i);
}
Collections.shuffle(range, rng);
return new ArrayList<Integer>(range.subList(0, drawNumberCount));
}
And the RNG is rng = SecureRandom.getInstance("NativePRNGNonBlocking");
RNG 是
rng = SecureRandom.getInstance("NativePRNGNonBlocking");
I found the reason behind the problem.我找到了问题背后的原因。 The
UniformRealDistribution
which I had used initially as this works with kolmogorovSmirnovTest(RealDistribution distribution, double[] data)
我最初使用的
UniformRealDistribution
与kolmogorovSmirnovTest(RealDistribution distribution, double[] data)
一起使用
For some reason however, UniformIntegerDistribution
is inclusinve-exclusive.然而,出于某种原因,
UniformIntegerDistribution
是包容性的。
When I changed IntegerDistribution uniform = new UniformIntegerDistribution(1, 81);
当我更改
IntegerDistribution uniform = new UniformIntegerDistribution(1, 81);
to IntegerDistribution uniform = new UniformIntegerDistribution(1, 80);
到
IntegerDistribution uniform = new UniformIntegerDistribution(1, 80);
it worked.有效。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.