The Two Famous Rabbis Experiments: How Similar is Too Similar?

Gil Kalai
Brendan McKay
Maya Bar-Hillel

Witztum, Rips and Rosenberg describe the outcomes of two experiments which purport to statistically prove the existence of a hidden code in the Book of Genesis. We show that these two experiments, viewed as two random samples from the same pop- ulation, yielded numerical outcomes which are more similar to each other than expected. We also show that the distributions obtained in some control experiments performed by Witztum et al. are flatter than expected. Our hypothesis is that Witztum et al. tailored their experimental procedures to meet naive expectations regarding how outcomes of ex- perimental replication and experimental controls should look. We give some statistical and empirical evidence supporting this hypothesis.

September, 1998