estingRandomNum berGenerators Dan Biebighauser Univ ersit y of Minnesota  Twin Cities REU Summer   estingRandomNum berGenerators In troduction This pap er is summary of the researc conducted during s

estingRandomNum berGenerators Dan Biebighauser Univ ersit y of Minnesota Twin Cities REU Summer estingRandomNum berGenerators In troduction This pap er is summary of the researc conducted during s - Description

This researc h men tored b y Prof aul Garrett w as initially directed to ards comparing di eren t random n um b er generators and presen ting the results This has certainly b een done b efore see the pLab pro ject at httpr andommatsb gacat for exten ID: 26840 Download Pdf

186K - views

estingRandomNum berGenerators Dan Biebighauser Univ ersit y of Minnesota Twin Cities REU Summer estingRandomNum berGenerators In troduction This pap er is summary of the researc conducted during s

This researc h men tored b y Prof aul Garrett w as initially directed to ards comparing di eren t random n um b er generators and presen ting the results This has certainly b een done b efore see the pLab pro ject at httpr andommatsb gacat for exten

Similar presentations


Tags : This researc men
Download Pdf

estingRandomNum berGenerators Dan Biebighauser Univ ersit y of Minnesota Twin Cities REU Summer estingRandomNum berGenerators In troduction This pap er is summary of the researc conducted during s




Download Pdf - The PPT/PDF document "estingRandomNum berGenerators Dan Biebig..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.



Presentation on theme: "estingRandomNum berGenerators Dan Biebighauser Univ ersit y of Minnesota Twin Cities REU Summer estingRandomNum berGenerators In troduction This pap er is summary of the researc conducted during s"‚ÄĒ Presentation transcript:


Page 1
estingRandomNum berGenerators Dan Biebighauser Univ ersit y of Minnesota - Twin Cities REU Summer 2000
Page 2
estingRandomNum berGenerators In troduction This pap er is summary of the researc conducted during six eeks at the REU Summer 2000 program at the Univ ersit y of Minnesota - Twin Cities. This researc h, men tored b y Prof. aul Garrett, w as initially directed to ards comparing di eren t random n um b er generators and presen ting the results. This has certainly b een done b efore (see the pLab pro ject at http://r andom.mat.sb g.ac.at for extensiv e results),

and in the course of reviewing others' results I b ecame more in terested in the actual tests used to compare di eren random n um b er generators. great source for explanations of these tests is the landmark w ork The rt of Computer Pr gr amming D. E. Kn uth (second edition, 1981). My pro ject shifted to ards studying this w ork (m uc h of this pap er is due to it) and preparing a summary of some of the tests used. This pap er b egins b yin tro ducing random n um b ers and random n um b er generators, then explains some empirical tests (with an initial fo cus on the c hi-square and Kolmogoro

v-Smirno v (KS) tests, the foundations for the empirical tests) and also the theoretical sp ectral test. WhatisaRandomNum ber? e all probably ha ean in tuitiv e feeling for what andomness is. or example, compare the follo wing sequences of heads and tails generated b y a fair coin ( (`heads') = 1 2): HTHTHTHTHTHTHTHTHTHT HTHTTHHHTHTHHTTTTHTH Most p eople w ould probably feel that the second sequence is random while the rst is not, ev en though b oth sequences ha e the same 1 20 probabilit y of exactly o ccuring (to b e honest, I did ip an actual coin for the second sequence and not the rst).

Some re ection will sho w that this `feeling' is v ery di∆cult to de ne in an y mathematical sense. De - nitions of randomness, o er time, ha e included h uman ignorance of initial conditions or h uman indi erence to what comes next in sequence of ev en ts. This is subje ctive andomness and implies that randomness is only presen tin h uman minds, not in the ob jectiv ew orld [Bennett 154]. Other de nitions rely on b eing unable to `predict' future ev en ts based on past ev en ts (lea ving the question of de ning `predictabilit y'; as Prof. Garrett men tioned, in a sequence suc

has1,2,3,4,5,6,7,8,9,0,1,2,... it's p ossible that someone could b e \to o dum b" to predict the next n um b er { do es that mak e the sequence `random' ?) Complexit y theory sheds a di eren t ligh t on the sub ject of randomness. Supp ose there is a nite set of ob jects and a nite set of descriptions of these ob jects Then let b e a function from to , and for eac ob ject in let there exist a description in suc that )= Then eac h ob ject has a description. urther, let eac h description b e a nite string. The descriptional c omplexity of an ob ject is the length of the string needed to describ

e it. This set-up a oids the w ell-kno wn Ric hard-Berry parado x [Li 1]. It also giv es another informal de nition of randomness - a string of n um b ers is `random' if its descriptional complexit y is the same as the length of the string. or example, the string: 1000100010001 00 01 00 01 00 01 00 01 00 010 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 can b e describ ed as `16 copies of 1000', while the string: 0101011110100 01 10 01 01 10 10 00 01 10 001 10 00 01 10 01 11 11 01 00 01 00 01 01 01 10 do esn't ha an ob vious description shorter than the string itself. With the ab o de nition,

this string could therefore b e considered random. Most strings of length do not ha ea description shorter than and listing the string itself is the b est p ossible description in these cases [Garrett 77]. If sequence is generated linear feedbac shift register (to be describ ed later), then the line ar omplexity of the sequence can b e determined, whic his m uc h shorter than the sequence itself for large sequences, b y use of the Massey-Berlek amp algorithm (the researc h fo cus of another REU 2000 studen t, Erin Casey) [Garrett 212].
Page 3
Kn uth, in his b o ok (section 3.5) lists

v e `insu∆cien t' de nitions of an in nite random sequence b efore coming to a sixth de nition that \surely meets all reasonable philosophical requiremen ts for randomness" [Kn uth 156]. This de nition is complex, but it migh t b e illustrativ e to examine an insu∆cient de nition of randomness to see wh y it fails: Ase quenc eof r al numb ers , ... in the interval [0,1) is de ne dtob e\r andom" if, whenever is pr op erty such that ([ ]) holds with pr ob ability one for se quenc ... of indep endent samples of r andom variables fr om the uniform distribution, then ([ ]) is true. (In this pap er,

w e let [ denote the en tire sequence.) This is a bad de nition, and here's wh y: no se quenc satis es this ondition. Let be the prop ert that no elemen t of the sequence is equal to a xed n um ber certainly has probabilit y one, and so an sequence with is not random b y this de nition. Examine our sequence [ ], and let Then our sequence is not random since ([ ]) is false [Kn uth 153]. Clearly randomness is not an easy thing to de ne. or more of the history of attempts to de ne randomness, as w ell as other de nitions, see Ch. 9of andomness y D. J. Bennett. RandomNum berGeneration No w that w

eha e a little bit of a handle on what random n um b ers are, w e can lo ok at w ys to create these n um b ers. Although there are man y metho ds of doing this, they all fall in to t o categories: deterministic and non-deterministic approac hes. e'll lo ok at the latter rst. Certainly ,an y deterministic metho d of creating sequences of n um b ers (a form ula where the input com- pletely determines the output) will not create random n um b ers [Garrett 212]. (It w ouldn't b e a pap er on random n um b ers if it didn't include the oft-quoted statemen t from v on Neumann: \An one who considers

arithmetical metho ds of pro ducing random digits is, of course, in state of sin" [Kn uth 1].) Our only hance to get truly random n um b er sequences (w eha en't de ned the meaning of `truly' here) is through a non-deterministic approac h. Examples of these, whic h either ha e b een used or are b eing used curren tly with arying degrees of acceptabilit include ipping coins, rolling dice, pic king slips out of ell-mixed hat (balls from an urn, etc.), keyb ar d latency (the time elapsed b et een k eyb oard actions on a computer), atmospheric noise pic ed up b y a radio receiv er (this is done at

http://www.r andom.or and it's where the second sequence in the complexit theory discussion came from), and p ersonal fa orite, the motions of la lamps (see http://lavar and.sgi.c om ). The b est source of random um b ers seems to be radioactiv materials, generated ho oking computer up to Geiger coun ter (this is b eing done at the ourmilab http://www.fourmilab.ch/ho tbits/ )). While these sources are in teresting in their o wn righ t, and the sequences they generate can b e analyzed y the tests w e'll discuss later, pseudo-r andom numb er gener ators (pRNG's), deterministic form ulas to

create sequences that lo ok random, are go o d enough for most applications if used carefully and ha e more in teresting mathematical prop erties. or the remainder of this section, assume that random um ber generator (RNG) is really pseudo-random. Later, in the discussion of testing RNG's, empirical tests will ev aluate the randomness of an y sequence, regardless of its source, and the (theoretical) sp ectral test w orks only for the linear congruen tial generator (LCG), a pRNG to b e describ ed shortly There are man yt yp es of pRNG's; the purp ose of this pap er is not b yan y means to giv e

a comprehensiv e list, but to list a few and some of their prop erties to giv e some p ersp ectiv e b efore describing the testing of sequences. Line ar c ongruential gener ators (LCG's) are de ned b y selecting the four in tegers: , the mo dulus, with m> , the m ultiplier, with 0 a , the incremen t, with 0 c , the seed, with 0 and de ning the sequence [ ] of `random n um b ers' b y the recursiv +1 =( ax )% (`%' in this sense means to reduce the result mo d , lea ving the smallest p ositiv e remainder.)
Page 4
or example, let =8, = 3 and =2. The sequence generated b y these v alues

is ; ::: Since the de nition is recursiv e, if a n um b er reapp ears in the sequence (as one ev en tually m ust, since is nite), then the en tire sequence b egins to rep eat. The erio of a sequence is a cycle of endlessly rep eating um b ers, and the length of the p erio d is de ned to b e the smallest suc h that for all indices (w e could force to b e greater than some to allo w for some early non-rep etition in a pRNG)[Garrett 213, 214]. The ab o e sequence therefore has a p erio d of length 4. It is desirable to ha e the p erio d of a pRNG b e as long as p ossible (to mak e it more

`random'). There are w ys to do this; without going in to an y of the details, w e brie y presen t them here for LCG's (for m uc more on LCG's and pro ofs of the follo wing details, see Kn uth, p. 9-25, and Garrett, p. 213-216, 219-222): (due to Carmic hael) With = 0 and prime, the maxim um p ossible p erio d of 1is ac hiev ed when is relativ ely prime to and is a primitiv e ro ot mo dulo is a primitive ot mo dulo m if for ev ery relativ ely prime to there is an in teger so that mo d [Garrett 68])[Kn uth 19]. A LCG has p erio d length if and only if: 1) is relativ ely prime to 2) 1is am

ultiple of , for ev ery prime dividing 3) 1is am ultiple of 4, if is a m ultiple of 4 [Kn uth 16] Another consideration is the p otency of a LCG with p erio d The otency is de ned as the smallest p ositiv ein teger suc h that 1) = 0 mo d will exist if the ab o e requiremen ts for a p erio d of length are met)[Kn uth 23]. Kn uth recommends a p otency of at least 4 and probably 5 or higher. This condition is necessary but not su∆cien ttoha e a `good' LCG [Kn uth 24]. While LCG's are su∆cien t for some applications, there are b etter pRNG's. Related to the LCG's are the line ar fe db ack shift r

gisters (LFSR's). The follo wing discussion is based on Garrett's w ork (p. 216-217, 222-225): With size and mo dulus ho ose co e∆cien ts ;c ; :::; c and se (lik the LCG's ;s ; :::; s ) (these will b e the rst n um b ers of the sequence). Then de ne +1 =( ::: 1) )% for all +1 (Note that if and then this is the Fib onacci sequence ; ::: ) This pro cess is more easily expressed in terms of matrices. As a simple example, let =3 (with co e∆cien ts ( ;c ;c )) and write Then the recursiv e de nition ab o e can b e written as +1 or instance, let =2, ( ;c ;c )=(0 1) and ( ;s ;s )=(0 1). Then the

sequence (including the initial 0,0,1) is ; ::: Note that in a LFSR, if a pattern of consecutiv e elemen ts that o ccured earlier in the sequence recurs later, the en tire sequence b egins to rep eat. Here the initial 0,0,1 recurs after 7 steps, and the LFSR has p erio d 7. These ideas are similar to those of the LCG's, but it can b e seen that it is easier to create longer p erio ds with LFSR's since more than one elemen t needs to recur for the sequence to b egin rep eating.
Page 5
The p erio d of a LFSR can also b e examined. Let the char acteristic olynomial of the matrix b e (as

usual) det ( xI e state without pro of (see Garrett) that if the c haracteristic p olynomial of in ] is primitiv e, then the asso ciated LFSR has the maxim um p erio d 1 for any initial seed other than all zeros. (A p olynomial of degree in ]is primitive if divides 1 but do es not divide 1 for an yin teger with ). Note the di erence in maxim um p erio d size for a LFSR with mo dulus compared to a LCG with the same mo dulus. Let's examine one more pRNG for comparison. The Blum-Blum-Sh ub generator is included here for three reasons. First, it's the pRNG used with the la lamps men tioned

earlier. Second, it's fun to sa out loud. Third, and most imp ortan t, it's a really go o d generator, pr ovably secure assuming that it is har (the algorithm do esn't run in p olynomial time) to factor large n um b ers in to primes (whic h is b eliev ed to b e true) [Garrett 217]. This fact mak es this pRNG m uc h more useful for cryptographic applications than the previous t o (whic h should not b e used for an y serious cryptograph y), but eac h bit in the sequence is more exp ensive (di∆cult) to compute, so the LCG's and LFSR's ould b e b etter for simpler applications when used with care.

The Blum-Blum-Sh ub generator w orks as follo ws: Select t ov ery large primes p; q b oth congruen t to 3 mo d 4 and compute the pro duct pq This is the mo dulus. Start with seed and compute the sequence ;s ;s ; ::: y the form ula +1 rom this sequence, create a sequence of bits b %2 This is the desired (pseudo)random sequence [Garrett 217]. Of course, there are man y other pRNG's in use, man y b etter than those describ ed here (or at least b etter than the rst o). The purp ose of this section as to pro vide some insigh as to ho random um b ers migh t b e generated. ApplicationsforRandomNum

bers eno w kno w some metho ds for creating random n um b ers, but an unansw ered question here is wh yw ould w an ttodosuc h a thing. This section will brie y list some applications of random n um b ers, and then e'll mo e on to the testing of random n um b er sequences. Here are some uses for random n um b ers (some uses are ob viously more mathematical in nature than others): 1. Simulation Random um b ers are almost alw ys necessary to mak realisitic mo del of natural phenomena. These sim ulations include economic, tra∆c, n uclear ph ysics and man y other mo dels. A b ene t of some of the

pRNG's is that the sequence can b e set so that it starts at the same place in the sequence eac h time, allo wing one to lo ok at the e ects of v arying certain parameters while exp osing it to the same `random' sequence eac h time [Bennett 150, Kn uth 1]. 2. Statistic al Sampling Ob viously , samples are sometimes needed to study a larger collection of things, and a random sample w ould lend itself to the b est p ossible accuracy of these statistical tests [Bennett 148, Kn uth 1]. 3. Crypto gr aphy Public ey crytograph systems mak use of large amoun ts of random data. or example, RSA requires

the use of large, random primes for its securit One-time pads require long eystream of random in tegers (if these in tegers are generated b y a p erio dic pRNG, the p erio d needs to b e longer than the message to b e enco ded, or the cipher e ectiv ely degenerates in to a Vigenere cipher, whic is vulnerable to attac k (unlik e a one-time pad)) [Bennett 149, Garrett 82, 213]. 4. Computer Pr gr amming Man computer algorithms in use require random um ber or random sequence, and random n um b ers can b e used as input to test the e ectiv eness of an algorithm [Kn uth 1].
Page 6
5.

Numeric al A nalysis Man y problems that are to o di∆cult to solv e (or quic kly solv e) can b e appro xi- mated b y tec hniques relying on random n um b ers (Mon te Carlo metho ds, for example) [Bennett 136, Kn uth 1]. 6. De cision Making In `real life' decisions, computer algorithms (men tioned ab o e) and game theory randomness can pla y an imp ortan t role [Kn uth 2]. 7. cr ation Finally , there are man y fun applications of random n um b ers (not that the `mathematical' ones aren't fun), including gam bling and computer games [Kn uth 2, www.r andom.or ]. An imp ortan thing to remem ber

when dealing with applications for random um b ers: no andom numb er gener ator is go for every applic ation [Kn uth 173]. As a simple example, a truly random in nite sequence of zeros and ones will con tain a million zeros in a ro w (in fact, this will happ en an in nite n um ber of times). This b eha vior is necessary for an application requiring extremely large amoun ts of random n um b ers (otherwise the sequence couldn't b e considered random), but it is dev astating for an application that just needed, sa thousand random digits and receiv ed thousand zeros from the RNG (this ties in to

the distinction b et een glob al nonrandomness and lo al nonrandomness). Caution should alw ys b e used when ho osing a RNG for a situation, and more than one RNG is sometimes needed for an application [Kn uth 145]. estingRandomNum berGenerators So far, w e'v e seen that there are man y di eren t applications requiring random n um b ers, and that there are man y di eren tw ys of acquiring them. e'v e men tioned ab o e that not all sources of random n um b ers b eha e in the same w , and that some are b etter than others, at least for di eren t applications. This b egs the question: ho w can w

e tell if a random n um b er generator is `go o d' (or `go o d enough')? e'v e already seen that there are w ys to maximize the p erio d of a LCG or a LFSR. Certainly , this is not the only requiremen ould demand of a RNG. or example, it ould b e imp ortan for a RNG designed to pro duce a long sequence of zeros and ones to pro duce them in roughly equal quan tities. It ould b e nice for there to be indep endence bet een elemen ts or subsequences of sequence pro duced RNG. In addition, RNG should b e fairly e∆cien t in order to b e of an y real use. Not surprisingly there are a lot of di eren

tests for RNG's and the sequences they pro duce. These tests can be divided in to distinct groups: empirical tests and theoretical tests. Empiric al tests are conducted on sequence generated RNG, and require no kno wledge of ho the RNG pro duces the sequence. The or etic al tests, whic h are b etter when they exist, are a priori tests in the sense that they require a kno wledge of the structure of the RNG but the sequence do es not necessarily need to b e generated [Kn uth 38, 75]. e will fo cus mainly on the empirical tests here. Before w e lo ok at these tests, there are t o ma jor tests

that pro vide the foundations for the empirical tests, the chi-squar test and the Kolmo gor ov-Smirnov test, and these will be discussed in detail. ollo wing this, e'll list some of the empirical tests with brief descriptions. The sp ctr al test is a theoretical test used for LCG's and will b e brie y describ ed as w ell. All of the follo wing has its ro ots in Kn uth's b o ok, p. 38-113, and the in terested reader is referred to this excellen ork for additional details and pro ofs of the follo wing material. TheChi-SquareT est The c hi-square ( test w as initially published y Karl P earson in

1900. earson's original notation in explaining the theory b ehind the test included use of the sym b ol; hence the name. This test can b e used in man y situations and basically , when giv en an outcome of an exp erimen t, can giv e an appro ximate probabilit yasto ho w lik ely that outcome is [Kn uth 39, 52, 54]. Supp ose w e had indep enden t observ ations (in our case, p erhaps elemen ts of a sequence generated b a p ossible RNG), eac h falling in to one of categories. Let b e the n um b er of observ ations falling in to the th category and b e the probabilit y that an observ ation falls in

to category Then w ew ould exp ect that np for large v alues of e'd lik eaw y to measure `ho w far o ' w e are from these exp ected v alues, so de ne the reasonable statistic (the here stands for `inadequate,' as w e'll shortly see) as the follo wing =( np +( np ::: +( np
Page 7
(the squaring is to mak e eac h term p ositiv e; without this, discrepancies could `balance out'). This will giv some measure as to ho w close the actual results are to the exp ected. Note that giv es equal w eigh ttoeac h category If not ev ery is the same, then can b e misleading b ecause it could `o

eremphasize' some discrepancies and `hide' others. e mo dify the statistic to the one that's actually used, whic giv es the correct w eigh tto eac h term. (called the chi-squar statistic of ,..., [Kn uth 40]) is de ned as np np np np ::: np np np np This solv es the w eigh t problem (if only it w ere so easy for h umans). can b e written another w , whic is often easier to compute. e tak e adv an tage of t o fairly ob vious facts: ::: ::: =1 and write as np np np np np np No w the imp ortan t question: what's a reasonable v alue for ew ould exp ect it to b e bigger than zero ( probably do

esn't qual np for large v alues of ), but it shouldn't b e to o large. e b egin to answ er this question b y referring to a table suc h as the follo wing [Kn uth 41]: ABLE 1 Selected P ercen tage P oin ts Of The Chi-Square Distribution 01 05 25 50 75 95 99 =1 00016 00393 1015 4549 323 841 635 =2 02010 1026 5753 386 773 991 210 =3 1148 3518 213 366 108 815 11 34 =4 2971 7107 923 357 385 488 13 28 =5 5543 1455 675 351 626 11 07 15 09 =6 8720 635 455 348 841 12 59 16 81 =7 239 167 255 346 037 14 07 18 48 =8 646 733 071 344 10 22 15 51 20 09 =9 088 325 899 343 11 39 16 92 21 67 =10 558 940 737 342

12 55 18 31 23 21 =11 053 575 584 10 34 13 70 19 68 24 73 =12 571 226 438 11 34 14 84 21 03 26 22 =15 229 261 11 04 14 34 18 25 25 00 30 58 =20 260 10 85 15 45 19 34 23 83 31 41 37 57 =30 14 95 18 49 24 48 29 34 34 80 43 77 50 89 =50 29 71 34 76 42 94 49 33 56 33 67 50 76 15 o use this table, read the line with 1. Then compare to the en tries in that ro w. or example, if =9 ( 1 = 8), then the fact that the 99 p ercen ten try is 20.09 means that 20 09 around 99 p ercen t of the time and w ew ould exp ect 20 09 to o ccur only ab out 1 p ercen t of the time (it should b e rep eated that the c

hi-square test giv es appro ximate probabilities), and a of 35 w ould b e prett y suspicious.
Page 8
One migh t ask wh yw e lo ok at the ( 1)th ro w. is called the n um ber of de gr es of fr dom [Kn uth 41]. This can b e explained m uc h more precisely (see Kn uth, p. 53), but in tuitiv ely , the ;Y ; :::; Y can b e seen as indep enden tP oisson random v ariables except for the fact that ::: ,soif w e kno w, sa ;Y ; :::; Y ,w e can compute , meaning that only 1ofthe 's are `indep enden t' [Kn uth 41]. Note that these um b ers in the table app ear to only be related to amazingly not

the um ber of observ ations or the probabilities of landing in eac category This is not en tirely correct, since the um b ers in the table are appro ximations that are only alid for large alues of [Kn uth 42]. Kn uth recommends that be big enough so that eac exp ected alue np is at least 5, although this seems to b e a fairly arbitrary c hoice (according to http://www.windsor.igs.net/ (full address in bibliograph y), it has recen tly b een demonstrated that in eac category is not necessary as long as there's something in eac category). The correct c hoice of for eac situation is not clear,

since really large can b etter detect global nonrandomness but `smo oth out' lo cal nonrandomness. Since the test really should b e run more than once on a sequence an yw y (recall the `million zeros' discussion; just b ecause something only happ ens 1% of the time do esn't mean it can't or shouldn't happ en), these di eren t tests could use di eren tv alues for hop efully making the results more accurate. Ho wdow ein terpret the results of a c hi-square test? Kn uth recommends a fairly arbitrary metho d (this will b e impro ed on later, although it's go o d enough for no w). Consider less

than the 1% en try or greater than the 99% en try to b e \nonrandom" bet een the 1% and 5% en tries or b et een the 95% and 99% en tries to b e \susp ect" bet een the 5% and 10% en tries (a more detailed table w ould b e necessary here) or b et een the 90% and 95% en tries to b e \almost susp ect" One w ould test parts of the `prop osed' random sequence at least three times, probably with a di eren eac h time, and if t o or more tests are \susp ect," the sequence is not su cien tly random [Kn uth 44]. Kn uth tested a few LCG's with the c hi-square test (see p. 44, 45); man y more LCG's and

other pRNG's ha e b een tested at the pLab pro ject ( http://r andom.mat.sb g.ac.at ). One ma jor question remains, and it w on't b e answ ered here. The reader migh tw onder ho w the ab o table is generated. The pro cess is not trivial and is to o complicated to include here, but an in terested reader should see Kn uth's b o ok, p. 53, 54, for the details. Again, the c hi-square test is a foundation for man y of the empirical tests w e'll describ e, and it is probably the most-used test for RNG's. TheKolmogoro v-Smirno vT est The Kolmogoro v-Smirno v (KS) test has its origins in a 1933 pap er

b A. N. Kolmogoro v, and N. V. Smirno v suggested some impro emen ts in 1939, leading to the join t name of the test [Kn uth 54]. The KS test is useful in areas where the c hi-square test is not, and can also b e used in conjunction with the c hi-square test. Because of this, it to o is foundational test for man of the empirical tests to follo w, and will be examined here. e rst in tro duce a common function in probabilit y theory Giv en a random v ariable , the cumulative distribution function (c df ) ), is de ned as ) = probabilit y that ( Note that an has range of [0 1] (sometimes

asymptotically) and will alw ys be increasing (or remain constan to er some in terv als) as increases from 1 to + [Kn uth 47]. Let tak eon the alues of the sequence generated b a RNG (w essen tially did the same thing in our discussion of the c hi-square test, except w e used ab o e). or the KS test, w e require that )be con tin uous. This is the exact opp osite of what did ab o e, where the ould ha b een nothing but `jumps' since as only allo ed to tak e on certain discrete v alues. The KS test deals with a di eren situation, where the n um b ers generated b y a RNG are allo ed to tak eonan

yv alue within a certain in terv al, leading to a con tin uous cdf. This ) is the theoretical distribution w ew ould exp ect our RNG to ha [Kn uth 45, 51].
Page 9
Supp ose w e mak indep enden t observ ations of , creating the v alues ;X ; :::; X e de ne the empiric al distribution function )as )= um ber of ;X ; :::; X that are The KS test compares )to )b y measuring the di erence b et een the t o distribution functions. When is su∆cien tly large, w ew ould exp ect the t o functions to b e similar if the sequence w e're examining is truly random. e therefore measure the di erence bet

een the t o functions b y forming the follo wing statistics: max )) , 1 max( )) , 1 In w ords, is the greatest deviation when is greater than , and is the greatest deviation when is less than (The factor is presen b ecause, for xed the standard deviation for is prop ortional to 1 ,som ultiplying b allo ws and to b e indep enden tof ) [Kn uth 47]. e use these statistics b y comparing them to the follo wing table in a manner similar to the c hi-square test [Kn uth 48]: ABLE 2 Selected P ercen tage P oin ts Of The Distributions Of And 01 05 25 50 75 95 99 =1 01000 05000 2500 5000 7500 9500 9900

=2 01400 06749 2929 5176 7071 0980 2728 =3 01699 07919 3112 5147 7539 1017 3589 =4 01943 08789 3202 5110 7642 1304 3777 =5 02152 09471 3249 5245 7674 1392 4024 =6 02336 1002 3272 5319 7703 1463 4144 =7 02501 1048 3280 5364 7755 1537 4246 =8 02650 1086 3280 5392 7797 1586 4327 =9 02786 1119 3274 5411 7825 1624 4388 =10 02912 1147 3297 5426 7845 1658 4440 =11 03028 1172 3330 5439 7863 1688 4484 =12 03137 1193 3357 5453 7880 1714 4521 =15 03424 1244 3412 5500 7926 1773 4606 =20 03807 1298 3461 5547 7975 1839 4698 =30 04354 1351 3509 5605 8036 1916 4801 e read the table as w e did with the test:

the probabilit y is 95 p ercen t that will b e less than 1.1463, and since the distribution is the same for b oth and ,w e could mak e the same statemen t for Unlik e the c hi-square test, the n um b ers in the ab o e table are not appro ximate v alues but are exact (within the rounding error) [Kn uth 48]. In addition, lik the hi-square test, need to b e careful ab out our selection of should be big enough so that w e detect if the distribution functions ) and ) are signi can tly di eren t, and y et b eing to o large will usually `smo oth out' lo cal nonrandomness. Kn uth recommends to b e

around 1000 (ob viously requiring a more extensiv table) [Kn uth 49]. Kn uth also describ es a pro cedure where a fairly large n um ber of 1000 statistics are calculated and then a KS test is made on these observ ations, thereb detecting b oth global and lo cal nonrandomness (see p. 49 for details). ha already seen some of the di erences bet een the hi-square and KS tests (namely that the hi-square uses discrete elemen ts while the KS w orks with a con tin uous sample space, although certainly a con tin uous sample space can b e `brok en up' in to distinct in terv als, allo wing for a c

hi-square test), but it is orth noting that they can be used together [Kn uth 50, 51]. In our discussion of the hi-square test, came up with a fairly inadequate w y of describing a sequence as \nonrandom," \susp ect," \almost susp ect" or supp osedly \random." No ww eha e a b etter pro cedure. Mak indep enden tests on di eren t parts
Page 10
of a random sequence and record the v alues ;V ; :::; V Then apply a KS test to these 's ( here is describ ed b y the plotted v alues of eac , and could b e found from an extension of T able 1). This is certainly b etter than our comparativ ely

arbitrary previous metho d [Kn uth 50, 51]. It's that time again in the section where the in terested reader migh tw an t to kno w the tec hnical details of wh y the KS test w orks, and once again the reader is referred to Kn uth's discussion of those details (see p. 54-56). or no w, w e will accept the v alidit y of b oth the c hi-square and KS tests, b oth individually and when used together, and use them as the building blo c ks of the empirical tests to follo w. EmpiricalT ests No w that the c hi-square and KS tests are in place, w e can mo e to a discussion of empirical tests used to

determine the `randomness' of a sequence. Again, empirical tests are used on a sequence pro duced b ya (prop osed) RNG and don't require kno wing exactly ho w the RNG op erates. e will expand the sequence notation in tro duced earlier. ] refers to the sequence ;U ;U ; ::: of real n um b ers b et een zero and one, supp osedly indep enden tly and uniformly distributed throughout that in terv al. refers to the sequence ;Y ;Y ; ::: of in tegers b et een 0 and 1 (with an in teger), using the rule dU and therefore preserving the same prop erties of b eing indep enden tly and uniformly distributed,

only that here this distribution is o er the in tegers b et een 0 and 1 [Kn uth 59]. Our purp ose here is to brie y describ e some of the empirical tests used. Kn uth do es this in his b o ok (p. 59-73), and the follo wing material is a summary of what he presen ts there. The reader should b e a are that Kn uth o ers more detail and probably b etter descriptions of the follo wing tests; ho ev er, this pap er is a summary of y researc ho er these past w eeks and therefore includes the follo wing material. Eac h test presen ted here will also include the pages where additional material can b e

found in Kn uth's b o ok. With this understanding, and the de nitions ab o e, pro ceed to describ e some of the empirical tests used to determine the `randomness' of a sequence. 1. Equidistribution or F quency T est [Kn uth 59] With an understanding of the material previously co ered, this is the simplest of the empirical tests. Giv en a sequence [ ], w ew ould require that its elemen ts b e uniformly distributed b et een zero and one. e can apply the KS test, with the ob vious )= for 0 1 (as it w ould ha e to b e for the sequence to ha e a uniform distribution; i.e., the probabilit y that 3

had b etter b e 2 3). Alternativ ely , select an yin teger and use [ ]. No ww e can get categories to use in a c hi-square test. or ev ery in teger suc that 0 r , coun t the n um b er of times that for 0 i Then eac h suc hin teger de nes a hi-square category , and w ew ould use the c hi-square test with (since there are in tegers b et een 0 and 1) and =1 =d (in order to b e uniformly distributed). This amoun ts to sa ying that, for example, if w e had a sequence of zeros and ones generated b y a RNG, w ew ould exp ect to get ab out as man y zeros as ew ould ones if w ew ere to tak e a sample

of the sequence. 2. Serial T est [Kn uth 60] The equidistribution test examines whether an individual elemen t in a sequence comes up more often than it should, but ould also lik pairs of successiv um b ers to be uniformly and indep enden tly distributed (for example, in binary sequence, (0,0), (0,1), (1,0) and (1,1) should all be equally lik ely). As Kn uth elo quen tly puts it, \the sun comes up just ab out as often as it go es do wn, in the long run, but this do esn't mak its motion random" [60]. do this, coun the um ber of times that the pair ;Y +1 q; r o ccurs for This should be done for

eac pair of in tegers q; r with q; r < d e then ha categories for a test, with a probabilit yof1 =d assigned to eac h category Instead of just pairs, this metho d can b e generalized to triples, quadruples and so on. The c hoice of has to b e a careful one, ho ev er, to meet Kn uth's ` v e in eac h category' requiremen t for the c hi-square test, and this gets more di∆cult as w emo e to larger subsequences. In practice, some of the follo wing tests are used in place of a generalized serial test. also note in passing that elemen ts of sequence are needed to mak observ ations. This is required;

if w ew ere to p erform the serial test on the pairs ( ;Y ;Y ; :::; ;Y ), our observ ations are b y no means indep enden t, and indep enden t observ ations are required for the test. 10
Page 11
3. Gap T est [Kn uth 60, 61] or eac in a certain range, this test examines the length of the `gap' b et een this elemen t and the next elemen t to fall in that range, and hence the name of the test. So, if and are t o real n um b ers suc that < 1, e're lo oking for the length of consecutiv e subsequences ;U +1 ; :::; U ;U +( +1) suc h that and +( +1) are b et een and but the other elemen ts in

the subsequence are not (this is a gap of length ). ew ould then p erform a test on the results using the di eren t lengths of the gaps as the categories, and the probabilities as follo ws: (1 ), (1 , ... (1 , ... Here , whic h is the probabilit y that an y elemen is b et een and Kn uth giv es the details of an algorithm to record the lengths of consecutiv e gaps. 4. Poker T est [Kn uth 62] As with the gap test, the name of the p ok er test suggests its description. e examine groups of v consecutiv ein tegers ( ;Y +1 ;Y +2 ;Y +3 ;Y +4 ) for 0 j and put eac h of these groups in to one of the

follo wing categories: All di eren t: abcde One pair: aabcd Tw o pairs: aabbc Three of a kind: aaabc ull house: aaabb our of a kind: aaaab Fiv e of a kind: aaaaa hi-square test is applied to these sev en categories, although less lik ely categories are sometimes group ed together in order to meet the ` v in eac category' requiremen t. Kn uth deriv es the necessary probabilities in his b o ok, and if categories w ere to b e com bined, their resp ectiv e probabilities w ould simply b e added together. Again, the groups of in tegers are not allo ed to o erlap in order to preserv e the indep

endence required b y the test. 5. Coup on Col le ctor's T est [Kn uth 62, 63] Using the in teger sequence [ ], this test examines the lengths of segmen ts +1 ;Y +2 ; :::; Y needed to get the complete set of in tegers from 0 to 1. The test is named b ecause, as Kn uth states, \w ema think of a b o y collecting yp es of coup ons, whic h are randomly distributed in his breakfast cereal b o xes; he m ust k eep eating more cereal un til he has one coup on of eac ht yp e" [63]. (I supp ose he w ouldn't ha eto at the cereal, but at least op en the b o x.) The lengths of these sequences mak e go o d

categories for (guess what?) the c hi-square test. Kn uth describ es the algorithm for implemen ting this test, as w ell as deriving the necessary probabilities. 6. Permutation T est [Kn uth 64] This test examines a [ ] sequence (that is, the elemen ts are real n um b ers b et een zero and one). tak e a p ortion of the sequence and divide it in to groups, eac h with elemen ts. eno wha e groups of the form ( jt ;U jt +1 ; :::; U jt +( 1) ) with 0 j In these groups, w e assume eac h elemen t is distinct from the others (a v alid assumption, since the probabilit yofan yt o elemen ts b eing equal

is zero) and put eac group in to a category dep ending on its relativ e ordering. or example, with = 4, the elemen ts of the group could be suc that +1 +2 +3 or +2 +3 +1 or an of the other p ossibilities. In this example, there are 4! p ossible relativ e orderings; in general, there are ! p ossibilities. hi-square test is then p erformed with ! and a probabilit yof 1 =t ! for eac h ordering (since eac h should b e equally lik ely in a random sequence). Kn uth giv es an algorithm for this test as w ell. 7. un T est [Kn uth 65-68] or this test, w e need a preliminary de nition. monotone sequence

is a sequence that has elemen ts that are either all increasing or all decreasing. or example, 8 and 9 0 are b oth monotone sequences, while 1 4 is not (although it con tains monotone subsequences). Here w e test a sequence for its `runs up' and `runs do wn' (again, w e see the descriptiv e test name); that is, w e examine the lengths of the sequence's monotone subsequences. 11
Page 12
As an example, w e divide the sequence 1 6in to `runs up' as follo ws: and w eha e a run of length 3, follo ed b yt o runs of length 1, another run of length 3 and a run of length 2. One w ould

initially think (esp ecially after the pattern of the previous tests) that the lengths of these runs could b e categories for a test. This is not the case, ho ev er, since long runs tend to b e follo ed b short runs, whic con ersely tend to be follo ed b long runs (w e're w orking either in a nite in terv al of real n um b ers or a nite collection of in tegers in our sequences, and it w ould b e unlik ely to ha e consecutiv long `runs up,' for example). This means that consecutiv e observ ations are not indep enden t from eac h other, whic hin alidates the use of a c hi-square test. Instead, w

e need to use a more complicated statistic whic his de ned and deriv ed in Kn uth's b o ok. e'll lea e that discussion out here, but at least w e understand the basic idea b ehind the test. 8. Maximum-of-t T est [Kn uth 68] This is the last empirical test w e'll describ e here. The idea is that w e break a sequence up in to subse- quences of equal length, tak e the maxim um v alue of eac h subsequence and apply the KS test (whic h nally returns) to those maxim um v alues. or 0 j , de ne y the rule max tj ;U tj +1 ; :::; U tj +( 1) (here e'v brok en the original sequence in to subsequences of

length ). no ha sequence ;V ; :::; V e apply a KS test to these v alues, using )= with 0 1 as our cdf for the comparison. Let's v erify that this is the v alid cdf. e're lo oking for the cdf of eac or a the probabilit that max tj ;U tj +1 ; :::; U tj +( 1) is the probabilit that tj and tj +1 and ... and tj +( 1) , whic h is the pro duct of the individual probabilities. So )= xx:::x and our KS test is v alid. This is b y no means an exhaustiv e list of the empirical tests used, and in fact Kn uth himself includes few more (p. 68-71), but it giv es a feel for the di eren kinds of empirical tests

used for RNG's. Most of these tests can also b e used on subsequences of the original sequence instead of just the elemen ts of the sequence [Kn uth 71], adding further to our testing capabilities. And, as migh t b e susp ected, some of these tests are b etter than others. or example, the equidistribution and serial tests tend to be `w eak er' (since most `random' sequences will pass them), while the run test tends to b e `stronger' in this sense [Kn uth 72]. e lea e the empirical tests for no w and examine the theoretical sp ectral test. TheSpectralT est As men tioned b efore, theoretical

tests di er from empirical tests in that they can predict ho w good a RNG will be without actually using a sequence generated b y the RNG. In addition, theoretical tests giv us more insigh t ab out the RNG itself, so they are b etter than empirical tests when they exist. Ob viously ho ev er, these tests cannot b e used for ev ery RNG (unlik e the empirical tests), they are usually hard to nd, and they tend to b e considerably more complicated than empirical tests [Kn uth 75]. or these reasons, and others, this pap er has so far fo cused on empirical tests and will a oid discussion of

theoretical tests except for the una oidable sp ectral test, \b y far the most p o erful test kno wn" [Kn uth 89]. A pap er on the testing of RNG's ould be glaringly incomplete without at least brief erview of this test, and that's exactly what w e'll pro vide here. Kn uth, of course, go es in to the full details and is w orth reading (p. 89-113). As alw ys, the follo wing material comes from Kn uth's b o ok. The sp ectral test has its ro ots in 1959 pap er N. M. Korob o v, and as dev elop ed and impro ed up on b y man y authors in m ultiple pap ers throughout the 1960's and 1970's [Kn uth

110]. It is a p o erful 12
Page 13
test b ecause \all go o d generators pass this test [and] all generators no w kno wn to b e bad actually fail it" [Kn uth 89]. The test deals with LCG's. Recall that a LCG is a pRNG de ned b y the follo wing: Select the follo wing four in tegers: , the mo dulus, with m> , the m ultiplier, with 0 a , the incremen t, with 0 c , the seed, with 0 and de ne the sequence [ ] of `random n um b ers' b y the recursiv +1 =( ax )% Recall also that the maxim um p ossible p erio d length of a LCG is if =0 or 1if is prime and =0. e will assume our LCG has the

maxim um p erio d length p ossible throughout this section. The sp ectral test examines the en tire p erio d of the LCG (and other theoretical tests examine the en tire p erio ds of pRNG's in general). In fact, ery few theoretical results ha b een pro en ab out LCG's that deal with less than their full p erio d (and ev en few er of these results are actually useful; see Kn uth (p. 109-110) for one of them) [Kn uth 75]. Since this yp e of test examines the en tire p erio d, it can detect glob al nonrandomness, while the empirical tests examine parts of the sequence and therefore can detect lo

al nonrandomness. Ob viously , b eing able to do b oth is nice, but w eha e to remem b er that theoretical tests don't exist in ev ery situation. or the sp ectral test, w e are w orking with consecutiv e elemen ts of a [ ] sequence (real n um b ers b et een zero and one), but w e assume here that the sequence is p erio dic with p erio d and can b e describ ed b ya LCG. W ew an t to lo ok at the set of all p oin ts in -dimensional space ;U +1 ; :::; U +( 1) The fact that e're not orking with in tegers ma seem to con ict with our statemen that e're orking with LCG's, but note that if w e

truncate the v alue of eac y a constan t amoun t (to a certain um b er of decimal places), w e can use a LCG to express this set of p oin ts. e need this LCG either to ha p erio d ,orw em ust add the p oin t (0,0,...,0) to the ab o e set if our LCG has p erio d 1 (this addition do es not a ect the results of the test in an y noticable manner when is large, as it should b e). Giv en this, e can express the ab o e set of p oin ts as x; s ;s )) ; :::; s )) x where )=( ax )% (The division b y1 =m ab o e is similar to our de nition of [ ], only w e're `going bac kw ards.') Again, w e are concerned

here with the set of p oin ts in -dimensional space, and not the order in whic they are generated. e plot this set of p oin ts in -dimensional space, and of course this can b e visualized in 2 and 3 dimensions (the reader is urged to see a visual represen tation of this pro cess for one LCG in Kn uth (p. 90) and for man di eren LCG's (most of whic are actually used in practice) at the pLab pro ject http://r andom.mat.sb g.ac.at )). Once these p oin ts are plotted, w e can analyze them with the sp ectral test. Certainly with an nite set of p oin ts in nite area (the unit square, the unit cub e,

etc.), can `co er' all of the p oin ts using a nite collection of parallel lines in 2-dimensional space, or a nite collection of parallel planes in 3-dimensional space, or a nite collection of ( 1)-dimensional h yp erplanes in -dimensional space. Let 1 = b e the maxim um distance b et een lines tak en o er all sets of parallel lines that co er all of the p oin ts x=m; s =m in dimensions. Similarly let = be the maxim um distance bet een planes tak en o er all sets of parallel planes that co er all of the p oin ts x=m; s =m; s )) =m in three dimensions. Extend this in an analogous manner for

-dimensional space; that is, = is the maxim um distance bet een 1)-dimensional yp erplanes er all sets of suc yp erplanes that co er all of the p oin ts x=m; s =m; :::; s =m in dimensions. Then w e call the t o-dimensional ac cur acy of the RNG, and, in general, is the -dimensional accuracy of the RNG [Kn uth 90, 91]. 13
Page 14
No w, the p oin t of the ab o e discussion is that the accuracy of truly random sequences is the same in all dimensions, while the accuracy of a p erio dic sequence decreases as the dimension increases. The sp ectral test, then, compares the alues of the

accuracy of sequence in di eren dimensions to determine ho `random' the sequence is. (It seems that not m uc hc hanges when 10, whic h is go o d from a computational standp oin t) [Kn uth 91]. The theory b ehind this (whic h really isn't to o complicated, in olving manipulations of ectors), in addition to an algorithm for computing suc accuracies, is describ ed in Kn uth's book (p. 92-101), but will not b e included here, except that w e state without pro of that min ::: ax ::: = 0 mo d with 2 ,0 and relativ ely prime to [Kn uth 98]. Again, this is complicated test, but it's the most po erful

test to use if the sequence meets the requiremen ts of our assumptions. e are also able to examine the en tire sequence without generating a full p erio d, whic h allo ws us to searc h for global nonrandomness, in addition to lo cal nonrandomness b y means of empirical tests. Conclusion A lingering question for the reader migh tbe: wh y are there so man y tests? (If not, try writing a pap er ab out testing RNG's and see if y ou don't ev en tually ask y ourself that question.) There are t o reasonable answ ers. First, a bad RNG can pass certain tests, so trying more than one test is essen tial

for making sure that a bad RNG is detected as suc h. or example, the nonrandom sequence 10101010101010 10 10 10 10 10 10 10 101 01 01 01 01 01 01 01 01 01 01 01 01 01 passes the equidistribution test with a p erfect score, but miserably fails the serial test. Secondly , in a related sense, w e can't consider a RNG to b e `go o d' unless it passes m ultiple tests (probably at least v e or six). So, w e need a n um b er of di eren t tests to sort out the bad RNG's from the go o d [Kn uth 73]. The in ten t of this pap er w as to in tro duce the reader to the testing of random n um b er

generators. In so doing, ha discussed some de nitions of randomness, di eren ys to generate random um b ers, and applications for these n um b ers. e then in tro duced the reader to around a dozen tests, b eginning with the foundational c hi-square and Kolmogoro v-Smirno v tests, then a v ariet y of empirical tests, follo ed b ya theoretical test, the most p o erful of them all, the sp ectral test. It is hop ed that the reader has dev elop ed an appreciation of this sub ject and has recognized the imp ortance of man y di eren t asp ects of mathematics in olv ed in this area. Finally ,ify ou'v

e read the en tire pap er, y ou kno who w imp ortan t and fundamen tal Kn uth's b o ok is to this sub ject. The reader is once more strongly encouraged to read and study the third c hapter of this w ork. The reader (as w ell as the author) w ould greatly b ene t from doing so. ersonal I curren tly attend Concordia College in Mo orhead, MN, and am ma joring in mathematics. My thanks to Concordia's Departmen t of Mathematics and Computer Science for making me a are of this opp ortunit and esp ecially to Dr. orde and Dr. omha e for writing letters of recommendation. Thanks also to Dr. Garrett for

his notes, lectures and help during these past w eeks. 14
Page 15
Bibliograph Bennett, Deb orah J. andomness Cam bridge: Harv ard Univ ersit y Press, 1998. Garrett, P aul. Intr duction to Crypto gr aphy Notes. 2000. Haahr, Mads. andom.or 30 June 2000. http://www.r andom.or Hellek alek, P eter. The pL ab Pr oje ct http://r andom.mat.sb g.ac.at Kn uth, Donald E. The A rt of Computer Pr gr amming ol. 2. 2 ed. Reading, MA: Addison-W esley 1981. avar and Silicon Graphics, Inc. 1998. http://lavar and.sgi.c om Li, Ming and P aul Vitan yi. n Intr duction to Kolmo gor ov Complexity and its

Applic ations New Y ork: Springer-V erlag, 1993. The Ro al Windsor So ciet y for Nursing Researc h. Chi Squar eR ese ar ch A nalysis < http ==www:w indsor :ig s:net= :nhodg ins=chi sq uar e r esear ch anal y sis:html > alk er, John. HotBits 10 June 2000. http://www.fourmilab.ch/hotbi ts/ 15