SIAM R EVIEW  Society for Industrial and Applied Mathematics Vol
102K - views

SIAM R EVIEW Society for Industrial and Applied Mathematics Vol

48No 3pp 569581 The25000000000Eigenvector TheLinearAlgebrabehindGoogle Kurt Bryan Tanya Leise Abstract Googles success derives in large part from its PageRank algorithm which ranks the im portance of web pages according to an eigenvector of a weight

Download Pdf

SIAM R EVIEW Society for Industrial and Applied Mathematics Vol




Download Pdf - The PPT/PDF document "SIAM R EVIEW Society for Industrial and..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.



Presentation on theme: "SIAM R EVIEW Society for Industrial and Applied Mathematics Vol"— Presentation transcript:


Page 1
SIAM R EVIEW 2006 Society for Industrial and Applied Mathematics Vol. 48,No. 3,pp. 569–581 The$25,000,000,000Eigenvector: TheLinearAlgebrabehindGoogle Kurt Bryan Tanya Leise Abstract. Google’s success derives in large part from its PageRank algorithm, which ranks the im- portance of web pages according to an eigenvector of a weighted link matrix. Analysis of the PageRank formula provides a wonderful applied topic for a linear algebra course. Instructors may assign this article as a project to more advanced students or spend one or two lectures presenting the material with

assigned homework from the exercises. This material also complements the discussion of Markov chains in matrix algebra. Maple and Mathematica $les supporting this material can be found at www.rose-hulman.edu/ bryan. Keywords. linear algebra, PageRank, eigenvector, stochastic matrix AMSsubjectclassifications. 15-01, 15A18, 15A51 DOI. 10.1137/050623280 1. Introduction. When Google went online in the late 1990s, one thing that set it apart from other search engines was that its search result listings always seemed to deliver the “good stuff” up front. With other search engines you

often had to wade through screen after screen of links to irrelevant webpages that !ust happened to match the search text. Part of the magic behind Google is its PageRank algorithm, which quantitatively rates the importance of each page on the web, allowing Google to rank the pages and thereby present to the user the more important (and typically most relevant and helpful) pages (rst. )nderstanding how to calculate PageRank is essential for anyone designing a web page that they want people to access frequently, since getting listed (rst in a Google searchleadstomanypeoplelookingatyourpage.

Indeed,duetoGoogle+sprominence asasearchengine,itsrankingsystemhashadadeepin,uenceonthedevelopmentand structure of the Internet and on what kinds of information and services get accessed most frequently. Our goal in this paper is to explain one of the core ideas behind how Google calculates webpage rankings. .his turns out to be a delightful application of standard linear algebra. Search engines such as Google have to do three basic things: 1. Crawl the weband locate all webpages with public access. 2. Indexthedatafromstep1,sothattheycanbesearchede3cientlyforrelevant key words or phrases.

Received by the editors .anuary 25, 2005; accepted for publication 0in revised form1 .anuary 5, 2006; published electronically August 1, 2006. 225,000,000,000 was the approximate market value of Google when the company went public in 2004. http://www.siam.org/journals/sirev/48-3/62328.html 5epartment of Mathematics, Rose-6ulman Institute of Technology, Terre 6aute, I7 47803 0kurt. bryan@rose-hulman.edu1. Mathematics and Computer :cience 5epartment, Amherst College, Amherst, MA 01002 0tleise@ amherst.edu1. 569
Page 2
570 K,RT BRYAN AN. TANYA LEISE Fig.1 An example of a web with only

four pages. An arrow from page to page indicates a link from page to page 4. Rate the importance of each page in the database, so that when a user does a search and the subset of pages in the database with the desired information has been found, the more important pages can be presented (rst. .his paper will focus on step 4. In an interconnected webof pages, how can one meaningfully de(ne and quantify the “importance” of any given page? .he rated importance of webpages is not the only factor in how links are pre6 sented, but it is a signi(cant one. .here are successful ranking algorithms other

than PageRank. .he interested reader will (nd a wealth of information about ranking algorithms and search engines, and we list !ust a few references for getting started (see the extensive bibliography in 798, for example, for a more complete list). For a brief overview of how Google handles the entire process, see 7:8, and for an in6depth treatment of PageRank, see 748 and a companion article 798. Another article with good concrete examples is 758. For more background on PageRank and explanations ofessentialprinciplesofwebdesigntomaximi=eawebsite+sPageRank, gototheweb6 sites 74, 11, 148. .o

(nd out more about search engine principles in general and other ranking algorithms, see 728 and 7?8. Finally, for an account of some newer approaches to searching the web, see 7128 and 7148. 2. Develo)ingaFormulatoRan,Pages. 2.1. TheBasicIdea. Inwhatfollowswewillusethephrase“importancescore”or !ust “score” for any quantitative rating of a webpage+s importance. .he importance score for any web page will always be a nonnegative real number. A core idea in assigning a score to any given webpage is that the page+s score is derived from the links made to that page from other webpages. .he links to

a given page are called the backlinks for that page. .he webthus becomes a democracy where pages vote for the importance of other pages by linking to them. Suppose the webof interest contains pages, each page indexed by an integer . A typical example is illustrated in Figure 1, in which an arrow from page A to page B indicates a link from page A to page B. Such a webis an example of a directedgraph. We+ll use to denote the importance score of page in the web. is nonnegative and >x indicates that page is more important than page (so A0 indicates that page has the least possible importance

score). A graph consists of a set of vertices 0in this context, the web pages1 and a set of edges . Each edge joins a pair of vertices. The graph is undirected if the edges have no direction. The graph is directed if each edge 0in the web context, the links1 has a direction, that is, a starting and an ending vertex.
Page 3
THE 025,000,000,000 EI1ENVECT3R 571 A very simple approach is to take as the number of backlinks for page .In the example in Figure 1, we have A2, A1, A 4, and A 2, so that page 4 is the most important, pages 1 and 4 tie for second, and page 2 is least important. A

link to page becomes a vote for page +s importance. .hisapproachignoresanimportantfeatureonewouldexpectarankingalgorithm to have, namely, that a link to page from an important page should boost page +s importance score more than a link from an unimportant page. For example, a link to your homepage directly from BahooC ought to boost your page+s score much more than a link from, say, www.kurtbryan.com (no relation to the author). In the web of Figure 1, pages 1 and 4 both have two backlinks: each links to the other, but page 1+s second backlink is from the seemingly important page 4, while page

4+s second backlink is from the relatively unimportant page 1. As such, perhaps we should rate page 1+s importance higher than that of page 4. As a (rst attempt at incorporating this idea, let+s compute the score of page as the sum of the scores of all pages linking to page . For example, consider the web of Figure 1. .he score of page 1 would be determined by the relation Since and will depend on , this scheme seems strangely self6referential, but it is the approach we will use, with one more modi(cation. Just as in elections, we don+t want a single individual to gain in,uence merely by

casting multiple votes. In the same vein, we seek a scheme in which a webpage doesn+t gain extra in,uence simply by linking to lots of other pages. If page contains links, one of which links to page , then we will boost page +s score by /n , rather than by .In this scheme, each webpage gets a total of one vote, weightedbythatwebpage’sscore that is evenly divided up among all of its outgoing links. .o quantify this for a web of pages, let ⊂{ ,...,n denote the set of pages with a link to page , that is, is the set of page +s backlinks. For each we require (1) where is the number of

outgoing links from page (which must be positive since if , then page links to at least page ). We will assume that a link from a page to itself will not be counted. In this “democracy of the web,” you don+t get to vote for yourselfC Let+s apply this approach to the four6page webof Figure 1. For page 1, we have 1+ 2, since pages 4 and 4 are backlinks for page 1 and page 4 contains only one link, while page 4 contains two links (splitting its vote in half). Similarly, 4, 4+ 2+ 2, and 4+ 2. .hese linear equations can be written Ax , where A7 and (2) 001 000 00 .his transforms the web ranking

problem into the “standard” problem of (nding an eigenvector for a square matrixC (Recall that the eigenvalues and eigenvectors of a matrix satisfy the equation Ax , by de(nition.) We thus seek an eigenvector with eigenvalue 1 for the matrix . We will refer to as the “link matrix” for the given web.
Page 4
572 K,RT BRYAN AN. TANYA LEISE It turns out that the link matrix in (2) does indeed have eigenvectors with eigenvalue 1, namely, all multiples of the vector 71 249:8 (recall that any non=ero multiple of an eigenvector is again an eigenvector). Let+s agree to scale these “impor6

tance score eigenvectors” so that the components sum to 1. In this case we obtain 12 31 4?G, 31 129, 31 290, and 31 194. Note that this ranking differs from that generated by simply counting backlinks. It might seem surprising that page 4, linked to by all other pages, is not the most important. .o understand this, note that page 4 links only to page 1 and so casts its entire vote for page 1. .his, with the vote of page 2, results in page 1 getting the highest importance score. More generally, the matrix for any webmust have 1 as an eigenvalue if the webin question has no dangling nodes

(pages with no outgoing links). .o see this, (rst note that for a general webof pages, formula (1) gives rise to a matrix with ij A1 /n if page links to page , and ij A 0 otherwise. .he th column of then contains non=ero entries, each equal to 1 /n , and the column thus sums to 1. .his motivates the following de(nition, used in the study of Markov chains. Definition1. A square matrix is called a column6stochastic matrix if all of its entries are nonnegative and the entries in each column sum to .he matrix for a webwith no dangling nodes is column6stochastic. We now prove the following.

Proposition1. Every column-stochastic matrix has as an eigenvalue. Proof. Let bean column6stochastic matrix and let denote an dimensionalcolumnvectorwithallentriesequalto1. Recallthat anditstranspose have the same eigenvalues. Since is column6stochastic, it is easy to see that , so that 1 is an eigenvalue for and hence for In what follows, we use ) to denote the eigenspace for the eigenvalue 1 of a column6stochastic matrix 2.2. Shortcomings. Severaldi3cultiesarisewithusingformula(1)torankweb6 sites. In this section we discuss two issues: webs with nonunique rankings and webs with dangling

nodes. 2.2.1. Nonuni0ueRan,ings. For our rankings, it is desirable that the dimension of ) equal 1, so that there is a unique eigenvector with A1 that we can use for importance scores. .his is true in the webof Figure 1 and more generally is always true for the special case of a strongly connected web(that is, you can get from any page to any other page in a (nite number of steps)J see Exercise 10 below. )nfortunately, it is not always true that the link matrix will yield a unique ranking for all webs. Consider the web in Figure 2, for which the link matrix is 01000 10000 0001 0010 00000 We

(nd here that ) is two6dimensionalJ one possible pair of basis vectors is 71 08 and A70 08 . But note that any linear combination of thesetwovectorsyieldsanothervectorin ),e.g., A74 08 It is not clear which, if any, of these eigenvectors we should use for the rankingsC
Page 5
THE 025,000,000,000 EI1ENVECT3R 573 Fig.2 A web of five pages, consisting of two disconnected “subwebs (pages and )and (pages ). ItisnocoincidencethatforthewebofFigure2we(ndthatdim( )) 1. Itis aconsequenceofthefactthatifaweb ,consideredasanundirectedgraph(ignoring which direction each arrow points),

consists of disconnected subwebs ,...,W thendim( )) ,andhencethereisnouniqueimportancescorevector with A 1. .his makes intuitive sense: if a web consists of disconnected subwebs ,...,W , then one would expect di3culty in (nding a common reference frame for comparing the scores of pages in one subweb with those in another subweb. Indeed, it is not hard to see why a web consisting of disconnected subwebs forces dim( )) . Suppose a web has pages and component subwebs ,...,W . Let denote the number of pages in . Index the pages in with indices 1 through , the pages in with indices +1 through , the

pages in with +1 through , etc. In general, let =1 for 1, with A0,so contains pages +1 through . For example, in the webof Figure 2, we can take A 2 and A5,so contains pages 1 and 2 and contains pages 4 and 5. .he webin Figure 2 is a particular example of the general case, in which the matrix assumes a block diagonal structure ... 0A 00 000A where denotes the link matrix for . In fact, can be considered as a web in its own right. Each matrix is column6stochastic and hence possesses some eigenvector with eigenvector 1. For each between 1 and construct a vector which has 0 components for all

elements corresponding to blocks other than block . For example, , .... .hen it is easy to see that the vectors ,1 , are linearly independent
Page 6
574 K,RT BRYAN AN. TANYA LEISE eigenvectors for with eigenvalue 1 because Aw .hus ) has dimension at least 2.2.2. DanglingNodes. Anotherdi3cultymayarisewhenusingthematrix to generate rankings. A webwith dangling nodes produces a matrix which contains one or more columns of all =eros. In this case is column-substochastic , that is, the column sums of are all less than or equal to 1. Such a matrix must have all eigenvalues less than or

equal to 1 in magnitude, but 1 need not actually be an eigenvalue for . Nevertheless, the pages in a webwith dangling nodes can still be ranked using a similar technique. .he corresponding substochastic matrix must have a positive eigenvalue 1 and a corresponding eigenvector with nonnegative entries (called the Perron eigenvector ) that can be used to rank the web pages. See Exercise 4 below. We will not further consider the problem of dangling nodes here, however. Exercise 1. Suppose the people who own page in the web of Figure are infuriated by the fact that its importance score, computed

using formula (1) , is lower than the score of page . In an attempt to boost page ’s score, they create a page that links to page ; page also links to page . Does this boost page ’s score above that of page Exercise2. Constructawebconsistingofthreeormoresubwebsandverifythat dim( )) equals (or ex ceeds) the number of components in the web. Exercise 3. Add a link from page to page in the web of Figure . )he resultingweb,consideredasanundirectedgraph,isconnected. Whatisthedimension of Exercise4. InthewebofFigure ,removethelinkfrompage topage . Inthe resulting web, page is now a dangling node. Set

up the corresponding substochastic matrix and find its largest positive (Perron) eigenvalue. Find a nonnegative Perron eigenvector for this eigenvalue, and scale the vector so that its components sum to Does the resulting ranking seem reasonable? Exercise 5. Prove that in any web the importance score of a page with no backlinks is Exercise 6. Implicit in our analysis up to this point is the assertion that the manner in which the pages of a web are indexed has no e,ect on the importance score assigned to any given page. Prove this as follows- .et contain pages, each page assigned an index

through , and let be the resulting link matrix. Suppose wethentransposetheindicesofpages and (sopage isnowpage andviceversa). .et be the link matrix for the relabeled web. Argue that PAP , where is the elementary matrix obtained by trans- posing rows and of the identity matrix. Note that the operation PA hasthee,ectofswappingrows and of , while AP swaps columns and . Also, , the identity matrix.
Page 7
THE 025,000,000,000 EI1ENVECT3R 575 Suppose that is an eigenvector for ,so Ax for some . Show that Px is an eigenvector for with eigenvalue Explain why this shows that transposing the

indices of any two pages leaves the importance scores unchanged, and use this result to argue that any per- mutation of the page indices leaves the importance scores unchanged. 1. ARemedy2ordim3 3A44 1. An enormous amount of computing resources are needed to determine an eigenvector for the link matrix corresponding to a web containingbillionsofpages. Itisthusimportanttoknowthatouralgorithmwillyield auniquesetofsensiblewebrankings. .heanalysisaboveshowsthatour(rstattempt torankwebpagesleadstodi3cultiesifthewebisn+tconnected. AndtheWorldWide Web, treated as an undirected graph, contains many

dis!oint componentsJ see 798 for some interesting statistics concerning the structure of the web. Below we present and analy=e a modi(cation of the above method that is guar6 anteed to overcome this shortcoming. .he analysis that follows is basically a special case of the PerronMFrobenius theorem, and we only prove what we need for this appli6 cation. For a full statement and proof of the PerronMFrobenius theorem, see chapter ? in 7108. 1.1. AModificationtotheLin,Matri5A. For an 6page webwith no dangling nodes , we can generate unambiguous importance scores as follows, including the case

of a webwith multiple subwebs. Let denote an matrix with all entries 1 /n . .he matrix is column6 stochastic, and it is easy to check that ) is one6dimensional. We will replace the matrix with the matrix (4) A(1 where 0 1. is a weighted average of and . .he value of originally used by Google is reportedly 0 15 79, 118. For any 70 18, the matrix is column6 stochastic and we show below that ) is always one6dimensional if (0 18. .hus can be used to compute unambiguous importance scores. In the case when A 0 we have the original problem, for then . At the other extreme is A 1, yielding . .his is

the ultimately egalitarian case: the only normali=ed eigenvector with eigenvalue 1 has A1 /n for all and all webpages are rated equally important. )sing in place of gives a webpage with no backlinks (a dangling node) the importance score of m/n (Exercise 9), and the matrix is substochastic for any m< 1 since the matrix is substochastic. .herefore the modi(ed formula yields non=ero importance scores for dangling links (if m> 0) but does not resolve the issue of dangling nodes. In the remainder of this article, we only consider webs with no dangling nodes. .he equation Mx can also be cast as (4)

A(1 Ax where is a column vector with all entries 1 /n . Note that Sx if A1. We will prove below that ) is always one6dimensional, but (rst let+s look at a couple of examples.
Page 8
576 K,RT BRYAN AN. TANYA LEISE Example For the webof four pages in Figure 1 with matrix given by (2), the new formula gives (with A0 15) 04G5 0 04G5 0 ??G5 0 4:25 420? 40 04G5 0 04G5 0 04G5 420? 40 4:25 0 04G5 0 4:25 420? 40 4:25 0 04G5 0 04G5 and yields importance scores 4:?, 142, 2??, and 202. .his yields the same ranking of pages as the earlier computation, but the scores are slightly different.

Example 2 shows more explicitly the advantages of using in place of Example As a second example, for the webof Figure 2 with A0 15, we obtain the matrix (5) 04 0 ?? 0 04 0 04 0 04 ?? 0 04 0 04 0 04 0 04 04 0 04 0 04 0 ?? 0 455 04 0 04 0 ?? 0 04 0 455 04 0 04 0 04 0 04 0 04 .he space ) is indeed one6dimensional, with normali=ed eigenvector components of A0 ,x A0 ,x A0 2?5 ,x A0 2?5, and A0 04. .he modi(cation, using instead of , allows us to compare pages in different subwebs. Each entry ij of de(ned by (4) is strictly positive, which motivates the following de(nition. Definition2. A

matrix is positive if ij for all and .his is the key property that guarantees dim( ))A1, which we prove in the next section. 1.2. Analysis o2 the Matri5 M. Note that Proposition 1 shows that )is nonempty since is stochastic. .he goal of this section is to show that )isin fact one6dimensional. .his is a consequence of the following two propositions. Proposition2. If is positive and column-stochastic, then any eigenvector in has all positive or all negative components. Proof. We use proof by contradiction. First note that in the standard triangle inequality | (with all real), the inequality is

strict when the are ofmixedsign. Suppose )containselementsofmixedsign. From Mx we have =1 ij and the summands ij are of mixed sign (since ij 0). As a result we have a strict inequality (:) =1 ij =1 ij Sumbothsidesofinequality(:)from A1to ,andswapthe and summations. .hen use the fact that is column6stochastic ( ij A1 for all )to(nd =1 =1 =1 ij =1 =1 ij =1 acontradiction. Oence cannotcontainbothpositiveandnegativeelements. If for all (and not all are 0), then 0 follows immediately from =1 ij and ij 0. Similarly 0 for all implies that each 0.
Page 9
THE 025,000,000,000 EI1ENVECT3R 577

.he following proposition will also be useful for analy=ing dim( )). Proposition 3. .et and be linearly independent vectors in )hen for some values of and , the vector has both positive and negative components. Proof. Linear independence implies that neither nor is 0. Let .If A 0, then must contain components of mixed sign, and taking A 1 and A0 yields the conclusion. If A 0, set A 1, and . Since and are independent, . Oowever, A 0. We conclude that has both positive and negative components. We can now prove that using in place of yields an unambiguous ranking for any webwith no dangling

nodes. Lemma3. If ispositiveandcolumn-stochastic,then hasdimension Proof. We again use proof by contradiction. Suppose there are two linearly independenteigenvectors and inthesubspace ). Foranyrealnumber , the vector must be in ) and so have components that are all negative or all positive. But by Proposition 4, for some choice of and the vector must contain components of mixed sign, a contradiction. We conclude that ) cannot contain two linearly independent vectors, and so it has dimension 1. Lemma4providesthe“punchline”forouranalysisoftherankingalgorithmusing the matrix (for 0 1). .he space

) is one6dimensional, and, moreover, the relevant eigenvectors have entirely positive or negative components. We are thus guaranteedtheexistenceofauniqueeigenvector )withpositivecomponents such that A1. Exercise7. Provethatif isan column-stochasticmatrixand then A(1 is also a column-stochastic matrix. Exercise 8. Show that the product of two column-stochastic matrices is also column-stochastic. Exercise9. Show that a page with no backlinks is given importance score by formula (4) Exercise10. Suppose that is the link matrix for a strongly connected web of pages (any page can be reached from any

other page by following a finite number of links). Show that dim( ))A1 as follows. .et ij denote the i,j -entry of Notethatpage canbereachedfrompage inonestepifandonlyif ij (since ij means there’s a link from to ). Show that ij if and only if page can be reached from page in exactly two steps. Hint- ij ik kj ;all ij are nonnegative, so ij implies that for some both ik and kj are positive. Showmoregenerallythat ij ifandonlyifpage canbereachedfrom page in exactly steps. Argue that ij if and only if page can be reached from page in or fewer steps (note that A0 is a legitimate choice—any

page can be reached from itself in zero steps!). Explainwhy isapositivematrixifthewebisstrongly connected. Usethelastpart(andExercise )toshowthat is positive and column-stochastic (and hence by .emma dim( ))A1 ). Showthatif ,then . Whydoesthisimplythat dim( )) A1
Page 10
578 K,RT BRYAN AN. TANYA LEISE Exercise11. Consider again the web in Figure , with the addition of a page that links to page , where page also links to page . Calculate the new ranking by finding the eigenvector of (corresponding to A1 ) that has positive components summing to . Use A0 15 Exercise12.

Addasixthpagethatlinkstoeverypageofthewebintheprevious exercise, but to which no other page links. Rank the pages using , then using with A0 15 , and compare the results. Exercise13. Constructawebconsistingoftwoormoresubwebsanddetermine the ranking given by formula (4) As of January 2005, the webcontains at least eight billion pagesPhow does one compute an eigenvector for an eight billion by eight billion matrix? One reasonable approachisaniterativeprocedurecalledthe powermethod (alongwithmodi(cations), which we will now examine for the special case at hand. It is worth noting that there is

much additional analysis one can do, and there are many improved methods for the computationofPageRank. .hereference7G8providesatypicalexampleandadditional references. 4. Com)utingtheIm)ortanceScoreEigenvector. .he rough idea behind the power method for computing an eigenvector of a matrix is this: One starts with a “typical” vector then generates the sequence Mx (so ) and lets approaches in(nity. .he vector is, to good approximation, an eigenvector for the dominant (largest6magnitude) eigenvalue of . Oowever, depending on the magnitude of this eigenvalue, the vector may also grow without

bound or decay to the =ero vector. One thus typically rescales at each iteration, say, by computing Mx Mx , where can be any vector norm. Oowever, to guarantee that the power method converges at a reasonable rate, one typically requires that the dominant eigenvalue A 1 for be simple. .hat is, the characteristic polynomial for should be of the form )A( 1) ) for some polynomial ) of degree 1, where ( 1) does not divide ). Alternatively, should possess no “generali=ed eigenvectors” for the eigenvalue A 1 other than the eigenvectors in ) (see 718 for more on generali=ed eigenvectors).

Actually,thefollowingpropositionprovideswhatweneedtoprovethatthepower method converges in this case, but with no explicit reference to the algebraic nature of the dominant A1 eigenvalue. It can also be used to show that this eigenvalue is simple. Definition4. )he -norm of a vector is Proposition 4. .et be a positive column-stochastic matrix and let denotethesubspaceof consistingofvectors suchthat A0 . )hen Mv for any , and Mv for any , where Amax 2min ij Proof. .o see that Mv is straightforward: Let Mv , so that =1 ij and =1 =1 =1 ij =1 =1 ij =1 A0 :ee =15] for a general introduction to the

power method and the use of spectral decomposition to $nd the rate of convergence of the vectors
Page 11
THE 025,000,000,000 EI1ENVECT3R 579 Oence Mv . .o prove the bound in the proposition, note that =1 =1 =1 ij where A sgn( ). Note that the are not all of one sign, since A 0 (unless , in which case the bound clearly holds). Reverse the double sum to obtain (G) =1 =1 ij =1 where =1 ij . Since the are of mixed sign and ij A 1 with 0 ij 1, it is easy to see that 1+2 min ij 2 min ij We can thus bound |≤| 2 min ij Let A max 2min ij . Observe that c< 1 and | for all From (G) we

have =1 =1 =1 || | =1 which proves the proposition. Proposition 4 sets the stage for the following proposition. Proposition5. Every positive column-stochastic matrix has a unique vector with positive components such that Mq with A1 . )he vector can be computedas Alim foranyinitialguess withpositivecomponentssuch that A1 Proof. From Proposition 1 the matrix has 1 as an eigenvalue and by Lemma 4 the subspace ) is one6dimensional. Also, all non=ero vectors in ) have entirely positive or negative components. It is clear that there is a unique vector ) with positive components such that A1. Let be

any vector in with positive components such that A1. We can write , where as in Proposition 4). We (nd that . As a result (?) A straightforward induction and Proposition 4 show that for 0 c< 1( as in Proposition 4) and so lim A0. From (?) we conclude that lim Example 3. Let be the matrix de(ned by (5) for the web of Figure 2. We take A70 24 41 0? 1? 198 as an initial guessJ recall that we had 70 2?5 2?5 048 . .he table below shows the value of for several values of , as well as the ratio . Compare this ratio to from Proposition 4, which in this case is 0 94.
Page 12
580 K,RT BRYAN

AN. TANYA LEISE 0.:2 0.255 0.411 0.144 0.?5 10 0.0591 0.?5 50 ?G 10 0.?5 It is clear that the bound is rather pessimistic (note that ?5 is the value 1 , and 0 ?5 turns out to be the second6largest eigenvalue for ). One can show that in general the power method will converge asymptotically according to Mx ≈| | , where is the second6largest eigenvalue of . Moreover, for of the form A(1 with column6stochastic and all ij A1 /n , it can be shown that | (see, e.g., 71, .heorem 5.108). As a result, the power method will converge much more rapidly than indicated by . Nonetheless,thevalueof

inProposition4providesaverysimplebound on the convergence of the power method here. It is easy to see that since all entries of are at least m/n , we will always have m/n in Proposition 4. As a practical matter, note that the positive matrix has no non=ero ele6 ments, so the multiplication Mv for will typically take ) multiplications andadditions, aformidablecomputationif A? 000 000 000. But(4)showsthatif ispositivewith A1,thenthemultiplication Mx isequivalentto(1 Ax .his is a far more e3cient computation, since can be expected to contain mostly =eros (most webpages link to only a few other

pages). We+ve now proved our main theorem. Theorem5. )he matrix defined by (4) for a web with no dangling nodes will always be a positive column-stochastic matrix and so have a unique with positive components such that Mq and A1 . )he vector may be computed as the limit of iterations A(1 Ax , where is any initial vector with positive components and A1 .he eigenvector de(ned by (4) also has a probabilistic interpretation. Consider a web6surfer on a web of pages with no dangling nodes. .he surfer begins at some webpage (it doesn+t matter where) and randomly moves from webpage to web page

according to the following procedure: If the surfer is currently at a page with outgoing links, he either randomly chooses any one of these links with uniform probability or he !umps to any randomly selected page on the web, each with probability (note that A1, so this accounts for everything he can do). .he surfer repeats this page6hopping procedure ad in(nitum. .he component of the normali=ed vector in (4) is the fraction of time that the surfer spends, in the long run, on page of the web. More important pages tend to be linked to by many other pages and so the surfer hits those most often.

Exercise14. ForthewebinExercise 11 ,computethevaluesof and for A1 10 50 using an initial guess not too close to the actual eigenvector (sothatyoucanwatchtheconvergence). Determine Amax 2min ij and the absolute value of the second-largest eigenvalue of Exercise 15. )o see why the second-largest eigenvalue plays a role in bounding ,consideran positivecolumn-stochasticmatrix thatisdiagonal- izable. .et be any vector with nonnegative components that sum to . Since is diagonalizable, we can create a basis of eigenvectors ,..., , where is
Page 13
THE 025,000,000,000 EI1ENVECT3R 581 the

steady state vector, and then write =1 . Determine , and then show that A1 and the sum of the components of each must equal . Next apply Proposition to prove that, except for the nonrepeated eigenvalue A1 , the other eigenvalues are all strictly less than in absolute value. Use this to evaluate lim Exercise16. Consider the link matrix 00 Show that A(1 (all ij A1 ) is not diagonalizable for m< Exercise17. How should the value of be chosen? How does this choice a,ect the rankings and the computation time? RE4ERENCES =1] A.BermanandR.Plemmons Nonnegative Matrices in the Mathematical Sciences ,

Academic Press, 7ew @ork, 1A7A. =2] M. W. Berry and M. Browne Understanding Search Engines: Mathematical Modeling and Text Retrieval , 2nd ed., :IAM, Philadelphia, 2005. =3] M. Bianchini, M. Gori, and F. Scarselli (nside )ageRank , ACM Trans. Internet Tech., 5 020051, pp. A2B128. =4] S. Brin and L. Pa,e The Anatomy of a Large-Scale ,ypertextual -eb Search Engine http://www-db.stanford.edu/ backrub/google.html 0August 1, 20051. =5] A. Farahat, T. Lofaro, J. .. Miller, G. Rae, and L. A. Ward Authority rankings from ,(TS, )ageRank, and SALSA: Existence, uni.ueness, and effect of

initialization , :IAM .. :ci. Comput., 27 020061, pp 1181B1201. =6] A.Hill Google (nside Out , Maximum PC, A 041 020041, pp. 44B48. =7] S.0am1ar,T.Ha1eliwala,andG.Gol23 Adaptive methods for the computation of )age- Rank , Cinear Algebra Appl., 386 020041, pp. 51B65. =8] A. 4. Lan,1ille and .. D. Meyer A survey of eigenvector methods of -eb information retrieval , :IAM Rev., 47 020051, pp. 135B161. =A] A. 4. Lan,1ille and .. D. Meyer Deeper inside )ageRank , Internet Math., 1 020051, pp. 335B380. =10] ..D.Meyer Matrix Analysis and Applied Linear Algebra , :IAM, Philadelphia, 2000. =11] ..Moler

The -orld’s Largest Matrix Computation , http://www.mathworks.com/company/ newsletters/news notes/clevescorner/oct02 cleve.html 0August 1, 20051. =12] J.Mostafa Seeking better web searches , :ci. Amer., 2A2 020051, pp. 66B73. =13] S. Ro3inson The ongoing search for ecient web search algorithms , :IAM 7ews, 37 0A1 07ovember 20041, p. 4. =14] 5. Ro,ers The Google )agerank Algorithm and ,ow (t -orks , http://www.iprcom.com/ papers/pagerank/0August 1, 20051. =15] W. J. Stewart An (ntroduction to the Numerical Solution of Markov Chains , Princeton Dniversity Press, Princeton, 7., 1AA4.