erication Thr ough the Principle of Least Astonishment Beth Isaksen aler ia Ber tacco Adv anced Computer Architecture Lab The Univ ersity of Michigan Ann Arbor MI bisaksenvaleria umich
150K - views

erication Thr ough the Principle of Least Astonishment Beth Isaksen aler ia Ber tacco Adv anced Computer Architecture Lab The Univ ersity of Michigan Ann Arbor MI bisaksenvaleria umich

e du ABSTRA CT Assessing the correctness of digital design is hallenging task hamp ered extremely large circuit netlists coun ter in tuitiv prop ert descriptions and illde57356ned sp eci57356cations In this pap er prop ose new eri57356cation metho do

Tags : ABSTRA Assessing
Download Pdf

erication Thr ough the Principle of Least Astonishment Beth Isaksen aler ia Ber tacco Adv anced Computer Architecture Lab The Univ ersity of Michigan Ann Arbor MI bisaksenvaleria umich

Download Pdf - The PPT/PDF document "erication Thr ough the Principle of Leas..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation on theme: "erication Thr ough the Principle of Least Astonishment Beth Isaksen aler ia Ber tacco Adv anced Computer Architecture Lab The Univ ersity of Michigan Ann Arbor MI bisaksenvaleria umich"— Presentation transcript:

Page 1
erication Thr ough the Principle of Least Astonishment Beth Isaksen aler ia Ber tacco Adv anced Computer Architecture Lab The Univ ersity of Michigan Ann Arbor MI bisaksen,valeria @umich.e du ABSTRA CT Assessing the correctness of digital design is hallenging task hamp ered extremely large circuit netlists, coun ter- in tuitiv prop ert descriptions and ill-dened sp ecications. In this pap er prop ose new erication metho dology inspired the principle of least astonishmen t. The under- lying idea is to pro vide an automatic assessmen of what

constitutes \common eha vior" for system, and use this to detect an anomaly in the design. Devian eha vior is presen ted to the erication engineer through in tuitiv e, compact diagrams whic lend themselv es to quic insp ec- tion for correctness. enable this metho dology in- tro duce Inferno, new to ol whic can analyze the results of logic sim ulation trace and automatically extract high- lev el diagrams represen ting the design's transaction activit across an user-dened in terface. In addition, Inferno can automatically generate hec er mo dule corresp onding to giv en

transaction, suitable for use in wide range of erica- tion metho dologies. en vision the deplo ymen of Inferno in closed-lo op constrain t-random sim ulation metho dology where an new transaction detected on the in terface is pre- sen ted to the user for analysis and, once deemed legal, it is merged in to an \appro ed transactions" hec er, whic ags the detection of an new yp of transactions. pro vide series of examples and exp erimen tal results to sho the eectiv eness of Inferno and some of its ossible uses. 1. INTR ODUCTION As hardw are designs con tin ue to gro in

complexit and time-to-mark et pressure in tensies, hardw are designers and erication engineers ust resp ond with ev er-higher stan- dards of pro ductivit and qualit In fact, in man cases the success of the pro ject dep ends on pro ducing sucien tly correct system at rst tap e-out. ormal and semi-formal erication can greatly reduce the risk of bugs escaping to silicon, but they require signican in estmen of time and eort on the part of the erication team. Moreo er, they ypically require the user to man ually sp ecify the prop er-

ties of the design whic are to pro en, deriving them from high-lev el, qualitativ sp ecication do cumen t, com- plemen ted their ersonally understanding of the system's functionalit riting suc prop erties or hec ers, when Permission to mak digital or hard copies of all or part of this ork for personal or classroom use is granted without fee pro vided that copies are not made or distrib uted for prot or commercial adv antage and that copies bear this notice and the full citation on the rst page. cop otherwise, to republish, to post on serv ers or to redistrib ute to

lists, requires prior specic permission and/or fee. ICCAD '06 San Jose, California USA Cop yright 2006 CM 1˝59593˝389˝1/06/0011 ... 5.00. used in sim ulation-cen tered metho dology is non-trivial in the est case, and, when the erication engineer is not completely familiar with the details of the implemen tation, it can erge on the imp ossible. Designers, who understand the inner orkings of their wn comp onen t, lac kno wledge of the system at global lev el and of the complex in terac- tions et een comp onen ts; hence they are not in osition to describ complex

prop erties, whic usually aect ulti- ple comp onen ts. Recen literature pro vides um er of real orld exp eriences rep orting on the complexit of running formal erication metho dology in an industry con text ev en on ery small part of the design, and on the hallenges of erifying complex system in general [12, 4]. Man factors con tribute to mak the erication task so error-prone and complex; ho ev er, more often than not these factors relate to the mo dels used to describ the system: Designs are commonly describ ed at the register-transfer lev el, whic attens the

design in to an amorphous structure from whic it is dicult to distinguish con trol signals from data busses and to discern the role of dieren con trol nets. The most common soft are to ols supp orting the under- standing of design's eha vior are eform view ers. Their use is ery cum ersome, requiring an engineer to visually insp ect complex eforms er millions of cycles in the at- tempt to understand the dep endency hain et een ev en ts and detect design error. Prop ert languages are usually declarativ e, making it par- ticularly hard for erication engineer to express

desired functionalit to eried. This is particularly true for cross-mo dule prop erties, whicj ma in olv man signals in complex expressions, with the result that often the prop ert is harder to debug than the design itself. Clearly an of reducing the user eort required the formal erication pro cess ould of great use. In particu- lar it ould greatly enecial if abstract design eha vior could extracted automatically and presen ted for anal- ysis to the erication engineer through structures whic are in tuitiv and compact, thereb oiding the need to in- sp ect

large quan tit of co de or eforms. erication could then fo cus on hec king the correctness of the high- lev el transactions, th us eliminating cum ersome activities and error-prone insp ections suc as those describ ed ab e. Con tributions The goal of the solution presen ted in this pap er is to i) lo er the barrier for the understanding of the activit of system across user-sp ecied comm unica- tion in terface, and ii) pro vide mec hanism to automatically detect when anomalous activit is observ ed at that in terface. In this con text, anomalous activit stands for in teractions

that engineers ha not et understo or seen, whic are as suc oten tial indicators of hidden bugs. ac hiev the rst goal pro viding no el algorithm and to ol to auto- matically extract the high-lev el comm unication activit at
Page 2
an user-dened in terface within design (it could the in terface et een design comp onen ts, or set of signals within single mo dule). By raising the lev el of abstraction of the in teraction proto col, it is easier to insp ect and ev alu- ate the correctness of the comm unication activit It is also easier to detect an anomalous transaction,

ossibly indica- tiv of hidden bug. This second observ ation suggests new erication metho dology where the correct eha vior of system is not dened priori through set of complex prop erties, but instead it is surmised analyzing the set of transaction activities detected our to ol, whic can also hec ed against sp ecication do cumen t. The second goal of this ork is to pro vide means to automatically de- tect when, in constrain t-based random sim ulation setting, new \uncertied" transaction is observ ed. ac hiev this means of an automatic transaction hec er gener-

ator whic can em edded in the system during sim ula- tion. This metho dology is inspired the informal principal of le ast astonishment whic h, applied to the orld of design erication, sa ys that bugs are lik ely to hide in the anoma- lous, or uncommon, eha vior of design. supp ort these goals dev elop ed Inferno, soft are to ol whic can analyze the activit of design in terface during logic sim- ulation and summarize the transactions observ ed through simple and in tuitiv diagrams. In addition, Inferno has the abilit to automatically generate hec er agging an un- certied

transaction, making it suitable for deplo ymen in closed-lo op random sim ulation en vironmen where eac new transaction is agged and presen ted to the user in diagram form. If deemed correct, it is merged in to the transaction- hec er, whic ecomes incremen tally complete. Otherwise, the design is up dated and the pro cess can restart. The remainder of this pap er is organized as follo ws: Sec- tion pro vides an erview of related solutions oth in the hardw are and soft are erication domains. Section giv es an erview of the Inferno arc hitecture and its use in the con text of

curren hec er-based and co erage-driv en eri- cation metho dologies. Sections to presen the extraction, analysis and hec er-generation algorithms whic enable In- ferno to pro vide the high-lev el transaction mo dels men tioned ab e. Finally Section and pro vide exp erimen tal results and outline future researc directions. 2. PREVIOUS ORK um er of previous orks ha dealt with the problem of automatically generating prop erties and sp ecications for oth soft are and hardw are. On the hardw are fron t, Hangal et al. [10] ha prop osed to ol to extract simple \proba- ble" prop erties

e.g. one-hot or utually exclusiv signals) through sim ulation trace analysis, whic can then fed to formal prop ert hec er for erication. In [9], the authors prop ose more general approac to automatic prop ert ex- traction, ev aluating wide range of ossible \time rela- tions" et een group of signals. Our solution shares with this line of researc the idea of extracting design eha vior automatically from sim ulation con text; ho ev er, at- tac the problem as high-lev el mo deling one, extracting transactions observ ed at user-selected in terface. Moreo er, although the authors of [9]

attempt to generate all ossi- ble prop erties, they do not dieren tiate et een data and con trol, th us common con trol sequence (or transaction) will lik ely go undetected unless it is observ ed man times with dieren data. In con trast, are able to recognize transaction ev en if it ccurs only once in the execution trace. The soft are erication comm unit faced with sim- ilar hallenges, has arriv ed at some solutions whic are also relev an t. Ernst et al. [8] ha prop osed Daik on, whic an- alyzes soft are execution traces to suggest list of ossible prop erties (or

annotations) for use with the static hec er ESC/Ja a. Prop erties can also generated using static analysis, as in [3], where, ho ev er, the approac requires that the program rst translated in to state mac hine. Ammons et al. erform analysis to generate sp ecication of the API in the form of \scenarios" describing common sequences of instructions [1], while ang deriv es constrain ts on the order of ccurrence of instructions [13]. The alue of suc scenario- or transaction-based sim ulation and anal- ysis is ell recognized. Brahme et al. for instance, ha dev elop ed system to allo

erication engineers to write testb enc hes and analyze results at the transaction-lev el [5]. Our to ol brings the enets of transaction-based analysis to register-transfer lev el testb enc hes. Tw additional im- ortan dierences of our con tributions with resp ect to the solutions outlined ab are that i) restrict our fo cus to con trol signals, so that can abstract the data and determine the set of transactions of the system, and ii) In- ferno is not restricted to iden tifying predetermined set of prop erties, but can analyze an con trol sequence observ ed in sim

ulation. ey idea driving the dev elopmen of our approac lies in the empirical observ ation that bugs are more lik ely to hidden in eha vior whic deviates from the norm; this is esp ecially true as the erication pro cess progresses and bugs ecome harder and harder to lo cate. In the erica- tion metho dology whic en vision for Inferno, transac- tions whic ha not een previously seen are susp ected to fault and hence ust insp ected user. This ob- serv ation has also een explored in the con text of soft are erication, for instance Engler [7]. An example rep orted in [7]

suggests that if oin ter dereference is normally as- so ciated with ull-c hec k, then the one lo cation where the ull-c hec is missing migh an ersigh t. Again, in or- der to detect suc ab erran eha viors, one ust rst ha go picture of what the exp ected eha viors are. Finally ha the additional ob jectiv er the previous literature of presen ting the user with high-lev el, in tuitiv mo del of the activit observ ed, to impro the understand- ing of the design and its correct eha vior in the form of transaction diagrams. This con tribution has similarities to the ork of Arts et al. in the

soft are orld [2]. 3. INFERNO ARCHITECTURE In this pap er presen Inferno, soft are to ol whic an- alyzes sim ulation trace to \learn" ab out the eha vior of design under erication. It then presen ts the user with in- tuitiv diagrams describing the observ ed eha vior as high- lev el mo del, in olving only ey con trol signals. Inferno can automatically split the sequence of signal activit observ ed during sim ulation in to tr ansactions basic sequences of ac- tivit whic h, when comp osed together, form ossible execu- tion scenarios. The user can use the transaction diagrams to learn ab

out the design's eha vior (when Inferno is run er set of kno wn-correct testb enc hes) or to quic kly ev alu- ate the correctness of the comm unication proto col across the in terface of in terest (for instance, when the trace is obtained from constrained-random sim ulation).
Page 3
In addition, Inferno can automatically generate prop ert hec ers, in the form of erilog assertions, corresp onding to the transactions observ ed (and appro ed the user) for direct use in an erication metho dology: In co erage-driv en erication metho dology the dis- tinct transactions

observ ed, and the um er of obser- ations at giv en in terface pro vides aluable metric to trac the progress of erication. In constrained-random sim ulation, Inferno can main- tain set of \appro ed" transactions, and the hec ers can ag the detection of an newly observ ed ones. new transaction is then transformed in to high-lev el diagram for user insp ection. If deemed correct, cor- resp onding hec er is generated, merged in to the set of appro ed transactions, and the sim ulation can con- tin ue. Otherwise, the design is mo died to correct the exp osed bug. ossible

terminating condition can set to xed um er of sim ulation cycles with no new transactions detected. high-lev el o of this eri- cation metho dology is illustrated in Figure 1. Note that the kno wn ol of transactions can re-used across ultiple ersions of the design to quic kly re- appro the correctness of an in terface. Finally once the user eliev es that the full set of ossi- ble distinct transactions has een observ ed, the hec er generated Inferno can used in formal erica- tion con text, to pro that, in fact, no other transac- tion can generated at the in

terface under analysis. Automatic generation Inferno (extract transactions) Transaction is legal: Add to the approved set Transaction is error: Fix design and continue (known correct Visual inspection Figure 1: Closed-lo op sim ulation with Inferno. In- ferno can used in sim ulation metho dology to automatically detect oten tial bugs. An initial set of appro ed transactions is extracted from kno wn-correct direct-sim ulation trace and the corresp onding set of generated hec ers is placed with the design. If, during the follo wing constrained-random sim ulation, an new transaction is detected,

it is submitted in diagram form to the user. If appro ed, the corresp onding hec er will expand the ini- tial kno wn set and the pro cess con tin ues; if not bug has een exp osed and the design mo died. rom structural standp oin (see also Figure 2) Inferno tak es as input i) small conguration le listing either design mo dule whose I/O in terface is the target of the anal- ysis or the sp ecic signals to consider, ii) sim ulation trace, and iii) the design under erication. This last input is only needed to determine the signals' directions at the in

terface of hoice. The in terface whic Inferno considers for its anal- ysis is ery exible concept: it can comm unication in terface of the design (suc as mo dule I/O) or it can custom-crafted from an com bination of signals within the design. When the in terface is sp ecied as mo dule I/O, additional lters can applied: for instance Inferno can automatically disregard all \busses", that is in terface sig- nals whose bit width is more than user-sp ecied constan t. Inferno then pro ceeds to pro cess the sim ulation trace to extract and record the alues observ ed er

time on the in- terface signals. Eac distinct \snapshot" of alues consti- tutes new ertex in the high-lev el diagram describing the in terface proto col. Tw ertices are connected an edge to indicate that the snapshots are subsequen t. this oin Inferno can pro ceed in ultiple ys: i) it can presen the structure it has already learned as monolithic diagram sho wing all the distinct activit observ ed at the in terface un- der study along with its observ ed time dep endency called Pr oto ol diagr am or ii) it can further analyze the snapshot sequence and break it in to distinct transactions presen

ted as ansaction diagr ams (for instance, in the case of bus proto col, ossible transactions are read, write, burst read, etc.). Section describ es in detail the algorithms in olv ed in this analysis. The resulting diagrams are presen ted to the user for visual insp ection and can automatically con- erted (and simplied) in to erilog hec ers. Inferno Design Simulation trace Transaction diagrams Verilog Checkers User specifies module to analyze or may select signals directly Figure 2: Inferno Arc hitecture. The user selects de- sign mo dule I/O or sp ecic list of signals to

monitor. Inferno observ es the alues assigned to these signals er the course of sim ulation run. It then analyzes the trace, extracting list of transactions and presen ting them in the form of high-lev el dia- grams. In addition, it generates from the diagrams set of opti- mized hec ers, whic can used to detect an new transaction. 4. EXTRA CTING TRANSA CTIONS This section presen ts the algorithms ha dev elop ed to extract the proto col and the transactions observ ed at the in terface under observ ation. build these diagrams, In- ferno extracts the direction of eac signal whic is part of the in

terface under observ ation and pro cesses the sim ulation trace to detect an alue hange on the signals of in terest. 4.1 Generating Pr otocol Diagrams Proto col diagrams are generated building directed graph with ertex for eac distinct pattern of alues ob- serv ed at the in terface under study Edges connect ertices whic are consecutiv in time; that is there is an edge from ertex to ertex if the in terface transitioned from con- guration to conguration during sim ulation. Practi- cally sp eaking, ertices in the Inferno diagrams represen ts \snapshots" of alues observ ed at the

in terface under analy- sis. Note that, due to our construction tec hnique, there are no ertices corresp onding to the same set of in terface alues in proto col diagram. Eac ertex is lab eled with the corresp onding alues observ ed at the in terface, separat- ing the ossible signal directions. In addition, lab el eac edge with the alue hanges whic lead from the source ertex to the sink one.
Page 4
Example 1. Consider bus in terface with the follo w- ing I/O signals: ack[1:0] as input, and cyc and stb as outputs. During sim ulation, observ the follo wing in ter- face sequence: (00 00)

@0 (00 00) @1 (00 10) @2 (00 11) @3 (00 11) @4 (00 11) @5 (10 11) @6 (10 11) @7 (00 10) @8 and (00 00) @9 (subscripts indicate sim ulation times). Inferno's proto col diagram generator will pro duce diagram with v ertices: A:00 00 B:00 10 C:00 11 D:10 11 and E:10 00 The directed edges in the diagram are: B, C, D, and A. Note that the proto col diagram abstracts the absolute time, and only trac ks time dep endencies et een ev en ts. This relativ ely simple pro cedure is already quite useful: it reduces trace, ossibly tens of millions of cycles long, to one compact image sho wing

transitions at the in terface. If an undesired eha vior ccurs only few times er the course of long regression suite, hances of iden tifying it through eform view er are ery slim, ho ev er it stands uc etter hance of eing detected as an anomalous ertex or edge in the corresp onding proto col diagram. The case study of Section sho ws an example of this situation. 4.2 Generating ransaction Diagrams The analysis to extract transactions starts pro cessing the en tire trace, lab eling eac distinct com bination of alues observ ed during sim ulation, and hence generating hain of lab els corresp onding

to the sequence of distinct cong- urations observ ed at the in terface. ransactions are then iden tied prop erly partitioning this initial hain of la- els. In informal terms, transaction is simply sequence of ev en ts corresp onding to particular high-lev el op eration, for instance read or write to bus, cac he, or memory In general transaction is sequence whic can easily understo as high-lev el op eration. On the other hand, giv en sequence of ev en ts, there are man dieren ys to group them in transactions. Our ob jectiv is to dene ys of grouping ev en ts whic

lead to man rep etitions of few distinct transactions. found that tec hnique whic giv es go results exploits the fact that most in terfaces are designed engineers reasoning through high lev el transac- tions, hence they tend to create ery ell-dened transaction oundaries in the design eha vior. ey asp ect in the con- cept of transaction is that system generates or pro cesses only one transaction at time. Hence, there ust one or more signal alues or transitions indicating the end of transaction and the eginning of the next. In the simplest scenario, the user can sp ecify whic signal

alues or tran- sitions corresp ond to transaction oundaries (for instance, in Wish one proto col, the assertion or de-assertion of the stb signal). Ho ev er, in general, Inferno ust infer the transaction oundaries from the trace itself. pro ceed rening the partitioning of the hain of lab els in to transac- tions through ultiple passes. Belo w, describ all phases of this pro cess, and rep ort the related pseudo co de in Figure 3. Boundary lab el (lines 2, 4-5) The rst pass iden ties the rst lab el of the hain that is rep eated (called the ound- ary lab el). Sp

ecically consider the rst lab el to re- eated in trace to mark the end of transaction. Hence, pro ceed in breaking the hain of lab els at eac ccur- rence of the oundary lab el. While this is not the only vi- able tec hnique to iden tify transaction oundaries, found exp erimen tally that it orks ell in practice. It can justi- ed noting that the stable in terface alue at the end of reset sequence almost certainly marks transaction ound- ary (though not necessarily the only one), and frequen tly it is also rep eated at the completion of eac transaction, hence it is observ

ed in sim ulation as the rst rep eated lab el. also considered an alternativ approac of setting the oundary lab el to the lab el with the highest um er of c- currences in the trace. In tuitiv ely this suggests that should extract the highest um er of simple transactions. Ho ev er, in practice, ha not found this second ap- proac to ork ell. eliev the reason ma lie in the fact that the rst solution creates etter corresp ondence with the transaction design in ten t. Lo op folding (lines 6-11) The second renemen iden- ties lo ops within transaction. ypical scenario

is burst-read transaction, where read sequence is rep eated ultiple times within transaction. Clearly all burst-reads should matc hed as the same transaction, regardless of the sp ecic um er of read op erations. can do so iden tifying lo ops and then matc hing transactions whic are iden tical except for the um er of lo op rep etitions. Boundary renemen (lines 3, 13-17) The transaction extractor algorithm describ ed so far orks ell in the case where all transactions end with the same lab el. When this is not the case, it fails to detect some of the oundaries, and or more

transactions ma clustered in to single one. The last renemen phase addresses this problem. It consists of one nal pass through all the transactions iden tied so far hec king if an transaction is the sux of another trans- action B. That is, if has pream ble after whic it matc hes completely In this case, can reasonably conclude that the oundary et een them constitutes new transaction oundary this oin t, rep eat the extraction pro cess using the new oundaries in addition to the original one (and rene them through lo op folding). The pro cess can rep eated

ultiple times un til it con erges. Ho ev er, found that usually one additional pass is sucien t. TransactionExtractor (labels chain) new boundary set first repeated label do boundary set new boundary set partition chain into segments,cut at boundary set for each segment identify all repeated sub-segments for each repeated sub-segment count number of occurrences 10 modify transaction to only one repetition 11 12 13 for each distinct segment pair (i,j) 14 if (j is contained in i) then 15 new boundary set += label ending (i-j)segment 16 17 while (new boundary set != boundary set) 18

Figure 3: ransaction extractor algorithm. the end of this pro cess automatically generate set of graphs, eac sho wing distinct transaction with er- tices corresp onding to the lab els and edges corresp onding to transitions et een them. or eac transaction indicate the um er of times it ccurred, and the initial sim ulation time at whic it as observ ed. Example 2. Consider scenario where the initial hain of lab els generated for the transaction diagrams is
Page 5
The rst rep eated lab el is hence the initial oundary segmen tation pro duces transactions: ), ), ). The lo op

folding algorithm will then fold the second transaction and lea only transactions: and ). Finally the last renemen disco ers that is also trans- action oundary since the rst transaction is sux of the second one. The nal set of transactions is then: and ). While ha emphasized the usefulness of these dia- grams for an engineer in terested in learning more ab out the op eration of the design, they ha other enets as ell. or instance, similar to what Ammons [1] suggests, can compare sev eral instan tiations of the same mo dule to hec if they generate the same

diagrams. If not, the test co er- age for some of them ma insucien t. Alternativ ely can pro duce new set of diagrams for eac revision of the design, enabling easy detection of unin tended hanges. can also use Inferno to compare dieren mo dules whic are in tended to follo the same proto col. If they do not app ear to corresp ond, either one or oth of the designs is incorrect or the testb enc stim ulus is inadequate. s1: -0 u0: 000 s1: 00 u0: 000 s1: 00 u0: -0- s1: 00 u0: 100 s1: 10 u0: 110 s1: 00 u0: 110 s1: 10 u0: 100 s1: 00 u0: 111 s1: 10 u0: 000 s1: 10 u0: 111 s1: 10 u0: 10-

s1: 00 u0: 10- s1: -0 u0: --- Figure 4: Proto col diagram for Wish one DMA Eac ertex represen ts distinct com bination of in terface al- ues. Edges corresp onds to transitions from one com bination to another. ertices are lab eled with the corresp onding in terface alues. Grey ertices and old edges sho the corresp ondence with the transaction sho wn in Figure 5. 4.3 Example: ishbone DMA Examples of oth proto col and transaction diagrams for an in terface follo wing the Wish one proto col are sho wn in Figures and 5. The proto col is primarily in tended to allo comm unication et een separate mo

dules in system-on-a- hip design, where um er of IP cores, ossibly of dier- en origins, ust in terface with eac other. The design from whic this diagram as extracted is DMA (direct mem- ory access) con troller but fo cus only on the Wish one in terface. Although the proto col pro vides for more complex cases, the only signals activ in this design are cyc stb and we from the master, u0 and ack and err from sla s1 cyc is asserted and ept high throughout the course of eac transaction, while stb is raised at eac op eration within the transaction. The we signal is asserted for writes and

de-asserted for reads, ack indicates that the sla has nished pro cessing the op eration on its end, and err ags error conditions. The proto col diagram sho wn in Figure includes all states and transitions observ ed er the course of an (extensiv e) regression test. Figure sho ws the burst- read transaction, one of eigh transactions extracted from the same run. The diagram sho ws that the transaction starts with cyc eing asserted, follo ed the assertion of stb to start the rst op eration. Since we is de-asserted, read is erformed. When the ack is receiv ed, stb is lo ered

to end the op eration. The burst-read transactions observ ed during sim ulation ma rep eat the read sequence up to three times. the end of the transaction, cyc is nally lo ered. Figure 5: ransaction diagram for burst-read The graph sho ws eac stage of the transaction through ertex corre- sp onding to the alues com bination at the in terface under study The dashed edges highligh the lo op, whic is to rep eated times as sp ecied. rom the diagram, it is easy to recognize the transaction's pream ble, core part and trailer. Edges are lab eled the corresp onding signal transitions,

where rising arro rep- resen ts rising edge and falling arro falling edge. 5. TRANSA CTION CHECKERS Once user has insp ected and appro ed the transaction diagrams, Inferno can automatically generate hec ers in register-transfer lev el (V erilog) form corresp onding to eac transaction. These hec ers can used in range of dif- feren erication con texts to impro the condence in the correctness of the design, as discussed in Section 3. It is not unreasonable to sa that extracting transaction diagrams from sim ulation trace, Inferno dev elops an \understand- ing" of the design, whic

it can use to generate transaction hec ers. In practice, attain this goal using the set of ertices and edges of transaction to establish the legal eha vior of the system. or eac ertex, generate an expression corresp onding to all legal outgoing edges from it; then build the transaction hec er as the disjunction of all these expressions. The initial ertex is used to ac- tiv ate the hec er. If the design erforms transition not describ ed in the hec er, the fail output signal is raised to ag oten tial problem. Our initial automatically gen- erated erilog description is then sen through

syn thesis to ol and optimized. In our exp erimen ts found that SIS [6] as sucien to handle the complexit of the netlists generated. Alternativ ely commercial syn thesis to ols, suc as Synopsys's Design Compiler, can used. hec er accepting um er of separate transactions can easily generated com bining the \fail" outputs of
Page 6
the individual hec ers. This divide-and-conquer approac to describ complex set of transactions has pro en to helpful in reducing the hec er complexit an ted to erify that no un-appro ed transaction as ccurring in the Wish one example of Section 6.

The hec er describing all ossible activit of the in terface as to complex to man- ageable in syn thesis and sim ulation. Ho ev er, when ex- pressed the hec er as comp osition of ultiple transaction hec ers, ere not only able to erify that the hec er as an in arian across the en tire sim ulation, but also ob- serv whic transactions as ccurring. The hec er gener- ator can also used to erify co erage. By expanding hec er to detect when the nal ertex of transaction has een observ ed and to include \complete" signal, can easily coun the ccurrence of eac yp of transaction in co erage-driv en

erication con text. Additional uses of the transaction hec ers ha een describ ed in Section 3. 6. LEARNING FR OM TRANSA CTIONS discussed in Section um er of erication metho d- ologies where Inferno can eectiv ely deplo ed. In the con text of design dev elopmen t, situations ma arise when clear sp ecication of the in terface proto col is missing. In this scenario Inferno can ey in dev eloping common understanding of the proto col through visual insp ection of the diagrams, hence eliminating oten tial complex in terface bugs. ypical example is that of in terfaces

et een in- house comp onen ts and third-part IP cores, where details of the in terface ma am biguous and access to the design- ers scarce. The enets of high-lev el transaction mo deling in Inferno are illustrated the follo wing case study While analysis results of stable designs pro vide in teresting examples, it is ultimately more alid to observ the ossi- ble uses of to ol in real orld setting. By lending our assistance to group of studen designers, ere able to observ the enecial eects of the proto col and transaction diagrams, oth in terms of direct bug-nding

i.e. iden tify- ing bugs simply examining the diagram), and in terms of hec king the equiv alence of instan tiations of the same mo dule er the course of single test (whic rev ealed ma jor dierence in the co erage of the mo dules). The studen ts ere engaged in pro ject to design dual- core Alpha pro cessor with indep enden oltage and frequency scaling incorp orated in to the cores. Eac core had its wn L1 instruction and data cac hes, and they shared L2 cac he using the MESI proto col. Only one concurren access to the L2 cac he as allo ed, leading to the need for arbi- tration. Moreo

er, since the L2 cac he alw ys op erated at the maxim um frequency and oltage, while the cores could op erate at lo er frequencies and oltages, the in terface e- een the L1 and L2 cac hes as async hronous. The initial division of lab or astly underestimated the complications of the cac he proto col and allo cated only one of the four engi- neers to design all of the cac he con trollers and arbitration logic. As the deadline approac hed and the cac hes still could not pass simple tests, it ecame clear that this had een mistak e. The other three designers ere recruited to aid in the debugging of

the cac hes. Ho ev er, they had ery lit- tle familiarit with that part of the design. The original designer and one other engineer egan together to analyze the system with eform view er. the same time, the other team mem ers (neither of whom as familiar with this part of the design) decided to put Inferno to use. The design as so far from eing op erational that it as imp os- sible to generate trace long enough to reliably determine the transactions; ho ev er, they could obtain proto col di- agram as the one sho wn in Figure 6, whic they pro ceeded to analyze. Ov er the course of the same nigh

t, despite the signican dierence in their bac kground kno wledge, oth teams indep enden tly disco ered the same bug. As indicated in Figure 6, the proto col diagram con tains transition from the idle state to state where the signals indicate that there is data ready from the L2 cac he. The most na v examina- tion suggests that this is suspicious, since there has ob vi- ously een no request for data from the L1 cac he. urther in estigation rev ealed that it as, indeed, bug. The dia- gram for the corrected ersion is the same except that the mark ed transition is missing.

This case study also exp osed the enet of Inferno in ev al- uating the qualit of testb enc suite. set up an ex- erimen whic compared the transactions detected at the in terface of separate instan tiations of the same mo dule since there ere cores, almost ev ery mo dule as du- plicated. Once the design as sucien tly stable to execute simple programs, transactions ere generated for the in ter- faces of the arbiter with eac of the cores. The results ere dramatically dieren t, with only one transaction in common. urther analysis sho ed that this as ecause the cores ere

executing the same program on the same data; hence, the one whic accessed the L2 cac he rst ould alw ys the rst to request new data, while the other ould alw ys nd the cac hed alues ready Consequen tly the transactions ere ery dieren at the in terfaces: one ould ha all misses and the other all hits. This result as ey in driv- ing the dev elopmen of more aried testb enc hes, leading to div ersied co erage on oth instances. 7. EXPERIMENT AL RESUL TS ev aluate the qualit of our solution, ha exercised Inferno on um er of widely arying designs: the Wish- one

DMA describ ed in Section 4.3, the PCI in terface of PCI bridge, one of the cac he arbiters in the dual-Core Alpha pro cessor from Section 6, Serial arallel In terface (SPI) in terface, the in terface et een the execute and mem- ory stages of Z80, set of FIF queues, Reed-Solomon deco der, and USB proto col design. All but the cac he ar- biter testb enc are ailable in [11]. estb enc gates FF cycles if.nets Wish one DMA 1,972 672 1,759,678 PCI proto col 334 95 8,298,177 13 Dualcore Alpha 109,441 22,608 9,310 10 SPI proto col 4,578 1,345 1,999 Z80 3,628 277 14,436 16 FIF Os 4139 2091 2612259 12

R.-S. Deco der 987 231 272 19 USB 292 98 1517565 13 able 1: estb enc hes haracteristics and setup. The table sho ws for eac testb enc h: um er of gates and ip-ops, the um er of sim ulation cycles in the input trace for Inferno, and the um er of nets (signals) whic are part of the in terface. able pro vides brief haracterization of the exp eri- men tal testb enc hes and able lists the results of our anal- yses. As sho wn in able 1, the um er of bits monitored on an one in terface ranges from to 16, and the lengths
Page 7
Figure 6: Proto col diagram for the cac he

arbiter in terface in the dual-core Alpha pro cessor. Notice that the lab eled edge sho ws the in terface lea ving idle state and en tering conguration where the arbiter signals data ready from L2 cac he. This is clearly bug, since no one has requested an data, and et it is there. This case study presen ts situation where direct insp ection of simple proto col diagram yields insigh tful understanding and debugging supp ort. of the sim ulation traces to analyzed ary up to mil- lion cycles. able sho ws that the bus in terfaces and cac he arbiter ha um er of ell-dened transactions

whic are able to detect, while the pip eline stage in terface of the Z80 do es not follo clear transaction pattern, as one ould exp ect. Only three oten tial transactions could iden tied for the Z80 in terface, and insp ection of the results suggests that these are not actual transactions, but simply coinciden tal lo ops of instruction sequences in the testb enc h. Surprisingly only three transactions ere iden tied for the SPI in terface. urther insp ection of the resulting diagrams rev eals that these do indeed corresp ond to eha vioral trans- actions, and the small um er just

reects the simplicit of the proto col: they are really the only transactions whic can ccur. The other proto cols all resp ond ell to the analysis, yielding um er of transactions. estb enc prot. distinct total rep eat nets transact. states states Wish one DMA 10 10 PCI proto col 13 208 310 60 Dualcore Alpha 10 13 SPI proto col 28 Z80 16 23 FIF Os 12 59 54 R.-S. Deco der 19 22 USB 13 40 25 20 able 2: ransaction analysis The table rep orts for eac test the um er of signals in the in terface, the distinct trans- actions whic Inferno could extract from the sim ulation trace, the ertices in

the proto col diagram, and the um er of in terface congurations whic are part of more than one transaction. also note that in all cases, including the Z80, the um- er of distinct congurations actually observ ed at the in ter- face (and therefore assumed to legal) is far smaller than the um er of ossible states, that is, (# sig nals monitor ed supp orting the argumen that proto col and transaction di- agrams are actually uc more compact to ev aluate the eha vior of system compared to eform view er. It also indicates that the hec ers generated are fairly com- pact, since they

only span the small set of congurations actually observ ed. Moreo er, the proto col diagram has few transitions compared to clique, suggesting that the ex- traction of transaction is indeed enecial in understanding the transition patterns et een distinct in terface congura- tions. This oin is also supp orted the observ ation that the conguration is frequen tly rep eated in more than one transaction, suggesting that it is not only the com bination of signals itself, but the con text in whic it app ears, whic should in terest us. Finally ha plotted the detection

of transactions er time (measured in sim ulation cycles) for three designs with signican um ers of transactions. In Figures 7, and mark ed the sim ulation cycle of rst disco ery for eac transaction. Note the erio dic bursts in new transactions observ ed, whic could corresp ond to when the regression testsuite switc hes to new phase, stim ulating dieren as- ects of the design. 8. CONCLUSION In this pap er presen ted Inferno, soft are to ol to au- tomatically extract, or infer, transactions from sim ulation trace er user-selected in terface. Inferno presen ts the re-

sults of its analysis through in tuitiv \proto col diagrams", rep orting the erall proto col observ ed at the in terface, and \transaction diagrams", sho wing the o of eac distinct yp of transaction detected. In addition, Inferno can au- tomatically generate transaction hec ers in erilog, that
Page 8
is, hec ers whic monitor the execution of transaction. Chec ers can used in the con text of constrained-random sim ulation, or co erage-driv en metho dology and also in formal erication setting. Dual-Core Alpha processor - arbiter protocol 500,000 1,000,000 1,500,000

2,000,000 2,500,000 3,000,000 3,500,000 4,000,000 simulation cycles # transactions Figure 7: Cac he arbiter. Num er of distinct transactions detected er time for the cac he arbiter of the dual-core Alpha. Wishbone 50,000,000 100,000,000 150,000,000 200,000,000 250,000,000 300,000,000 350,000,000 400,000,000 simulation cycles # transactions Figure 8: Wish one DMA. Num er of distinct transactions detected er time for the Wish one DMA design. In the con text of random sim ulation-based erication, Inferno enables no el erication metho dology inspired the principle of least

astonishmen t, automatically ex- tracting an new transaction observ ed at the selected in ter- faces, and presen ting it to the user for ev aluation through simple, high-lev el diagrams, th us allo wing erication en- gineer to fo cus in on the uncommon asp ects of design's eha vior, in the hop of unco ering hidden bugs. The case study in debugging dual-core Alpha pro cessor demonstrates Inferno's usefulness in practice, ev en for large designs. In addition, exp erimen tal ev aluation on range of designs in- dicates that Inferno is eectiv in summarizing the common eha vior of

system and presen ting it to the user through simple and in tuitiv diagrams. plan to explore this metho dology further dev eloping additional tec hniques to decomp ose the inheren structure of an in terface activit in to simple comp onen ts and in estigating more scalable solutions for our assertion generation engine. 9. REFERENCES [1] G. Ammons, R. Bo dik, and J. R. Larus. Mining sp ecications. In Symp osium on Principles of Pr gr amming anguages pages 4{16, 2002. [2] T. Arts and L.-A. redlund. race analysis of erlang programs. In CM Sigplan Notic es pages 18{24, 2002. Figure 9: PCI

bridge. Num er of distinct transactions de- tected er time for the PCI bridge design. [3] S. Bensalem, Y. Lakhnec h, and H. Sadi. erful tec hniques for the automatic generation of in arian ts. In Pr dings of the 8th International Confer enc on Computer ide eric ation pages 323{335, Aug. 1996. [4] B. Ben tley and R. Gra alidating the In tel en tium Micropro cessor. Intel chnolo gy Journal pages 1{8, 2001. [5] D. S. Brahme, S. Co x, J. Gallo, W. Grundmann, C. N. Ip, W. aulsen, J. L. Pierce, J. Rose, D. Shea, and K. Whiting. The transaction-based erication metho dology ec hnical

rep ort, Cadence Design Systems, Inc., Aug. 2000. ec hnical Rep ort No. CDNL-TR-2000-0825. [6] E. M. Sen to vic h, K. J. Singh, L. La agno, C. Mo on, R. Murgai, A. Saldanha, H. Sa j, R. Stephan, R. K. Bra yton and A. Sangio anni-Vincen telli. SIS: system for sequen tial circuit syn thesis. ec hnical rep ort, 1992. [7] D. Engler, D. Y. Chen, S. Hallem, A. Chou, and B. Chelf. Bugs as devian eha vior: general approac to inferring errors in systems co de. In SOSP '01: Pr dings of the eighte enth CM symp osium on Op er ating systems principles pages 57{72, New ork, NY, USA, 2001. CM Press. [8] M.

D. Ernst. erication for legacy programs. In erie ols: The ories, ols, Exp eriments uric h, Switzerland, Octob er 10{13, 2005. [9] G. ey and R. Drec hsler. Impro ving sim ulation-based erication means of formal metho ds. In ASPD C, Pr dings of the Asia South Pacic Design utomation Confer enc pages 640{643, Jan. 2004. [10] S. Hangal, N. Chandra, S. Nara anan, and S. Chakra ort Io dine: to ol to automatically infer dynamic in arian ts for hardw are designs. In '05: Pr dings of the 42nd annual onfer enc on Design automation pages 775{778, New ork, NY, USA, 2005. CM

Press. [11] ttp://www.op [12] T. Sc ub ert. High-lev el formal erication of next-generation micropro cessors. In Pr c. pages 1{6, June 2003. [13] J. ang and D. Ev ans. Automatically inferring temp oral prop erties for program ev olution. In Pr dings of the 15th International Symp osium on Softwar eliability Engine ering (ISSRE'04) pages 340{351, No v. 2004.