Achie ving Principled Assur edly rustw orth Composable Systems and Netw orks Peter G
128K - views

Achie ving Principled Assur edly rustw orth Composable Systems and Netw orks Peter G

Neumann Computer Science Lab SRI International Menlo ark CA 940253493 NeumannCSLSRIcom Abstract Hug halleng es xist with systems and networks that must dependably satisfy string ent equir ements for security eliability and other attrib utes of trust

Download Pdf

Achie ving Principled Assur edly rustw orth Composable Systems and Netw orks Peter G

Download Pdf - The PPT/PDF document "Achie ving Principled Assur edly rustw o..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation on theme: "Achie ving Principled Assur edly rustw orth Composable Systems and Netw orks Peter G"— Presentation transcript:

Page 1
Achie ving Principled Assur edly rustw orth Composable Systems and Netw orks Peter G. Neumann Computer Science Lab, SRI International Menlo ark CA 94025-3493 Abstract Hug halleng es xist with systems and networks that must dependably satisfy string ent equir ements for security eliability and other attrib utes of trustworthiness. Dr awing on what we have learned ver the past decades, our CHA TS pr oject seeks to establish coher ent common-sense appr oac towar trustworthy systems. The appr oac en- compasses compr ehensive sets of equir ements, inher ently

sound ar hitectur es that can be pr edictably composed out of well-conceived subsystems, highly principled de velop- ment tec hniques, good softwar engineering disciplines, sound oper ational pr actices, and judiciously applied assur ance measur es. Although suc an appr oac is lik ely to seem completely old-hat to some esear her and totally impr actical to commer cial de veloper s, the wisdom thus embodied is seldom used consistently (if at all) in pr actice; if it wer used wisely muc of the untrustworthiness in today systems would simply disappear This paper briefly summarizes our appr

oac and its potential benefits. 1. Intr oduction summarize the primary task of our tw o-year project under Contract N66001-01-C-8040, as part of ARP Composable High-Assurance rustw orthy Sys- tems (CHA TS) program. The ARP Program Manager is Dr Douglas Maughan. Our project spans the goals of the ARP CHA TS program including trustw orthiness, composability and assurance. The final report [10 is ail- able on the eb, in draft form prior to June 28, 2003, and in final form thereafter: http://www tml http://www df

http://www In our project, we are confronting an xtremely dif fi- cult problem namely ho to attain demonstrably trust- orthy systems and netw orks that need to operate under stringent requirements for security reliability survi abil- ity and other critical attrib utes. In particular we pursue sound foundations for the creation of trustw orthy systems and netw orks that can be easily composed out of subsys- tems and components, with predictably high confidence that the meet their requirements and also do something sensi- ble when forced to operate

outside of the xpected normal range of operational conditions. ard this end, we x- amine arious principles for achie ving trustw orthiness, con- sider constr aints that might enhance composability propose ar hitectur es and trustworthy subsystems that are inherently lik ely to result in trustw orthy systems and netw orks, con- sider constraints on administr ative pr actices that can reduce the risks of bad beha vior and xplore approaches that can significantly increase assur ance also outline needs for ne esear and de velopment that could significantly im- pro the future for

dependably trustw orthy systems. ith respect to the future of trustw orthy systems and netw orks, perhaps the most important recommendations in- olv the ur gent establishment and consistent use of realis- tic highly disciplined and principle-dri en architectures, as well as de elopment practices that systematically encom- pass trustw orthiness and assurance as inte gral parts of what must become coherent de elopment processes and sound subsequent operational practices. Only then can we ha an realistic hopes that our computer -communication in- frastructures and consequently our national

infrastruc- tures will be able to beha as needed, in times of crisis as well as in normal operation. De eloping systems with critical trustw orthiness re- quirements is inherently more complicated than ordinary softw are. The risks are typically much greater [9 ], and the challenges ha no simple turn-the-crank solutions. Abil- ity understanding, xperience, education, and enlightened management are crucial. Success can be greatly increased in man ays, including dependable hardw are, rob ust sys- tem and netw ork architectures, perv asi use of good soft- are engineering practices, careful

attention to human- oriented interf ace design and especially ease of operation and system administration, sound and properly used pro- gramming languages, trustw orthiness-enhancing compilers, techniques for increasing interoperability among heteroge- neous systems and subsystems, methods and tools for anal- ysis and assurance, and other actors. The absence or rel- ati deficienc of each of these actors today represents potential weak link in process that is currently riddled with too man weak links. On the other hand, much greater
Page 2
emphasis on these actors can result in

substantially greater trustw orthiness, often with more predictable results. The approach of our project is strongly inspired by his- torical perspecti es of fruitful research ef forts and xtensi xperience (both positi and ne gati e) relating to the de el- opment of trustw orthy systems. It is moti ated by the prac- tical needs and limitations of commercial de elopments as well as by CHA TS successes in inserting greater discipline into the open-source orld. It pro vides useful guidelines for disciplined system de elopments. It also identifies ari- ous recommendations for future ef

forts. As consequence of the inherent comple xity associated with the challenges of de eloping and operating trustw orthy systems and net- orks, it is impossible to represent the breadth and depth of scope of our ork in this brief summary which merely touches on the problems and the potential solutions. Thus, we ur ge you to read the cited report [10 and see ho much of it might be applicable to you. The main thrust of the ef fort considers lessons dra wn from past research and prototype de elopments, and ari- ous approaches that can lead to much greater trustw orthi- ness. Man of these

concepts will be well kno wn to you, at least in the small ho we er the need to be applied vigor ously and consistently in the lar The ould-be lessons of the past ha been widely ignored, especially in com- mercial de elopments with multitude of xcuses being of fered: for xample, percei ed irrele ance of trustw orthi- ness in the mass mark etplace; lack of customer demands for systems satisfying critical requirements; absence of le gal li- ability for consequential damages; dif ficulties in using good softw are engineering practices, high-assurance techniques, and aluation criteria; le

gac compatibility constraints; weak system-oriented computer science education; and perv asi tendenc to blame users and system administra- tors for the human ailures that produce inherently fla wed systems, weak netw ork protocols, and badly designed hu- man interf aces. Short-sightedness abounds. In particular the percei ed importance of short-term cost increases and delayed deli eries could be lar gely mar ginalized if the long- term costs of deb ugging, inte gration, recoding, incremental maintenance, frequent upgrades, and critical dependence on highly skilled administration were

included in the analysis. It is widely recognized that up-front ef forts can greatly re- duce the erall ef fort. do not wish to preach to the choir or to proselytize the unw ashed. This summary paper is written in the hopes that you will read the report, irrespecti of your softw are de elopment predilections. or de elopers who seriously seek to ercome the typical practice of seat-of-the-pants perhaps-just-good-enoug softw are creation, particularly for critical applications, we of fer fe ca eats about the report. If you are looking for erall simplicity you may be sadly disappointed; indeed,

each chapter of the report may con vince you that there are fe if an easy answers. Ho w- er there are man answers if properly applied. If you are looking for some practical advice on ho to de elop systems and netw orks that are substantially more trustw or thy than what can be achie ed today by patching together of f-the-shelf fla wed systems, then you may find some encouraging directions to pursue. belie that diligent observ ance of the approaches described in our report [10 can greatly impro the situation. The opportunities for this within the open-source community are

considerable; the are also potentially applicable to closed-source proprietary systems if the xcuses noted abo can be esche wed. Ho we er in an case, what is needed is greater discipline and attention to the fundamentals we discuss relating to the entire life ycle, including the presence of meaningfully comprehensi critical requirements, sound system and netw ork architectures, highly principled softw are de elop- ment, and operationally are de elopment. This approach is neither simplistic nor erwhelmingly complicated. 2. ppr oach of the Pr oject Our starting point in olv es the desired roles

of principles rele ant to the conceptualization, design, im- plementation, operation, and use of information systems and netw orks ha ving critical requirements for security reliability and survi ability also identify obstacles to achie ving acile composability and interoperability and to consider approaches that can contrib ute to the de elopment of significantly greater composability in systems with critical requirements. then consider characteristics of architectures that are lik ely to predictably satisfy the CHA TS goals, based on the discussion of principles and the analysis of

composability If serious measures of assurance of the required trustw orthiness are desired, then ariety of coherent and mutually supporting techniques needs to be applied. There is no one approach that fits all. Whene er assurance techniques are used, their strengths and limitations need to be understood from the outset, and applied specifically where the can be most ef fecti e. arious techniques are considered. 3. Attaining rustw orthiness belie that man R&D directions are important for the short- and long-term future for computer and netw ork communities, for de elopers, and for

ARP A. The basis of our project is the xploration of fe of the potentially most timely and significant approaches, which are summarized roughly as follo ws. (See [10 for xtensi discussion of each item.)
Page 3
Principles. re visit fundamental principles of trustw or thy system de elopment, cull out those lik ely to be most ef fecti e, xplore their practical limitations, and pro vide basis for principled architectures, principled de elopment, and principled operation. or xample, the 1975 Saltzer Schroeder principles [16 are still mostly rele ant today On one hand, some of the

most basic principles seem to be treated merely as arcana from the ancient literature; the are found occasionally in uni ersity curricula, and are intuiti ely appealing to some de elopers. On the other hand, these principles are surprisingly absent in commer cial programming practice, and therefore unobserv ed in sys- tem architectures, system implementations, programming languages, compilers, softw are engineering disciplines, and softw are de elopment tools. Certain principles are of great potential benefit to com- posability trustw orthiness, and assurance; these include (among

others) abstraction, modularity encapsulation, layered protection, separation of polic and mechanism, and separation of concerns to the xtent that the are sensibly applied to the architecture and the implementation, and that the do not interfere with one another In practice, some of the principles can sometimes be contradictory and therefore should not be erendo wed or considered as ends in themselv es. Composability xplore obstacles to achie ving seam- less composability and techniques for attaining practical composability in the future. Composability is meaningful at man layers of

abstraction, for components, subsystems, netw ork ed systems, and netw orks of netw orks. It is also applicable to policies, protocols, specifications, formal rep- resentations, and proofs. Subsystem composability tak es on ariety of forms, including sequential (e.g., with or with- out feedback, and with or without recursion) and parallel ecution. Composability has long been crusade of the research community ut has typically been hindered by short- sighted de elopment practices. Obstacles to seamless composability typically include (for xample) hidden state interactions; emer gent

properties that xist be yond what is rele ant to constituent subsystems and that manifest themselv es only because of composition; nonmodular and poorly designed subsystems that seriously hinder decomposition; and compositions that simply do not scale properly These and further dif ficulties often arise because of poor design abstraction and the lack of appropriate mod- ularity and encapsulation. Thus, principled architecture and softw are de elopment both can play significant role in enhancing composability articularly rele ant are the techniques and de elopment tools that induce

softw are engineering discipline, sound programming-languag constructs, ecution compatibility and interoperability Ho we er in certain cases, compositions of seemingly compliant subsystems can actually compromise the ability of the resulting system to satisfy its requirements. rustw orth composable ar chitectur es. characterize composable open distrib uted-system netw ork-oriented ar chitectures capable of fulfilling critical security reliability survi ability and performance requirements, while being readily adaptable to widely dif fering applications, dif ferent hardw are and softw are

pro viders, and changing technolo- gies. By architecture, we specifically mean both the struc- ture of systems and netw orks and the design of their func- tional interf aces (at each layer of abstraction). Although dif ferent architectures may be needed for dif- ferent classes of applications, the can share man com- mon principles and attrib utes. consider se eral alter nati architecture approaches. articularly appealing are architectures in which great attention is paid to minimiz- ing the xtent to which all subsystems must be trusted, in which trustw orthiness can be concentrated

primarily in fe particularly critical components within layered and dis- trib uted systems, with emphasis (for xample) on trustw or thy serv ers and highly constrained well-defined interf aces that can greatly enhance composability and interoperabil- ity Ideally we or architectures in which man com- ponents do not ha to be completely trustw orthy (and in which some may be completely untrustw orthy as in Byzan- tine agreement), ut where the erall system can still be adequately trustw orthy An xample of such an architec- ture is distrib uted and netw ork ed system in which fe really

critical special-purpose components (e.g., dedicated serv ers) serv particular purposes for which the are x- tremely trustw orthy It should be possible to compose those special-purpose systems out of just fe trustw orthy com- ponents, and to omit the ast majority of the bloatw are that might otherwise be found in con entional operat- ing system together with its def ault applications. The clue to the decomposition problem (getting rid of all the func- tionality you do not need, and consequently getting rid of most of the corresponding security vulnerabilities) lies in the composition problem

that is, being able to construc- ti ely include just the essential functionality and nothing else (achie ving stark subset ). Examples of such architec- tures are pro vided by Rushby and Randell [15 and Proc- tor and Neumann [13 in implementing multile el security without ha ving to trust ery end-user system to enforce multile el security (MLS), although similar approaches are alid for single-le el systems. dif ferent xample is found in SeaV ie [7 ], where high-assurance MLS database man- agement system as demonstrably achie able by putting Oracle on top of an MLS security ernel, with no MLS-

Page 4
trustw orthiness required in the DBMS. Such architectures are needed to pro vide alternati es to the relati ely undisci- plined mass-mark et operating systems in which essentially all of the ernel code, utility programs, application code, test softw are, and user beha vior must be trusted en when the are not trust worthy Also of considerable interest is the concept of combining subsystems in ays that actu- ally increase the resulting trustw orthiness; we enumerate 22 such basic trustw orthiness-enhancing mechanisms in [10 ]. or truly comple sets of requirements, comple xity

can still be managed with combination of sound composable architectures, hierarchically layered and horizontally distrib uted abstraction, sensible modularity together with encapsulation, stark subsetting for critical subsystems by remo al of unnecessary functionality proacti ely olv able ut well-concei ed conceptually simple interf aces that mask local comple xity and principled softw are engi- neering. As result, inherently comple systems can be reduced to relati ely simple interconnections of relati ely simple components with relati ely simple interf aces and relati ely simple xception

conditions. (F or historical xample of this, see the Pro ably Secure Operating System design [4 11 ], in which each module as formally defined in fe pages of formal specifications, and the interlayer dependencies were also defined in fe lines of formally defined abstract implementations [11 14 ], ut where the erall system as quite comple x.) rustw orth oundations and assurance. seek sound bases for requirements, specification, implementa- tion, trustw orthiness, and assurance for predictably com- posable interoperable components. Whate er assurance measures are

deemed desirable in increasing trustw orthi- ness (as opposed to merely being trusted), those measures need to be distrib uted throughout the de elopment process from conceptualization and requirements through de- sign, implementation, and operation and inte grated thor oughly throughout. Some aspects of formal methods are certainly rele ant, especially when critical requirements are in olv ed. Per haps surprising to some people, significant progress has been made in the past 30 years in formal methodologies and supporting tools, particularly for applications requiring safety reliability

and security One area of particular rele- ance that as xplored in the 1970s ut that has ne er been widely used is the ability to formally map hierarchical ab- stractions at one layer onto abstractions at other layers (as for xample in [12 14 ]). This approach could be xtremely aluable in the present conte xt, as it can enable the formal analysis of compositions (e.g., systems of systems). The use of formally or semiformally based analytic tools is ery promising area. As one recent xample, under our CHA TS project, Hao Chen and Da agner at the Uni- ersity of California at Berk ele ha de eloped

model- checking en vironment that xamines source code for pres- ence or absence of certain types of characteristic implemen- tation fla ws in code (e.g., in olving setuid-lik calls and root pri vile ges); the ha already applied this tool to send- mail, OpenSSH, and wu-ftpd [1 ], and are continuing to xtend its applicability Looking to future in which special-purpose and general-purpose applications might become routinely composable out of more-or -less compatible demonstrably trustw orthy components, an analysis tool ould be highly desirable that can analytically determine the

composability (among other properties) of softw are components not just for the initial creation of composed system, ut also in subsequent reconfigurations, upgrades, and en dynamic installation of mobile code. This approach could (for xample) tak adv antage of specifications and softw are formally sho wn to be consistent with those specifications, including descriptors relating to pre viously aluated characteristics of the components (such as internal locks that might cause deadly embraces in certain conte xts, assumptions re garding dependence orderings, identified

interf ace limitations, and other attrib utes that might af fect the compositionality). Ideally this approach could then be used iterati ely for xample, initially pairwise or -wise, and then er successi ely wider scopes, possibly ascertaining obstacles to the desired compositions, or en potential ailure modes that ould suggest that specific composition should not be permitted because of its identified deficiencies. Other properties could also be included, such as dynamic trustw orthiness, configuration stability and operational actors. realize that there are all sorts

of limitations of such an approach, ut en small steps forw ard could be ery useful. rustw orth pr otocols. need to de elop ne protocols and/or xtend xisting protocols that ef fecti ely mask the peculiarities of arious netw orking technologies where er possible, ut able to accommodate wide range of tech- nologies (e.g., wireless and wired, optical and electronic), and capable of addressing all rele ant critical requirements. This is ery dif ficult challenge, and necessitates the in olv ement of NIST standards ef forts, the de elopment communities, and or ganizations such as the Internet

Engi- neering ask orce (IETF). Principled operational practice. need to bring the abo concepts into the realm of operational practice, which is seriously in need of greater dependability and controllability including perhaps some formal dynamic analyses inline with ould-be reconfigurations and dy-
Page 5
namic system changes. Man of the concepts considered here ha considerable potential to ard that end. 4. Historical Reflections Throughout the history of ef forts to de elop trustw or thy systems and netw orks, there is an unfortunate short- age of observ able long-term

progress. Significant research and de elopment results are typically soon for gotten or else widely ignored in practice. Computer systems ha come and gone; programming languages ha come and (occasionally) gone; certain specific systemic vulnerabili- ties ha come and gone. Ho we er man generic classes of vulnerabilities seem to persist fore er such as uf fer erflo ws, stack mismanagement, race conditions, of f-by- one errors, mismatched types, di vide-by-zero crashes, and uncheck ed procedure-call ar guments, to name just fe classes within ery long list. Ov erall, it is

primarily only the principles that ha remained in violable at least in principle despite their ha ving been widely ignored in practice. It is time to change that unfortunate situation, and honor the principles. This is particularly frustrating to researchers who kno better ut should also be ery em- barrassing to commercial de elopers who don (although it seems not to bother them ery much). As an xample of lessons that might ha been learned from the past, consider the Multics de elopment that be gan in 1965, with considerable support from ARP A, MIT Bell Labs, and Hone ywell. Multics made some

xtremely important early adv ances. Its architecture included truly independent se gmentation, paging, and concentric pro- tection rings in the hardw are, hierarchically structured directories, access-control lists, and dynamic linking of symbolic file names, and highly principled de elopment process that enforced appro al of specifications before implementation and the use of higher -le el programming language (a subset of PL/I). See also Fernando Corbat o uring lecture [3 for surv of lessons learned from the ef fort. The system architecture took significant adv antage of

abstraction, modularity and encapsulation, and of the constraints imposed by PL/I handling of strings, arrays, and structures. Because of the fle xibility uilt into the design, it as subsequently relati ely straightforw ard to retrofit multile el security into the deli ered production ersion of the system. Kar ger and Schell ha recently reconsidered their 1974 Multics security aluation [5 with fresh analysis of the aluation xperience. Their 2002 paper [6 notes (for xample) that the use of PL/I, the underlying hardw are protection, and the stack discipline almost completely oided uf

fer and stack erflo ws, data interpretable as ecutables (because of the absence of the ecute bit), rojan horses, and other characteristic security problems and generally greatly enhanced the security and the ease of aluation. Unfortunately man of the lessons observ ed by Corbat and more recently by Kar ger and Schell and man of the principles enumerated by Neumann [8 in 1969, Saltzer and Schroeder [16 in 1975, and others ha been widely ignored in more recent system de elopments. ARP should tak considerable credit for ARP vision during the 1960s in supporting the Multics ef fort, ut

should no also recognize the importance of the lessons that should ha been learned by others along the ay to the present and should encourage greater observ ance of those lessons. (Man additional references are gi en in [10 ].) 5. Dir ections or the Futur Much ef fort remains in demonstrating the practical rele ance of the outlined approach. If intelligently applied, significant long-term benefits are lik ely to result, including inherently sound system and netw ork architectures that minimize the dependence on the trustw orthiness of all systems, subsystems, and users (local or

remote, whether malicious or not); highly disciplined de elopment prac- tice; fe wer fla ws and less need for frequent patches and upgrades; and systems that do not erly rely on the constant vigilance of an inordinate number of skilled system administrators. People are clearly our most critical resource, ut there is serious shortage of computer professionals with sense of history and theory plus strong grasp of the discipline that is necessary for sound de elopment practice and sound operational practice. Hardw are that enhances softw are security and reliability can also be ery

beneficial. Although certain mainstream processors ha some security-related functionality that could approach the capabilities of the Multics hardw are noted abo e, little of that functionality is actually used for trustw orthiness. Some hardw are aids are also clearly desirable, such as special-purpose co-processors in digital commerce, cryptography and management. Finally demonstrations of ho techniques for high assurance can realistically be incorporated into mainstream softw are de elopments ould be xtremely aluable. Our CHA TS project is addressing all these directions. 6.

Conclusions The task of designing and implementing assuredly trust- orthy systems and netw orks is inherently comple x. It re- quires great diligence, ef fort, xperience, and abo all areness of past mistak es and commitment to oid- ing them. Obli viousness to the past is high-probability path to untrustw orthy systems. hope that our ARP
Page 6
CHA TS report will pro vide you with useful pointers to the past and the future. Please read the cited report [10 ], and share your thoughts with us on its rele ance within your wn en vironments. It is vident that enormous benefits should

result from system and netw ork de elopments that are highly principled and that use inherently sound architectures and sensible softw are engineering practices. This is not ne concept, although it seems to be seldom practiced. One enormous benefit therefrom ould be the attainment of predictably seamless composability of systems (or perhaps analytically identifiable interactions) and starkly subsetted systems that oid the usual plethora of xploitable vulnerabilities. This can entail compositions of demonstrably trustw orthy sub- systems, or in some cases compositions of less trustw

or thy subsystems whereby it is possible to xplicitly demon- strate the attainability of greater trustw orthiness (as dis- cussed in [10 ]). Refer ences [1] H. Chen and D. agner MOPS: An infrastructure for xamining security properties of softw are. In Ninth CM Confer ence on Computer and Communications Security ashington, D.C., No ember 2002. CM. [2] H. Chen, D. agner and D. Dean. Setuid demystified. In Pr oceedings of the 11th USENIX Security 2002 San Francisco, California, August 2002. USENIX. [3] .J. Corbat o. On uilding systems that will ail (1990 uring Aw ard Lecture, with follo

wing intervie by Karen Frenk el). Communications of the CM 34(9):7290, September 1991. [4] R.J. Feiertag and .G. Neumann. The foundations of Pro ably Secure Operating System (PSOS). In Pr o- ceedings of the National Computer Confer ence pages 329334. AFIPS Press, 1979. [5] .A. Kar ger and R.R. Schell. Multics security al- uation: ulnerability analysis. In Pr oceedings of the 18th Annual Computer Security Applications Confer ence (A CSA C), Classic aper section Las gas, Ne ada, December 2002. Originally ailable as U.S. Air orce report ESD-TR-74-193, ol. II, Hanscomb Air orce Base,

Massachusetts. [6] .A. Kar ger and R.R. Schell. Thirty years later: Lessons from the Multics security aluation. In Pr oceedings of the 18th Annual Computer Secu- rity Applications Confer ence (A CSA C), Classic a- per section Las gas, Ne ada, December 2002. http://www .acsac.or g/ [7] .F Lunt, R.R. Schell, .R. Shockle M. Heckman, and D. arren. near -term design for the SeaV ie multile el database system. In Pr oceedings of the 1988 Symposium on Security and Privacy pages 234 244, Oakland, California, April 1988. IEEE Computer Society [8] .G. Neumann. The role of motherhood in the pop art of

system programming. In Pr oceedings of the CM Second Symposium on Oper ating Systems Principles, Princeton, Ne er se pages 1318. CM, October 1969. [9] .G. Neumann. Computer -Related Risks CM Press, Ne ork, and Addison-W esle Reading, Mas- sachusetts, 1995. [10] .G. Neumann. Principled assuredly trustw or thy composable architectures. echnical re- port, Computer Science Laboratory SRI Inter national, Menlo ark, California, 2003. Fi- nal report, SRI Project 11459, June 28, 2003; see http://www .htm l; also and chats4.pdf. [11] .G. Neumann, R.S. Bo yer R.J.

Feiertag, K.N. Le vitt, and L. Robinson. Pro ably Secure Operating Sys- tem: The system, its applications, and proofs. ech- nical report, Computer Science Laboratory SRI Inter national, Menlo ark, California, May 1980. 2nd ed., Report CSL-116. [12] S. Owre and N. Shankar Theory interpretations in PVS. echnical Report SRI-CSL-01-01, Computer Science Laboratory SRI International, Menlo ark, CA, April 2001. http://www wre. [13] N.E. Proctor and .G. Neumann. Architectural impli- cations of co ert channels. In Pr oceedings of the if- teenth National Computer Security Confer ence

pages 2843, Baltimore, Maryland, 1316 October 1992. (http://www 2.h tml). [14] L. Robinson and K.N. Le vitt. Proof techniques for hierarchically structured programs. Communications of the CM 20(4):271283, April 1977. [15] J.M. Rushby and B. Randell. distrib uted secure sys- tem. IEEE Computer 16(7):5567, July 1983. [16] J.H. Saltzer and M.D. Schroeder The protec- tion of information in computer systems. Pr oceed- ings of the IEEE 63(9):12781308, September 1975. (http://www .multicians.or g).