Design of the control software for the muon readout system in NA Stephen F
145K - views

Design of the control software for the muon readout system in NA Stephen F

Takach and Kirsten Nelson Cavendish Laboratory Cambridge University Madingley Road Cambridge CB3 0HE United Kingdom Received 25 October 1996 accepted 29 April 1997 We present a brief description of the readout system for the muon detector in the CER

Download Pdf

Design of the control software for the muon readout system in NA Stephen F




Download Pdf - The PPT/PDF document "Design of the control software for the m..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.



Presentation on theme: "Design of the control software for the muon readout system in NA Stephen F"— Presentation transcript:


Page 1
Design of the control software for the muon readout system in NA48 Stephen F. Takach and Kirsten Nelson Cavendish Laboratory, Cambridge University, Madingley Road, Cambridge CB3 0HE, United Kingdom (Received 25 October 1996; accepted 29 April 1997) We present a brief description of the readout system for the muon detector in the CERN NA48 experiment. This is followed by a discussion of the design of the control software for that system. We particularly emphasize the object-oriented technique of encapsulation. © 1997 American Institute of Physics. S0894-1866 97 01804-X

INTRODUCTION The NA48 experiment at CERN is searching for direct CP violation in the system. The objective is to mea- sure the double ratio of the decay rates of and into the charged and neutral 2 ˛nal states. The NA48 collabo- ration has been conducting R&D and constructing the de- tector for the measurement for the last ˛ve years. What has emerged consists of a sophisticated beam line, detector, trigger, and data-acquisition system. 1±4 For the muon system, the readout consists of 56 chan- nels of time-to-digital converters TDCs housed in FAST- BUS modules of 8 channels each. A TDC

channel samples the activity of its given photomultiplier tube in 1-ns bins and discriminates it against a preset threshold. The TDC then stores the digitized samples in a dual-ported memory for later extraction. This 1-GHz TDC was developed at Cambridge as part of the responsibility for the muon sys- tem. Upon receipt of a trigger, a readout controller ROC collects data from the TDC modules and transfers it over an optical link to the data-acquisition system DAQ . A so- called ‘‘trigger generator TGEN module interfaces the muon readout system to the experiment’s system clock and to the

trigger. A central control program CP oversees all the sub- detector readout systems. The CP imposes a ˛nite state model Fig. 1 on the collection of readout systems via a ‘‘harness’’ process that runs in single-board computers SBCs that reside within the readout crates. Upon request from the CP, the harness will fork child processes that pre- pare the readout systems and enact the state changes. For the muon system, the SBC is an ALEPH event builder con- taining a 68020 microprocessor and running the OS9 oper- ating system. Although the readout system for the muon detector is rather

modestly sized, from a control point of view it contains enough complexity to warrant serious thought about organizing the tasks that the harness must fork to enable changes of state in the system. This com- plexity is one reason for turning to object-oriented model- ing and design. I. OBJECT-ORIENTED MODELING AND DESIGN Object-oriented modeling and design OOMD is a tech- nique in the creation of software. One bene˛t of OOMD is that it forces the programmer to focus attention on the prob- lem domain before writing a single line of code. The inten- tion in this technique is to design a

model that adequately describes the system, without regard to details of imple- mentation such as computer language, ‘‘do-loops,’’ etc. In other words, in OOMD the programmer ˛rst obtains an adequate organization and understanding of the solution to a given problem, removed from implementation constructs. Only later, after the model is complete, does he or she pursue implementation. This approach has at least four advantages. First, it naturally provides a way for the programmer to think through all aspects of a given problem thoroughly. The programmer’s ˛rst job is to model

adequately the situation presented. Second, implementation comes later rather than sooner. By re-examining and criticizing the model the pro- grammer can discover mistakes, dif˛culties, and omissions before they appear in a program itself. Third, the diagrams of the model act as a good source of documentation during the development of the system. When several people are working on the same project, it gives them a common focus for communication. Last, OOMD allows the programmer to limit the scope of side effects generated by de˛ciencies in his/her model or by later changes in

implementation. Even after coding has begun, the designer may probably will discover errors. By encapsulating information, the designer can ensure that an error in one part of a system does not propagate widely to many other parts of the system. In understanding the problem domain via OOMD, the designer creates three models of the system under consid- eration. The ˛rst, and probably most important, is the ob- ject model . It describes the objects the ‘‘things that in- teract with each other in a given system, the operations that each object allows, the data that each object contains, and

the relationships between the sets of objects in the system. Currently at the Dept. of Physics, Wayne State University, Detroit, MI 48201. E-mail: takach@physics.wayne.edu E-mail: nelson@hep.phy.cam.ac.uk 508 COMPUTERS IN PHYSICS, VOL. 11, NO. 5, SEP/OCT 1997 © 1997 AMERICAN INSTITUTE OF PHYSICS 0894-1866/97/11 /508/6/$10.00
Page 2
The second model is the dynamic model . It describes how the objects interact in time or in response to a sequence of events. The last model is the functional model . It describes the ˇow of data through the system, the transformations that it

undergoes, and the functions required to make those transformations. Object-oriented implementations have three character- istics. Encapsulation is the process of distinguishing an ob- ject’s internals from its externals in the context of a given model. Internal data and algorithms are hidden. External data and functions characterize the object to the outside world. In other words, each object has a well-de˛ned inter- face. Polymorphism is the characteristic of having the same conceptual operation apply to multiple objects. For ex- ample, ‘‘inversion’’ applies both to scalar numbers and

to matrices. But the details of inversion obviously depend on the object to be inverted and could be hidden encapsu- lated in the operation itself. Finally, inheritance also char- acterizes object-oriented programs. An object derives or inherits some of its de˛ning properties from another ob- ject. For example, the object ‘‘home appliance’’ may be characterized by data such as model number, manufacturer, and cost. The object ‘‘refrigerator’’ can inherit these prop- erties as well as have other properties more speci˛c to it such as present temperature. II. APPLICATION TO THE MUON

READOUT SYSTEM A. Object model In developing the control system for the muon readout, we have applied the OOMD characteristic of encapsulation. Our model characterizes each object by a set of attributes or external data values and by a set of methods or functions. The data values describe the information available to other objects in the system. The functions are the operations that a given object may perform or that other objects can per- form on a given object. In Fig. 2, a box represents each object and contains two sections. The upper section is a list of the object’s attributes, and the

lower section is a list of its functions. Lines connecting the boxes specify a relation- ship between the objectsÐthat one object may access the attributes or request an action from another. The entirety of the object model for the muon readout consists of four submodels. The ˛rst is the object model of the TDC module. The second and third are the models of the TGEN and ROC modules. The last is the model of the error system. Figure 2 shows the object model for the TDC modules. In this case the model serves largely as a list of subcomponents of the TDC. The same is true for the object

models of the TGEN and ROC modules, not shown. The TDC module contains a module clocking system, a ˛rst-in, ˛rst-out FIFO module for storing data, a set of channels, and a pipeline memory for each channel. The ‘‘pipeline memory’’ is the circular memory that holds data for up to 200 s. These are the primary objects that require control in the TDC module. The other objects in the model pertain to the interac- tion of these primary objects with the hardware. The object is the TDC software mirror TDC SWM . It acts as a mirror of the FASTBUS HARDWARE REGISTERS by providing input-output

buffering for them. Note that the TDC SWM is hierarchical. It is broken into two objects: the mirror of the FASTBUS control space registers CSRs , TDC SW CSR, and the mirror of the FASTBUS data space registers DSRs TDC SW DSR. The TDC SW DSR is further split into component registers A and B. For some operations that the objects perform, using the buffering is mandatory. For others, direct access to the hardware is better. Therefore, objects TDC, TDC MOD CLK, TDC MODULE FIFO, and PIPELINE all have direct access to FASTBUS HARDWARE REGISTERS along with access to the TDC SWM. On the other hand,

object CHANNEL is an example of an object that only requires access to TDC SWM. The last object model is that for the error system; see Fig. 3. If a SBC discovers any errors in a readout system, it must save them somewhere and report them at a convenient time. In the muon system, the initialization and run tasks collect relevant information about an error, pack that infor- mation, and report to the CP with the error message utility EMU Because the ERRORS model is not a ‘‘list of parts’’ for a hardware module in the readout system, its basis is not in hardware. Nevertheless, its model follows

similarly from characterizing it by a set of attributes and functions. Section III C describes error reporting in more detail. B. Dynamic model The intent of the dynamic model is to display the response of the objects to a sequence of events. The notation used in the diagrams helps to describe this sequence. Figures 1 and 4 provide examples of the notation. A box represents a state, which may be further divided into many substates. Arrows show the sequence of state changes, and text next to each arrow describes the reason for the change. Text in parentheses is a parameter that must pass from

one state to Figure 1. Finite state model for the readout systems of NA48. COMPUTERS IN PHYSICS, VOL. 11, NO. 5, SEP/OCT 1997 509
Page 3
the next, and text in square brackets de˛nes a logical con- dition that must be true to make the indicated change. In the muon control system, the most complicated be- havior occurs in the highest level of the run-time dynamic model, Fig. 4. After preliminary initialization, the run task enters an idle state. At the next burst, the run task enters the RUN IN BURST state. While in this state, the run task repeatedly monitors each of the objects

belonging to the TDC, the ROC, and the TGEN. As the looping line shows, while doing this, it must be ready to capture a stop signal from the run control CP , a so-called private-out-of-burst signal, or an error condition. The private-out-of-burst signal de˛nes the beginning of a time period between beam bursts. Readout systems may refuse to respond to the trig- ger system during this period, and they may privately ex- ecute self-tests. Upon capturing one of those conditions, it returns to its present state and continues with what it was doing before one of those signals interrupted it. If

it com- pletes its in-burst sequence before the private-out-of-burst signal arrives, it re-enters the RUN IN BURST state from the top. If the run task receives a private-out-of-burst signal, it leaves the in-burst state and enters the RUN OUT BURST state. There it goes through a sequence of out-of-burst steps to monitor each of the objects belonging to the TDC, the ROC, and the TGEN. If the ‘‘guard period,’’ the period just Figure 2. The object model for the TDC module. 510 COMPUTERS IN PHYSICS, VOL. 11, NO. 5, SEP/OCT 1997
Page 4
before the next burst, begins during the sequence, or

if the sequence is completed, the run task continues from the COLLECT BURST SUMMARY state. Finally, it delivers an end-of-burst summary to the DAQ and reports collected errors to the CP. If a stop signal arrived sometime before reaching the end of the REPORT ERRORS TO CP state, the run task terminates upon leaving this state. Otherwise, the run task then returns through the burst initialization state to the idle state and awaits the next burst. If the arrival of the next burst caused the transition from state REPORT ERRORS TO CP to state BURST INIT, then state IDLE will be metastable. After

burst initialization is completed, the run task will pass without delay through state IDLE into the next state. Figure 3. The object model for the error system. Figure 4. Run-time dynamic model of the control system. See the beginning of Sec. II B for an explanation of the notation and symbols. COMPUTERS IN PHYSICS, VOL. 11, NO. 5, SEP/OCT 1997 511
Page 5
C. Functional model Although the object model and the dynamic model are the two most important models for the muon control system, the functional model also plays a role. The object attributes are particular data values or ˇags

that specify some piece of information, yet the data returned from the hardware mod- ule arrive packed into 32-bit words. Similarly, the hardware expects values packed into 32-bit words during data trans- fers to it. Thus, the attribute values undergo a transforma- tion in passing them from object space to FASTBUS space and vice versa. Although this is not conceptually dif˛cult, the functional model manifests the transformations that oc- cur. One method of handling the transfer from FASTBUS space to object space is to ask an object to read a 32-bit word from the FASTBUS hardware, to mask

off a particu- lar bit in the data word, and to return that information to the user. Such a solution would be wasteful of FASTBUS cycles, for each object would have to read the same 32-bit status word from the hardware data bus before extracting its relevant piece of information. The functional model of the data transfers required by the system leads to the solu- tion of buffered input. A status register in the TDC module is read only once. Each of the objects then accesses the data word from memory. In fact, the object model of Fig. 2 originally did not contain the hierarchy of objects

starting with TDC SWM. In the ˛rst model, all the objects directly accessed the hard- ware. Only in making the functional model did the current solution become apparent. This is an example of how re- examining the entire model helps to uncover dif˛culties and omissions before they appear in a program. III. IMPLEMENTATION A. Language choice Encapsulation is advantageous even in situations in which the code is written in a nonobject-oriented language. Be- cause time constraints forced us to move forward quickly with implementation, we forewent the learning of an object-oriented

language such as C 11 . In the control sys- tem for the muon readout, we used only standard C. Let us recall that both attributes and functions characterize an ob- ject. Each object consists o f a C structure to hold its at- tributes and a set of functions. To preserve data encapsula- tion, we only declare the structure of attributes in the functions that belong to that object. These functions then serve attributes to other objects, and other objects may ac- cess attributes only through these server functions. We packed the set of functions belonging to an object into a library. The collection

of all libraries formed a toolkit from which we built the programs that perform the initialization and run-time control of the muon readout system, as well as related test programs for hardware and software develop- ment. At the level of the FASTBUS hardware, the attributes of the FASTBUS HARDWARE REGISTERS are the data contained in the hardware memories themselves. The ALEPH event builder FASTBUS library provides a set of functions for accessing FASTBUS data and control space registers. These are the object functions of the object FASTBUS HARDWARE REGISTERS. For the objects of the so-called

software mirror TDC SWM and related ob- jects in Fig. 2 , the software mirror requests data from the FASTBUS HARDWARE REGISTERS. The C structures of the software mirror then contains copies of the contents of the hardware registers. The objects of the software mir- ror then serve this data to other objects upon request. B. Burst synchronization and interrupts As shown in the modeling of Fig. 4, while the run task is performing the duties of a given state, it must be alert to external signals. The run task may receive a stop signal from the CP. Via Ethernet the CP will send the stop signal to

the harness program running in the Aleph event builder AEB . The harness program will then send a kill signal to the run task. The run task may also receive burst timing signals such as a beginning-of-next-burst signal a period of ap- proximately 15 s , a private-out-of-burst signal, or a guard- period-begins signal from the hardware of the NA48 burst timing system. To receive burst signals, we utilized a hard- ware interrupt driver written for the AEB. The AEB has an emitter coupled logic ECL input on its front panel that provides an interrupt to its microprocessor. Additionally, the AEB has

a parallel port that accepts transistor± transistor logic inputs. A custom, external module receives the burst timing signals from the NA48 timing system. It then provides these signals to the AEB’s parallel port and their logical-OR to the interrupt line. When the interrupt driver receives an interrupt, it reads the parallel port, deter- mines which signal arrived, and delivers it to the run task. To implement the signal capturing modeled in Fig. 4, during initialization we prepared the run task to receive and handle signals. We used the OS9 interrupt facility to allow the run task to

intercept signals. We also de˛ned an inter- rupt handler for the run task. When the run task receives any external signal, it jumps to the handler. So that the handler is quick, its only job is to set a few global ˇags. The ˇags identify the signal to the run task and control the ˇow through the main part of it. After the handler is done setting ˇags, the run task resumes from where it left off. C. Error collecting and reporting The requirements for collecting and reporting errors are different in the initialization and run phases. If an error appears during the

initialization phase of the muon readout system, the initialization task reports it immediately and initialization ceases. If an error appears during the run phase in burst, for example , the run task need not report it right away. It is extremely unlikely that an operator could intervene during the burst duration of approximately 2.5 s in which an error is discovered. Therefore, the run task can save the error in a list for the given burst and report it during the out-of-burst period. Figure 4 illustrates how we have modeled this behavior for the case of the run task. Upon detection of an

error, the run task saves the given error and remains in its present state a loop that returns to the state from which it started At the end of the out-of-burst period see Fig. 4 , the run task will enter the state REPORT ERRORS TO CP. 512 COMPUTERS IN PHYSICS, VOL. 11, NO. 5, SEP/OCT 1997
Page 6
From there it will report errors discovered since the last time that it entered that state. The run task asks the ER- RORS object to send a message to the EMU object. Object EMU then injects a message to the central message system. During a burst, the run task will continuously monitor for

errors in the readout system. If a serious problem oc- curs, a given error may appear repeatedly. It could easily ˛ll the error list in a short time. The error system need not report a superˇuous number of errors because a shift opera- tor would ignore an excessive number of them in any case. At present, we have simply limited the overall number of errors allowed in the list. One bene˛t that encapsulation provides is that modi˛cation and upgrading can usually oc- cur seamlessly. One possible path of upgrade in the future is to ˛lter the number of each kind of error

reported. IV. CONCLUSION The technique of object-oriented modeling and design is certainly useful, but at ˛rst it is dif˛cult if one is used to a less disciplined style. To stop thinking in terms of ‘‘do- loops, subroutines, and arrays’’ takes time. Furthermore, suppressing the urge to stop the modeling and jump into the coding requires self-control. Despite the change in style, creating the object, dy- namic, and functional models forces one to examine a given problem from all angles and to understand how the entire software should ˛t together. The system grows by direction,

not by accretion. Building the objects and their functions yields a handy toolkit for constructing other pro- grams. In our case, we have emphasized encapsulation in our object model and in its implementation, for it allows a simple model that we can implement straightforwardly in a nonobject-oriented language. Encapsulation provides at least two bene˛ts. It allows for splitting the work into nearly orthogonal pieces with clear interfaces such that little interference results during development. Moreover, it does prevent mistakes or changes of implementation in one section of code from

spreading through other areas. A case in point is the implementation of the underly- ing buffer between FASTBUS HARDWARE REGISTERS and the characterizing functions of the objects. After most of the object functions existed, we decided to change the implementation of the buffer so that reading from it and writing to it could occur without interfering with each other. We did not incur undesirable side effects from this change. None of the object functions that interact with the buffer required a single modi˛cation. Independent of the ˛nal means of implementation, the technique of

object-oriented modeling and design is a use- ful and powerful approach to software development. We hope that it will ˛nd increasingly more widespread use for developing programs in physics. ACKNOWLEDGMENTS We thank Cambridge NA48, particularly David Munday, Andy Parker, Tom White, and Steve Wotton, for advice during the entirety of this project. We thank Ken Baker and especially Sven Katvars for many discussions on the func- tionality and design of the TDC hardware. K. Nelson grate- fully acknowledges support from the U. S. National Sci- ence Foundation. S. Takach thanks the United

Kingdom’s Particle Physics and Astronomy Research Council for sup- port. REFERENCES 1. N. Doble, L. Gatignon, and P. Grafstrom, ‘‘A Novel Application of Bent Crystal Channelling to the Produc- tion of Simultaneous Particle Beams,’’ CERN SL 95-16 EA 2. P. Grafstrom et al. , Nucl. Instrum. Methods A 344 , 487 1994 3. P. Debu, talk given at 27th ICHEP, 20±27 July 1994, Glasgow, Scotland, published in conference proceed- ings. 4. G. Barr et al. , Nucl. Instrum. Methods A 370 , 413 1996 5. A. Marchioro, ‘‘Event Builder User’s Guide,’’ ALEPH internal memo ALEPH 88-59, DATACQ 88-11, 6 June 1988. 6.

J. Rumbaugh et al. Object-Oriented Modelling and Design Prentice Hall, Engelwood Cliffs, NJ, 1991 7. F. Meijers, ‘‘EMUX: Error Message Utility for OS-9 and POSIX,’’ CERN internal note. 8. A. Castro and A. Miotto, ‘‘Aleph Event Builder FAST- BUS Library Ver. 2.0,’’ ALEPH 88-146, DATACQ 88-21, 14 November 1988. 9. R. Fantechi, ‘‘I/O Port Interrupt Driver User’s Guide, NA48 internal note, 2 April 1996. COMPUTERS IN PHYSICS, VOL. 11, NO. 5, SEP/OCT 1997 513