SPS RF Control System demonstration

on: 17 November 1999.


E. Bracke


*** --- ***






























People present:


      Pierre Charrue

      Michel Jonker

      Marc vanden Eynden


      Erik Bracke

      Andy Butterworth

      Edmond Ciapala

      John Molendijk

      Frode Weierud






      SPS RF Control System presentation of 17 November 1999.



            1     Short overview of the system.

                  1.1   Presentation layer on HP workstations.

                  1.2   Process layer in LynxOS PCA frontend.

                  1.3   Equipment layer in G-64 controllers.

                  1.4   Central processes in the RF Control System.

                  1.5   Conclusions.


            2     On the SPS2001 upgrade.

                  2.1   Our vision.

                  2.2   Questions.

                  2.3   Services currently available

                        from the Controls group.


            3     Demo of TWC_200_Power and RF_Synchro.



1        Short overview of the system.


1.1       Presentation layer on HP workstations.


                A command is formulated with the GUI of the SPS RF Control System. Commands to the system can take considerable time to execute. This is dependent (required) on the RF hardware and cannot be influenced by the control system or the operator.

                The presentation layer GUI is configured by means of Oracle database generated config files.


                Selection of a command goes through:


-        Selecting an RF System. This determines to which of the frontend servers the formulated command will be issued.

-         Selecting a control function; possibly select or enter some data.

-         Send the command to the hardware.


                The protocol, that connects the presentation layer with the underlaying PCA layer is implemented in standard CERN RPC. It has the same functionality in our control system as the now proposed 'middleware' has for the SPS2001 software unification project.


                In the SPS RF Control System, two types of results can be generated by issuing a command:


-        A 'report'. This is a short string, that says something of the executional status of a step in the actual control algorithm that was activated by the operator. Each of the steps, that build together the total control algorithm implementing the command, issues obligatory such a report.            
The GUI shows these reports and keeps a history of all executional status of all steps.
It is possible for an operator to determine how much (how deeply nested) reports he will receive.

-        A 'reply'. This is the data that was requested by an operator; the reason for him issuing the command. A command doesnot necessarily generate a reply; not always a command generates data.


                An operator can, through his instance of the GUI, subscribe (keep an eye on) to the reports issued by other RF Systems than the one he actually has selected for his control.


                An operator can re-set an RF System with settings that have previously been saved into an archive (see: 'Central processes'). It should be noted that the operator doesnot, under this circumstance of re-setting, require to have the same operational 'capability'. This means that writing certain settings to the RF hardware, that normally can only be done by RF specialists, can still be done through re-storing by e.g. the SPS operations group (which has generally a lower 'capability').



1.2       Process layer in LynxOS PCA frontend.


                The frontend servers are considered to be the 'entry point' for a remote operator to the RF hardware; the machines on which they run are physically distributed in BA3, BA4 and BA2.

                All processes (see later) that together make this process layer are configured by Oracle database generated files at boot time only. All necessary data structures are created in memory at that moment (runtime ressource preservation).


                The proces layer's (PCA) function is to translate a command into a corresponding control algorithm that has been designed to be as generic as possible. It will be the equipment (ECA) layer (see later) that takes care of 'ironing out' the inevitable little differences between hardware of different makes which, as seen from the operator, perform however a same function in the accelerator. E.g. SPS RF has transmitters for the Travelling Wave System of brands: Siemens, Philips and Valvo.


                Commands from the presentation layer 'clients' are received by the 'kernel' process, which is therefore the PCA resident 'server' process of the SPS RF Control System.


                Kernel has several functions:


-        Checking the validity of the command. Kernel knows all commands of the current PCA. It will refuse badly formulated (in the RPC protocol) or unknown commands from the presentation layer.

-        Checking the access rights of the operator. Here we must not only think in terms of 'capability' or 'privileges' but also in terms of 'reservation'. Indeed certain RF hardware require only one operator in charge at a time (high voltage; security!).

-        Queing the succesive commands in front of one or several 'Executer' processes that do the actual sequencing of the command steps.

-        Informing the 'Report and Reply' process for upcoming 'reports' and possibly 'replies'.


                Communication between Kernel and its child processes Executer(s), Report and Reply is implemented by means of shared memory buffers, managed by the 'Shared Memory Buffermanager'. The latter is also a child process, launched by Kernel on boot time. There exists an RPC connection between Kernel and R and R as well.


                Actual command sequencing is done by one or several Executers. A 'switch on' of all RF Travelling Wave lines e.g. can therefore proceed in parallel, each Executer taking care of one of the 6 RF lines for which he carries the responsibility.


                The following functionality and features are provided by the Executer process:


-        The Executer operates on command sequences. These are tables that on every step (i.e. table entry) reference a sub-command. Complex commands are generally built from simpler commands (nesting), a simpler command could here be another command sequence or a C-code routine.

-        Commands can be 'aborted'. This is important for long command sequences (several minutes) that are considered obsolete at a certain point in time.   
The Executer inspects an 'abort flag' in its input buffer after execution of each step in a command sequence. Whenever this flag is set by kernel, it skips to a special sequence step in the current command sequence. An 'abort' sequence is here referenced. If this flag is not set, this special sequence step is normally always skipped. The abort sequence is defined as: 'create a safe state for this command sequence'. Control is also automatically transferred to the abort sequence in case of an error occurring while executing the command.

-        Commands can be 'stopped'. This situation occurs when a required state (e.g. the reason for the operator of launching the command) has already been met. This is important if such a command is a precondition for a more complex command (runtime ressource preservation).

-        The Executer is capable of conditionally skipping a certain number of steps in a sequence. Conditions could be dependent of e.g. a status acquisition (note: 'stopping' a sequence is a special case of this!).

-        The Executer can conditionally 'loop' within a command sequence. A status acquisition within the loop, together with a time-out can provide the information for 'breaking' the loop in case of an anormality.

-        The Executers create of each command sequence a (very) verbose log file on the PCA's local harddisk. Each sequence step generates an entry in the logfile, identifying the step and the C-routine that executes with all of its comments. This facility proved to be very useful especially during debugging of newly installed RF Systems in our Control System.


                One or several sequence steps will require communication to the equipment layer software in the ECA. The protocol between Executer and ECA is currently the standard 'equip' API as used for our MIL1553b fieldbus. SPS RF uses the local EDU service / equip driver, we do not use the network and SLequip + its on-line database for our fieldbus access (network and run-time ressource preservation).

                Another protocol exists on top of this between the Executer fieldbus access sequence step and the ECA resident software. This protocol uses ASCII, human readable informative strings.

                In 99% of the cases, fieldbus access is standardized at the SPS RF on the 'ReadBytes' mode with the data or subcommand to be written to the ECA in the 'UserOption'. This guarantees that always an ECA resident application software 'report' will be received in the Executer buffer (acknowledge from the actual hardware). Besides, 'UserOption' allows any data of any size or format to be written to the hardware.


                Each control sequencestep generates at least a report: the executional status of the step (succeeded/not succeeced). This report (coming from the equipment layer in the case of a fieldbus access) is just interpreted and put into a shared memory buffer by the Executer. According to the returned executional state of the sequence step the Executer decides whether to continue the current sequence, to stop or to abort. The buffer, containing the report, is sent off to the Report and Reply process.


                Report and Reply, duely informed by Kernel of the upcoming buffers from the Executer(s), will dispatch the buffer contents by RPC via the two return paths (either the report or the reply channel) back to the command issuing and/or report subscribed GUI's on the presentation layer.



1.3       Equipment layer in G-64 controllers.


                The ECA layer translates the functional control algorithm of the command issued by the process layer to specific hardware dependent instructions. The ECA fullfills the following five requirements:


-        Real-time context switching.      
Allowing for SPS timing driven reconfiguration of the RF hardware (e.g. for another sub-cycle, particle).

-        Automatic set-up at start-up time.            
Initializing the ECA and the connected hardware into a known and safe state in which commands from the control system can be accepted.

-        Automatic surveillance.             
Interfaces to the SPS surveillance system. Triggers alarms for the alarmscreen in MCR.

-        Communication interface.          
It is here where the functional commands from the process layer of the SPS RF Control System are translated into the actual hardware dependent equivalents. Sometimes even further sub-instruction sequencing or decision taking is required by the particular hardware connected to this level of control.

-        System log. Accumulating a history of past events for analysis after problems.


                The G64 ECA's use an operating system of their own, based on the AMX RT-kernel.



1.4       Central processes in the RF Control System.


                Apart from the processes already mentioned, there exist 2 other, central processes that are integrated part of the SPS RF Control system. These processes run in the SPS RF central server machine rfsrv1.


-        The process 'actif'.      
Whenever an operator command sucessfully writes data settings into the RF hardware, a copy of these are kept in memory by the report and reply process at the frontend server (PCA). The R and R process is polled by the central 'actif' process that then fetches this data by RPC and stores it in shared memory. It is then available to the presentation layer GUI (RPC request) which will use it as the 'default value' for the data entry window; giving an operater a reasonable startvalue for re-issueing that same command.

-        The process 'archive'. 
An operator might decide that the current set of 'settings' of an RF System constitutes a stable situation. He then can make a snapshot of these and save it as a named archive (a file) for later reconstitution of the same state of that RF System. For such an archive will be used the current settings copy held by the 'actif' process in shared memory.  
Several of these archives can be managed by the 'archive' process. They can be restored via the presentation layer GUI.



1.5       Conclusions.


                In short, the SPS RF Control System has the following features and provides the following services:


-        A three layered control system: Presentation, Process and Equipment.            

-        Process and Equipment layers are distributed around the SPS auxiliary buildings, close to the actual RF hardware.               
All layers get their configuration data from a coherent database (Oracle). This data consists of a set of files especially taylored to needs of each layer (sql extraction scripts) and are read at boottime only. (runtime ressource preservation.)

-        Communication between presentation and process layer goes via standard CERN RPC. The process layer is considered to be the 'entry point' to the RF hardware. Communication between process and equipment layer goes via local calls to the standard CERN equip driver and MIL1553b fieldbus.           
The RF hardware is directly cabled to the RF ECA's.

-        The RF Control system provides for facilities for user capability check and RF hardware reservation. These checks are done on both presentation and process layers. On the process layer the command validity is checked as well.

-        A possibility of operator subscription to reports from another RF System than the one which the operator is currently controlling.

-        A system for archiving operational settings allows for quick restore of the operational situation in case of e.g. a power cut.

-        The SPS RF Control System provides a facility for long running commands (tens of minutes) to be operator aborted if considered obsolete while being in execution. The abort will leave the equipment in a safe state, however without reverting to the state of before command execution start.

-        For proper construction of commands the Executer provides facilities of 'looping within a sequence', 'skipping sequence steps', 'sleep with time-out', 'stop on required state met', 'abort on problem'.

-        At the Executer level logfiles are created for each command in which each C-routine on a sequence step logs its findings.



2        On the SPS2001 upgrade.


2.1       Our vision.


                We think the 'compromize' as shown in Michel Jonker's slide show (number 17) is closest to our implementation of RF control.


                This means that the 'device server' in the first approach will communicate through standard CERN RPC with our PCA kernel process such that the original presentation layer GUI can still connect the usual way as well (important for RF specialists). It also means that careful study has to be made as to what is defined as a 'device' (see also 'Questions').


                As the presentation layer GUI is configured by files coming from the Oracle database, the device server as well should be capable of using a set of configuration files coming from the same database.


                The SPS RF Control System currently implemented 'services' of 'reservation' (take/release) of RF hardware, access to the latter as a function of operator 'capability' and 'subscription' to reports as well as 'archiving' of successfully written operational settings could be of general use. In the case of these implemented in 'contracts' by the SPS2001 concept, it would certainly ease future migration of our control system towards this new standard (file: middlew1.doc).



2.2       Questions.


-        How do we build complex 'devices' from simpler sub-'devices'? Especially if those sub-devices can under certain circumstances be devices in their own right.

-        What facilities / services are offered in the infrastructure of SPS2001 to arrive to this end?



2.3       Services currently availablefrom the Controls group.


                For correct operation of our scheme, with minimum modification to, and the same functionality of the SPS RF Control System, SPS RF expects the following CERN standard software packages to remain in existence for considerable time and, at least, maintained at their current functionality:


-        The standard CERN RPC.

-        The standard equip library for local access to the EDU / MIL1553b fieldbus.