Softwareandsecureconfigurationforthepoweringinterlocksystem.doc

lhc project document no. lhc-s-pic-0001 rev 0.3 cern div./group or supplier/contractor document no. ab/co edms document no. 999999 date: 2006-12-12 the large hadron collider project cern ch-1211 geneva 23 switzerland functional specifcation the commissioning of the hardware in the lhc sectors software and secure configuration for the powering interlock system abstract this document describes the software functionalities of the industrial controllers and the associated supervision system of the powering interlock system. it describes in detail the implemented protection mechanisms and the way to securely configure and change the associated operational parameters. several failure and maintenance scenarios are described and the procedures for a correct reconfiguration of the system are defined. after their initial deployment for the powering interlock system of sc magnets (pic), the same mechanisms will be put in place for the protection of nc magnets (wic). prepared by : markus zerlauth robert harrison julien mariethoz juan blanco sancho alejandro castaneda ivan romera frederic bernard checked by : bruno puccio verena kain jorg wenninger hcc approved by : rudiger schmidt hermann schmickler lhc project document no. lhc-s-pic-0001 rev 0.3 page 2 of 18 history of changes rev. no.datepagesdescription of changes 0.1 0.2 0.3 2006-07-18 2006-11-03 2006-12-04 13 13 15 first draft including comments of rudiger and a discussion with verena on the interaction with mcs (management of critical settings) including comments of rudiger and including new pvss implementation lhc project document no. lhc-s-pic-0001 rev 0.3 page 3 of 18 table of contents 1.introduction 5 2.scope.5 3.plc software .6 3.1flow charts for circuit types.6 3.2global protection mechanisms6 3.3history buffer / unicos6 3.4system information6 3.5function blocks .6 3.6configuration data6 3.7versioning and cvs 6 4.supervision application 6 4.1gui.6 4.2background functionality (cryo_ok, qps_ok) 6 4.3history buffer / archive / logging6 4.4configuration data6 4.5versioning and cvs 6 5.secure configuration .7 5.1generation of configuration data7 5.2crc for the plc configuration file .8 5.3crc for the cpld mask 9 5.4crc for the supervision application .9 5.5use of configuration data during machine operation and maintenance9 5.6the cvs repository .10 5.7integration with management of critical settings (mcs)11 6.failure cases and maintenance procedures .12 6.1changes of powering layout12 6.1.1adding equipment or electrical circuits.12 6.1.2removing equipment or electrical circuits defined as auxiliary circuits12 6.1.3removing equipment or electrical circuits defined as essential circuits12 6.2hardware faults, replacement of components .13 6.2.1replacement of printed circuit boards13 6.2.2replacement of power supplies for the plc and the pic.13 6.2.3replacement of the plc, ethernet module or memory card13 6.3changes of operational parameters14 6.3.1changing the flag for the powering subsector off 14 6.3.2changing the beam dump flag 14 6.4re-installation or upgrade of generic codes 15 7.references .15 lhc project document no. lhc-s-pic-0001 rev 0.3 page 4 of 18 table of figures figure 1: generation and deployment of configuration data for the powering interlock system 8 figure 2: cvs directory structure for the powering interlock project .10 figure 3: content of cvs repository for interlock system cip.ua83.al8 11 lhc project document no. lhc-s-pic-0001 rev 0.3 page 5 of 18 1.introduction as part of the lhc machine protection system, the powering interlock system (pic) assures the protection of superconducting magnets in the lhc by interfacing with the quench protection system (qps) and the power converters. it also interfaces to the beam interlock system by providing user_permit = true if the whole powering system is operational 1. in total, 36 industrial controllers (siemens 319 series) are installed in the lhc, protecting around 860 electrical circuits in 28 independent powering subsectors 2 3. normal conducting magnets in the lhc are used in all eight insertion regions of the lhc and are protected by eight dedicated interlock controllers (wic), using a safety plc (siemens 300 f). although the basic protection functionality is guaranteed via hardwired links in between the 3 main systems 4, each industrial controller has to be configured according to the protected hardware in the powering subsector. in a similar way, the pvss supervision application has to be mapped to this instance of the interlock controller. while the basic software packages are kept generic, a block of configuration data is used to configure the system for its specific location 5. secure ways of reconfiguration and a change of operational parameters have been implemented, based on configuration data from the controls configuration database. 2.scope this document describes in detail the software functionalities of the powering interlock system along with the mechanisms that have been put in place to safely reconfigure any part of the interlock system relying on software or configuration data. this will be of vital importance after the completion of the dedicated interlock commissioning phases (hca:pic1 and hca:pic2 6), as it is inevitable that failing hardware needs to be exchanged during the lifecycle of the machine. in addition, a certain number of operational parameters which are part of the configuration data are likely to be optimised during machine operation and certain flexibility has to be implemented without decreasing the safety of the system or requiring a lengthy re-commissioning of the system. for quality assurance reasons however it is envisaged to maintain the hardware commissioning procedures throughout lhc operation to re-commission the interlocks after e.g. tunnel interventions due to maintenance or changes of the powering system. the same mechanisms are to be put in place in a second stage for the warm magnet interlock system (wic) of the lhc and the sps-lhc/cngs transfer lines. lhc project document no. lhc-s-pic-0001 rev 0.3 page 6 of 18 3.plc software introduction (circuit types, hw choice, scope, ) 3.1flow charts for circuit types 3.2global protection mechanisms 3.3history buffer / unicos 3.4system information plc restarts, pic id, etc 3.5function blocks short description of each fb (maybe in appendix) 3.6configuration data 3.7versioning and cvs 4.supervision application introduction (framework, scope, implementation ) 4.1gui 4.2background functionality (cryo_ok, qps_ok) 4.3history buffer / archive / logging plc restarts, pic id, etc 4.4configuration data 4.5versioning and cvs lhc project document no. lhc-s-pic-0001 rev 0.3 page 7 of 18 5.secure configuration ensuring consistency and correctness of configuration data for machine protection systems is a major challenge and has to start at the very source of the configuration process. a unique source of data and clear interdependencies in between the various parts of the database model are a pre-requisite for a reliable configuration process. to ensure the correctness of layout data in the lhc functional layout databases, a rigorous versioning scheme is applied, including a series of verification mechanisms and an approval process before the migration to a new data version is approved. 5.1generation of configuration data based on the layout and configuration data of the powering interlock system, a dedicated generation script will be used to produce the set of configuration files for each of the 36 instances of the powering interlock system as shown in figure 1. it is important to note that the generation scripts are part of the controls configuration database and that the same mechanisms for the tracking of changes are applied as for the data itself. a set of configuration files consists hereby out of three main files: one for the configuration of the scada system (in our case pvss) one for the plc process one for the configuration of a programmable logic device even if format or content change from one file type to another, for consistency the same data source is used depending only on the previously defined layout. in addition to these three basic configuration files, four other files are used for the powering interlock system: to configure laser (alarm system) to configure the transfer of data from the pvss archive to the lhc logging database to configure cmw (controls middle ware server used during hardware commissioning) to debug versions of the plc configuration file for the first phases of commissioning in order to assure coherency of the individual files at any time, the set of the three main configuration files will always be created from the database at the same time and signed with cyclical redundancy check (crc) checksums of 32 bits for the pvss file and the plc file and 8 bits for the file to mask the unused inputs of the programmable logic device. the individual checksums are written to each individual configuration file. the configuration file for the scada system will contain all checksums of a system, as it will continuously verify and assure the overall coherency during operation. lhc project document no. lhc-s-pic-0001 rev 0.3 page 8 of 18 figure 1: generation and deployment of configuration data for the powering interlock system 5.2crc for the plc configuration file a specific function block fc10 of the plc program will contain the required configuration data. this function block is generated from the layout db and compiled into the program before its download into the corresponding plc. the data contained in fc10 is split into two parts: operational parameters (powering subsector off, essential /auxiliary circuit for beam operation) hardware related configuration data (holding the input/output addresses, circuit types, etc). each of these parts will be signed with a 32 bit crc, which will be published by the plc in the data blocks db92.dbd96 (for the operational parameters) and db92.dbd100 (for the hardware related parameters). the split of the configuration data into a (more or less static) hardware part and operational parameters will allow the operators to change the less safety critical operational parameters whilst being sure that the hardware addresses remain the same. the supervision application can then display a warning that operational parameters have changed but there will not be the need for a re-commissioning of the system. changes to these operational parameters need to be done in the database (according tools will be provided to the operators) and the newly created configuration file needs to be stored in cvs and downloaded into the according controller. in addition to the checksums described above, the plc will communicate lhc project document no. lhc-s-pic-0001 rev 0.3 page 9 of 18 its current software version (db92.dbd84) the pic id (db92.dbd82) and the version of the configuration file (db92.dbd92) to the pvss supervision application. 5.3crc for the cpld mask the cpld integrated into the boards cipaa and cipab is a fast logic controller used for the generation of beam dump requests in case of failure. the according vhdl code represents an and logic of all available 96 inputs and is connected in parallel with the plc output to the cibu interface. to configure the cpld for its use in a certain powering subsector, the unused inputs have to be masked in order not to influence the output. the according mask is generated in a hex format from the configuration database (e.g. ff80c3ffffffffffb7ff1680 for al4) and has to be compiled into the vhdl code along with an 8 bit crc before being loaded into the cpld. after the download, the crc can be displayed via 8 monitoring outputs of the cpld by setting a defined address at the level of pvss (bit pattern 00010 at the address bits a4a0). in a similar way the version of the vhdl code can be retrieved (bit pattern 00000 at the address bits a4.a0 for the high word of the sw version and bit pattern 00001 for the low word of the sw version). 5.4crc for the supervision application as the highest instance in the configuration process, the pvss application will verify the consistency of the configuration data in the underlying plc and cpld. the pvss configuration file for an instance of the interlock controller will therefore contain the crcs of the corresponding plc configuration and the matrix mask along with the sw versions which are to be used together with the current pvss version in the plc and the cpld. in case the sw versions change for e.g. in the plc to a higher version, special mechanisms might also specify the lowest version allowed to run with the current pvss version. as the overall coherency will depend on the correctness of the pvss configuration file, special attention has to be given to avoid loading a wrong configuration file to an interlock device. when importing a configuration file, the device name contained in the configuration file should be cross-checked against the device being currently configured. additional safety mechanisms such as the involvement of the mac address of the underlying plc might be included (from january 2007 onwards, it will require a valid mac address specified in the netops database for front-end within the technical network). 5.5use of configuration data during machine operation and maintenance once the powering interlock controllers have been commissioned for a certain configuration, the according set of configuration data is stored in a dedicated repository, allowing for a chronological history of the file versions. once the pvss, plc and matrix are configured, the scada system will verify the coherency of all active configuration files and the according software version at every startup of an electrical circuit. it is not possible to change configuration files without a previous stop of all electrical circuits in the powering subsector (with the exception of the pvss configuration file which is not considered part of the safety functionality of the system). lhc project document no. lhc-s-pic-0001 rev 0.3 page 10 of 18 to allow for a continuous verification, the various components of the interlock system publish the versions and checksums of their current configuration and the scada system will compare this data against its own configuration file, containing the expected checksums of the related components. as by design the scada system is not involved in the safety critical functions of the system but only required for monitoring and start-up procedures of magnet powering, a verification of the configuration data is performed every time before circuit powering is permitted for any of the electrical circuits. due to the larger number of manipulations to be performed by the scada system, the crc and version of the cpld is verified in time intervals not exceeding one hour (configurable in expert mode of the scada system). this is acceptable as a reconfiguration of this logic device cannot be done remotely but requires a tunnel intervention. if any part of the configurations containing hardware addresses has changed when starting a new machine cycle, the scada system will inhibit powering of any of the circuits of this device. in case operational parameters have been changed, the scada system might display a warning that the operational parameters have changed with respect to the latest commissioning but will take no further action. 5.6the cvs repository configuration data downloaded from the database will be stored centrally in a cvs repository. the cvs repository will contain folders for the various versions of the plc, pvss and vhdl source codes and folders for each instance of the powering interlock controller, holding the currently commissioned (and archived) versions of the downloadable software as shown in figure 2 and figure 3. the changes applied in the new files with respect to previous versions will be indicated in the description of the latest version, along with the timestamp and person performing the modification. like this, a unique location is available for reloading of configuration data. figure 2: cvs directory structure for the powering interlock project lhc project document no. lhc-s-pic-0001 rev 0.3 page 11 of 18 figure 3: content of cvs repository for interlock system cip.ua83.al8 5.7integration with management of critical settings (mcs) the system for the management of critical settings (mcs) is using very similar procedures for the transfer and authorisation/authentication of configuration data towards the front ends (mainly vme). nevertheless no mechanisms are currently available for the configuration of industrial controllers such as plcs (requiring dedicated sw tools for their configuration), which is why for the time being the two systems are going to co-exist beside each other. it is envisaged to include the management of versions and operational parameters to the critical settings db at a later stage, once the unstable phase of hardware commissioning is terminated and changes of operational parameters become less frequent. a communication via cmw with the scada system of the interlock systems will allow verifying the current configuration in the front ends against the approved ones in the database and create a sw interlock in case of mismatch. in addition, the inclusion of the operational parameters in this db will allow for easy read access for the operators. lhc project document no. lhc-s-pic-0001 rev 0.3 page 12 of 18 6.failure cases and maintenance procedures during the commissioning and later operation of the machine protection systems one will encounter situations where a configuration file has to be reloaded (e.g. after the replacement of a faulty plc unit) or operational parameters are to be changed to optimize beam operation. the following use cases give a first list of expected interventions and the way to proceed in order to safely restore the initial configuration. the procedur