Progress of the Machine Control Upgrade at Cosy/Jülich
Abstract
The Cooler Synchrotron COSY [1] operated at the Research Center Jülich is undergoing staged machine control upgrades driven by the requirements of the JEDI (Jülich Electric Dipole moment Investigations) collaboration [2]. The upgrades aim towards better beam control e.g. beam orbit, tune, and chromaticity control improvements. A better orbit control was achieved through the upgrade of BPM electronics and migration from initial Tcl/Tk based control system to Control System Studio (CSS) utilizing EPICS [3]. Currently, a design for improved beam tune control is in development. The main part of work is the transition towards a faster and less restrictive magnet control. It further includes improved tune measurement tools as well as the migration of control for quadrupole magnets to EPICS. Ultimately the control of all systems should be centralized around EPICS to enable ease of operation, automation, setup of services, etc. The decision path, technical details of the upgrade and performance of the upgraded sub-systems are presented. We also showcase how the COSY team’s physics and research goals are complemented by Cosylab’s technical design and implementation to form a synergetic collaboration.
Introduction
The Cooler Synchrotron COSY-Jülich was taken into operation in 1993 [1]. COSY was operated 24/7 as a user facility until 2014 providing polarized and unpolarized particle beams to a community of scientists primarily concentrated around the universities in the federal state of North RhineWestphalia, Germany and international users. COSY is a multipurpose facility serving experiments that use beams circulating in the storage ring or extracted and transported to three external experimental stations. Special emphasis is made on beam cooling [4] [5], ultra-slow extraction [6], spin control [7] and operation of internal targets of different types [8]. Since 2014 COSY is a test facility for FAIR and precision experiments. Electron and stochastic beam cooling, beam instrumentation, cluster-jet target technologies are developed and tested at COSY. The JEDI collaboration uses the facility for research and developments towards a dedicated high-precision EDM storage ring and the first measurement as proof-of-principle at COSY.
The machine control system was developed in-house. During initial commissioning and first years of operation HP-UX running on HP-9000 workstations was used. The front-end hardware was mainly based on VME and G64 bus systems. A Real-time operating system (RT/OS) was developed at COSY for the VME systems. The Single Command Single Response (SCSR) data exchange protocol is a major component of RT/OS. Most of the GUIs were written in Tcl/Tk. To minimize the cost of technical support the control system was migrated to SuSE 8 running on Intel PC in 2002. Later on OpenSUSE and Ubuntu were used.
In the early phase of the control system design a decision was made to use Ethernet as the main transport mechanism. This fact had many implications, as the availability of the Ethernet-capable hardware was rather limited at that time. On the other hand, this is a significant advantage for the upgrade presented here.
Upgrade motivation
With the JEDI experiment, new requirements concerning the overall RMS beam orbit deviation were introduced [9]. Therefore an automated beam orbit control system had to be developed. Furthermore, other components were identified as being in need to be upgraded or to be added. One example is the analogue BPM readout electronics, whose signal offsets prevented an accurate position determination, especially around the 0 position. The decision was made, instead of implementing the new sub-systems and features into the old control system, to upgrade the latter in a step-wise manner, in order to avoid long down-times of the machine. In addition, the following considerations were taken into account:
- Add a logging and archiving mechanism
- Use software developed and supported by a larger community
- Software and compatible hardware available without protocol adaption
In order to satisfy the above-mentioned requirements EPICS was chosen as a new control system environment.
Orbit Correction Implementation
At COSY, the orbit can be measured using beam position monitors (BPM) and corrected using steerer magnets. The orbit correction controller has to synchronize with COSY operating cycles. A typical cycle is depicted in Fig. 1.
Figure 1: Schematic COSY operation cycle.
It starts with injection, followed by a synchronous ramping of the magnets initiated by a timing signal and ends with a ramp down phase. While ramping, the magnet power supplies are controlled by function generators that follow pre-defined waveforms. During this time, the magnet current cannot be set manually, so during normal operation the orbit cannot be adjusted in the middle of the machine cycle. The left part of the diagram in Fig. 2 depicts a newly developed algorithm that uses information broadcasted by the timing system to synchronize to the machine operation, switches the magnets from ramp mode to DC mode, enabling direct current manipulation in flat top and enabling orbit correction.
Figure 2: The cycle synchronization algorithm on the left and the orbit correction algorithm on the right.
The same algorithm has to ramp the magnets back to the starting values before the end of the cycle and re-arm the function generators so they can start ramping when the timing system signals the beginning of the next cycle.
The orbit correction itself follows a common singular value decomposition correction algorithm. When the magnets are in the DC mode, a procedure implemented in an EPICS Input/Output Controller (IOC) uses the SCSR protocol to collect the orbit measurement values, calculate new correction values based on the difference between the current measured orbit and the desired orbit, calculate new corrector strengths, and send them to the steerer magnets using the SCSR protocol. A custom GUI for CSS [10] was developed in Java to allow control of the orbit correction algorithm and allow users to view the results. It can be seen in Fig. 3.
Figure 3: Orbit Correction GUI
Migration of Steerer Magnet Controls to EPICS
While operators were using the orbit correction implemented as described above, it was discovered that the process is cumbersome for day to day usage. The operators had to use the Tcl/Tk based version of the control system to define the base ramp parameters, turn off this old system, modify (shorten) the ramp for orbit control purposes, load it to the targets (custom scripts running from the command line) and then start the new orbit correction software (based on EPICS). To mitigate the usability issue, the ramp definition parameters for the steerer magnets, ramp generation, loading onto the targets, and the GUIs, shown in Fig. 4, were migrated to EPICS and CSS. This now allows operators to focus on the status and control of the accelerator and not to spend time on the software itself. This system also serves as a test bed for migration of other Tcl/Tk based systems to EPICS.
Figure 4: Steerer magnet GUI.
Quadrupole Magnet Controls, Save and Restore Machine State
During the testing of the enhancements described above, a need to transfer the accelerator states saved during earlier runs to new system’s restore function was identified. This would allow for older experiment-specific configurations to be restored on the new control system. In addition to the migration of earlier accelerator states, we are looking into options for the migration of configuration for all future systems that will be migrated to EPICS. We are also preparing for a migration of the Quadrupole magnet control to EPICS. This will simplify tune control and the implementation of a Tune Scan algorithm. To achieve an automated Tune Scan, the control of the quadrupole magnets needs to be improved so that there is no limitation to pre-defined ramps. We will start by developing a simulator for the magnet power supply, migrating the quadrupole control from the Tcl/Tk-based control system to EPICS and verifying its operation with the simulator. In the second step, we will test the IOC with a power supply and controller that are in a designated test area and not connected to production magnets, observe and verify the operation on the real device without any impact on the production environment. Once we are satisfied with the performance on the test bench, we will migrate the production environment to the EPICS-based control system and evaluate the behaviour on the production system. While the system is in production and support is ongoing, we will start implementing other features such as migration of the controller-specific code to the magnet power supply controller, making on-the-fly changes to the predefined ramps with synchronization between the various controllers, defining additional recipes that will allow easier automation of the scanning for the ideal tune of the beam.
On Collaboration Between IKP and Cosylab
The COSY upgrade requires close collaboration between the Institute for Nuclear Physics (IKP) of the Research Center Jülich and Cosylab. The upgrade plan utilizes a staged approach. Inevitably, the newly introduced technologies and upgraded parts must comply with the existing control system technologies. The split and interface between the two has to be firstly well understood, all the risks assessed, the existing and new functionality designed, implemented, and thoroughly tested before putting into production. For all this to eventuate, Cosylab and IKP are working together on the requirements, design documents and testing and profiling the functionalities. It is important to describe existing machine features, find the possible means for introducing new functionalities, review, iterate and converge on the final design shape. For implementation and during the testing period, we found it invaluable to work together, live on the machine. Then, as issues arise, additional features are needed or the performance has to be improved. We achieve the latter by continuous and dynamic weekly work prioritization, planning and reporting.
Conclusion and Outlook
Facing new requirements, mostly from the JEDI experiment, made upgrades of the COSY control system and other sub-systems necessary. This was done by introducing an EPICS / CSS based system in a step-wise approach. This procedure made it necessary for the new parts of the control system to interact with the initial control system and the legacy hardware. First sub-system to be replaced was the BPM readout electronics, together with an orbit correction mechanism. This mechanism adapted the old protocol towards the correction magnets. This effort allowed so far to achieve RMS orbit deviation better than 1.2 mm. The next enhancement will be the EPICS integration of the quadrupole magnets in order to better control the tune of the machine. A subsequent next step might be to improve the control of the other magnets, more precisely the dipole and sextupole magnets, so all magnet controls are transferred to the new system. As some other systems are facing higher error rate because of the hardware reaching end-of-life, further upgrades are being prepared.
References
[1] R. Maier et al., “Status of COSY”, in Proc. EPAC’94, London, UK, Jun.-Jul. 1994, pp. 165–170.
[2] http://collaborations.fz-juelich.de/ikp/jedi/
[3] Experimental Physics and Industrial Control System, https://epics.anl.gov/
[4] A. J. Halama and V. Kamerdzhiev, “Model Development for the Automated Setup of the 2 MeV Electron Cooler Transport Channel”, in Proc. COOL’17, Bonn, Germany, Sep. 2017, pp. 28–31. doi:10.18429/JACoW-COOL2017-TUM22
[5] R. Stassen, B. Breitkreutz, and N. Shurkhno, “Recent Results of HESR Original Stochastic Cooling Tanks at COSY”, in Proc. IPAC’18, Vancouver, Canada, Apr.-May 2018, pp. 913– 915. doi:10.18429/JACoW-IPAC2018-TUPAF078 (preprint) J. Phys.: Conf. Ser., vol. 1067, pp. 052013. doi:10.1088/ 1742–6596/1067/5/052013
[6] B. Lorentz, “Stochastic Noise Extraction at COSY”, Slow Extraction Workshop 2017, CERN, Switzerland
[7] N. Hempelmann et al., “Phase Locking the Spin Precession in a Storage Ring”, Phys. Rev. Lett. vol. 119, pp. 014801, Jul. 2017. doi:10.1103/PhysRevLett.119.014801
[8] A. Khoukaz et al., “Technical Design Report for the PANDA Internal Targets”, PANDA Collaboration, Mar. 2012, GSI RE-TDR-2012–002.
[9] M. Rosenthal and A. Lehrach, “Spin Tracking Simulations Towards Electric Dipole Moment Measurements at COSY”, in Proc. IPAC’15, Richmond, VA, USA, May 2015, pp. 3764– 3766. doi:10.18429/JACoW-IPAC2015-THPF032
[10] http://controlsystemstudio.org/
10th Int. Particle Accelerator Conf., IPAC2019, Melbourne, Australia.
JACoW Publishing, ISBN: 978–3–95450–208–0
DOI: 10.18429/JACoW-IPAC2019-WEPGW007
Content from this work may be used under the terms of the CC BY 3.0 licence ( © 2019). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI.