Submitted:
25 August 2023
Posted:
29 August 2023
You are already at the latest version
Abstract
Keywords:
1. Introduction
- Controls-kt [1] core and modules, that allows creating device servers (software modules that provide communication with a physical device) with minimal effort.
2. SCADA systems by scale and application
- Centralized.
- Distributed.
3. SCADA architecture
- Communicate with devices directly (using the protocol provided by the device).
- Interact with devices through an intermediate hardware link in the face of a programmable logic controller (or PLC).
- Standardization and unification of communication with I/O modules. This feature is important because device manufacturers use different specifications and communication protocols. Being able to simply plug these devices into a PLC without having to create additional device drivers is important for the industry.
- The PLC can be set up with simple process control instructions that can operate autonomously even in the event of an accident in the control system ([5]).
4. Device server design
- Device contains some properties (called attributes in TANGO).
- Properties have fixed types: numbers, strings, booleans, and lists of all of the above. Usually, there is also an option to define custom types as combinations of basic types.
- Properties could be writeable or read-only.
- Some systems ([6]) also allow to subscribe on properties changes. In this case, the device reports updates at fixed intervals or following device logic.
- In some systems it is also possible to define actions (or commands), which have an argument and a result (both use types that are available for properties).
4.1. Device property design
- Remote state. The value that is sent via network and which is observed in monitor software and database.
- Logical state. The actual value is passed to a property setter or read from the property getter.
- Physical state. The state of a physical device that corresponds to a given property value.
- the name of the property,
- the time of the change,
- the value of the property after the change.
4.2. Conversion between asynchronous and synchronous modes
4.3. Property value types and communication
5. Controls-kt framework
5.1. Device API overview

5.2. Device server implementation


- Three mutable properties: timeScale, sinScale and cosScale. Mutable properties could be read and written. Meta converter ensures that one could use a type-safe way to access them.
- Two mutable properties: sin and cos that could not be written, but could only be read.
- An action resetScale that has no arguments or outputs and just resets the scale.

5.3. Implementation of device communication protocols in Controls-kt
- Direct serial interface. The Controls-serial module provides synchronous and asynchronous phrase-based communication with a device.
- Network interfaces. Controls-kt supports synchronous and asynchronous communication with raw TCP and UDP protocols. There are two implementations of such communication: one uses JVM internal implementations and is available only on JVM. Another one uses the Ktor-network module ([26]) and will be available also for native compilation in the future.
- Modbus. Controls-Modbus module supports Modbus-RTU and Modbus-TCP protocols via j2mod library ([27]). Modbus is the most used protocol for low-level communication. The Controls-kt device could be used both as a Modbus-master (client) and Modbus-slave (server). The interaction is based on a type-safe registry mapping.
- OPC-UA. Controls-kt supports OPC-UA client and server modes with Eclipse Milo ([28]). In a server mode Controls-kt devices server connects to a device and translated OPC-UA properties to DataForge Meta objects. In server mode, it exposes the Controls-kt device server as an OPC-UA device.
5.4. A real-life example of stand-alone device server
- Antenna motors are controlled directly with a PCB based on an STM32 processor ([29]). The embedded software is written in C++ and performs basic movement and gyro-stabilization tasks.
- STM32 board is connected via 2-wire RS485 interface to Raspberry Pi 4 microcomputer ([30]). The communication is done with Modbus-RTU protocol.
-
The Controls-kt software runs on Raspberry Pi JVM and consists of several modules:
- –
- A Modbus registry that allows to access Modbus properties by name instead of register number and allows storing complex objects in Modbus registers.
- –
- A device server that wraps the registry (the tools for creating this server are already included in Controls-kt SDK).
- –
- A tuner controller that allows measuring the level of the signal for automatic antenna position adjustment.
- –
- Visualization server based on VisionForge framework ([31]).
6. Distributed system implementation with Magix


- Network discovery, which is one of the most complicated parts of traditional SCADA systems is not required. The endpoint needs only the event loop connection address to run.
- A subscriber does not have to know all properties and devices it listens to. For example, the database can record all events from all sources without the discovery of all of the devices.
- Asynchronous communication allows to avoid deadlocks and other synchronization issues ([35]).
- Lack of delivery guarantees. Since messages are sent in asynchronous mode, the sender does not receive a notification if the message is not delivered. The communication protocol usually guarantees that the message is delivered to the event loop, but there are no simple ways to determine if the message is received by the specific subscriber.
- Multiple replications of messages. The message is delivered to all subscribers, who have filters that accept the message.

6.1. Magix loop connections implementations
- RSocket ([38]). There are both endpoint and loop-side connections for RSocket. The protocol is recommended for small and medium systems.
- HTTP/2 connection (using SSE for backward communication). This mode is implemented in [3], but is not recommended for larger systems.
- RabbitMQ (AMQP protocol). Controls-kt supports only endpoints-side integration. The loop is the RabbitMQ itself.
- MQTT. Controls-kt supports only endpoints-side integration.
6.2. Inter-system connections with Magix
7. Visualization tools
- No-code or low-code tools to design connections and system interactions. This kind of system is frequently incorrectly called HMI (human-machine interface).
- Dashboards for monitoring and controlling specific devices and their measurements.
- Waltz-controls framework ([40]) was specifically designed to work with Magix. It allows to design of client-side dashboards using a variety of widgets.
- Plotly-kt framework ([41]) that allows both client-side and server-side dashboard management.
- Any other framework used for dashboards like Plotly-dash, Grafana, etc.
8. Performance tests for Magix and Controls-kt
- A virtual device specification with a single floating point value property, which gives a random value on each read. A random value is used because the algorithm does not send an event if the new value is the same as the one before read. The device is configured to re-read the value each 10 ms, which is considered a very good time sensitivity for DCS (the usual requirements start from 50 ms).
- A bootstrap procedure to load 100 such devices simultaneously.
- A Magix event loop, supports RSocket-TCP, RSocket-WS, and ZMQ protocols. The connection with devices is done via RSocket-TCP protocol.
- A visualization service based on Plotly-kt, which is connected to the Magix loop with RSocket-WS. The Plotly-kt uses WebSocket inside to communicate between the backend and frontend. The visualization service receives all events from all devices, computes the difference between the event origin time and the time it has been received (latency), then shows the maximum difference accumulated in 200 ms time for each device.
9. Conclusion
References
- Nozik, A.; Samoilov, V.; Klimai, P.; SPC-code. SciProgCentre/controls.kt: 0.2.0, 2023. [CrossRef]
- Khokhriakov, I.; Nozik, A. Magix-RFC. https://github.com/waltz-controls/rfc, 2023. [Online; accessed 20-August-2023].
- Khokhriakov, I.; Merkulova, O.; Nozik, A.; Fromme, P.; Mazalova, V. A novel solution for controlling hardware components of accelerators and beamlines. Journal of Synchrotron Radiation 2022, 29, 644–653. [CrossRef]
- Koval, N.; Alistarh, D.; Elizarov, R. Fast and Scalable Channels in Kotlin Coroutines, 2022, [arXiv:cs.DS/2211.04986].
- Wikipedia contributors. IEC 61131-3 — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=IEC_61131-3&oldid=1161871613, 2023. [Online; accessed 23-August-2023].
- Götz, A.; et al. TANGO Heads for Industry. In Proceedings of the 16th International Conference on Accelerator and Large Experimental Physics Control Systems, 2018, p. THCPL05. [CrossRef]
- Burckhardt, S. Principles of Eventual Consistency. Foundations and Trends® in Programming Languages 2014, 1, 1–150. [CrossRef]
- Lazidis, A.; Tsakos, K.; Petrakis, E.G. Publish–Subscribe approaches for the IoT and the cloud: Functional and performance evaluation of open-source systems. Internet of Things 2022, 19, 100538. https://doi.org/https://doi.org/10.1016/j.iot.2022.100538.
- collaboration, T. TANGO PubSub. https://tango-controls.readthedocs.io/projects/rfc/en/latest/12/PubSub.html, 2023. [Online; accessed 20-August-2023].
- Lange, R.; Johnson, A.N. ADDING FLEXIBLE SUBSCRIPTION OPTIONS TO EPICS*. 2011.
- Breslav, A.; Elizarov, R.; Reshetnikov, V.; Erokhin, S.; Ryzhenkov, I.; Zharkov, D. Kotlin coroutines design proposal. https://github.com/Kotlin/KEEP/blob/master/proposals/coroutines.md, 2023. [Online; accessed 20-August-2023].
- Chaize, J.M.; Gotz, A.; Klotz, W.D.; Meyer, J.; Perez, M.; Taurel, E. TANGO - an object oriented control system based on CORBA. Conf. Proc. C 1999, 991004, 475–479.
- Goloborodko, S.; Grygiel, G.; Hensler, O.; Kocharyan, V.; Rehlich, K.; Shevtsov, P.V. DOOCS : an Object Oriented Control System as the Integrating Part for the TTF Linac.
- Sam, S. A performance study of protocols used in a print on demand server, 2015.
- Viotti, J.C.; Kinderkhedia, M. A Benchmark of JSON-compatible Binary Serialization Specifications, 2022, [arXiv:cs.SE/2201.03051].
- Nozik, A.; Pantuev, V. Direct Search for keV-Sterile Neutrino in Nuclear Decay. Troitsk Nu-Mass (Scientific Summary). JETP Letters 2019, 110, 91 – 96. [CrossRef]
- Abeln, A.; et al. Conceptual design of BabyIAXO, the intermediate stage towards the International Axion Observatory. JHEP 2021, 05, 137, [arXiv:physics.ins-det/2010.12076]. [CrossRef]
- Nozik, A. Kotlin language for science and Kmath library. AIP Conference Proceedings 2019, 2163, 040004, [https://pubs.aip.org/aip/acp/article-pdf/doi/10.1063/1.5130103/14196335/040004_1_online.pdf]. [CrossRef]
- Nozik, A.; Postovalov, I.; Zelenyi, M. mipt-npm/dataforge-core: 0.5.2, 2021. [CrossRef]
- Nozik, Alexander. DataForge: Modular platform for data storage and analysis. EPJ Web Conf. 2018, 177, 05003. [CrossRef]
- Stein, B.; Clapp, L.; Sridharan, M.; Chang, B.Y.E. Safe Stream-Based Programming with Refinement Types. In Proceedings of the Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, New York, NY, USA, 2018; ASE ’18, p. 565–576. [CrossRef]
- Wikipedia contributors. MapReduce — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=MapReduce&oldid=1171373785, 2023. [Online; accessed 22-August-2023].
- TANGO collaboration. TANGO device server. https://tango-controls.readthedocs.io/en/latest/development/device-api/device-server-writing.html, 2023. [Online; accessed 20-August-2023].
- TANGO collaboration. TANGO pogo class generator. https://tango-controls.readthedocs.io/en/latest/tools-and-extensions/built-in/pogo/description.html, 2023. [Online; accessed 20-August-2023].
- TANGO collaboration. TANGO device examples. https://gitlab.com/tango-controls/device-servers, 2023. [Online; accessed 20-August-2023].
- Ktor contributors. Ktor raw sockets. https://ktor.io/docs/servers-raw-sockets.html, 2023. [Online; accessed 20-August-2023].
- O’Hara, S. J2mod. https://github.com/steveohara/j2mod, 2023. [Online; accessed 20-August-2023].
- Heron, K. Eclipse milo. https://projects.eclipse.org/projects/iot.milo, 2023. [Online; accessed 20-August-2023].
- STMicroelectronics. STM32. https://www.st.com/en/microcontrollers-microprocessors/stm32-32-bit-arm-cortex-mcus.html, 2023. [Online; accessed 20-August-2023].
- Raspberry pi foundation. Raspberry pi 4 model b. https://www.raspberrypi.com/products/raspberry-pi-4-model-b, 2023. [Online; accessed 20-August-2023].
- Nozik, A.; Dmitrieva, K.; Klimai, P.; Postovalov, I. mipt-npm/visionforge: 0.2.0, 2022. [CrossRef]
- Narkhede, N.; Shapira, G.; Palino, T. Kafka: The Definitive Guide Real-Time Data and Stream Processing at Scale, 1st ed.; O’Reilly Media, Inc., 2017.
- Dobbelaere, P.; Esmaili, K.S. Kafka versus RabbitMQ: A Comparative Study of Two Industry Reference Publish/Subscribe Implementations: Industry Paper. In Proceedings of the Proceedings of the 11th ACM International Conference on Distributed and Event-Based Systems, New York, NY, USA, 2017; DEBS ’17, p. 227–238. [CrossRef]
- Wikipedia contributors. ZeroMQ — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=ZeroMQ&oldid=1170894095, 2023. [Online; accessed 22-August-2023].
- Coulouris, G.; Dollimore, J.; Kindberg, T. Distributed Systems: Concepts and Design (International Computer Science), 4th rev. ed. ed.; Addison-Wesley Longman, Amsterdam, 2005.
- Mohd. Noor, A.S.; Mat Deris, M. Extended Heartbeat Mechanism for Fault Detection Service Methodology. In Proceedings of the Grid and Distributed Computing; Ślęzak, D.; Kim, T.h.; Yau, S.S.; Gervasi, O.; Kang, B.H., Eds., Berlin, Heidelberg, 2009; pp. 88–95.
- Wikipedia contributors. Watchdog timer — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=Watchdog_timer&oldid=1162902143, 2023. [Online; accessed 22-August-2023].
- RSocket contributors. RSocket. https://rsocket.io/, 2023. [Online; accessed 20-August-2023].
- Persaud, A.; Regis, M.J.; Stettler, M.W.; Vytla, V.K. Control Infrastructure for a Pulsed Ion Accelerator. IEEE Transactions on Nuclear Science 2016, 63, 2677–2681. [CrossRef]
- Khokhriakov, I.; Merkulova, O.; Wilde, F. Waltz - A Platform for Tango Controls Web Applications. In Proceedings of the Proc. ICALEPCS’19. JACoW Publishing, Geneva, Switzerland, 08 2020, number 17 in International Conference on Accelerator and Large Experimental Physics Control Systems, pp. 1519–1524. [CrossRef]
- Nozik, A.; Samorodova, E.; ArtificialPB.; SPC-code.; Joseph-Hui.; Nieboer, T.; Henry, Z. SciProgCentre/plotly.kt: 0.6.0, 2023. [CrossRef]
- Nozik, A. MassDevice.kt. https://github.com/SciProgCentre/controls.kt/blob/dev/demo/many-devices/src/main/kotlin/space/kscience/controls/demo/MassDevice.kt, 2023. [Online; accessed 20-August-2023].







Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).