Replacing DDS with Apache Kafka as middleware technology for the Rubin Observatory control system

Tiago Ribeiro; Russell E. Owen; David J. Mills; Michael A. Reuter; Andy W. Clements; William O'Mullane

doi:10.1117/12.3020002

25 July 2024 Replacing DDS with Apache Kafka as middleware technology for the Rubin Observatory control system

Tiago Ribeiro, Russell E. Owen, David J. Mills, Michael A. Reuter, Andy W. Clements, William O'Mullane

Author Affiliations +

Proceedings Volume 13101, Software and Cyberinfrastructure for Astronomy VIII; 1310118 (2024) https://doi.org/10.1117/12.3020002
Event: SPIE Astronomical Telescopes + Instrumentation, 2024, Yokohama, Japan

Abstract

Once in operation, the Vera C. Rubin Observatory will execute a ten-year-long survey of the Southern sky known as the Legacy Survey of Space and Time (LSST). The Rubin Observatory Control System (Rubin-OCS) is a distributed system with each component in charge of a particular sub-system e.g.; the mount, the M1M3 mirror support system, etc. Each component is designed as an independent part of the system, and they must work together during operations. Communication between the components is done by means of a software middleware. The software middleware is the backbone of the system, allowing components to communicate with each other in a seamless way. The highly distributed nature of the Rubin-OCS places tight constraints in terms of latency, availability, and reliability for the middleware. The baseline implementation of the Rubin-OCS adopts the Data Distribution Service (DDS) technology for the middleware. In the Rubin-OCS, the middleware is encapsulated with a layer of abstraction known as the Service Abstraction Layer (SAL), which currently uses the ADLink-OpenSpliceDDS implementation of the Data Distribution Service (DDS) message passing system. Recently we performed a study of using Apache Kafka to replace DDS as the middleware technology for the Rubin-OCS. This study was motivated by the middleware-related challenges we faced while integrating the system as well as the recent announcements indicating that the adopted library may be deprecated during the lifespan of the project. The study involved throughput and latency studies and a proof of concept of our core libraries. Overall Kafka proved to be a suitable replacement for DDS.

Conference Presentation

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Tiago Ribeiro, Russell E. Owen, David J. Mills, Michael A. Reuter, Andy W. Clements, and William O'Mullane "Replacing DDS with Apache Kafka as middleware technology for the Rubin Observatory control system", Proc. SPIE 13101, Software and Cyberinfrastructure for Astronomy VIII, 1310118 (25 July 2024); https://doi.org/10.1117/12.3020002

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE