Challenges of Managing Real-Time Financial Market Data Storage

Friday, June 16, 2023 - 1:40 pm2:35 pm

Kiran Kasichayanula and Nishith Nedungadi, Bloomberg LP


Are you looking to manage the storage of real-time market data? Or maybe you are interested in knowing how Bloomberg manages the storage of 300 billion unique events per day. Allow us to take you on our journey from local disk to cloud storage. We will talk about our motivations for the transition and explain how and why we decided to invest in a custom data scraping and chunk streaming solution to publish data into the cloud. We will talk about lessons learnt in migrating a complex production environment to leverage cloud storage.

Kiran Kasichayanula, Bloomberg LP

Kiran is System Reliability Engineer (SRE) in Bloomberg’s Feeds Engineering group. Kiran has worked on stability of real-time market data pipelines through automation, process standardization, and continuous process improvement. Kiran loves to play table tennis.

Nishith Nedungadi, Bloomberg LP

Nishith is an SRE team leader in Bloomberg’s Feeds Engineering group. He has 20 years of experience building and managing real-time data streaming systems in the finance industry. Nishith is interested in finding solutions to scalability and reliability problems related to the market data space. He loves robotics and spends most of his free time mentoring FIRST teams.

@conference {288321,
author = {Kiran Kasichayanula and Nishith Nedungadi},
title = {Challenges of Managing {Real-Time} Financial Market Data Storage},
year = {2023},
address = {Singapore},
publisher = {USENIX Association},
month = jun

Presentation Video