Effective Regression Testing Framework Built with Open Source Tools

Due to the evolving Coronavirus/COVID-19 situation, SREcon20 Americas West has been rescheduled to June 2–4, 2020.
More information is available here.

Tuesday, March 24, 2020 - 11:00 am11:50 am

Jay Qiu and Chi Chin Chen, Bloomberg

Core Principles

As the agile development methodology gets more popular, it is a big challenge to release new versions of components into distributed systems without introducing new faults or degrading system performance. In this presentation, we discuss an effective testing framework we built using record-and-replay and log analysis techniques for regression tests and capacity tests on large scale distributed systems. A stack of open source tools are used to build this testing framework. Through replaying the incoming events and comparing systems responses, we can verify the correctness of the systems. System performance metrics are collected through log analysis in replay for performance criteria check and performance comparison with the previous version of systems. This testing framework is also an example of effectively using open source tools to build a practical application.

Jay Qiu, Bloomberg

Jay Qiu is a senior SRE at Bloomberg. He has been working on testing and monitoring fields for many years and has involved in architectural design and implementation for equity, fixed income and commodity trading and valuation systems at major financial companies for over 20 years.

Chi Chin Chen, Bloomberg

Chi Chin Chen is a senior SRE at Bloomberg. He has been a software engineer consultant for financial companies in NYC for over three decades, specializing in application system design and implementation using latest technologies.

@conference {247312,
author = {Jay Qiu and Chi Chin Chen},
title = {Effective Regression Testing Framework Built with Open Source Tools},
year = {2020},
address = {Santa Clara, CA},
publisher = {{USENIX} Association},
month = mar,