Mainframes evolved into the relational database in the 1970s with the core tenet of providing users with an easier-to-use abstraction, an expressive query language, and a vertically integrated system. With the explosion of data in the early 2000s, we created the big data stack and decoupled storage from compute. Since then the community has gone on to build the modern data platform that looks like a deconstructed database. We survey the different technologies that have been built to support big data and what a modern data platform looks like, especially in the era of the cloud.
Modern data platform architectures are spurring a wave of innovation and intelligence by enabling new workloads that weren’t possible before. We will review three main phases of technology evolution to highlight how the user experience of working with data has changed over time. The article concludes with a review of the current state of data architectures and how they are changing to better meet demand.
