cfaed Distinguished Lecture Series

Abstract

Flash memories have been in widespread usage for a while but they have had some performance and reliability problems which have made them unsuitable for long term storage of traditional database data. A new class of memory called Storage Class Memories (SCMs) are emerging which are built using different technologies than flash devices. SCMs overcome many of the shortcoming of flash devices while approaching the cost of flash memories. SCMs fall in between DRAM and traditional disk storage along many dimensions (performance, cost, energy usage, ....). As a result, large SCM-based memory systems will be built. While main memory database management systems (MMDBMSs) companies like TimesTen and SolidDB have been around for a while, those companies have been acquired recently by Oracle and IBM, respectively. SCMs will permit the sizes of databases managed by MMDBMSs to be very large while being cheaper than those using only DRAM. SCMs may be viewed as disks or as memory from an architectural perspective. Depending on the viewpoint, the implications on DBMS architectures will be very different. Some preliminary ideas on usage of a small amount of non-volatile memory realized by using battery-backed DRAM was presented in a paper design called Safe RAM in VLDB 1989. Technology has evolved tremendously in 2 decades and it is time for us to revisit system architectures.

In IBM Research, we have been working on multiple projects to understand the implications of SCMs on software and hardware architectures in general and on DBMS architectures in particular. Traditional locking, recovery, storage management and query processing ideas would need to be extended to take advantage of SCMs. In this talk, I will discuss what we have learnt from our investigations and what needs to be further explored. I believe this presentation will generate a lot of discussions and debates. This talk should be of interest to hardware, software, systems and storage people both in industry and academia.