In-Memory Database – Simplifying a Complex Data Management Technology

October 9, 2014
Editorial Staff


Editorial Staff

In-memory database (IMDB) technology, or main memory database (MMDB), is not as revolutionary as most publishers would suggest. As the name implies, IMDBs house data kept or manipulated within a machine’s main memory rather than disk storage found in traditional database management systems.

Another term for main memory is RAM (Random Access Memory), where the computer can only manipulate main memory data. Therefore, every program you execute and every file you access must be copied from a storage device into main memory. The amount of main memory on a computer is crucial because it determines how many applications can be executed at one time and how much data can be readily available to a program.

Main memory databases are faster than disk-based databases as fewer CPU instructions are required to access the data for reading and writing. To give a perspective, in fine-tuned Oracle systems, 95% or more of all data requests are already available in main memory. A difference of 5% is certainly not a valid reason to speak of a revolution. Not a single technology used in IMDBs is new. What is new, however, is the idea to combine several technologies which, so far, were only available in separate products.

In-Memory Versus Disk Storage

The figure below identifies the traditional disk-based storage against the efficiency of IMDB. The disk-based approach requires several intermediary steps before retrieving data for analysis. However, with in-memory database, you can fetch data directly from the calculation engine instead of wading through data marts, pre-aggregations/calculations, and segmented data storage/integration.

In-Memory Database Market Trends and Forecasts

Gartner considers in-memory databases amongst the top 10 technology trends impacting the IT infrastructure. At the rate application infrastructure technologies are maturing, and as Moore’s lawbrings us faster and more affordable semi-conductors, mainstream use of in-memory computing (IMC) is more likely in the future.

The in-memory computing market has two major segments:

  1. In-memory data management system
  2. In-memory application platform

At present, the IMDB system has greater market share, yet the in-memory application platform is projected to increase significantly. According to Research and Markets, IMC is likely to reach $13B by 2018 from $2.21B (2013) – a staggering 43% year-over-year growth. In a separate analysis from Gartner, in-memory data management is projected to reach $1 Billion by 2016.

Most of this forecast is coming from a sharp rise in transactional and analytical requirements for voluminous big-data. IMDB solutions are quickly finding homes in almost every database warehousing and analytics application. Today database management and analysis technologies are ubiquitous. From private sector organizations to public/government sector, data management and analysis technologies play a vital role for achieving operational efficiency. Organizations, today, need technologies which can store more data in less space while analyzing at higher speeds.

Should I Consider Using In-Memory Database?

Numerous criteria require consideration before leaping to an IMDB solution. Items such as existing infrastructure, data persistence requirements, application architecture, IT budget, etc. must all be aligned toward your strategic goals for implementing IMDB. But if I have to keep it simple, you should ask yourself:

Can my current persistent database meet my performance requirements?

If your current traditional database is struggling to handle your current load, you should research more into in-memory database and computing. However, if your current IT environment is doing fine and scaling it would keep your IT applications in the green zone, it would be wise to wait and let the market mature some more. While considering IMDB, keep in mind that you are evaluating a true in-memory database if you need the performance only an in-memory database could provide. Caching a persistent database is not the same. If your high performance requirements are only limited to certain columns which are highly utilized by your analytics application, then perhaps a column in-memory database could be a more suited option for you.

Leave a Reply