How a growing demand for data tier scalability spawned the NoSQL movement

There was a time when even the suggestion of storing an organization's data in anything but a relational database would seem foolish. But NoSQL has flipped that equation on its head, as organizations who are looking for speed and scalability are looking for new ways to integration non-relational technology into their solutions.

Though NoSQL originally developed as a flexible and agile alternative to relational databases, non-relational databases haven’t yet gained wide acceptability in the large enterprise segment. The market for NoSQL is growing, but good old fashioned relational databases still have a large clientele, and this is likely to remain so for the foreseeable future. But as more and more organizations look for speed and scalability, the NoSQL option is becoming a more and more likely option for enterprise organizations.

Why NoSQL?

Given the ever-increasing volumes of data and the need to interact between different databases, quite a few companies are beginning to adopt data technologies that enable easy scalability, non-relational capabilities and speed of processing.  NoSQL databases have proliferated in recent years to cater to the growing need for both quick storage and access to very large volumes of data across a multiplicity of servers. This requires scalability at a level that relational databases are simply not equipped.

Rather than scaling vertically and spending millions of dollars on an Oracle box, you can buy a federation of cheap hardware and distribute your database. 

Eric Redmond, author of Seven Databases in Seven Months

Despite this, many enterprises still prefer using SQL database systems for a number of reasons, the primary one being their compliance with ACID (atomicity, consistency, isolation and durability) assurances, which guarantees a high level of reliability. Another reason that probably discourages large organizations from deploying NoSQL databases is the scarcity of NoSQL developers, due to the fact that NoSQL is still a relatively new technology. Relational SQL is a mature technology and there’s no dearth of RDMS experts.  Also, many NoSQL database vendors are start-ups that are perceived as lacking the resources to offer global support and maintenance along the lines of what is offered by large companies such as IBM and Oracle.  Though NoSQL technology offers developers the flexibility and agility necessary to process different data types in real time, the relatively low penetration of NoSQL databases in the large enterprise environment is also due to the fact that SQL databases have been in use at the enterprise level for years now and have proved to be relatively reliable.

However, most businesses today can’t afford to ignore the potential that huge amounts of unstructured data generated through Facebook, Google, Twitter and Amazon hold in relation to delivering insights into customer behavior and ever-evolving market trends. In order to stay competitive, companies will need to invest more in Big Data analysis. According to IDC, roughly 90% of data generated globally over this decade is likely to be unstructured.   

As more and more business processes now involve processing various data types, it is much simpler for developers to combine real-time data in one API. And NoSQL databases, systems that can often run on cheap, commodity hardware, will make managing and maintaining that massive volume of data much cheaper. “Rather than scaling vertically and spending millions of dollars on an Oracle box, you can buy a federation of cheap hardware and distribute your database,” said Eric Redmond, author of Seven Databases in Seven Months

The unification of NoSQL

Of course, managing data across a disparate set of NoSQL databases is still a problem. With a view to address the problem that developers face when working with data from a number of different sources, EspressoLogic recently announced that it had introduced a new capability that would enable developers to access and combine NoSQL and SQL data in one REST API. Though this recently added functionality is applicable only to NoSQL data in MongoDB at present, Espresso Logic CEO, R. Paul Singh said that the company was open to including other NoSQL databases if the need arose in future. But still, while a significant amount of work is being done to get different NoSQL systems to co-exist, the industry is a long way away from having a true NoSQL standard. “Standards are beautiful, wonderful things, but you cannot predict when they’re going to be adopted. There are uncontrollable events that suddenly allow them to take off,” said Dan McCreary, coauthor of Making Sense of NoSQL.

Can relational and NoSQL worlds combine and co-exist?

In order to cater to the growing demand for scalability and architectural flexibility, a number of SQL database vendors have upgraded their relational databases to include NoSQL capabilities. The Oracle 12 version has the added functionality needed to process JSON data. Earlier this year, EnterpriseDB added NoSQL features to its PostgreSQL version 9.4.

On August 19, even EnterpriseDB announced that it was offering a free NoSQL-compatible development environment, Postgres Extended Datatype Developer Kit (PGXDK), on AWS (Amazon Web Services). The developer kit, which is scheduled to be launched on September 10, seeks to enable programmers "to use Postgres for the kinds of applications that until recently required a specialized NoSQL-only solution," according to EnterpriseDB. The kit is designed to help developers use PostgresSQL’s NoSQL capabilities to develop Web.2 applications.

Adding NoSQL functionality to relational databases has some advantages. Whereas NoSQL-only databases are not ACID-compliant, SQL systems with NoSQL functionality enable NoSQL processing with atomicity and consistency guarantees. Also, developers have the option of using SQL rather than querying through NoSQL. Relational databases with NoSQL functionality might even be faster than NoSQL databases. At least that’s what EnterpriseDB claims. According to the company’s principal system engineer, Postgres performs faster than MongoDB when processing the same JSON data. On the other hand, Splice Machine offers a NoSQL Hadoop database with SQL querying functionality. According to the company, this delivers greater speed than a conventional SQL database.

Going forward, it’s likely that both SQL and NoSQL solutions will develop alongside in order to address the different needs of customers. As Internet computing grows, so also will the demand for processing unstructured data.

What tools and resources have you used to  master NoSQL? Let us know.


Dig Deeper on Big data architecture

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.