NoRAM DB => “If It Does Not Fit in RAM, I Will Quietly Die For You”

Out of 472 NoSQL databases / distributed caches that are currently available, highly buzzed and scream that only their precious brand solves the world toughest problems.. There are only a few that have less screaming and more doing that I found so far:

Choosing a Drink


See.. Choice makes things better and worse at the same time =>

if I am thirsty, and I only have water available, 
I'll take that, satisfy my thirst, 
and come back to focusing on whatever it is I was doing

at the same time

if I am thirsty, and I have 472 sodas, and 253 juices, and 83 waters available
I'll take a two hour brake to choose the right one, satisfy my thirst, 
and come back to focusing on whatever it is I was doing

It may not seem as two different experiences at first, but they are different approaches.

Especially bad is when out of those 472 sodas, #58 has such a seductive ad on a can, and it promisses you $1,000,000 prize, if only you make a sip. So you are interested ( many people drink it too ), trying it… and spitting it out immediately => now there are 471 left to go.. remaining thirst, and such a dissatisfaction

If there is CAP, there is CRACS


NoSQL is no different now days. Choosing the right data store for your task / project / problem really depends on several factors. Let’s redefine a CAP theorem (because we can), and call it CRACS instead:

  • Cost:                    Do you have spare change for several TB of RAM?
  • Reliability:            Fault Tolerance, Availability
  • Amount of data:   Do you operate Megabytes or Terabytes, or maybe Petabytes?
  • Consistency:         Can you survive two reads @ the same millisecond return different results?
  • Speed:                  Reading, which includes various aggregates AND writing

That’s pretty much it, let me know if you have you favorite that is missing from CRACS, but at least for now, five is a good number. Of course there are other important things, like simplicity ( hello Cassandra: NOT ) and ease of development ( hello VoltDB, KDB: NOT ), and of course fun to work with, and of course great documentation, and of course great community, and of course… there are others, but above five seem to nail it.

Distributed Cache: Why Redis and Hazelcast?


Well, let’s see Redis and Hazelcast are distributed caches with an optional persistent store. They score because they do just that => CACHE, they are data structure based: e.g. List, Set, Queue.. even DistributedTask, DistributedEvent, etc. Again they are upfront with you: we do awesome CACHE => and they do. I have a good feeling about GemFire, but I have not tried it, and last time I contacted them, they did not respond, so that’s that.

NoSQL: Going Beyond RAM


See, what I really learned to dislike is “if it does not fit in RAM, I will quietly die for you” NoSQL data stores ( hello MongoDB ). And it is not just the index that should entirely fit into memory, but also the data that has not yet been fsync’ed, data that was brought back for querying, data that was fsync’ed, but still hangs around, etc..

The thing to understand when comparing Riak to MongoDB is that Riak actually writes to disk, and MongoDB writes to mmap files ( memory ). Try setting Mongo’s WriteConcern to “FSYNC_SAFE”, and now compare it to Riak => that would be a fair comparison. And having LevelDB as Riak’s backend, or even good old Bitcask, Riak will take Gold. Yes, Riak’s storage is pluggable : ).

Another thing besides that obvious Mongo “RAM problem” is JSON, I mean BSON. Don’t let this ‘B‘ fool you, a key name such as “name” will take at least 4 bytes, without even getting to the actual value for this key, which affects performance, as well as storage. Riak has protocol buffers as an alternative to JSON, which can really help with the document size. And with secondary indicies on the way, it may even prove be searchable : ).

Both Riak and MongoDB struggle with Map/Reduce: MongoDB does it in a single (!) SpiderMonkey thread, and of course it has a friendly GlobalLock that does not help, but it takes a point from Riak by having secondary indicies. But both: Mongo and Riak are in the process of rewriting their MapReduce frameworks completely, so we’ll see how it goes.

Cassandra Goodness and Awkwardness


Cassandra is however a solid piece of software, with one caveat: you have to hire DataStax guys, who are really, really, really good, by the way, but you have to pay up good money for such goodness. Otherwise, on your own, you don’t need to really have a PhD to handle Cassandra [ you only really need to have a PhD in Comp Sci, if you actually can and like to hang around in school for a couple more years, but that is a topic for another blog post ].

Cassandra documentation is somewhat good, and the code is there, the only problem is it looks and feels a bit Frankenstein: murmurhash from here, bloom filter from here, here is a sprinkle of Thrift ( really!? ), in case you want to Insert/Update, here is a “mutator”, etc.. Plus adding a node is really an afterthought => if you have TBs of data, adding a node can take days, no really => days. Querying is improving with CQL roll out in 0.8, but any kind of even simplistic aggregation requires some “Thriftiveness” [ read "Awkwardness" ].

CouchDB is Awesome, but JSON


CouchDB looks and feels like an awesome choice if the size of the data is not a problem. Same as with MongoDB, “JSON only” approach is a bit weird, why only JSON? The point of no schema is NOT that it changes ALL the time => then you have just a BLOB, content of which you cannot predict, hence index. The point is, the schema ( and rather some part of it ) may/will change, so WHY the heck do I have to carry those key names (that are VARCHARs and take space) around? Again CouchDB + alternative protocol would make it in my “Real NoSQL Dudes” list easily, as I like most things about it, especially secondary indicies, Ubuntu integration, mobile expansion, and of course Erlang, but Riak has Erlang too : )

VoltDB: With Great Speed Comes Great Responsibility


As far as speed, VoltDB would probably leave most of others behind (well, besides OneTick and KDB), but it actually is not NoSQL (hence it is fully ACID), and it is not NotJustInRam store, since it IS Just RAM. Three huge caveats are:

1. Everything is a precoded store procedure => ad hoc querying is quite difficult
2. Aggregation queries can’t work with data greater than 100MB ( for a temp table )
3. Data “can” be exported to Hadoop, but there is no real integration (yet) to analyze the data that is currently in RAM along with the data in Hadoop.

But it is young and worth mentioning, as I think RAM and hardware gets cheaper, “commodity nodes” become less important, so “Scale Up” solutions may actually win back some architectures. There is of course a question of Fault Tolerance, which Erlang / AKKA based systems will solve a lot better than any “Scale Up”s, but it is a topic for another time.

Dark Hourses of NoSQL


There are others, such as Tokyo Cabinet ( or is it Kyoto Cabinet now days ), Project Voldemort => I have not tried them, but have heard good stories about them. The only problem I see with these “dark horse” solutions is lack of adoption.

Neo4j: Graph it Like You Mean It


Ok, so why Neo4j? Well, because it is a graph data store, that screws with your mind a bit (data modeling) until you get it. But once you get it, there is no excuse not to use it, especially when you create your next “social network, location based, shopping smart” start up => modeling connected things as a graph just MAKES SENSE, and Neo4j is perfect for it.

You know how fast it is to find “all” connections for a single node in a non graph data store? Well, it is longer and longer with “all” being a big number. With Neo4j, it is as fast as to find a single connection => cause it is a graph, it’s a natural data structure to work with graph shaped data. It comes with a price it is not as easy to Scale Out, at least for free. But.. it is fully transactional ( even JTA ), it persists things to disk, it is baked into Spring Data. Try it, makes your brain do a couple of front splits, but your mind feels really stretched afterwards.

The Future is 600 to 1


There is of course HBase, which Hadapt guys are promising to beat 600 to 1, so we’ll see what Hadapt brings to the table. Meanwhile I invite Riak, Redis, Hazecast and Neo4j to walk together with me the slippery slope of NoSQL.

14 comments

  1. Is Intersystems Cache one of those 472?
    And if so, how does it fare?

    I’m curious since it has been around for ages

  2. @nicki,

    To be honest with you that would be #473 that I have not worked with. Do you have any experience with it that you can share? What would be things that really rock? Things that really suck? Would be great to know both.

    /Toly

  3. Any experience with amazon’s dynamo db??

  4. VoltDB engineer here. I’d like to address the three points you’ve made.
    1. Ad hoc querying was initially not a focus at VoltDB, however, we’ve made great strides in performance and flexibility in the past few months, and there’s much more coming in 2012. This is a real priority for us.
    2. The 100mb temp table limit is a tunable number, and only affects certain kinds of queries. We also support materialized views that can aggregate on hundreds of GB or even a few TB with typical VoltDB performance. If you get stuck on a particular kind of query, let us know and we’ll see if we can help.
    3. Many users are not keeping today’s data in VoltDB and yesterday’s in in Hadoop, though they start off with that thought. Most push data either into both systems simultaneously, or they push data “through” VoltDB into Hadoop (with seconds of latency). In either case, the data is “query”-able in Hadoop no later than if you didn’t have VoltDB. We also have 2012 improvements planned for our Hadoop integration.

  5. A question from a non-techy: And in general security is not well-handled by NoSQL databases, right? including the above?

  6. Sadly, actually working with those rather than just testing them is a complete different experience.
    Neo4j for example can’t really handle big data.

  7. John,

    1. So now it is possible to execute ad hoc queries as it possible with traditional RDBMS/NoSQL?

    2. 100MB was (is?) the max for temp tables, hence no real “big data” aggregation is possible. As to the materialized views => they only support four basic aggregates: SUM, COUNT, MIN and MAX, hence it does not really matter how large they can go, since no real aggregation can by done on them.

    3. They point is not where the data gets pushed from/to, but that the abstraction is broken. When I have a durable store, it should allow to _seamlessly_ query across days without caring whether the data lives in Hadoop, VoltDB or somewhere else.

    Thank you,
    /Toly

  8. @Andrew,

    Depends what you mean by “general security”. NoSQL is a _different_ architecture, hence requires a different way to secure it. Whoever says that NoSQL cannot be secured just needs to hire people who know how to do it.

    Thank you,
    /Toly

  9. @Nicolai,

    “Neo4j for example can’t really handle big data” => can you support this claim with an example / use case?

    Thank you,
    /Toly

  10. Is ‘=>’ some new kind of fancy punctuation?

  11. @”your name”

    {key1 => value1, key2 => value2 …} is one of Ruby’s hash syntaxes(?).

  12. @”your name”,

    I invent my own punctuation :)

    I use “=>” for several things:

    * Follows: “today is raining => streets are wet”
    * Given That: “you said 42 => do you have anything to back it up?”
    * Which Is: “And we all love Pi => 3.14159265358979323846264″

    and more..

    In general, I love languages not because they constrain us, by things like punctuations, but because they open minds and facilitate conveyance of ideas.

    /Toly

tell me something...
  1. (required)
  2. (valid email - optional)
  3. Captcha
  4. (required)