Back to blog

Getting started with FoundationDB

2014-02-02 - Posted in Uncategorized Posted by:

by Rinat Abdullin

During the last week at HPC my focus has been on FoundationDB. FDB is a nice NoSQL database which has a bunch of great properties:

  • It stores key-value pairs of bytes, where keys are always sorted. You have usual GET/SET/DELETE operations along with range operations that come from sorted key nature
  • Multiple key operations can happen in a transaction
  • Many advanced operations can be implemented as Layers on top of that storage abstraction. There is even a SQL layer for that.
  • FDB scales nicely as you add new nodes to the cluster
  • Cluster of up to 6 nodes can be used for free in production
  • We (or Tomas, to be more precise :]) managed to get 75k write transactions out of a small cluster we setup at the digital Ocean
  • Setting up a cluster is a no-brainer even for a Linux noob like me
  • FDB handles load distribution automatically, it moves data as necessary, too
  • FDB has client libraries for python, golang, erlang, Node.js and even .NET
  • Their team is extremely helpful and humble
  • You can configure level of replication (e.g.: single, double, triple) before write is ACKed
  • FDB can be configured to store views in memory or on disk, transaction store is always durable

I personally really like that FDB is extremely opinionated about what it does (just a replicated transactional key-value storage), but it does this extremely well so far.

We are planning to use FDB as our event store and for persisting view models (which will be replicated across the cluster). I’m actually the one having fun with that implementation. Event Storage itself is a simple abstraction, however making implementation work properly with FDB key-value storage is something that requires better insight into inner workings of FDB. Plus, I get to do that in a go language.

My next week will focus on getting our full planned stack to play together (in an extremely jacky way), so that we could start developing components for the HPC2.

PS: my current development environment looks like this (Ubuntu LTS + “awesome” tiling manager + sublime + GoSublime):

Screenshot 2014 02 01 15 27 47

7 Comments

Jonathan Oliver 6 years ago

FoundationDB looks awesome. It seems to have all of the properties I would want in a key/value store. The only thing that I’m trying to figure out is how to work around the maximum 100K per value size limit. I have documents/projections/view models that I’m storing which are about 1MB or so.

Reply

Rinat Abdullin 6 years ago

Jonathan, thanks for the spelling corrections, fixed.

Regarding limits of value sizes – you can just store different parts of the model in different adjacent keys and then simply read entire key range in one operation. Since keys are ordered, this is a fast operation.

Reply

Jonathan Oliver 6 years ago

Also, your dev environment is Ubuntu *LTS* … and *GoSublime*.

Reply

Mikhail Zelenin 5 years ago

Rinat, why do you use linux instead of mac os? and which tiling manager do you use? Thank you in advance.

Reply

Rinat Abdullin 5 years ago

Mikhail,

I’m using Linux because it is closer to our production environment. Besides, I really like flexibility and simplicity of Linux. Re tiling manager – “Awesome” tiling manager and “zsh” as an awesome shell.

Reply

Kaarel 5 years ago

FoundationDB looks great. Some of the things listed as known limitations seem unusual though, maybe just if I compare it to a more traditional SQL engine. Wonder if you have have any comments:

– Long transactions (where long is 5 seconds)
– A songle slow machine can slow down the entire cluster
– No Zero-Downtime Upgrades (contrast with the micro service architecture with multiple versions of the same service that you discussed in a recent podcast. When I upgrade FDB I would still bring the whole system to a halt. No escape from Saturday night releases 🙂 )

https://foundationdb.com/documentation/known-limitations.html

Reply

Rinat Abdullin 5 years ago

Kaarel, thank you for the comments and the insight. You have a lot more experience with running FoundationDB in production than we do (we’re just getting started).

However, while moving forward with our project, we plan to design around these limitations (e.g. having FDB is nice, but completely killing a machine or even a datacenter should not be a big deal). This would harden code against downtimes (planned or not) and would allow to have graceful degradation. Purely, in theory so far 🙂

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *