Some tips on using MongoDB (with Lithium examples)

International Church of the Foursquare Gospel

Four Squares Image gratefully obtained via Wikipedia

I didn’t rush into trying out MongoDB at first, when the buzz began to grow on NoSQL. The reason being that so many developers didn’t bother to research and understand SQL properly, a bread and butter technology proven over decades… and when those same developers move into something shiny and new, proclaiming how great it is, they scare me.

Databases are not simple or trivial. It’s like a Mountain. You have to respect the Mountain or Nature will force you to show respect.

What you don’t know can lead to disaster in production situations. It’s the type of thing that you want to let others, like Foursquare, fail painfully at, identify and analyze the bottlenecks for you, and then you come in and try it out as the issues start to get resolved and the technology matures a bit.

I think we’re getting there and the time has come to get involved. There are lots of clever things I really like about Mongo and still some gotchas. This post will give you an example of both. I urge you to do your research on how MongoDB works so that you can use it like a champ.

OK, let’s dig in.

MongoDB was built to scale horizontally. Their goal was to make that easy. To help highly trafficked websites have a smaller burden when they grow quickly. And to be as fast as possible. There are no shortcuts. You sacrifice one thing to achieve another. So just keep in mind that 10gen made deliberate choices. And there are times when you want to override the defaults.

One such example is safe mode. In order to optimize for speed, MongoDB does not, by default, wait for confirmation that a write was successful. So you go on your merry way, only to find out later that your write never happened. I encountered this very issue yesterday when an update wasn’t happening, and I didn’t know why. If you turn on safe mode, Mongo will wait until a confirmation comes in (or an error).

You need that error to find out that something went wrong and figure out how to fix it.

That’s why their choice of default is controversial. But there’s nothing inherently wrong with it. In fact, this approach is getting popular in other areas. The caveat is that it’s up to you to handle it properly. And that message is getting lost along the way…

Check out spine.js, a javascript library by a Twitter engineer. Instead of waiting for confirmation from the server on ajax calls, they update the UI first and have to build in error handling later.

If all you lose is some Paris Hilton or Kim Kardashian tweets, no biggie. The world won’t go a-hurtin’. But if you just failed on a credit card payment, well then my friends, it’s another story entirely.

My point is, it’s great that you can optimize the hell out of the web experience you’re building. But if you aren’t even aware that you need to handle errors, then what the hell are you doing anyway?

OK, I think I’ve nagged you enough about fully researching your database of choice, now it’s time to get practical.

Here’s tip #1. When you are working on your dev machine, safe mode should be turned on for all DB operations. It’s similar to strict mode.

Additionally, when you are processing an important request in production, this too should be done with safe mode ON. For example user registration.

This is how you set it up on Lithium ( aka #li3 ). I ran into a bit of a gotcha. Lithium has some defaults for (‘test’, ‘development’, ‘production’) environments. And I’ve been working with ‘local’. It turns out that setting up a custom environment in Lithium is a bit unintuitive (remember Lithium isn’t even on version 1.0, so keep expectations in check).

Instead of:

Environment::add(‘local’, array(‘foo’ => ‘bar’);

Environment::set(‘local’), you actually need to add it with the set command and then set it with a second call to the set command.

So I have an environments.php file under the config directory. It contains this:


use lithium\core\Environment;


if (preg_match('/^local/', $_SERVER['HTTP_HOST'])) {
Environment::set('local', array('host' => 'local.example.com'));
Environment::set('local');
}

Now in a moment, I’m going to show you what my connections.php looks like. You may need to adjust yours a bit. Post a comment if it isn’t working right.

connections.php:
Connections::add('default', array(
'local' => array('type' => 'MongoDb',
'host' => 'local.example.com',
'database' => 'example_LOCAL'),
'test' => array('type' => 'MongoDb',
'host' => 'localhost',
'database' => 'example_TEST'),
));

Now you’re going to want to “filter” the “create”, “update” and “delete” methods. Hat tip to @mehlah for the approach. So right underneath that code, add this (updated to make more DRY, per @mehdi’s comment):

if(Environment::is('local')) {
Connections::get('default')->applyFilter(array('create', 'update', 'delete'), function($self, $params, $chain){
$params['options']['safe'] = true;
return $chain->next($self, $params, $chain);
});

} // end if

And voila, safe mode is on for local. Now play around with your app. Some of you may be surprised to find some exceptions being thrown. If so, that means Mongo has been silently dropping some db calls. This is something you’ll need to fix😉

Now that that’s done, you need to know how to manually turn on safe mode for important queries, like credit card processing.

Let’s say you have an activity model. And you’re updating a credit card. Here is a call with safe mode on:

if (Activity::update($query, $conditions, array(‘atomic’ => false, ‘safe’ => true))) {
$success = true;
}

Now here’s tip #2. You don’t need to create a field called “created”. Because the _id that Mongo creates contains a timestamp! This is one of those clever things I really like. It would take very little to find out how via Google, so I’ll leave you a link or two and this task, as an exercise for you, dear reader.

4 responses to “Some tips on using MongoDB (with Lithium examples)

  1. Thanks for the mention😉
    to DRYup things, applyFilter() happily takes an array of methods

  2. Thanks @Mehdi, so this should work?:

    Connections::get(‘default’)->applyFilter(array(‘create’,’update’,‘delete‘), function($self, $params, $chain){
    $params[‘options’][‘safe’] = true;
    return $chain->next($self, $params, $chain);
    });

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s