lessons learned using mongodb
Hold on Cowboy
This blog post is pretty old. Be careful with the information you find in here. It's likely dead, dying, or wildly inaccurate.
Back in 2012, there was a new way to build web apps fast and packed full of features. It was MEAN. MongoDB, Express, Angular, and Node. I built a number of apps with this technology, and it worked. However, as time passed they began to show their weaknesses. Today we’ll look at a few lessons learned from MongoDB.
What MongoDB Kicks Butt At
Most technologies cannot be just dismissed outright, even though you read article after article telling you “Why you should not X”. That’s just stupid, narrow minded, and immature.
You should use X for what X was best designed to do.
MongoDB is a Document Store, not to be confused with a database as you are probably thinking. Relational Databases have been around since the dawn of unix timestamp. MongoDB is really good for two types of data.
- Data that does not change much
- Data with different nested structure you don’t really care about consistent fields
For example. Let’s say you have a device and it logs a large number of things about itself every second. Say, a phone that logs its latitude, longitude, battery charge, network information, and more. Later you’ll want to query or search this information, but it doesn’t need to be updated since you are recording a log. MongoDB would be great for this. Since it can handle huge amounts of data and it really doesn’t have to be consistent.
What MongoDB doesn’t do well
The appeal to me initially was being able to keep the schema of my data in the app and have the app worry about field names. MongoDB doesn’t care if you feed it different types of data with each insert into a collection. It will happily create the corresponding field in a related data format. It might not be the type of data you want, but life goes on.
Missing or misspelled fields
While migrating the data I realized that one of the timestamp fields was named creatdAt
. Not a big deal since I wasn’t really using this field on the front end at the time, but just think if I would have realized this in the code and updated it? Then I would have a boat load of records with a creatdAt
field, and another set of the same records with a createdAt
field. You now have to write a script to fix all the incorrect spellings.
Fields with different data types
While migrating the data I had number fields that were supposed be numbers, but in fact some were strings, some floats, some integers. I had to adapt my import script to look for different types of data (string, number, float) and then convert them into integers. Notice the $numberLong
, that’s sometimes when the integer is too long for Javascript, so it will use that type of indication. numberLong.
function getAmount(amount) {
if (amount === undefined) {
return 0
}
if (isNumber(amount)) {
return parseInt(amount, 10)
}
if (isString(amount)) {
return parseInt(amount, 10)
}
if (amount && amount['$numberLong']) {
return parseInt(amount['$numberLong'], 10)
}
return 0
}
Inconsistency in your data
For some weird reason, a few of the passwords in the database were being stored as plain text instead of a hash. This was very disconcerting. I’ll write more later on why you should not be doing password hashing in the app (hint: let the database do it). In other cases, since I didn’t do a lot of data validation on some fields, I noticed that people were using the app in very strange ways. Like storing weird meta values in a person’s email record. I guess they were just being creative in how they stored data.
Conclusion
Building a web app with users, data, settings, etc? Use Postgres as your database. If you need to store massive amounts of log or complex JSON object data, then use MongoDB. As an added bonus, Postgres has so many built in features, it can actually do many of the things you were doing in the app. Such as updating timestamps, computing user password hashes, and much more.
But What About Migrations of Schema
Since I use Node.js mostly, I use the wonderful Knex module that you can write migrations in this manner
const TABLE = `charges`
exports.up = function (knex, Promise) {
return knex.schema.alterTable(TABLE, t => {
t.jsonb('shipping_contact').comment('JSON object of shipping details')
})
}
exports.down = function (knex, Promise) {
return knex.schema.alterTable(TABLE, t => {
t.dropColumn('shipping_contact')
})
}
I have an npm script that looks like this "start": "npm run migrate && node ./web"