when mongoid .first slows you down
"How soon not now, becomes never" - says Martin Luther and it perfectly describes my procrastination and ignorance towards getting back at writing. so here it is.
we use mongodb as the datastore for our product (sell.do). coming from an activerecord background (yes rails, yayy!!), most of us almost blindly use {model}.first/last to find our expected result and it (almost always) works without any issues. and then suddenly one fine day you start experiencing degraded response times in production and your good friend newrelic points out that your database is the culprit (well its you :P).
indexing and pro-active database profiling is of utmost importance for any application. given our dataset size (our largest collection has > 22 million documents), we had made sure we covered our queries (read more about query coverage here) with good indexes and expected blazing fast performance but no :\
on closer examination, we noticed that mongoid quietly inserts a "$orderby"=>{:_id=>1} whenever we did User.where(******).first (or lack of better resource name). wow, where did this come from? (existing bug raised with mongo) well you don't really care about it initially but trust me sorting (specially when not required) can lead to lot of slow long running queries when you have huge collections. on digging further, we found that in version 2.4 the optimizer can't use the indexes we set, if a $sort or $orderby is mentioned in the query but not specified in the index. changing our query pattern to User.where(******).limit(1).entries.first avoided the unnecessary sort operation (which we didn't really need anyways) and started using the index again.
... and the world was back to normal :P
ps: here's quick way to analyse the queries in rails console. Thanks Steve McHail!
# This initializer adds a method, show_mongo, when running a rails console. you can include it in a file in your config/initializers folder. When active, all # moped commands (moped is mongoid's mongodb driver) will be logged inline in the console output. # If called again, logging will be restored to normal (written to log files, not shown inline). # Usage: # > show_mongo if defined?(Rails::Console) def show_mongo if Moped.logger == Rails.logger Moped.logger = Logger.new($stdout) true else Moped.logger = Rails.logger false end end alias :show_moped :show_mongo end