#delayed_jobs

[red-green-red]

delayed jobs and scripts

Today I was facing a rather annoying issue with delayed jobs in rails. I had previously worked with them and as far as I remembered, they were really easy to get working. 

The difference this time: I was running a library script (which I had placed in the root lib directory) in my rails application. I was loading the files above the class structure and trying to get delayed jobs to work as follows:

Dir[File.expand_path(Rails.root.join 'lib/parse_scripts','**','*.rb')].each {|f| require f}

class Api::V1::PostsController < ApplicationController

.......

.......

 ParseScripts.delay.scrape

This was correctly inserting a delayed job entry into my table. But after a bit, I realized that it was silently failing and deleting itself automatically from the table (An issue that was mentioned in the delayed job wiki). The reason being, the delayed job script has no idea about my ParseScripts module which was not being loaded in the current environment! 

Hence, the fix was to require the module files as a rails initializer (config/initializers/load_scripts.rb) so that it was available to the delayed job script as well. And voila! everything works as normal again :)

[qiscus]

Rails Implementation of Delayed_jobs

From previous post regarding Rails Introduction of Background Process, we already knew the usage and importance of background process. It helps us to optimise the performance without sacrifice the lost of data or process.

First step, we need to add Delayed_jobs gem in our Gemfile

gem 'delayed_job_active_record'

then we just bundle install it.

after that we need to generate a table in our database to record all the delayed_jobs activities

$ rails generate delayed_job:active_record

then as usual, 

$ rake db:migrate

in localhost, we need to run the rake jobs:work

$ bundle exec rake jobs:work

and we need to put this line on Procfile, so Heroku can identify and run worker to do that command

worker: bundle exec rake jobs:work

done, all the requirement set-up already, we can straight away use it in our controller

For implementation in controller, we used to use this 

UserMailer.notification_email(mymail).deliver

to send the notification email, now we can queue this process in the background simply by put .delay method on it  and remove the deliver, rails will automagically queue and deliver it

UserMailer.delay.notification_email(mymail)

for other method, we can just simply put .delay on that method

Post.send_notif(user)

become

Post.send_notif(user).delay

done.

 So far, we satisfy with Delayed_jobs, however we also looking at Resque and Sidekiq (which both use Redis instead of Active_record) for anticipation of scalability issues later on. We tried implement Sidekiq for some of our process and saw that the performance was awesome and very fast, however currently we switch it off first for some reason.

[qiscus]

Rails Introduction of Background Process

Qiscus web application is written in Ruby on Rails. It is an amazing framework to build modern web application. Previously, Phidecks is written in PHP (Codeigniter) until we pivot and thinking it was a good time to rewrite everything in Ruby. As a semi-private discussion platform (which rapid conversation and discussion happen everytime and in real-time) , we need to ensure that Qiscus can handle request/responds very fast.  meanwhile, we need to make sure all the functions in Qiscus work. In this kind of application we definitely need a server which can handle background process while still process the web engine. Fortunately, Heroku provide this kind of process and Rails make it easier. Rails has plugin module called Delayed_jobs. It is very easy to setup and perform quite well and good enough to handle millions of data transaction in the background. In what case we need this delayed_jobs? Simple example of this usage: User submit a post or message, and in the same time we have to give notification to all user related to the post and sending some email related to this event.  Actually it is possible to do everything in one Rails controller through Ajax, but that means user have to wait until everything finished to be able do the next event. For us, sacrifice user experience is not the option. By using Delayed_jobs, we can throw this notification and sending email things to the background and let the worker server handle this for us, meanwhile the user does not need to wait the background server to finish their work. The most important thing, the data will not lose because we will process that anyway. It is just the event that user send is being delayed in the worker server and process immediately after first process finish. How is the implementation? we will explain our sample implementation in the other blog post, so stay discuss with us!