#background job

[saptutorials]

Mastering SAP Background Job Processing

SAP background job processing is an indispensable feature that automates and schedules time-consuming tasks such as reporting, program execution and data processing without impacting real-time system performance. This guide offers step-by-step instructions on efficiently creating, scheduling and managing these jobs in SAP. What is a background Job? SAP background jobs are automated processes…

[joelmeador]

Another podcast with me. I feel like I was pretty on fire here. This is like my special interest and something I've been working around for almost my entire career.

[joelmeador]

Today on Elixir Wizards Office Hours, SmartLogic Engineer Joel Meador joins Dan Ivovich to discuss all things background jobs.

Co-workers and me talking about background processing for about an hour.

[sandeep45-blog-blog]

Setting up Resque in Rails - Step By Step

Here are steps for setting up Resque (based on Redis). The environment also includes Rails, unicorn and activeRecord.

gem 'redis' gem 'resque', :require => 'resque/server' These two gems are needed to setup Redis & Resque. You also need to install Redis. That can be done by brew install redis. Follow the steps to ensure that redis is running.

gem 'dotenv-rails', :groups => [:development] This is needed to load the .env variables in the rails applicaiton. When the app is run using foreman, the .env file is loaded automatically, but other times I found this gem useful.

Edit the ProcFile to Have: worker: env TERM_CHILD=1 RESQUE_TERM_TIMEOUT=10 bundle exec rake resque:work QUEUE=* Here you are saying that worker will run this command. In the command you giving 10 seconds for the worker to finish off any work it has already started All is is doing is running the resque rake task and telling teh rake task to run all queues.

Create a folder in /app called jobs Here add the following

require 'resque/errors'

module CsvExportFileMaker @queue = :exports

def self.perform(widget_id) Rails.logger.info "doing Work"

rescue Resque::TermException Rails.logger.error "an error happened. job asked to terminate while doing work" Resque.enqueue(CsvExportFileMaker, widget_id) end end

Here one, we are including the errors file so we can requeue the task incase it is terminated while it is still runnning. Then we are specifyingt eh queue from which it will pull off work. We are also defining a perform method which does the work.

Add dotenv.rb in the initializers directory. In the file add

if Rails.env.development? Rails.logger.info "We ar in development, lets load dotenv" require 'dotenv' Dotenv.load end

Here we r just loading the dotenv gem in dev.

Add resque.rb file in the initializers directory.

Rails.logger.info "In resque.rb conecting to resque url #{ENV["REDISTOGO_URL"]}"

uri = URI.parse(ENV["REDISTOGO_URL"])

Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password, :thread_safe => true)

Dir["/app/app/jobs/*.rb"].each { |file| require file }

Here we are connecting to redis on init. Then we are loading all the job workers.

Now add resque_auth.rb as another initializer

Resque::Server.use(Rack::Auth::Basic) do |user, password| user == "1" password == "2" end

We are doing this to add basic auth to resque admin portal

In our routes file add

mount Resque::Server, :at => "/resque"

This mounts the resque admin portal to our app

Now in uncorn.rb add the following to disconnect and reconnect to Redis.

Add the following to the before_form block

if defined?(Resque) + Resque.redis.quit + Rails.logger.info('Disconnected from Redis') + end

Add the following to the after_fork block

if defined?(Resque) + uri = URI.parse(ENV["REDISTOGO_URL"]) + Resque.redis = Redis.new(:host => uri.host, :port => uri.port, + :password => uri.password, :thread_safe => true) + Rails.logger.info('Connected to Redis') + end

Now add the following rake tasks:

dotenv.rake

if Rails.env.development? + require 'dotenv/tasks' + task :mytask => :dotenv do + # things that require .env + end +end

resque.rake

+require "resque/tasks" + +task "resque:setup" => :environment do + ENV['QUEUE'] = '*' + + Resque.before_fork do + Rails.logger.info "in resque.rake before fork" + defined?(ActiveRecord::Base) and + ActiveRecord::Base.connection.disconnect! + end + + Resque.after_fork do + Rails.logger.info "in resque.rake after fork" + defined?(ActiveRecord::Base) and + ActiveRecord::Base.establish_connection + end +end + +desc "Alias for resque:work (To run workers on Heroku)" +task "jobs:work" => "resque:work"

Here we are adding resque rake tasks to our tasks. Then we are making sure to disconnect and connect to db in every worker.

Now in our controller add the code to add a job

Resque.enqueue(CsvExportFileMaker, widget_id)

In the .env file add REDISTOGO_URL=redis://localhost:6379/

Upon pushing to heroku

add the redisToGo plan add a worker.

[bitsandpuzzles-blog]

[vonrails]

Background Job in Ruby

Background Job在Rails的處理方式有很多. 早期的Delayed Job, Background Job.. 乃至近期比較熱門的 Resque. 或比較屌.. 直接用EventMachine或QueueServer去處理..

但.. 今天要介紹一款利用Threads來處理BackgroundJob的Gem. 它好用,速度快. 威. 重點是很簡單.. 前後不用5分鐘你就可以繼續你的工作.. 預設是in-memory,但比起reqcue 開5個worker就耗掉你500m的ram,它吃ram吃得非常小. 如果怕in-memory資料會遺失,還可以接redis或mysql等. 這麼好用的東西怎能不推一下呢. ;)

Girl_Friday

Github : https://github.com/mperham/girl_friday

安裝 : gem install girl_friday

記得 :

development.rb & test.rb加入 :

GirlFriday.immediate!

application.rb加入 :

config.threadsafe!

使用 :

在initializer.rb加入 girl_friday.rb

YOUR_JOB_NAME = GirlFriday::WorkQueue.new(:give_this_job_a_thread_name) do |opts| do_some_works_in_here end

在Model或Controller裡呼叫 :

YOU_JOB_NAME.push(some_hash_values_that_you_want_send_in)

這樣就可以了. 很簡單. 不是嗎? ;)

後台 :

當然,我們可以監控目前的使用情況. 預設是使用Sinatra. 如果我們是在Rails裡面使用,那麼直接Call出來顯示一下就可以了.. 直接使用下列命令則我們可以得到一個Hash :

GirlFriday.status

下面是將回傳的Hash整理過後之畫面 :

很簡單明瞭吧.. ;)