Are your workers working?

Most of the Rails developers choose Sidekiq for background processing in their Rails applications and setup the Sidekiq workers to do the long running work in background.

But as a Rails developer have you ever wondered whether your workers are doing their job? Now question may pop up in your mind, How to know whether my background workers doing their job? No worries now you can do that easily with sidekiq-status gem, How? Lets see how …

Bit about sidekiq-status:

  • An extension to the sidekiq message processing to track your jobs.
  • sidekiq-status implemented sidekiq middleware and uses Redis underneath for storing statuses and meta information about jobs.
  • In latest release gem added some really cool features like more status types, Web UI enhancements  which makes it really cool to have in your projects for tracking background jobs.

Let’s setup sample worker:

To track job progress of worker add sidekiq-status gem to the Gemfile and after `bundle install` include `Sidekiq::Status::Worker` module in your worker.

Lets say one wants send  emails to 100 different users in background Sample `UserWroker` will look like,

  class UserWorker
    include Sidekeq::Worker
    include Sidekiq::Status::Worker #Important
    def perform
      User.find_each do |user|
        UserMailer.weekly_summary(user).deliver_now
      end
    end
  end

Now make your worker more proactive by using following methods from `Sidekiq::Status::Worker` module to set the current progress of the worker.

  • total : Total number of actions that your worker going to perform default is 100 eg. If your worker sends email to 100 users set this to 100.
    total 100
  
  • at : Sets current number of actions performed by worker eg. If job has completed sending email to 10 users then worker sets this value to 10.
    at 10, "some message"
  
  • retrieve : Method to retrieve values by method name or stored keys eg. If you want to retrieve value of ‘at’ then `retrieve :at` will return latest value of ‘at’.
    value = retrieve :at #returns 10
  
  • store : Associate any custom data with job, eg. If you want to store delivery count in status hash.
    store delivery_count: 10
    delivery_count = retrieve :delivery_count #returns 10
  

Let’s use above methods to track job status in our sample worker which will turn into following

  class UserWorker
    include Sidekeq::Worker
    include Sidekiq::Status::Worker #Important
    def perform
      # Setting total number of users
      total User.count
      store delivery_count: 0 
      User.find_each do |user|
        # Setting 'at' for progress
        status = retrieve(:at).to_i
        delivery_count = retrieve(:delivery_count).to_i
        at (status.to_i + 1), "User: #{user.id}, delivery_count: #{delivery_count}"
        UserMailer.weekly_summary(user).deliver_now
        delivery_count += 1
        store delivery_count: delivery_count
      end
    end
  end

After setting your worker like above, visit /sidekiq/statuses tab on sidekiq web you should see your workers progress.

sidekiq-status-blog

On cautious note, make sure you follow all steps to integrate Sidekiq  Web UI and sidekiq-status Web UI.

I have created a template gist for above sample worker along with detailed steps to setup sidekiq worker with sidekiq-status.

If you got stuck in some problem while implementing this, Please let me know It will be my pleasure to help you out and suggestions are most welcome.

Have a nice day and keep watch on your Workers!

Advertisements

One thought on “Are your workers working?

Add your thoughts...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.