Run Sidekiq with multiple queues in Ruby on Rails

,

I’m working for a ticketing system team, recently we want to generate a huge number e-tickets (more than 100) from multiple events in a campaign, and then send these e-tickets to agencies.

We discussed together to find out the good way to implement this feature. Skip the business logic, basically we need to loop all events in a campaign, select seats and generate e-tickets. This feature will be run as background jobs.

We’re using sidekiq for background processing. We have a “ticket” queue to generate e-ticket whenever a customer buy a tix. A little bit introduce about “ticket” queue: It is a queue to run important background jobs.

So with e-ticket generating, we’re going to start a new queue.

We can specify the queue for the worker that will run e-ticket generating by sidekiq_options

1
2
3
4
5
6
7
8
class TicketGeneratingWorker
  include Sidekiq::Worker
  sidekiq_options queue: "klook"

  def perform(args)
    # blah blah
  end
end

Alright, now we have two different queues:

  • ticket queue for normal e-ticket generating when a custom buy a tix.

  • klook queue for auto generate 100 e-tickets from multiple events in a campaign. “klook” is kind of promotion that we run for the campain.

The next question is how we run Sidekiq’s queues in worker servers.

In our end, we’re using Rails for the ticket generating and Capistrano for deployment. And there’s a capistrano-sidekiq plugin. Therefore the problem is now easier, we just need to configure sidekiq to start with multiple processes and set queues will run on these processes.

1
2
3
4
5
6
set :sidekiq_config, "config/sidekiq.yml"
set :sidekiq_processes, 2
set :sidekiq_options_per_process, [
  "--queue ticket",
  "--queue klook",
]

That’s all I want to share. See you guys in the next post!