Heroku provides a highly scalable background worker framework via delayed_job, but leaves the burden of scaling it to the user. Several people have tried to solve this by either forking the delayed_job gem or hooking into the gold standard collectiveidea gem. I like the latter approach, but haven’t seen an implementation that meets my requirements.

Specifically, I want the following in my scaling library:

  1. Don’t require any special third party forks.

  2. Let me set minimum and maximum worker counts – and change them without redeploying.

  3. Let me set how busy I want my workers to be.

  4. Work out of the box with minimal configuration.

To achieve these things, I built middle_management.

You can get up and running with it very quickly:

  1. Add “middle_management” to your Gemfile

  2. Set the required heroku environment variables (MIDDLE_MANAGEMENT_HEROKU_USERNAME, MIDDLE_MANAGEMENT_HEROKU_PASSWORD, MIDDLE_MANAGEMENT_HEROKU_APP, MIDDLE_MANAGEMENT_MIN_WORKERS, MIDDLE_MANAGEMENT_MAX_WORKERS).

  3. Deploy.

The code is on github and the gem is on Rubygems. Check out the readme for further details.