Web application development agency

Testing Ruby rake tasks is crucial

Rationale

Covering rake tasks with unit tests is not merely a nice-to-have, but a crucial part of testing on a way to robust code and safe production data. Consequences of having untested tasks can vary from annoyed or angry customers to critical system failures.

Imagine that you have a task that sends out a payment reminder every midnight. If the task is buggy, it can happen that customers with zero outstanding invoices will still receive a reminder. Chances are that you will discover a failure only once your customers start wondering or complaining, at best in the morning, since often rake tasks are run in the background by CRON every midnight. You will not be able to just undo it and the only solution will be to send another email apologizing for the first one. I doubt your customers will be happy about either.

Even worse if your tasks alter or delete rows in a database. It will be extremely hard to reveal and restore modifications made by broken tasks, if possible at all (well, at least unless you have some sophisticated change tracking mechanism).

Solution with code examples

Being armed with the knowledge of why we need to test rake tasks in the first place, we can now move on to writing actual tests.

The code examples are written in Ruby using Minitest framework (built-in). They do not depend on Rails; however, it takes very little effort to adapt them to other application and test frameworks.

Also, for the sake of examples imagine a CRM system that has a notion of overdue invoices which can be cancelled and we got an assignment to create a rake task for that.

First, we need to understand what type of test we might use. A unit test is out of the question since too many moving parts are involved in a rake task. We can’t and don’t want to mock or stub anything in this test. The key point is to ensure that the task works as a whole, therefore an integration test seems like a natural choice.

Second, we need to know what exactly to test. I see two main parts here: core functionality and output.

Testing core functionality

Core functionality of the task is basically the reason why it exists in the first place. The only thing we are interested in this test is to verify that an overdue invoice has indeed been cancelled. Consider the following example.

def test_cancels_overdue_invoice
  invoice = overdue_invoice

  capture_io do
    run_task
  end

  assert invoice.cancelled?
end

As a side note, capture_io is actually meant to be used to capture stdout and stderr streams into strings, as its name suggests, but in this case it is used to suppress any possible output the task may produce, which would otherwise get printed in the test results. A much better name would be mute_io since this is what we need. Nevertheless, capture_io is used here for brevity. Intention-revealing method names will be covered in another article.

Testing output

The second part is about verifying that a task reports to a user about what has been done (and potentially what has not). It is by no means less important than testing core functionality as users may tend to either run the task repeatedly or even consider it broken if they get misleading or incorrect feedback.

def test_output
  invoice = overdue_invoice

  assert_output "Invoice ##{invoice.id} has been cancelled\nCancelled total: 1\n" do
    run_task
  end
end

The point here is to capture everything the task writes to the stdout. Luckily Minitest already has assert_output assertion just for that.

Full code examples with additional comments are available at Github.

Conclusion

It’s much easier to have a safety net in the form of tests than dealing with subtle bugs and corrupted production data in emergency mode. That’s a great idea in general, but it gains additional importance when it comes rake tasks since they are often used to process mass data and therefore the potential harm is much greater. The same applies to one-time tasks, so it is not a valid excuse to leave them untested.

Take care of your customers and don’t let them be your free beta testers! Spare your coworkers who may need to deal with the consequences of failures made by your tasks with zero tests!

Tags: ruby, unit testing, rake tasks, minitest