Web application development agency

Fixing low code coverage when using test helper with minitest, simplecov and rake

Introduction

You might face low code coverage in non-Rails environment ¹ (for example, when using Sinatra) when running rake test task, whereas running single tests via ruby -I test test/unit/some_test.rb works fine. This article explains why it happens and how to fix it.

The article assumes you have already configured Simplecov and Minitest Rake test task.

Analysis

Most of the suggestions found in Google will be about including and calling SimpleCov at the very top of your test helper (the entry point for all tests), but if you followed its documentation mentioned above, it is already there, so there is more than that.

In order to debug, we first need to know what actually happens when we run rake test. Minitest has handy rake test:cmd task for this purpose, so let’s run and see what it will tell us:

$ rake test:cmd
ruby -Ilib:test:. -w -e 'require "minitest/autorun"; require "test/unit/pg_user_test.rb";
require "test/unit/owned_note_test.rb"; require "test/test_helper.rb";
require "test/integration/rake_tasks/db/migrations/new_test.rb"'

This is the command that will be executed when you run rake test ². Can you spot at a glance what is wrong there? If you run the task multiple times you may notice that all the testcase files are required in random order and, most importantly, after a test helper (in most cases), which calls SimpleCov.

This happens because Minitest test runner does not distinguish a test helper from regular testcase class files.

Therein lies a problem: Simplecov ³ merely has no chance to fire before the very first line of our application code is loaded. So putting SimpleCov.start at the beginning of the test helper is not the whole story: The test helper itself must also be the first thing to be executed when the test suite runs.

The cure

Fortunately, fixing the issue is quite simple: You just need to tell Minitest to load your test helper first by customizing the framework attribute of the test task, like so:

Minitest::TestTask.create do |t|
  t.framework = %(require "test/test_helper.rb")
end

and make sure your test helper has require 'minitest/autorun' or otherwise the test suite will not run.

Tags: ruby, minitest, simplecov, rake, code coverage, test helper

https://github.com/rails/rails/issues/27088#issuecomment-261385679.