Use canary tests for dependencies

This is part of the Semicolon&Sons Code Diary - consisting of lessons learned on the job. You're in the testing category.

Last Updated: 2024-03-02

For years I used a gem (Ruby library) called attribute_normalizer that took user input (e.g. "description" or "name") and formatted it in certain ways (e.g. removing trailing whitespace). It worked via an initializer where I listed all the attributes it was supposed to normalize.

This worked for years.

Then they removed the feature without any warning. Only six months later, did I notice my data was looking odd in places.

I could have caught this mistake by having had a policy in place that for any library I use, I add at least one "canary test". The idea is that this test would execute a telling example of the gem's action. Once this test fails, the entire subsystem should be suspect and all instances deemed incorrect.

it 'normalizes name using attribute_normlizer (canary test for whole codebase)' do = ' Rome '
  expect( eq('Rome')


For important dependencies, it's worth putting a test in place to ensure functionality keeps working between versions.