Nicole is a software engineer and writer, and recently wrote about the trade-offs we make when deciding which tests to write and how much testing is enough.

We talk about:

Balancing schedule vs testing
How much testing is the right about of testing
Should code coverage be measured and tracked
Good refactoring can reduce code coverage
Is it worth testing error conditions?
Are rare error codes ok to just monitor?
API drift and autospec
Mitigating risk
Deciding what to test and what not to test
Focus testing on key money-making features
If there's a bug in this part of the code, how much business impact is there?
Performance testing needs to approximately match real world workloads
Cost of a service breaking vs the cost of creating, maintaining, and running tests
Keeping test suites quick to minimize getting distracted

Links:

Too much of a good thing: the trade-off we make with tests
Load testing is hard, and the tools are... not great. But why?
Yet Another Rust Resource (YARR!)
Goodhart's law - "When a measure becomes a target, it ceases to be a good measure"

Sponsored by Mailtrap.io

An Email Delivery Platform that developers love.
An email-sending solution with industry-best analytics, SMTP, and email API, SDKs for major programming languages, and 24/7 human support.
Try for Free at MAILTRAP.IO

Sponsored by PyCharm Pro

Use code PYTEST for 20% off PyCharm Professional at jetbrains.com/pycharm
Now with Full Line Code Completion
See how easy it is to run pytest from PyCharm at pythontest.com/pycharm

The Complete pytest Course

218: Balancing test coverage with test costs - Nicole Tietz-Sokolskaya 28:48 Share

Test & Code

Shownotes Transcript

218: Balancing test coverage with test costs - Nicole Tietz-Sokolskaya