Comment by danjl
11 hours ago
Why just unit tests? Integration tests seem much more valuable as documentation of what the users will do in the app. Unit tests have limited benefits overall, and add a bunch of support time, slowing down development. If you have good (90%+) coverage just from integration tests, you are likely doing 90%+ coverage of the unit tests at the same time, without the extra effort or support burden. You can use the same reasoning to describe the benefits for understanding the code, you get a clear understanding of the important usage cases, plus you get the unit-level "documentation" for free.
Your point is valid, and some of the dialog in the replies to your comment is also valid. So, I'm just responding to the root of the dialog. What architectures are you working with that suggest higher integration test strategies?
I'd suggest that the balance between Unit Test(s) and Integration Test(s) is a trade-off and depends on the architecture/shape of the System Under Test.
Example: I agree with your assertion that I can get "90%+ coverage" of Units at an integration test layer. However, the underlying system would suggest if I would guide my teams to follow this pattern. In my current stack, the number of faulty service boundaries means that, while an integration test will provide good coverage, the overhead of debugging the root cause of an integration failure creates a significant burden. So, I recommend more unit testing, as the failing behaviors can be identified directly.
And, if I were working at a company with better underlying architecture and service boundaries, I'd be pointing them toward a higher rate of integration testing.
So, re: Kent Dodds "we write tests for confidence and understanding." What layer we write tests at for confidence and understanding really depends on the underlying architectures.
unit vs integration tests is not an either/or. you need both, and in appropriate coverage amounts.
a common way to think about this is called the "test pyramid" - unit tests at the base, supporting integration tests that are farther up the pyramid. [0]
roughly speaking, the X-axis of the pyramid is number of test cases, the Y-axis is number of dependencies / things that can cause a test to fail.
as you travel up the Y-axis, you get more "lifelike" in your testing...but you also generally increase the time & complexity it takes to find the root-cause of a test failure.
many times I've had to troubleshoot a failure in an integration test that is trying to test subsystem A, and it turns out the failure was caused by unrelated flakiness in subsystem B. it's good to find that flakiness...but it's also important to be able to push that testing "down the pyramid" and add a unit test of subsystem B to prevent the flakiness from reoccurring, and to point directly at the problem if it does.
> Unit tests have limited benefits overall, and add a bunch of support time, slowing down development
unit tests, _when done poorly_, have limited benefits, require additional maintenance, and slow down development.
integration tests can also have limited benefits, require additional maintenance, and slow down development time, _when done poorly_.
testing in general, _when done well_, increases development velocity and improves product quality in a way that completely justifies the maintenance burden of the additional code.
0: https://martinfowler.com/articles/practical-test-pyramid.htm...
> unit vs integration tests is not an either/or. you need both, and in appropriate coverage amounts.
Agreed. But I also agree with the commenter that for documentation purposes, integration tests are an order of magnitude more useful.
> a common way to think about this is called the "test pyramid" - unit tests at the base, supporting integration tests that are farther up the pyramid.
I used to be a believer in that pyramid, but my experience has shown me it depends on the project. Wherever it's feasible (i.e. doesn't involve long test times), I've found integration tests to be far more useful than unit tests. I've had experiences where I'd do a project and have really high unit test coverage, only to unveil fairly trivial bugs. The reverse hasn't happened - if I start a project with solid integration tests, I almost never encounter trivial bugs.
Generally, I now write integration tests and mock away time consuming/resource heavy parts (e.g. network calls, DB calls, etc). Better for documentation. Better for testing.
If you look for edge cases in integration tests, you will have a combinatorial explosion of integration tests and you will be adding much more work. Unit tests save time, not lose it.
I make this part of my filtering potential companies to work with now. I can't believe how often people avoid doing unit tests.
That's funny, since I wouldn't code at a place that mandates unit tests. Sure, they have a very minor role, in very specific cases, but I'd say 90% of projects can get 90% of the benefits by writing only integration tests with 90% coverage. If you'd like a more in-depth discussion of why integration testing is better: https://kentcdodds.com/blog/write-tests
I think unit testing if you're testing in a blackbox manner. Whitebox unit testing tends to be very fragile and nowhere near as valuable as an integration test.