Comment by 0xbadcafebee
11 days ago
> If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests,
Actually that's wrong too. The production database will be different than the "testing Postgres instance", leading to bugs.
It turns out that whatever testing solution you use, if it's not the actual production instance and you're not using real production data, there will be bugs. Even then there's still bugs.
This is the simple truth: you can't catch all the bugs. Just put in Good Enough testing for what you're doing and what you need, and get on with life. Otherwise you will spend 99% of your time just on testing.
This doesn't mean the solution of testing with a separate Postgres instance is wrong.
> The production database will be different than the "testing Postgres instance", leading to bugs.
It never happened to me to be honest. This reads an argument for "if you can’t do perfect, just do it badly" but it’s nonsense. Running tests against a local Postgres instance with the same major.minor version and same extensions as your prod instance WILL work.
And testing your storage layer against the database is probably the most reliable safety net you can add to an app.
> Running tests against a local Postgres instance with the same major.minor version and same extensions as your prod instance WILL work.
A team I worked with recently said the same thing. But, as I predicted, they ran into bugs because the CloudSQL Postgres was different than their Dockerized Postgres, even though it was the same core version.
There will always be testing problems you can't anticipate. Especially with systems that are not your own code. Just be ready to adapt your testing when it doesn't work as expected, and don't invest too much in the testing if it's not worth it.
Which bugs?
I’ve been ready for 5 years as it’s the duration I’ve been testing my storage layers with 90% integration tests (the 10% being the hard to reproduce error cases, these tests have low value but are easy to test with mocks so I still test them). The only issue I’ve encountered was with time zones (shocking), and it made me ensure I got full control of the time zones in my app, in my deployment, in my local Postgres and in my prod Postgres, so net benefit in the end.
2 replies →