Block Pull Requests that made tests slower

Block Pull Requests that made tests slower(github.com)

5 points by zupo 3 years ago | 5 comments

bfeynman 3 years ago |

Would've thought for most services you usually just continue to add unit tests and not change/update them, as that is dangerous for backwards compatibility. Time taken for tests has never really been something I've seen anyone care about, since you can upgrade your build hardware. Regardless, this stuff is usually configured to be outputted when you run tests anyway, nice feature to make it easier to see in a more friendly way than just terminal.

latchkey 3 years ago | |

> Time taken for tests has never really been something I've seen anyone care about

https://en.wikipedia.org/wiki/Argument_from_ignorance

It is an absolute quality of life issue for development. The longer tests take to run, the more loathe developers are to write and work with them. The worst is when developers start to add calls to `sleep()` in order to make async tests work properly... it just balloons out of there and becomes an extremely brittle test suite.

bfeynman 3 years ago | | |

Setting aside the egotistical link to misused philosophical point (I specifically did not say no one cares about it, nor did I assert I've never seen someone who does care about it), the major bottleneck for most software has always been dependency resolution and build time (which might include running tests).

HN has been chock full of blogs for years from big and small firms that talk about how they sped up the build process. Updating hardware, compilers, architecture refactoring etc, compilers and adding optimization flags could affect the run time of a test, outside of one that literally has some `sleep(...)` call in it, in which case this report is also fairly useless.

zupo 3 years ago | |

I've seen the following multiple times in the past: 1. something changed in test layer setup that made all tests slower 2. something changed in application code that made all tests slower (!!)

And most of the times, they were avoidable, and fixed after we realized what happened.

zupo 3 years ago |

Unit tests tend to get slower with time, wasting a lot of developer hours. BlueRacer helps you keep them fast by posting a unit test performance summary on every Pull Request.