What I have changed my mind about in software development(henrikwarne.com) |
What I have changed my mind about in software development(henrikwarne.com) |
Over the years I have realized that some comments are
needed and useful. These days I add comments when there
is something particularly tricky, either with the
implementation, or in the domain.
Ah, yes. Another convert. There are dozens of us. Dozens!!Nothing makes me feel crazier than trying to get fellow engineers to comment their code. Code alone can only tell you the "what." If the "why" is not obvious, comment it.
Unit testing private methods.
I am always dismayed by engineers who vote a hard and unyielding "no" on this issue.I guess ideally your private methods should not be tested directly. If you work at an ideal shop doing ideal things under ideal conditions, please let me know if you are hiring.
In all or most of these cases the tests for private code will hopefully be somewhat temporary; perhaps think of them as scaffolding used during construction or renovation.
- For example, perhaps you have a good reason to write the private methods first and you would like to make sure they are sound before proceeding.
- Perhaps you have a division of labor due to a time crunch and you are writing the private methods while somebody simultaneously writes the public methods.
- Perhaps you are encountering some thorny preexisting code with no test coverage and you would like to just make sure things work.
- Perhaps the public methods are undergoing a lot of flux and you would like to make sure the private methods do not suffer regressions during this flux
- Sometimes it's just easier to test private methods directly rather than indirectly via a public interface. Maybe this is a code smell, but also maybe you don't have time for a full refactor.
I've actually moved further and further away from unit testing over the years (after being a pretty big TDD fan for a long time). In terms of bang for buck, integration tests across your public API are the best IMO. You're testing how your API is actually used. The problem historically has been that they're fragile to refactoring and difficult to run, but with the right tooling you can get around that.
Unit tests are deeply coupled to the internal structure of your system - refactoring often implies changing unit tests, which opens the door to bugs where the code and test both change to match each other.
As you say, integration tests validate your public API, which from a 'correctness' point of view is really the only thing you care about, not the internal structure of the system. That's why I love integration tests, you can make sweeping refractors without needing to change the tests, because the test will still tell you whether the behaviour of the whole system is correct.
You really shouldn’t care how a function is tested. That there exists a test for that particular function isn’t particularly useful measure of how well the function is tested. So you need a methodology that tells you which of your functions have been covered by your tests. If it’s not covered you should make a judgement about how to get that coverage.
Getting your private methods covered by the public interface is good because it encourages you to write thorough tests for those public methods that covers all the ways the method might be used.
But in my experiences, I've had major issues with a reliance on integration tests. This is almost exclusively in Rails.
- They're slow, because there wasn't an easy way to stub out the slow stuff
- If you stub out the slow stuff now you have no coverage for the stubbed stuff, unless you also have unit tests covering the guts of the app
- When integration tests fail, it can take a lot of work to investigate the cause, as opposed to granular unit tests
I always focus on integration testing. If I have time I'll write unit tests, but there needs to be a good reason. I'll unit test an area that has proven to be buggy even with integration tests, but this is rare.
Finally, some code is just really complex and/or critical. These are good candidates for unit tests.
Unit tests for each unit of functionality, and a small number of integration tests to validate all works well together.
The usually called unit tests indeed should not directly test implementation details, like private methods, but then we have to find a different name for the former because I do think they are useful sometimes
Adding the single biggest reason I have encountered: Because the overall functionality is too complex to test through the API alone, so the alternative is to extract part of the "private" functionality to a separate class that would then be public itself, making functionality public that shouldn't be. By that, the discussion about "testing private methods" ignores the elephant in the room, which is that the language used doesn't have a rich concept of public vs. private in the first place.
As evidence for that I'd like to mention that I never had that problem in Rust, because I can simply extract the private functionality into helper functions within a private sub-module and unit-test them there, without that functionality becoming public to the rest of the code.
When your solution to not wanting to write tests for important behavior is to trick yourself into wanting to by adding extra layers of code for no other reason, maybe the rule stopping you from doing it was wrong.
Ideally tests should be run in a specific order, from most low level to most high level. So tiny basic functions first, then functions that use those functions, then functions that use THOSE functions, etc until you have end-to-end tests.
Unfortunately I know of no testing system that has such a hierarchy.
But yeah, then you've changed it from private to public.
Why you shouldn't test private methods: Because the implementation might change, obsoleting the tests. But everything might change (except the end-user requirements, haha), so by this argument you shouldn't test anything except what is in the spec/what the user will see.
The only benefit of not testing private methods, is the same benefit that not testing anything else brings.
Test private methods when you have sufficient behavior in them that it's worth testing in isolation.
And that was ok because everyone was happy to have a lot of shit written fast.
I guess maybe what I'm trying to say is that shitty tests are sometimes worse than no tests.
However, I do agree that maybe there is a time during which it's OK to just not have tests. If you are just spitballing it, prototyping, code jamming, maybe even blasting out an MVP for alpha or beta testers. Sure.
Part of mastering a craft is making reasoned decisions about breaking rules.
I write the documentation that I would like to see myself.
But isn't that the common view? I think most programmers would agree that the "why" deserves documenting. The problem is that in the heat of the moment, most forget.
I’ve been happy testing private behaviour with tests
directly alongside the code.
I'm intrigued but I don't understand. What does this look like? What language is this?"Comment the how, not the why."
When you read programming books, like "Learn Language XYZ" books, there are generally no comments whatsoever because there is no need. Which makes sense in the scope of such books, but I think it accidentally sets a precedent for eschewing comments in the minds of many.
But.
Out in the "real world?"
I'd say 90-95% of coders don't comment a damn thing. At least that has been my experience working in the industry since the 90s.
Some coders are in the obnoxious and toxic "code should be self-explanatory" camp, which is objectively dumb. Code can't explain intent, such as business rules or weird shit you're doing to work around weird shit in external libraries or APIs or hardware bugs.
A greater number of coders are of the mindset that inline comments are bad and that things should be explained in the commit message and/or pull request. This is more noble, but I think it is not nearly as practical as inline comments for a variety of reasons.
Many coders also believe that comments make code harder to read. I find this baffling and dumb. Get a bigger monitor or get an editor that lets you collapse/hide comments with a keystroke.
Not sure how many times I’ve been looking into a new performance issue as we grow, and come across one of my own comments.
Sort of falls into the Good/Fast/Cheap trifecta. When you have to do the Fast/Cheap bit, note when you know it’s not optimal so you can identify it quickly when you or someone else comes back to it.
This is because I define “good code” as simple code. If code gets to the point that it needs to be explained with comments, then there’s a good chance there is opportunity for simplification.
In other words, if I can’t read code and follow what it’s doing — or why it’s doing it — then I won’t ask them to add comments, but to instead refactor/simplify the code. (This is assuming the code can’t be made more readable with simple name changes, which a lot of times is all that is needed.)
Yes, there are times when the “why”, even with simple code, needs to be explained. Comments in this case are fine. But ideally this is the exception and not the norm.
My day job is mostly PHP (yeah yeah, I know) and the general attitude towards Xdebug in the community seems to lie somewhere between apathy and active avoidance.
To me, a debugger is such an invaluable tool not only for identification of bugs but also for familiarising oneself with new code. The value of being able to not only trace execution from start to finish but also to modify state in a running program is immense, and gives me a small taste of what Lisp folks rave about.
Littering print statements everywhere feels like banging rocks together when there’s a perfectly good lighter in your pocket.
Comments are needed where there is more to the code than just reading it. Where there is some external reason WHY. Where it is written in a specific way for non-obvious reasons. Where there are pitfalls and dangers in changing the code. Where this specific approach is the result of fixing some problem and if you change it then you might be reintroducing a problem.
There's lots of reasons to comment your code, but mostly I think code should be the documentation.
The fewer comments the better, because then developers who come later will see comments and thing "this must be important because there is a comment here". Too many comments dilutes the value of comments.
When it really matters I start my comment with a couple of lines like this:
// LISTEN UP!!!
// LISTEN UP!!!
I have to say, it seems strange that the author EVER thought that no comments should ever be needed - that seems like a strange and dogmatic conclusion to have come to.For me, this is the kind of stuff I questioned in my first few years. After I productionalized a few real world systems for a business, the whole “dev street cred” thing lost it’s appeal. It wasn’t about some imaginary “dev purity” thing anymore, it was about being efficient and making sure I was contributing to the bottom line.
Computers aren’t digital pencil and paper. They’re levers for the mind. If the 3 GHz processor just sits there idling, waiting patiently to copy some bytes from one buffer to another, it’s being wasted. It could be checking the syntax, looking up documentation, or chasing memory references in the debugger so I don’t have to.
From my point of view, ChatGPT is simply an anti-spam algorithm which is finally restoring "web search" back to being useful.
The problem is that there is no money in that. So, everybody is trying to apply all the ML/LLM/ChatGPT stuff to absolutely anything in the hopes of making some money.
The article just made me think of some devs that I’ve worked with that were obsessed with particular tooling (insert comment about a hammer and everything being a nail).
I do keep an eye on emacs. It turned into mainly my org-mode editor, but the v29 article that hit the HN front page recently has me curious about trying more dev work with it.
It's on the About page:
"I have been programming professionally for more than 30 years"
> Unit testing private methods.
As someone else pointed out this leads to accidental testing. I'm not a test zealot, I think 100% coverage is a fool's errand, and I think TDD is an abomination, but well-structured, well-thought-out tests can be a game changer when used appropriately. Testing things by accident has inevitably lead me to finishing some piece of work then spending half a day or more tracking down why some test failed inexplicably.
> Using an IDE.
I think a better point here is to get really good with whatever tool you use. If you know every incantation in vim you're going to be amazingly productive. If you know every keyboard shortcut in IntelliJ you'll be as effective, but probably not much more. The person who knows vim or emacs in and out will beat the person clicking around in an IDE every day of the week.
That being said some of it is spot on in my admittedly limited experience (only about 13 years or so, in a handful of industries, never FAANG-level scale). The point about commenting problem areas in the domain has really changed my approach to comments. I don't write any comments about what the code is doing unless it's a "here be dragons, don't change $X unless you're free the rest of the week" kind of warnings. But I comment extensively why the business or regulation requires A or B to happen instead of the more straightforward C.
The ChatGPT bit as well matches my experience. For well-defined things where it's hard for ChatGPT to make the answer up, and easy for you to verify if it does (or at least low-damage), it's worth the $20/mo IMO. I tried using it to learn CDK and while I'm not sure it saved me any time, it did save me from having to trawl through AWS documentation.
For some languages using commercial IDEs is a very smart choice. Refactoring TypeScipt, for example, with Jetbrains IDE is a breeze, and I think it should be available in developers toolkit whatever their preference is.
For some, more dynamic/niche languages (Hello Elixir!), IDEs stand in a way because they take longer to set up and they still don’t produce results as good as glued together scripts and editor macros (side note: macros aren’t only for inserting text, one can pick text under cursor and search for a specific pattern using rg or even send refresh signal to a browser on the other screen).
There are also two other layers that I always mention as arguments against IDEs.
First is that IDEs change often and no matter how much you try your workflows are going to change. It’s hard to get high proficiency when things are changing and having new IDE feature replacing Your Way is a pain and a learning deterrent (thing that I experienced multiple times).
Second is something I call GPS Development. When I work with IDE I tend to not pay attention where am I and where I should be going because hopping navigation is super easy. And then when I am deprived of those tools I’m completely lost and not productive at all. With arguably dumber tools I can open shell and still navigate and edit with whatever editor I have and it still works well. Thing is that with my line of work stuff breaks often and IDEs rightfully decline to work on broken codebase.
My current stance is to use whatever you’re most comfortable with (on a unit by unit basis). Struggling with editor or IDE, even the coolest/smartest one, is going to interrupt your thinking process and cause performance hit much bigger than whatever gains it could ever produce.
As a longtime and ongoing Emacs user... I have to agree.
Sure, Emacs Can Do It [tm?], but having to setup the appropriate tag system, and ensuring it's kept up-to-date with the codebase is a pain.
For example, our codebase is embedded C with #ifdefs depending on which target we're building from. This means that a naive, cscope(regex)-like search will get confused about which is the proper definition for the active target... when it even can find a definition (it's shocking how often it fails for no clear reason).
So I turned to RTags, which meant having to generate compile-commands.json, which... I'll stop here. Suffice to say setup wasn't trivial, and for some reason it eventually broke and I never bothered again.
Rinse and repeat for every other language.
Nowadays with the popularity of treesitter and LSP even small-community command line text editors can have most of what IDEs do built-in.
What if I told you that you don't need private methods?
This is one of my favs that is all over out codebase.
private float chanceOfRain = 0.1f
public float GetChanceOfRain() {
return chanceOfRain;
}
Perhaps somebody can explain it too me because I'm too afraid to ask.You can get your Emacs to act like Jetbrains in a number of ways, but that can sometimes end up being quite complicated. I very much like the experience of opening any Jetbrains IDE and having this working relatively easily compared to Emacs.
All that being said, investing in Emacs gives you benefits you won't find anywhere else.
For example, I was able to make my Emacs find by reference and find by definition functionality work well and exactly the way I wanted it to.
I basically set it up to fall back on resources if the previous one didn't return anything. It went from Language Server Protocol -> CTags -> Regex with rg.
It was also cool to pick multiple sources and priority order them for suggestions. Mine, from highest to lowest priority, were: Language Server Protocol -> local buffer matches -> open buffer matches (or something like that, it's been a while since I touched my config).
This made it so that I'd get suggestions from my LSP for code, but if I were typing a comment and repeating a word I've used before then LSP would come up empty but Emacs would be auto-completing that word for me.
That's one of the main reasons I picked up Emacs back in the day -- it provided a nice integration with gdb.
The distinction between an IDE and a text editor now mainly comes down to the configuration time, where both (neo)vim and Emacs support the classical IDE features as long as you configure them to.
The whole point is ignoring the dogmatism of our profession full of various shamans with magic formulas to solve everything from scrum to xp to tdd to rust to functional programming to ood etc.
Then it’s always the same shit, but it’s your fault because you didnt do this or that
It reminds me of something…
Was also sceptical about ChatGPT and changed my mind like OP. I was less pragmatic on this one and brought ChatGPT over to Emacs https://github.com/xenodium/chatgpt-shell. Pretty happy with the result so far.
VS Code and other setups make those opt in.
Meaning, when you see a bunch of typos and really poorly written code you know it is from one of the DIY setup people.
So then you have to move all those checks to the next stages like SCM hooks and CI.
Having to start a heavy, proprietary IDE to build code and other assets is a royal pain in the hole.
Debuggers provide much more information, but stopping and stepping through the code is much more time intensive than printing out variables at various points and running the function and doing a quick double check that they're correct.
It also allows you to check the state of the program earlier for problems that may only emerge after the fact. Instead of having to stop what I'm doing and retry and step through with the debugger, I can just look back at the log and see what went wrong where.
I think both debug logs and debuggers have valuable use cases.
Debuggers are basically useless for both of those imo.
Debugger break points introduce synchronous behavior into your async application, which basically forces race conditions to fail when debugging.
And I'll admit to not being a debugger expert but I don't know of any that can watch multiple executibles at once.
(IMHO no programming language should add features which make interactive debugging harder, it's never worth it in the long run)
Debuggers tend to be more useful in cases of corrupted state that isn't the result of broken control flow logic. These are much rarer, usually the product of something like broken bit-twiddling logic. And these days that kind of code is rarely written manually. When I write a non-trivial data container, I usually provide a standard public method that does a deep inspection of the integrity and consistency of the internal state. These are much too expensive to use as a typical assertion, but turning this on usually isolates the problem so well that a debugger isn't required to figure out what went wrong.
I used to use debuggers a lot more than I do now because the nature of the typical bug has changed. The types of bugs are also much less diverse than they used to be.
I'd add many people don't work in languages with good debuggers. I work in TS and the debugging async node code is an absolutely garbage experience in many cases.
If I want to check that the dependency injection engine creates & disposes correctly while navigating the app, I only need printing.
If I want to intercept some process and check value on the fly, or even change it, debugger is the way to go.
It's kinda ironic that you're discarding the matches because you have a good lighter.
Also "modern" debuggers ("modern" as in "this century") offer a lot more than just simple on/off breakpoints: conditional breakpoints, data read/write breakpoints (e.g. who wrote this broken piece of data), "tracepoints" which don't stop execution but instead log out a message or run some scripted action, etc etc...
A good debugger is essentially an interactive program state explorer and visualizer, ideally it would be fully integrated with the actual development process (e.g. edit, compile, run, break somewhere to inspect program state, tweak some code while program is suspended, rewind a program state a bit and continue running with the modified code, rinse, repeat...).
I can't even remember when was the last time I was on a project where debugging would've been even a bit doable. Maybe, just maybe, during development when it's running on my computer.
But it worked on my computer, that's why it ended up in production =)
With proper logging (or just print statements) I can go in after the fact and narrow down what happened when it didn't work in a specific situation.
From your comment I feel like I am missing better ways to debug.
Either stop the program snag the state of a variable and go. Or set a sample point where when it gets hit the debugger takes a sample of local state and continues. It aught to be able to do that within a few hundred clock cycles.
Myself I use error/debug printf, a command line interface, debugger, and sometimes an oscilloscope.
With error/debug printf I leave them in and just turn them on/off with a compiler switch. Over time they tend to evolve into something that provides good info on what's going on.
The command line interface allows me to inspect the program state on the fly and run tests on the fly.
The debugger allows me to do random look sees of what the program state is and follow the call tree.
Oscilloscope allows me to inspect/debug/verify timing issues.
My suggestion to you is implement a command line interface to you program so you can interact and inspect your program while it's running. It's really not hard at all. Implement commands as a table. { "command", handler, extra_data}
I’m really not familiar with Java but given PhpStorm supports concurrent debugging sessions I assume IntelliJ does too.
Because logging provides a different experience and can be sometimes much faster. I can run the code once and then analyze all the collected information later. The log allows me to go back and forth very fast, or very quickly see that e.g. a 150th iteration went sideways even if I don't exactly know what to expect. Another area where logging is tremendously helpful is debugging concurrency related bugs.
There’s a neat trick I like to do when I’m trying to triage a bug that only shows up with some input data. I’ll run the passing test and the failing test with detailed logging, then diff the logs to understand how the code is behaving differently in each case.
I have an expectation of state, the execution can be checked against that expectation via observation of the output (which usually has serialized state). It's not a fancy True False or checkmark, but it's an assertion (which constitutes a test of an expected state) nonetheless.
From my experience, simply having xdebug enabled comes with a significant performance hit, which is why I generally avoid it unless I really need it.
Every year more good engineers start refusing to interview for most php gigs if it's not FAANG. You're with what's left of the rest
That was a trend many years ago when people started seeing how comments are often out-of-date, sometimes been copied pasted around so many times it's not even saying anything useful about the code next to it, and sometimes the author is just a bad writer and makes it more confusing than the code itself. So people became pissed off and kind of decided that the code needs to be clear enough to be self-documenting: short functions with verbose names, good variable names etc. making comments unnecessary.
I do agree with that, but IMHO there's still a place for comments: as many others are saying: when you need to document why something has been written that way, not just what (which the code should be able to tell by itself)... and I believe OP is also claiming that, and you appear to be missing this context.
Distressingly common, though. I'd say most of my last few positions have been at places where "self-documenting code" was the mantra even though it was blatantly clear that the code and what it was doing was far too complex for that to work.
I’ve noticed that different language have various ideas about this - like
Closure would be close to a fetish to build abstractions so that the actual code that does “stuff” just explains the business process, and the rest are just libraries.
Or golang goes in the other extreme where there it is idiomatic to have as little abstraction as possible, writing up everything as you go.
Haven’t coded much in either so this is just a beginner’s observation.
> it seems strange that the author EVER thought that no comments should ever be needed - that seems like a strange and dogmatic conclusion to have come to.
But just before that you say:
> There's lots of reasons to comment your code, but mostly I think code should be the documentation.
Exactly that's why it's not at all strange. As you yourself write, code should be the documentation. Need to add comments => your code is unclear, and unclear code => bad code, and so instead of writing comments your time is better spent improving the code.
The reality of course is that things need to be done and there's no time to fix all layers and have perfectly organized and readable code and therefore clarifying comments are needed.
So no need to shame the author, it's just a typical progression from youthful maximalism to more mature shades of grey thinking.
E.g. developers might forget to update comments when they fix a bug.
There are also the "// set b to 0" brigade, which is an exaggeration but makes the point that commenting the painfully obvious is a trap too many fall into.
I think there was also an argument that methods called "setWidgetsToMaximumSoThatStartOfDayChecksWorkOnWeekends()" were a Good Thing, and definitively Self Documenting, a viewpoint which seems (thankfully) to be losing popularity.
These days, I'm of the opinion that you should test at the highest level possible (if testing a web service, for example, at the level of the actual public API to the service), thoroughly define the behaviours expected for those APIs, and leave the insides of the service entirely untested, unless you have _very_ good reason to. (Critical code, payment code, hard-to-test code, etc.) This ensures you have the important behaviours tested, but leaves you free to modify implementation details as necessary without needing to alter or rewrite entire swathes of test code that suddenly starts failing because you decided to break up or consolidate an inner method.
If you disagree, maybe an illustrative example would help. I couldn't think of one where I want to test a private method in detail that is not worth exposing.
https://learn.microsoft.com/en-us/dotnet/api/system.runtime....
<InternalsVisibleTo Include="assemblyname" />
Which means you can do stuff with msbuild props, like auto including a `$(assemblyname).Tests` into your projects.One of the lessons I've learned is that very little needs to be private, and there's a lot of advantage of not having things private.
Putting that in a clearly defined namespace allows access when needed, for testing or workarounds, while conveying the needed "if you use this you're on your own".
There is “public” within a module, and “public” for other modules. This is not on a per method basis, though.
And you can make a method “package-private”, and you can then test it right a way.
Class Access Modifiers were for controlling property and method inheritance, which has been bastardized into "CAMs tell you the API", because it's convenient. An object's API can be defined by an interface (following traditional OOP conventions), allowing CAMs to be used for their original orthogonal purpose. In practice, developers don't want to make an interface when they can slap a CAM on it and say "good enough". Some language don't even support interfaces, pushing the practice. So here we are with the wrong abstraction.
You also can simulate it in JavaScript by using a closure and a separate package for your "private" functions. By mixing the "private" code into the closure you can make it private in usage but still available to unit tests.
What exactly is being referenced here?
I'm intrigued. TDD may bring way more pomp and circumstance than it deserves, but when you get right down to it all it says is:
1. Write a test first so that its failure state is proven.
2. Test behaviour, not implementation.
I have certainly been caught in haste writing a test after the fact, messing it up – in a way that saw it pass, and then later realized that it wasn't actually testing anything. #1 solves a real problem.
Which must mean the abomination is the behaviour testing? Do you really care about the implementation, though? Surely that can be considered throwaway? You are not going to gain anything asserting that you used bubble sort and have that assertion fail when you update your code to use quick sort.
This is useful for to me only when fixing existing code but not when writing new code. Before I write the implementation, it is obvious the test would fail. By not having to code the test first, I have more flexibility on changing the API if needed.
(1) is the distinguishing characteristic of TDD, but it is not an end in itself, it's a mere way to achieve (2). You write the test after you have the API design, but before you have the implementation, and thus supposedly decouple your test from the implementation, since you don't have the implementation yet.
This approach never sat right with me.
* It makes design iteration more costly: if you want to change the API design as you're implementing the thing, now you have to change a bunch of tests to use the new API, and that's when you don't even have the implementation to test yet! This makes you subconsciously prefer quick and dirty design patches instead of making bigger but better changes to the design. So you're statistically trading potentially better design for potentially better tests. I'd rather have the former than the latter, because it's much more insidious.
* Just like TDD encourages writing tests for behavior instead of implementation, it also encourages relying on tests as the sole arbiter of code correctness. The supposed better quality of tests is offset by over-reliance on them, including by not adding tests to check the weak points of your implementation (because you already wrote all the necessary tests before you came up with it... right?). I've worked with some TDD-loving developers who didn't even feel the need to run the code they wrote on their dev box, in the application context. Tests pass, therefore straight to production. The bugs caused by this attitude offset all other advantages of TDD.
* TDD pretty much relies on 100% unit test coverage, because since you (supposedly) don't think about what the implementation will be, you don't know in which methods the bugs are likely to be. This causes you to over-invest in writing unit tests, but writing and maintaining tests is not free. The risk of bugs is not the same in all of the code, and same for the cost of bugs. Not to mention that some of the code is better tested at a higher level with integration tests or end to end tests, but IME nobody takes TDD there, probably because it's too annoying.
* Overall, TDD seems like an overly elaborate way to proactively "gotcha" yourself with various unhappy-path edge cases before even thinking about the implementation – thinking that would, in fact, expose more of such edge cases. I find that instead of this weird backwards overhead-heavy process, the usual development process works just fine. Start with the big picture, and leave messing with small details for last. Whereas TDD requires you to bring up all the small details / edge cases upfront (otherwise it's indifferent from regular testing).
Like with most ideologies, there are some good high level principles to take away from TDD, but at the end of the day, most ideologies like this are designed to replace unreliable thinking with reliable process. And while that process may indeed reliably produce some kind of consistent output, it's not necessarily better than what you'd have without this process, or with a different process.
I used to unit test all the components individually. But these days I test with most of the componentry wired up, only mocking the i/o layers. I end up writing fewer tests, they have been less brittle, and refactoring is easier.
If all the public interface test cases pass, isn't that enough?
Sometimes there are well-defined processes for performing a task, and that task is performed in only one place in the system. Therefore the details of the process can be kept class private inside of the only consumer. That doesn't mean that the processes should never be tested. If the task is cumbersome to set up or runs slowly then there is good reason to test the internal parts of the process which can be tested with dozens, hundreds or even thousands of permutations of input data cheaply and efficiently. Always relying on the large-scale tests to hit every combination of inputs for a well understood subroutine can be inefficient.
You could make the argument that this well-understood process could be broken out into its own class/package/module and tested with its own public interface, but if there really is only one consumer then that's kind of a strange trade-off to make in many cases.
But even deeper, it is "accidental" because you are not testing contracts on the public APIs, you're testing the private functions through a layer higher. When the implementation changes, you have coupled your tests so tightly to the implementation that they are useless for regression testing. If not, then you actually aren't testing the private function at all, you're just hitting some parts by coincidence.
When I see tests that are orders of magnitude larger than the units they’re testing and contain mocked references that are used 7 stack frames deep I know the author had a fundamental misunderstanding of how to write helpful and maintainable tests.
He was right to change his mind about the other things.
I was also skeptical about remote working, and while I think it is worse in most ways than office working, it's not a lot worse, and the lack of commute is sooo big a benefit that overall it's totally viable.
When I did C++ programming, I would test via public interfaces. If it didn't seem sufficient, or got messy, the reason would always be "This is a big class that is doing a lot of stuff." I would then identify all the things the class was doing, make classes for each one, and have instances of them in the original class as private members.
This way I could test the individual classes easily, and still test the (formerly) big class using only the public interface (with appropriate mocking as needed).
The main thing I learned from unit testing is to pay little attention to the word unit. Insisting on testing the private methods after the exercise above is usually a symptom of aiming for a level of intellectual purity that does not benefit the code.
(Of course, a better solution may be not to use OO to begin with, but that's a whole other discussion).
I worked in a place that disallowed friend functions and classes generally, but explicitly for unit testing. I never wrote another private function. Instead, there was an explosion at the tiny-free-function-that-could-have-been-a-private-function factory, and the parts ended up everywhere.
So you throw away smarter tools like IDEs because they do not understand your code when it's broken, but then you start using dumber tools that NEVER understand your code?
I would say an IDE can still do anything your dumber tools can (grep, dumb find-replace) even when your code is broken, but when it's fully working it gives you so many superpowers it's incredible you would not want that.
I had 2 use cases lately - one was to investigate usage of all function bodies that used specific library for the last 6 months. JetBrains IDE has semantic search but it requires a lot of fiddling (syntax feels very foreign, might be Groovy thing or their own DSL and doesn’t span across multiple commits.
Second was to navigate through a custom pre-transpiration layer, which JetBrains could pick up around 30% of the times.
Those are two cases solved quickly with list refiltering (first one took 15 minutes, second took less than 3). And they were portable, as sharing was a simple copy & paste.
The gist of it is that dumb and smart is marketing thing. ripgrep is dumb tool just as fzf is, but they can be made into a very smart setup.
IDEs can make high general work impact but from my experience suck badly when you enter road less traveled.
You can have access to the same navigation tricks as VS Code in any editor that supports it.
If you just prefer IDEs or you have some other interesting tools you want then fair enough, but navigation? That shouldn't be a motivator any more.
I "love" btw those code bases rich with tests but 99% of the time not finding a bug, but breaking on the tiniest detail change.. not supporting refactoring like it should be but making it slow and everyone wanting not to refactor because of the crap test rat tail.
These days (at least the way we do it) the integration test code gets refactored alongside the rest of the code, and compile-time errors catch 90% of issues.
That's part 1 of what sold me on integration tests. Part 2 is the ability to simply and efficiently spin up a DB per test, removing the need for mocking the persistence layer.
Slowness is only a problem when it's a problem, and can sometimes be solved with better tooling. I'd feel more confident in a system with 1000 integration tests that take 2 hours to run over one with 10,000 unit tests that take 10 minutes to run.
On your last point, this is positive and negative. Yes, it can be harder to find the issue, but IME integration tests actually find the issues that will cause things to break in production, and it's easier to find out why a test is failing than why an error is being logged.
The one place I still use unit tests is around calculation algorithms where you want to push a lot of different data through. You could do it with an integration test, but the test run overhead of say 100 parameterized integration tests is far too high. In these cases the algorithm will still be exercised by an integration test, but the nitty-gritty will be done in a unit test.
Setup and teardown can be an issue, but we've found a good balance. We spin up a clean DB for each test in less than a second. Running out full suite does take a couple of hours, but that's what CI servers are for.
I just can't see a genuine reason to not use a real IDE and debugger. You can even fall back on the same tools you had before with them! Using a debugger doesn't mean you can't also use print statements, but if I could only have one I'd pick the debugger every time.
The most obvious case are API docs which includes internal APIs (means: any public method, in my codebases). Comments are much faster to read and understand than the source code, can be quickly pulled up by using the right hotkey in your IDE, and can be browsed via generate API doc pages in a browser. I don't want to read your function to figure out what it does, I don't want to guess how I'm supposed to use it from the name. I want proper docs.
The second most obvious case is where the reason for a piece of code isn't obvious. Typically this means bug fixes or workarounds, but can also be performance optimizations. If something isn't explained by a comment then sooner or later it's going to get accidentally changed. Unit tests aren't a complete replacement because you don't necessarily know what tests are for what lines of code, and often it's hard to unit test an obscure bug fix anyway (e.g. workarounds for OS level problems).
Personally I can't imagine coding in clojure without scope capture
But I also might not be running one of the services in my personal dev environment.
Further, there are cases where variables are optimized out of builds so you have to add code to ensure they stick around (in which case you may as well print). Again, this applies to iOS development. Maybe others.
> suck badly when you enter road less traveled.
It seems you just decided that's the case without any sort of rationale behind it. Please give a real example I can try instead of hand waving.
What this means is that you can have tests for implementation details, that are cordoned off from your main test suite (which defines your code's real requirements and public API). The benefit is that if you change implementations and suddenly a function is no longer needed, you can just delete the test code right there in the same file, so refactoring is very slightly easier and more obvious. You can also decide whether they should be run during your build process or just at dev time.
I deal with this kind of bs every day in a large oss C++ codebase - some style guide at Google says everything needs to by default go into an anonymous namespace and so you can't reuse anything that the original braingenius deemed useful. Of course this means very few things are actually decomposed/composable. I have lately resorted to forking and carrying patches that expose all the functionality in the anon namespaces.
I've worked on enough codebases that were made extremely fragile by "smart" people fiddling with (or worse, just relying on) notionally private variables, so I think encapsulation is pretty great. OTOH, I've worked on some fairly large scale js/ts where encapsulation is pretty loose, and 95% of the time it's not an issue.
The reason the "chanceOfRain" example is so revealing if this is that there's no reason to treat data like that as if it's an aspect of some entity with its own intimate knowledge. Why have this theoretical class wrapping around weather info as if we must ask a meteorologist? What's wrong with chanceOfRain simply being a function with weather data passed to it, or even just a piece of data that's already been computed by something? The latter options undoubtedly would result in less code and an identical outcome of data at-rest.
What's seemingly wrong with it is this idea you are referring to, which is that the authors of code must know better than the end-developers using said code. This mostly erroneous way of thinking is often justified as a means of preventing other developers from wasting their time doing something "incorrect", but the more distal reason is actually to prevent the code authors from having their time wasted answering questions by other developers encountering bugs doing things those authors don't approve of. Just like DRM, it only serves the authors and punishes fair use, while the "pirate" (in this case the pissed-off developer) finds a way to circumvent the system. If a developer is determined enough, they can write patches to get the behavior they want from the code the author attempted to lock away; in many cases, this is the most appropriate engineering decision since submitting a pull request is usually futile and inviting one's self into a debate with some rando.
There's of course other problems with private members on classes. In many languages, private members on a superclass are not available to the subclass extending it.
The worst part of all of this is how common it is for professional developers to subject their colleagues to this poppycock. It's one thing to try and prevent third-parties from doing "the wrong thing", but it baffles me how developers believe in making access to the "intimate" data highly difficult for other developers working on the same codebase. What the purple f***k? That data wouldn't be "intimate" if developers didn't blindly follow the theology of OOP; the only reason any data can be "intimate" to an object is if that object is thought of as a thing with behavior rather than as a shape of data. Someone may of course argue that encapsulation prevents entanglement, but it actually doesn't; entanglement is based on behavior, not data. At the end of the day, data is data, and your program is millions of bits in memory or storage, none of which have a necessary metaphysical relationship with one another. Behavior, on the other hand, is dependent on other behavior. You can hide away all your instance-specific data, but that will not* stop you from writing code that is highly interdependent. In other words, it doesn't make that much of a difference where you get "chanceOfRain" from, because entanglement doesn't stem from using the wrong data but by making things highly interdependent and not separating concerns. In other words, the argument that private members prevent entanglement is crap. At best, it discourages causing unexpected behavior and limits unexpected breakage, but even that doesn't always work given how common it is for breaking changes to be made to minor or even fix version changes.
In my opinion, data should rarely if ever be made private, and the only thing that might make sense to ever be made private is functions. Note that this suggests the exact opposite of "chanceOfRain" example. Obviously, not all behavior necessarily needs to be exported from a module, but just what is expected to be relevant to the end-developer. The beauty behind this approach is that it becomes a lot easier for an end-developer to patch the code or even just copy the code for functions they don't have explicit access to. Even better if the functions are pure* functions that don't rely on the state of module-level or global data structures.
So many problems would be solved in general if developers would knock it off with OOP and especially privatization. These are legitimate tools to have in programming, but their typical usage is a result of mass psychosis.
That's how I develop in general: a "component" does not exist because it has multiple-clients, but because it is a conceptual piece of logic that makes sense to document and test in isolation. It allows to define what is the public API of this component and what isn't. This is how software scales and stays maintainable over time IMO.
I often find that when I have a very complicated structure with lots of behavior, there’s some other cohesive unit of functionality living inside. That part is often private, but extracting it into a separate structure with public interfaces makes complete sense. From the perspective of consumers of the original class, all of this is still implementation detail that can’t be directly accessed. But there’s nothing wrong about accessing that underlying bit directly through its own public interface.
Every line of code will last some amount of time. Some lines will be maintained in production for decades. And others will be thrown out in a few weeks. For example, if I’m mocking up a UI, the css I write won’t outlast the mock-up. Or doing a game jam. Or prototyping a different way to structure some code.
Unit tests are useful for long lived code but they slow down your capacity to do scrappy iterations. Whether you need to prioritise short term velocity or long term reliability depends on the needs of the project you’re working on.
It always amuses me when HN users play Top Trumps with age. Reminds me of the "32 year veteran distinguished engineer level at FAANG" [1].
It's like whoever's the oldest wins. Bonus points if you mention you're a parent and lack the free time to use the more pernickety tools.
Though if I had to rate them I would put JetBrains IDEA at the top because the refactoring capabilities are so good.
And anyway any project that uses issues tracking also has a lot of context in the issue tracking - by your point of view the developer writing the piece of code that needs commenting should somehow cram the whole issue inside the codebase. We are already using external system anyway - there is no harm in putting information there - the system won't be complete without it anyway.
In my experience (and I have been on any kind of failed at any scale) there is no good way to manage information and context - just slightly less terrible ones.
And yes - I do exactly think that the developer should cram the whole issue in the codebase! Those are your acceptance tests, or explicit representations of use-cases or strategies, or service layers, or at the very least some comments lamenting what would otherwise be inscrutable. If your codebase doesn't have a first-class way of describing what it does and why, I don't think it's well-factored.
I totally agree that it's easy to mess up, I just continue to think this terrible workflow of externalising key information about code and why it exists is fundamentally defeatist. I don't think we should settle for it.
I agree in principle that the metadata and code should be linked and easily pop in view on demand. But that is not how the tooling currently works and we find only crutches.
This is cool paradigm that could make amazing startup. Add in some LLM for buzzword to hook investors. And actually LLM that can trawl all the current sources of knowledge and spitting a one paragraph to glance what and why is going on could even work.
But we can't do it while constraining to text files. We need the equivalent of hypertext for code ... and it will be mess in the end.
For now, I imagine that advertisers are currently trying to find some way to adapt their platforms to have an LLM spit out their product via some paid prompt injection that a publisher can sell them.
2. Microservices is a team separation technique, allowing different team to live in isolation, sharing only API documentation. It's the same as regular web services, except within a single organization instead of across organizations. If you happen to have access to the code (not a guarantee!), are you really going to be able to jump into a codebase you're not familiar with to start adding print lines with more expediency than other possible approaches?
Seriously asking; I am well versed in everything gdb, but I often revert to logging and "import pdb; pdb.breakpoint()" when I have to debug Python.
How does this show you "issues in the code base's architecture?"
In the end you'll have an implementation of the interface with quite complex logic in the methods' implementation to handle differing returns, which will couple your mock implementation to the test cases setup, or you'll reimplement a mocking framework.
I prefer to use a mocking framework to deliver stuff instead of stabbing myself to keep purity of interface-based APIs.
perhaps relatedly, i like to build my code from the bottom up, and make sure each layer is solid so that i can use it to construct the next layer. there is often not a public api at all until i am well into the project, so for me that would involve writing fair bit of code with no tests at all simply because it was all private code.
i do use the "public methods on private helper objects" pattern a lot, but there is a fair mix of times where it's the best way to write the code and times where it's just to keep the test framework happy.
Eg: I’ve done some deep algorithmic work on CRDTs lately and my code has a lot of internal parts which are all quite fiddly to implement correctly. For example, I’m using a custom btree with some unusual features as an internal data structure. Btrees are famously difficult to implement correctly, so my btree has both unit tests and fuzz tests to make sure it does what I expect. Having that test suite makes integration tests easier to triage, since I know any failures in my integration tests probably don’t come from the btree. And some btree code paths are probably rarely or never executed by my integration tests. But I still want that code to be correct. Testing it in isolation is the best way.
They’re a lot of other small pieces that I test like this - like saving & loading code, my graph traversal utility code (for comparing versions semantically) and so on. Low level unit testing can find bugs in utility modules while you write the module itself, not later (when you start using the module in some other code).
I go even further - for the browser level tests, avoid using “testid” or css classes or anything that is “implementation”, but rely in your test on solely things that the user can read / interact with.
So don’t “Press button with id “generate”, but the button that says “save” inside the content element titled “generation”.
This way any refactoring work would not require test changes (as it should) and any change of the visible ui / workflow to the user would require an adjustment to the test.
This is a style that I learned from ruby’s integration testing framework “capybara”, and have been replicating it wherever I can since.
A nice bonus is that if you switch rendering technologies, you can reuse the tests (like react native for example).
> This way any refactoring work would not require test changes
This is the purpose of a testid, your letting people know removing it can/will break tests.
(Especially e2e tests that might not only be ran by you/ might not live alongside the code)
> and any change of the visible ui / workflow to the user would require an adjustment to the test.
I really don't follow this reasoning; why would I want my tests to fail when the Accept button is renamed to Agree?
> Python has this (_ access)
Bypassing CAMs is useful for testing. Python has a mechanism, which is used for that^...regardless of the intent of the feature.
^Underscores are a partially enforced convention for private methods. If your import stripped your access to an attribute, you can still access via __getattribute__() which reaches into the hierarchy. There are other tricks as well. You might call this a special method lookup.
You may have a unit test that makes assertions about logs though? I would definitely expect somebody that isn't a junior programmer to understand this.
I don't think your explanation is compelling.
> a log is just some subsystem of your overall program that prints some output with no pass/fail condition.
The pass fail is interpreted. A unit test result is interpreted, be it with a big green/red or an exception or any other way you wish to express it.
Running something manually and checking the logs in an environment, when there is a reported issue, is modus operandi of every developer today. Did A happen and B happen and what did C say before exit? A looks good (because it has an expected state), etc.
> I would definitely expect somebody that isn't a junior programmer to understand this.
I didn't say logs were a replacement for unit tests (that's a previous poster's strawman). Regardless, they are a form of testing, because we use them to observe state after execution. Manual testing is testing, manual testing and observation of logs, is testing.
Executing a suite requires a manual interaction (ie > run suite or > make build -> runs suite or > edit cron -> runs suite). In a way, all testing is still manual, but test suites allow us to scale testing, which logs do not (1million log files is impractical to use as testing output). That's why unit test frameworks are a good thing.
This can also be a great way to write your unit tests. Once you like what your code does in some situation, you can copy paste the input and expected output into a unit test to make sure you don’t break that correct behaviour later.
When there's too much noise it becomes harder to see what's happening.
You can turn down the log level but then you loose that visibility altogether.
More simply, it takes how the macro economy scales software (and, really, most everything) and tries to apply the same processes in the micro.
I would say "Observe the state of an execution stack". Tomato tomato.
> which has a very specific meaning
Unit testing is a methodology, not a specific thing.
How you observe it, be it through a pass fail indicator or a "This looks right", is immaterial. Splitting hairs on the output methodology does not change the utility.
> If your import stripped your access to an attribute
Python doesn't have class access modifiers, and in particular import does not strip access to attributes in Python. There are two ways Python modifies behaviour in the presence of leading underscores:
1. If there is a module attribute (i.e. a variable in a module) starting with an underscore, it will not be imported when doing `import * from some_module` -- which is not recommended anyway. Importing `some_module` directly will give normal access to the attribute, as `some_module._foo`.
2. If a class or instance attribute's name begins with a double underscore, it will not be accessible directly; however its name is simply mungled at compile time:
> class Foo:
> __foobar = "__foobar"
>
> def __init__(self):
> self.__bar = "__bar"
> self.__baz__ = "__baz__"
>
> Foo.__foobar
AttributeError: 'Foo' object has no attribute '__foobar'
> foo = Foo()
> foo.__bar
AttributeError: 'Foo' object has no attribute '__bar'
> Foo._Foo__foobar
'__foobar'
> foo._Foo__bar
'__bar'
> # Note that does not hold for attributes both starting and ending in double underscores
> # (so-called 'dunder' attributes):
foo.__baz__
'__baz__'
You are correct that underscores are a convention, but in Python one simply cannot count on the kind of access control that exists in other languages, especially when writing code that is supposed to be executed in unknown contexts (i.e. a library). When writing tests, one presumably has full control over both the code and the execution context, so this kinds of tricks can be used relatively safely; that being said, test runners like pytest tend to provide their own abstractions and tools around those tricks anyway.Ironically, there's a story on the frontpage of HN right now, regarding attribute handling in python, which may be of interest: https://lwn.net/SubscriberLink/943619/eaa8a4496fcba1fd/
I.e., assuming my_module has function foo, instead of:
from my_module import *
foo()
Use this: import my_module
my_module.foo()
Or this: import my_module as mm
mm.foo()Is the "only way" not property-based testing (and maybe fuzz testing)? If that doesn't get you there, it is likely that the API is poorly designed.
Some function behavior is intentionally and intrinsically tied to temporal access patterns in the API usage. Unless you can simulate a broad cross-section of real-world runtime API access timing patterns with your testing framework, you won't test all of the code paths. You often see this test problem with scheduler-like functions, where the correct function behavior varies based on internal resource pressures that are an interaction between temporal access patterns of the API and the runtime environment. It is a single function, self-contained, and quite simple, maybe not more than 100 LoC, but test environments are so sterile that usually only a single code path is actually used no matter what you throw at the API.
Some functions have critical code paths that dynamically switch strategies to mitigate when certain types of contention or resource starvation conditions are detected internally. These can be extremely rare cases across the set of possible inputs, such that fuzzing is unlikely to trigger them, or the set of inputs that can trigger them is dependent on exogenous environmental details e.g. the machine where the test is run.
As a fun side-effect, sometimes these functions do not have deterministic result. Getting the same result out of the function all the time does not imply correctness, you also have to know why you got the result you did.
All of the above does not apply to writing business logic in Java or similar. But for high-performance and/or high-reliability systems software, these cases come up often enough that it is a well-understood testing problem. Even if you expose all of the internal implementation detail to the unit tests it is not always possible to reliably trigger all the code paths from an arbitrary test absent purpose-built test tooling.
i am currently marking all functions as public for simplicity, but i feel like it's a failing of elixir and/or its test framework that i cannot say "private, but an associated test module should be able to see them", and once the code is done i will be exploring some of the third-party solutions people have come up with to hack around the issue. (the other option, of course, is to have a new module that just contains a couple of public methods, and regard the entire implementation module as a private module with public functions)
I'm not sure why all testing frameworks don't have this.
One issue that leads to private methods in Java is the lack of any way to extend the set of operations on built-in classes: a problem that Kotlin and C# solve with extension methods.