The Design of Software is a Thing Apart(pathsensitive.com) |
The Design of Software is a Thing Apart(pathsensitive.com) |
Documentation also (can) tell you why the code is a certain way. The code itself can only answer "what" and "how" questions.
The simplest case to show this is a function with two possible implementations, one simple but buggy in a subtle way and one more complicated but correct. If you don't explain in documentation (e.g. comments) why you went the more complicated route, someone might come along and "simplify" things to incorrectness, and the best case is they'll rediscover what you already knew in the first place, and fix their own mistake, wasting time in the process.
Some might claim unit tests will solve this, but I don't think that's true. All they can tell you is that something is wrong, they can't impart a solid understanding of why it is wrong. They're just a fail-safe.
You start saying you did X because of Y, and Y is weird because of Z, and so X is the way it is because you can’t change Z... hold on. Why can’t I change Z? I can totally change Z.
Documentation is just the rubber duck trick, but in writing and without looking like a crazy person.
Also, writing it down lets someone else, later, take the role of the duck and benefit from your explanation to yourself.
I've done this to myself. It sucks. Revisiting years old code is often like reading something someone else entirely wrote, and you can be tempted to think when looking at an overly complex solution that you were just confused when you wrote it and it's easily simplified (which can be true! We hopefully grow and become better as time goes on), instead of the fact that you're missing the extra complexity of the problem which is just out of sight.
Every step of the process seemed like a local improvement, and yet I ended up where I started. It was like the programming equivalent of the Escher staircase: https://i.ytimg.com/vi/UCtTbdWdyOs/hqdefault.jpg.
Yes. Tests will solve this. Your point is perfect for tests.
If another experienced coder cannot comprehend from the tests why something is wrong, then improve the tests. Use any mix of literate programming, semantic names, domain driven design, test doubles, custom matchers, dependency injections, and the like.
If you can point to a specific example of your statement, i.e. a complex method that you feel can be explained in documentation yet not in tests, I'm happy to take a crack at writing the tests.
Sometimes you just have to pick the right tool for the job, and sometimes that tool is prose. I think if you get too stuck on using one tool (e.g. unit tests), you sometimes get to the point where you start thinking that anything that can't be done with that tool isn't worth doing, which is also wrong.
OTOH, system tests provide a realm where external implicit/explicit requirements may actually be validated. Perhaps.
// This implementation is unnatural but it is needed in order to mitigate a hardware bug on TI Sitara AM437x, see http://some.url
(that's an obvious one; but there are plenty of other cases where documentation is easier than a test)
Combining the two is good, but let's not act like the tests themselves immediately solve the problem.
import cPickle
def transform(data):
with open('my.pkl', 'r') as f:
model = cPickle.load(f)
return model.predict(data) float FastInvSqrt(float x) {
float xhalf = 0.5f * x;
int i = *(int*)&x; // evil floating point bit level hacking
i = 0x5f3759df - (i >> 1); // what the fuck?
x = *(float*)&i;
x = x*(1.5f-(xhalf*x*x));
return x;
}
The only way to test an expected input/output pair is to run the input through that function. If you test that, you're just testing that the function never changes. What if the magic number changed several times during development, do you recalculate all the tests?
You could create the tests to be within a certain tolerance of the number. Well how do you stop a programmer from replacing it with
return 1.0/sqrt(x);
Here's a commented version of the same function from betterexplained.com.
float InvSqrt(float x){
float xhalf = 0.5f * x;
int i = *(int*)&x; // store floating-point bits in integer
i = 0x5f3759df - (i >> 1); // initial guess for Newton's method
x = *(float*)&i; // convert new bits into float
x = x*(1.5f - xhalf*x*x); // One round of Newton's method
return x;
}
I actually just found this article [0] where someone is trying to find the original author of that function, and no one on the Quake 3 team can remember who wrote it, or why it was slightly different than other versions of the FastInvSqrt they had written.
> which actually is doing a floating point computation in integer - it took a long time to figure out how and why this works, and I can't remember the details anymore
This made me chuckle. The person eventually tracked down as closest to having written the original thing had to rederive how the function works the first time, and can't remember exactly how it works now.
I think the answer is both tests and documentation. Sometimes you do need both. Sometimes you don't, but the person after you will.
It matters more when designing libraries/frameworks than one-off apps.
Switching to a new framework/platform/language at the point the one you were on before finally matured enough that it was hard to ignore the need for good design doesn't actually help. you'll still be back eventually.
"So you update the code, a test fails, and you think “'Oh. One of the details changed.'"
Some of the concerns they raise about writing tests are covered by Uncle Bob here: http://blog.cleancoder.com/uncle-bob/2017/10/03/TestContrava... and here: http://blog.cleancoder.com/uncle-bob/2016/03/19/GivingUpOnTD...
From my own limited experience it can make explaining a program to someone new almost trivial. You just use the various flows defined as almost visual guides to what is happening. I don't want to say FBP is a silver bullet, but I think it points to the idea that it is possible capture much more of the theory and design of the program in the code.
Basically our philosophy is this: a small system like a booking system which gets designed with service-design, and developed by one guy won’t really need to be altered much before it’s end of life.
We document how it interfaces with our other systems, and the as-is + to-be parts of the business that it changes, but beyond that we basically build it to become obsolete.
The reason behind this was actually IoT. We’ve installed sensors in things like trash cans to tell us when they are full. Roads to tell us when they are freezing. Pressure wells to tell us where we have a leak (saves us millions each year btw). And stuff like that.
When we were doing this, our approach was “how do we maintain these things?”. But the truth is, a municipal trash can has a shorter lifespan than the IoT censor, so we simply don’t maintain them.
This got us thinking about our small scale software, which is typically web-apps, because we can’t rightly install/manage 350 different programs on 7000 user PCs. Anyway, when we look at the lifespan of these, they don’t last more than a few years before their tech and often their entire purpose is obsolete. They very often only serve a single or maybe two or three purposes, so if they fail it’s blatantly obvious what went wrong.
So we’ve stopped worrying about things like automatic testing. It certainly makes sense on systems where “big” and “longevity” are things but it’s also time consuming.
And that's the problem. We need ways to make those higher level designs (~architecture) code.
How do you get the product you want when you don’t know what you want?
Wouldn't it be better to use data abstraction instead of abusing primitive types?
For instance dates are often abstracted as a Date type instead of directly manipulating a bare int or long, which can be used internally to encode a date.
So, age, which isn't an int conceptually (should age^3 be a legal operation on an age?), could be modelled with an Age type. This, on top of preventing nonsense operations, also allows automatic invariant checking (age > 0), and to encapsulate representation (for instance changing it from an int representing the current age to a date of birth).
Would be better than
return x >= ASCII_A;
surely. ASCII_A could be set incorrectly, or have a dumb type, and is more verbose anyway. By using the character directly, the code speaks its purpose.
Fatal mistake? Really? An unrecoverable failure?
So, none of the software I've written in the last decade worked, despite all evidence to the contrary?
Right.
Sometimes it gets worse still: you can have different theories according to (a) scientists doing basic research into physics or human perception/cognition, (b) computer science researchers inventing publishable papers/demos, (c) product managers or others making executive product decisions about what to implement, (d) low-level programmers doing the implementation, (e) user interface designers, (f) instructors and documentation authors, (h) marketers, (h) users of the software, and finally (i) the code itself.
Unless a critical proportion of the people in various stages of the process have a reasonable cross-disciplinary understanding and effective communication skills, models tend to diverge and software and its use go to shit.
There are some great comments about this buried in https://hn.algolia.com/?query=naur%20theory&sort=byPopularit....
The OP makes excellent points concerning the relative independence of design and code in the context of the "extreme programming" paradigm having become very common if not dominant.
-- Linus Torvalds
Isn't this definition circular, using "programmed" in defining "programming"?
[1] You could have probably guessed that the name was going to be "programming", but it might not have been.
I love them, I can express some things very concisely and even clearly. But there's no direct connection to the code and so keeping things synchronized (like keeping comments synchronized with code) is nigh impossible.
We need the details of these higher level models encoded in the language in a way that forces us to keep them synced. Type driven development seems like one possible route for this, and another is integrating the proof languages as is done with tools like Spark (for Ada).
This will reduce the speed of development, in some ways, but hopefully the improvement in reliability and the greater ability to communicate purpose of code along with the code will also improve maintainability and offset the initial lost time.
And by keeping it optional (or parts of it optional) you can choose (has to be a concious choice) to take on the technical debt of not including the proofs or details in your code (like people who choose to leave out various testing methodologies today).
My admittedly very brief experience with formal methods was that they were actually less close to the "design" of the software than the code. So not sure that's a direction that will get us anywhere we need to go.
>in a way that forces us to keep them synced.
Why "synced"? Wouldn't it be better if those higher level designs were actually coded up and simply part of the implementation, but at a level of abstraction that is appropriate to the design.
We used to increase our level of abstraction, but now we appear to have been stuck for the last 30-40 years or so. At least I don't see anything that's as much of a difference to, let's say Smalltalk as Smalltalk is to assembly language.
I think if it took something like JSdoc and have it more teeth you could do something like this in just about any of the dynamically typed languages.
https://fsharpforfunandprofit.com/series/designing-with-type...
> We need model based editing environments that will allow us to have a much richer set of software building blocks.
And looking at rust, I can start to imagine a future where macros are powerful enough to support a lot of declarative coding.
When coding javascript today I write code like:
// can be imported, and api.router() mounted in express
let api = new API(...);
module.exports = api;
api.declare({
method: 'get',
route: '/hello-world',
description: `bla bla bla...`,
scopes: {AnyOf: ['some-permission-string']},
// (more properties)
}, (req, res) => {...});
Declaring JSON + function is super powerful in JS. In rust macros might allow us to make constructs similar to my "API" creator, but with static typing. And who knows maybe macros can expose meta-information to the IDE...
As far as we can tell, the technology that can create a piece of exact code from a vague specification is called strong AI.
Heck, we don't even have a language to describe vague specifications without loss of fidelity. We don't know if such a language can exist.
Of course I could be wrong.
This is not unlike the domains of philosophy, morality, ethics, and law. Attempting to express or enforce philosophy and morality via legalism is an exercise in futility, and even ethics which appears to be on the same level as law actually isn't since the presumption of ethics is behavior even in the absence of a law.
I disagree. ASCII_A speaks it's purpose (we purposefully want an ASCII A stored here). And one can check the constant's definition, and immediately tell if it's correct. E.g.
const ASCII_A = 'A' // correct
const ASCII_A = 'E' // wrong
return x >= ASCII_A
Whereas:
return x >= ‘A’;
So, by those two lines:
const ASCII_A = 'E';
(...)
return x >= ASCII_A;
These line, on the other hand:
return x >= ‘A’;
(If you say it's because it's written twice, well, that's only a valid clue if ASCII_E doesn't happen to be defined too.)
Gets the whole message across in one line, as does using 65 with the comment.
Of course, all of this is likely overkill for your specific example. If I'm writing a to_hex routine, I'm not going to extract those constants as the context & commonplaceness of the algorithm makes it redundant. For the same reason that one might write i++ in a for loop instead of i += ONE. However, extracting inline constants to named variables is frequently something I look out for in code review, especially the more frequently the same constant appears in multiple places, the more difficulty a reader might have trying to understand why that value is the way it is (or if there's any discussion at all), or if it's a value that will potentially change over time. The negative drawbacks of extracting constants is typically minimal & with modern-day refactoring it's a very small ask of the contributor.
> ASCII_A
It comes down to naming and purpose.
The example, ASCII_A, is terrible because it doesn't describe the purpose with its name.
What will end up happening in any large codebase is ASCII_A will get reused in dozens of different places for dozens of different reasons.
If it was named minValidLetterForAlgorithmX it would convey intent and its more likely to be used correctly.
I'm not so sure it's a straw man, I often see defining constants like this cargo culted even if there are only one or two uses. In that case 'A' is great because it's value is right there, I don't have to look at the assignment and then go look up what the actual value is, so it's more readable.
When it's used in several disparate places then ASCII_A is better and your arguments about correctness should take precedence, we sacrifice some readability but it's worth it.
But you’re channeling some crazy madness suggesting that someone would use ‘A’ to mean 65. Shudder. I guess we’ve all seen some horrors over the years.
> ASCII_A (usually spelled just 'A')
Of course, they are not the same thing. In the last 6 months I've worked on a very old system that uses not-quite-ASCII. 'A' was 65 but '#' wasn't 35.
If A signifies something else, use that name; otherwise just use plain 'A': it already gives us as much information as needed, and has one less place where the programmer can screw up.
As an aside, if someone changes the constant value of ‘A’ now, the world will be broken for a while. (But my code would recompile correctly unchanged with the new standard header.)
My experience is that documentation is generally a shorthand word that means non-runnable files that do not automatically get compared to the application source code as it changes.
Of course there are some kinds of blurred lines among tests and documentation, such as Cucumber, Rational, UML, etc.; but that's not what the parent comment was talking about when they described the function with a naive/buggy implementation vs. an enhanced implementation that handles a subtle case.
> but let's not act like the tests themselves immediately solve the problem
I'm saying that yes, the tests do immediately solve the problem in the parent comment's question: a test for the "subtle" case in the parent comment immediately solves the problem of "how do we ensure that a future programmer doesn't write a simplified naive implementation that fails on this subtle case?"
As noted, the comments are for the why, which tests don't tell you without some additional information.
Bigger still, is what happens when a project spills beyond a single repo, but not even Google is that big :) :). Apache projects are good models for that kind of documentation IMO, even though the pages have ugly css.
That's a form of written-in-prose documentation like the OP was arguing for, and unlike writing unit tests. Documentation doesn't have to be in a separate file.
Ultimately you don't, but ASCII_A requires double the intentional actions to name it and have it also be 'A', whereas 'A' vs 'E' or whatever else is a much easier typo.
It's the whole idea behind NOT having magic values in your code. That is, that:
if (temp > 212)
if (temp > WATER_BOILING_TEMP)
WATER_BOILING_TEMP = 275
if (temp > 275)Unless, as I wrote after, you have both ASCII_A and ASCII_E declared, which wouldn't be surprising.
I don't find the "spot the error" argument to be very convincing; I still name stuff, but just for the semantic value.
Or 275°C at around 60 bar.
So for a lexer, you'd create a dummy program with the output of each token on a newline, and then test that the tokenizer's output matches what you expect from the input. But you shouldn't test whether the functions themselves are correct. The tests cases should be designed to throw up any bugs in any of the internal functions, and the system should catch it as defined.
Otherwise you end up documenting what the system currently is, rather than that the goal of the system is met.
A unit test should test a specific unit; commonly a class. You should have unit tests for the interface of that unit (never private or even protected methods). This absolutely does not prevent internal restructuring but it drive you toward seeing your classes as individual service providers with an interface.
That's the point. If the dummy program fails then your implementation is bad.
> Further, if the test fails you now have to try and figure out what component caused the failure.
Have you heard of logging?
> A unit test should test a specific unit; commonly a class. You should have unit tests for the interface of that unit (never private or even protected methods).
This is almost exactly what I suggested. I stated 'dummy programs' for individual tests because it's more modular and closer to actual usage than mocking. If your language has an equivalent, use that.
One way to implement this is by using two methods: one normal and one with the hardware mitigation code, with a dispatch that chooses between them.
This separation of concerns ensures that your normal code runs normally on normal hardware, and your specialized code runs in a specialized way on the buggy hardware.
This separation also makes it much clearer/easier to end-of-life the mitigation code when it's time.
B. Your solution is not obviously better, it's a different trade-off. And it's not immediately apparent to me how exactly you would write the test for the affected hardware, in the first place. What if the hardware bug is extremely hard to reproduce?
A. When I encounter areas that need specialized workarounds (such as a mitigation for a security vuln) then I advocate using two methods: one of the normal condition and one for the specialized workaround. Same reasons as above, i.e. separation of concerns, easier/clearer normal path, easier to end of life the workaround.
B. In my experience tests are better than documentation for any kind of mitigation code for an external dependency bug, because these are unexpected cases, and also the mitigation code is temporary until the external bug is fixed.
Imagine this way: if a team uses just documentation, not tests, then what's your ideal for the team to track when the external bug is fixed, and also phasing out the mitigation code?
But also a workaround, right? Because it's not actually declarative, it's APIs that are made to look declarative-ish.
So there's several layers of mismatch, for example most of the active ingredients being strings, meaning you're coding mostly in the string-language embedded into JS.
We do that a lot.
Probably time to start looking at our workarounds (and Macros are another workaround) and figure out what we are actually trying to do.
I'm not entirely sure what you mean...
In an ideal world "API" would be a keyword, similar to how "class" is a keyword for declaring/defining classes. Example:
API MyApi {
constructor(defaultName) {
this.defaultName = defaultName;
}
/** some description */
method: get
route: /hello-world
scopes: some-permission-string || other-permission-string
{
let name = request.query.name;
if (!name) {
name = this.defaultName;
}
response.send('hello ' + name);
}
}
module.exports = MyApi; // similar to how you would export a class in JS
Whilst my JS code, where I create an API object and call API.declare(...) for each route isn't as neat as having an "API" keyword and code-generation for said keyword, it's pretty close.
I just write the API.declare(...) as something that collects arguments, and then those can be instantiated multiple times... Similar to how a class can be instantiated multiple times.
I do similar things for loading components that depend on each other: declare dependencies, define function for loading using said dependencies -- then have some library code construct a function that loads any desired component with maximum concurrency (by analysing the DAG).
Text is essential for humans and could be a significant part of the interface (both reading and typing).
The problem is more about the underlying unit being text.
The problematic part of text is when we just write it freely and then have to parse it back and make sense of it which has very severe consequences.
This is because in the programming environment world we are forever stuck with a "text editor" mindset.
This means we shift the complexity away from the environment and pass it on to the compiler and the programmer which is not a good place for it to be.
So you can use "nano" to write very complex programs. Some people consider that a benefit, and in one way it is. However it is also simultaneously very problematic. Because it perfectly demonstrates just how far removed the authoring tool is from the problem domain.
The way I see it is that to unlock a new generation of software development experiences there's no way but to accept that we can't forever confine the act of programming to the primitive act of writing text like a book in a text editor.
The environment needs to be much more closely connected to the problem domain so that it can further empower you to do stuff.
I think richer editing, in effect editing the AST, rather than the concrete syntax is interesting.
But I'm not convinced that it's a blocker for making more declarative code. I think macros will get us very very far. The procedural stuff in rust, has me really excited about the future.
The downside of not using text as the underlying unit of truth is that you have to reinvent an IDE, version control, conflict resolution, review tools, etc. That takes a LOT of buy-in.
If you can 90% of syntactic sugar required from macros (and the like), you can work on the actual abstractions / declarations instead of the compiler toolchain.
Just my two cents.
Or just an encoding scheme.
So your solution is to dig through log files to find the problem instead of read the unit test name(s) that demonstrate the failure?
>This is almost exactly what I suggested.
Well, it wasn't clear from your wording. It sounded like you were advocating integration testing instead of unit testing (which is a thing that people commonly do, so that's why I reacted to it).
- not all external bugs are fixed, some are permanent (e.g. the hardware issues).
- tests can tell you when something is wrong, but not when something is working. I'm not really sure how a test could tell you that "external bug is now fixed" - and even in cases where that _might_ work, I'm not sure it's a good idea to use a test for that.
return x >= 'A';
already and only means ascii A. Is there a C compiler anywhere where or likely in future where 'A' in C is NOT ascii A? The comment is redundant if correct, and could be wrong after an edit, so it has no value.
See, here's where you are wrong.
ASCII_A = "A"
alphas = ["Α", "А", "Ꭺ", "ᗅ", "ꓮ", "Ａ", "𐊠", "A", "𝐀", "𝖠", "𝙰", "𝚨", "𝝖"]
for c in alphas:
print c == ASCII_A
False
False
False
False
False
False
False
True
False
False
False
False
False
Using Ruby and its built-in minitest gem:
1. Write a test that does minitest assert_in_epsilon(x,y,e)
2. Write a minitest benchmark test that compares the speed of the fast function with the speed of the naive function.
Notice the big advantage for long term projects: if the hack ever ceases to work then you'll know immediately. This actually happens in practice, such as math hacks that use 32-bit bit shifts that started failing when chip architecture got wider.
> no one on the Quake 3 team can remember who wrote it
Exactly. We have the code file, but not any documentation separate from the code, such as notes, plans, attempts, reasoning, etc.
I agree with you that it can be tested. But it doesn't explain anything about why it works, or what methods the author tried that didn't work as well. If you ever had to make it faster, or make it work better on different hardware, you'd be starting from scratch again.
Optimizations like these a great area for tests because the test files can keep all the various implementations and can benchmark them as you like.
This enables the tests to prove that the a new implementation is indeed optimal over all previous implementations, and continues to be optimal even when there are changes in external dependencies such as hardware, libraries, etc.
> But it doesn't explain anything about why it works
IMHO it does, for all the areas expressed in the original link and the parent comment.
Here are the original link examples:
1. "What [TDD] doesn’t do is get you to separate the code’s goals from its implementation details."
IMHO first write the code's goals as tests, then write the method implementations. The tests may need new kinds of instrumentation, benchmarking, test doubles, etc.
2. "[D]oes the description of what Module 3 does need to be written in terms of the description of what Module 1 does? The answer is: it depends."
IMHO write modules with separation of concerns, and with clear APIs that use API tests. If you want to integrate modules, then write integration tests.
3. "Quick! What does this line of code accomplish? return x >= 65;"
IMHO write code more akin to this pseudocode:
return age >= US_RETIREMENT_AGE
return letter >= ASCII_ASo someone comes along later, look at your test, and wonders: why did you go through all that trouble? You can definitely write tests for a lot of that stuff, but they still don't fluently communicate the why of your choices.
return 1.0f / sqrt(x)
You can also test execution time too, but that's finicky and doesn't help explain how to fix it if you break that test (if there's no accompanying documentation).
Speed is the entire purpose of this method, not the numerical accuracy.
int main() { printf( "%d %d\n", 'A', "A" ); return 0; }
produces: 65 197730221
since the value of string "A" is its base address.