def rename_user(user_id: str, name: str) -> User | Exception:
# Consume the function
user = get_user(user_id)
if isinstance(user, Exception):
return user
user.name = name
return user
Perspective: I've written primarily in Python for over 20 years, as well as coded extensively in many other languages (C, Objective-C, JavaScript, Java, Bash) and have familiarity with a bunch more (Go, Ruby, Gradle, TCL, Lua, Kotlin, C++).
In practice, exception handling, declared exceptions or not, just isn't that big of a problem. When I'm writing code, I have to reason about it, and exceptions and return/error values are just one aspect of that reasoning.
I think there's still an argument here that this is just a symptom of trying to force functional paradigms on python, which is happening more and more, which I like but am sometimes worried about how pythonic it is... For a one odd script it works well, but when I want to maintain something and have to manage many dependencies and onboard new devs it becomes quite challenging.
You shouldn't catch interrupt signals regardless of whether you're throwing or returning errors. This is why your error classes should extend Exception and not BaseException. The interrupt errors (e.g. KeyboardInterrupt) extend BaseException
As such, writing idiomatic code for the language of the SDK really should be a concern, rather than implementing some idiosyncratic half-baked version of Go or Rust error handling in Python.
on the other hand, if someone's writing non-pythonic python because pythonic code doesn't work well for their program, it might be a program they shouldn't be writing in python, which in my experience happens way more often than it should.
> It's impossible to know which line might throw an error without reading the functions themselves... and the functions those functions call... and the functions those functions call. Some thorough engineers may document thrown errors but documentation is untested and therefore untrustworthy. Java is a little better because it forces you to declare uncaught errors in method signatures.
The author's proposal doesn't change this as much as they think it does. You still don't know what type of errors a function will throw without inspecting the code and thus how to resolve them. Unless, you have a blanket switch for every possible error anything could return which is the very thing they are complaining about.
Some of us used to wrap php "errors" to convert them into exceptions. Then I switched to Python and was pleased to see the pointless distinction between errors and exceptions gone... not gonna go backwards on this.
Did exactly that back in the day, this comparison is flawed: PHP errors (and warnings, notices) by default went completely outside the flow of the program, and converting them to exceptions was a means of forcing developers to deck with them right away, sometimes at all.
We didn't need many wrappers given the nature of our SDK, but some programs will need many wrappers and that could get unwieldy
Based on your decision to write wrappers internally, presumably you'd argue that the sensible thing for them to do is to write wrappers either for your SDK or for every other library to get back to a consistent style and less mental overhead.
A `Result` can contain either a non-error value (Result::Ok) or and error value (Result::Err), never both.
But, more generously: why not simply return an error, and use isinstance(val, Error) for error handling? Making objects and calling functions is quite costly, and that can largely be avoided.
I don’t get it. Languages that use exceptions for all kinds of errors will also use exceptions for routine errors that happen as a matter of course—the happy path is not the overwhelmingly most common branch, and errors are not exceptional. In turn not zero-cost for all but the exceptional case.
Expected errors - like "user not found" - should use a value instead of exception. In Python, you can use sentinel objects, or tuples, or None.. lots of options really. Occasionally there is a good reason to use exceptions for flow control even for know errors (various hooks come to mind), but this should be pretty rare compared to number of places that can raise an unexpected errors.
Unexpected errors should not be caught at all, except maybe at the very top level (to record them and return 500 to user). The examples in post, where you catch exception and re-raise Exception back are terrible - what's the point of them? There is no extra clarity, just verbosity. I would defect them in any code review.
Coarse-grained error handling is great as long as exceptions are meaningful and stack traces are good, which is the usual case in python. All that matters for unexpected errors is that (1) user sees an error message and (2) the real cause is recorded for later analysis. A single top-level try block does both.
That's why exceptions are called exceptions, not errors. If a routine called openFile() can't open a file, that's a pretty exceptional situation, and it's up to the caller to decide whether the exact reason is an error in their case.
The exception object is already a value that can have not only a message text, but also any data members, so why reinvent the wheel with sentinel objects, tuples, etc.?
> The examples in post, where you catch exception and re-raise Exception back are terrible - what's the point of them? There is no extra clarity, just verbosity. I would defect them in any code review.
Typically, you re-raise an exception after adding some content to it. This may be less important in Python, which gives you a great stack trace, but in a language like C++, the lack of context information makes the exception basically useless.
Engineering is about tradeoffs.
There is more advantage in doing the accepted python solution (exceptions) than inventing your own (which you claim to be better, but I personally think is worse). If you are developing in a team, stick to established conventions and spend your time focusing on your business problem.
> Regardless of the specific approach, returning errors as values makes us consider all of the places an error could occur. The error scenarios become self-documenting and more thoroughly handled.
This is so true. Most Java exception handling is a try/catch around about 20 lines of code, with superficial logging/rethrowing and no context about exactly what was being done to what when the exception occurred, just a filename/line# and a probably cryptic error message.
In Go, best practice is something like:
bytes, err := os.ReadFile("myfile.json")
if err != nil {
return nil, fmt.Errorf("reading file %s: %v", "myfile.json", err)
}
var data map[string]any
err = json.Unmarshal(bytes, &data)
if err != nil {
return fmt.Errorf("unpacking json from file %s: %v", "myfile.json", err)
}
If you are going to replicate this with exceptions, it would require much more boilerplate, as his example demonstrates, which is ironic given that is the charge levelled at Go.
Ok(2) + Ok(3) = Ok(5)
Ok(2) + Err() = Err()
Err() + Ok(3) = Err()There's a pretty decent explanation in the readme. If you're more interested in Monads, I am not sure I'd cover that in a HN comment very well, but I would encourage you to take a look.
(A “Result” is Rust-speak for a sum type which is either A or B, where (say) B is conventionally an error)
While I like Rust — and Result works really well with `?` — it doesn't actually look like that's the best pattern for Python?
try:
thing.set_name("Doodad")
except Exception as err:
raise Exception(f"failed to set name: {err}") from err
How is this safer than the original?
If the caller wasn't catching exceptions thrown by this function before, it's not catching exceptions thrown by this function now. What is being gained by catching an exception to do nothing but throw another exception?
This feels like a strawman.
def get_user(user_id: str) -> User:
rows = users.find(user_id=user_id)
if not rows:
raise Exception("user not found")
return rows[0]
def rename_user(user_id: str, name: str) -> User:
user = get_user(user_id)
user.name = name
return user, NoneI couldn't agree more! When you throw exceptions it's unclear where the control flow will go
if err:
return nil, err
I won’t claim to know what your intent was but to me the brand placement changed your post from a comment to an ad.
tuple[User, None] | tuple[None, Exception]
The original is more like Go approach which is a big flaw with the language.
> Why does the author want to swallow these exceptions?
Sometimes you want to swallow exceptions and sometimes you don't. The examples in the article may be a little contrived, but there are situations where logging an error and continuing is better because it prevents data loss.
> Let them propagate and it's obvious where the issue lies.
If they propagate down the stack then you lose context. You also may not have all the data you need to properly recover (e.g. still write to a table but make the errored field null).
> You still don't know what type of errors a function will throw without inspecting the code and thus how to resolve them.
You're right that we don't know which error occurred with the approach in the article, but you at least know that there could be an error. This is better than try/catch because that doesn't tell you whether an error could happen
You capture `Exception' in your examples which is a serious anti-pattern in nearly all cases. One of the rare places you'd want to do that is at the top level of a service that has to clean up safely before bailing out. None of the contrivances in your example demonstrate that requirement.
If you need to capture an exception, you capture the narrowest one that matches the business/technical requirement you have. The rest should bubble up and get handled by another part of your code or, indeed, cause your app to crash if it's unhandled.
Exceptions don't mean "error" either; they're just exceptions.
> If they propagate down the stack then you lose context
I cannot think of any instances where you 'lose context' by propagating them. You should raise Foo from e if you're re-raising, though, which you clearly did not bother doing. That does preserve context.
Passing an instance of an Exception object around? What is this madness... are you building a debugger? No? A complex framework or tool like pytest where you may want to manipulate the stack frames to benefit the end user? No? Then don't pass exception objects around and raise them somewhere else.
It's really not that simple. Don't forget that the concept of exceptions (and more powerful things, like CL's condition system) were invented because of the problems encountered with errors-as-value approaches.
It's a fundamental trade off without a right answer, and people have been arguing about it now for 50+ years without resolution.
Maybe I’m misunderstanding you, but that’s what the Python stack traceback is for. It works pretty well, I prefer it over JS.
Is this not solved by using raise from?
In particular, the ReadFile example in Python will raise OSError, and those already include filename and error message. So you'd get the same result with 0 extra lines.
For the second example, the json unmarshalling will not auto-add filename, but its easy enough to do using exception chaining:
try:
data = json.loads(bytes)
process_1(data)
process_2(data)
except:
raise Exception(f"Error while parsing file {filename}")
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "somefile.py", line 4, in somefile
Exception: Failed while parsing file.json
(an anecdote: we've had the team which converted the CLI tool from Python to Golang. The first time they ran the tool, it printed a single line:
invalid character '"' after top-level value
I accept it's largely personal preference, but having used both mechanisms for many years, I find Go best practices for error handling are simple and easy to follow, and results in easily maintainable code, compared to exception handling which doesn't really come with a simple set of best practices, meaning it is often badly put together or added as an afterthought.
dtodf, err := os.Open(filepath.Join(filepath.Dir(file), "diffid-to-descriptor"))
if err != nil {
return nil, fmt.Errorf("opening diffid-to-descriptor: %w", err)
}
best practices are nice, but fully automatic is even better. The minimum-effort path in Python produced vastly more useful traces than in Go, and unfortunately too many programmers go mininum-effort path.
[0] https://github.com/ko-build/ko/blob/cfc13deeb6417d7e1582f031...
This can also be done with error passing, and is a design failure there too:
_, ex = perform_batch_operation(on: list_of_data)
if ex:
# Oh no, we don't know which entry in the list failed
# and can't update the appropriate row
# (This, of course, should be handled in
# perform_batch_operation, not bubbled up here)I've also seen APIs that allow returning both error and a value ("something went wrong, but here is a fallback or something") which are of course extremely confusing given the usual style.
We don't return errors in our public methods so our SDK still feels Pythonic. We're debating whether to add `_safe` suffix methods that return errors to give people the option, but for now our library only throws errors to consumers.
You might ask "well what's the point if you still throw errors to consumers?" We feel that forcing our engineers to deal with errors where they can happen is worthwhile. Our team writes a lot of Go and we love how it forces you to think about every spot that can error, so we wanted the same experience in Python
I might express a slight preference that your internal code is idiomatic if I'm going to be poking around in there at some point. But I wouldn't be bikeshedding about your C coding style if you'd provided a Python module written in C, so I don't feel I should have a strong opinion as a user here.