Go 1.21 Release Candidate(go.dev) |
Go 1.21 Release Candidate(go.dev) |
What a mistake.. reserved keywords are words I can no longer use for myself...
Zig does it better by requiring a prefix @ for most their builtin needs
You can continue to declare your own entities with these names.
I don't like this design..
In fact, you can enable warnings/logs that indicate whether code that is affected by the loopvar experiment results in a stack-allocated or heap-allocated loop variable: https://github.com/golang/go/wiki/LoopvarExperiment#can-i-se...
I imagine that the current workarounds for this issue also end up with heap-allocated variables in many cases.
The fine details resemble the analysis of correctness - all the evidence shows people expect per-iteration semantics with considerable frequency, and don’t rely on per-loop semantics with measurable frequency. But it’s impossible to completely automate that assessment. Likewise, it’s impossible to automatically detect code that will spuriously allocate because of the semantic transition.
However, I wonder what it will mean if someone who mostly writes Go will now use another language? Will they be more prone to make that mistake?
After the change, escape analysis figures out if the changed iteration variable actually needs heap allocation; in an internal sample of code that was actually buggy (i.e., biased, guaranteed to have at least one loop like this) for 5/6 of the loops escape analysis decided that heap allocation wasn't needed.
The reason this optimization isn't part of the language change proposal is that escape analysis is "behind the curtain"; ignoring performance, a program should behave the same with or without it, and it is removing heap allocations all over the place already. Escape analysis is also extremely difficult to explain exactly, so you would not want it in the spec, and "make escape analysis better" (that is, change it) is one of the prominent items in the bag of things to do for Go.
in some of my use cases, I need make sure source code is fully protected, neither Node nor Django can do that well, Go will be perfect as it is compiled, however there is nothing like MERN or Django in Go(yet). Another option will be Java, but I do not know Java.
(This is the same behavior as the append built-in function today, for example. These things in Go are _not_ reserved keywords, they are simply global functions that can be overridden at other scopes.)
min and max are common variable names so depending on the version of go and the scope you should expect min and max to mean different things.
No reason these functions couldn’t have been part of the stdlib.
Nice to see their going in a good direction.
package main
import (
"fmt"
"path/filepath"
)
func main() {
filepath := filepath.Dir("./")
//filepath.Dir('./") -> This is now a string. Can't use filepath package anymore
fmt.Println(filepath)
}
you can still allow this, of course, by aliasing the package import
but needing to do this is "terrible"
is that correct?
The only lesson to be learned here is that languages are different. But I think the new Go behaviour is more ergonomic.
Loop variable capture is a foot-gun that in the last six years has cost me about 10-20 hours of my life. So happy to see that go. (Next on my list of foot-guns would be the default infinite network timeouts — in other words, your code works perfectly for 1-N months and then suddenly breaks in production. I always set timeouts now; there’s basically no downside)
Interesting to see them changing course on some fundamental decisions made very early on. The slices *Func() functions use cmp() int instead of less() bool, which is a huge win in my book. Less was the Elegant yet bizarre choice — it often needs to be called twice, and isn’t as composable as cmp.
The slog package is much closer to the ecosystem consensus for logging. It’s very close to Uber’s zap, which we’re using now. The original log package was so minimal as to be basically useless. I wonder why they’re adding this now.
I’ve already written most of what’s in the slices and maps packages, but it’ll be nice to have blessed versions of those that have gone through much more API design rigor. I’ll be able to delete several hundred lines across our codebase.
What’s next? An http server that doesn’t force you to write huge amounts of boilerplate? Syntactic sugar for if err != nil? A blessed version of testify/assert? Maybe not, but I’m happy about these new additions.
Beware of a naive http.Client{Timeout: ...} when downloading large payloads. I've always set http.Client.Timeout since day one with Go due to prior experience, but was bitten once when writing an updater downloading large binaries, since the Timeout is for the entire request start to finish. In those scenarios what you actually want is a connect timeout, TLS handshake timeout, read timeout, etc.
https://blog.cloudflare.com/the-complete-guide-to-golang-net... does a good job explaining how to set proper timeouts, except there's a small problem: it constructs an http.Transport from scratch; you should probably clone http.DefaultTransport and modify the dialer and various timeouts from there instead.
In general, setting timeouts beyond the entire request timeout is pretty involved and not very well documented. Wish that can be improved.
I just started my first Go tutorials this week. One of them was go.dev's Writing Web Applications [0]. I was actually struck by the lack of boilerplate (compared to frameworks I've used in Java/Python/etc.) involved.
I get that it's a toy example, but do you know of any better write-ups on what a production Go web server in industry looks like?
You asked for an example, and here is one. This is my side project "ntfy", which runs a web app and API and handles hundreds of thousands of requests a day and thousands of constantly active socket connections. It uses no router framework, and has a modified (enhanced version of the http.HandlerFunc) that can return errors. It also implements a errHTTP error type that allows handler functions to return specific http error codes with log context and error message.
It is far from the most elegant, but to me Go is not about elegance, it's about getting things done.
https://github.com/binwiederhier/ntfy/blob/main/server/serve...
The server runs on https://ntfy.sh, so you can try it out live.
I sort of see what you're saying, but then again, the addition of a couple of small generic packages (slices, map, cmp) and one larger package (log/slog) isn't exactly a huge amount of new surface area. Definitely not as big a qualitative change as generics themselves, which added I think it was about 30% more content to the Go spec.
> The slog package ... I wonder why they’re adding this now.
Because it's very useful to a ton of people, especially in the server/service world where Go is heavily used. To avoid a 3rd party dependency. To provide a common structured logging "backend" interface. See more at https://go.googlesource.com/proposal/+/master/design/56345-s...
I agree we can be enthusiastic, but the Go team is still spending a lot of time getting APIs right, finding solutions that fit well together, and so on. I don't think it's the downward spiral of "let's pull in everything" we've seen in P̶y̶t̶h̶o̶n̶ some other languages.
> the changes demonstrate a transition from the traditional go philosophy of almost fanatical minimalism, to a more utilitarian approach.
This change demonstrate that "to stay as a mainstream" programming language, you can't preach minimalism , has to adopt utilitarian approach.
https://ashishb.net/all/infinite-network-timeouts-in-java-an...
If all goes well, you won't have different libraries using different loggers anymore, in some not too distant future, which should improve easy composability.
Also a bit surprised how fast it was added to the stdlib, but perhaps there was a lot more consensus on the api compared to other golang proposals.
I'm changing logging on the service right now and it just makes sense to use it now, but entire service can't move to pre-release version of go.
I hope the experimental fix makes it into the next version of Go by default.
Does someone knows why Go uses env variables (like GOOS and GOARCH) instead command line arguments?
I cannot even begin to tell you how many different itemInSlice functions I've written over the years.
The WASI preview shows Google is committing engineering resources to WASM, which could grow the community a touch.
For example, I'm working on a custom WASM host (non-browser) and have a tinygo WASM package with import bindings like this:
//go:wasm-module rex
//export wait_for_event
func wait_for_event(timeout_usec uint32, o_event *uint32) bool
When I add Go's desired `//go:wasmimport rex wait_for_event` directive, it complains about the types `*uint32` and `bool` being unsupported. Tinygo supports these types just fine and does what is expected (converting the types to uint32). On the surface, I understand why Go complains about it, but it's such a trivial conversion to have the compiler convert them to `uint32` values without requiring the developer to use unsafe pointer conversion and other tricks.
Hopefully I can find a way to keep both tinyo and Go 1.21rc2 happy with the same codebase going forward and be able to switch between them to evaluate their different strengths and weaknesses.
Zerolog will still be relevant for raw performance (slog is close to zap on perf - doesn’t win benchmarks, doesn’t look out of place either), fewer really need it but some really do.
Personally, I’m most excited about log/slog and the experimental fix to loop variable shadowing. I’ve never worked in a language with a sane logging ecosystem, so I think slog will be a bit personally revolutionary. And the loop fix will allow me to delete a whole region of my brain. Pretty nice.
> New maps package for common operations on maps of any key or element type.
> New cmp package with new utilities for comparing ordered values.
Pun intended? =D
Note that the order mimics variable assignment. You copy an integer with:
var src, dest int
dest = src // dest first, src second
Also really excited to see loop capture variables finally getting sorted out. It is a constant pain point with new devs, and I have no good answer when they ask "but WHY is it like this?"
More information about loop capture here for those interested https://github.com/golang/go/discussions/56010
Because, historically, it's been like that all over, it's not just Go. For example, Python has the same loop variable reuse.
Probably comes from a time when compilers were a lot simpler, and all local variables were allocated stack space for the whole duration of the function call.
max := something()
package main
func main() {
arr := make([]int, 0, 10)
make := 1
arr = append(arr, make)
len := func(arr []int) int { return -1 }
println(len(arr))
// Output: -1
}
Java: We added structured concurrency and virtual threads!
Golang: We added a min function!
Most of the standard lib still doesn't properly support generics, and at this pace, it will be another 5 years at least before it does.
Tbh I don’t see most of the standard lib benefitting from generics. For example, json.Unmarshal wouldn’t be dramatically better with generics — in practice, I rarely see runtime errors where I passed the wrong kind of thing to that function.
I personally love the slow pace of go development. I love that I don’t need to refactor my code every year to take advantage of whatever new hotness they just added. The downside is that stuff that’s annoying now will be annoying forever (like those times when you want a more expressive type system), but I’m willing to live with that.
Doubly-so when `clear` on a map actually seems to follow the convention of removing all contained elements.
I suppose it could have been x = clear(x) or clear(&x), but certainly if you understand Go semantics then seeing any function call do Foo(slice) already signals that the call can't modify the length since there's no return value.
Slowly walking back dogmatic positions is just how the Go team works.
I say this as a person that wrote Go full time for a handful of years.
Those were always bad alternatives to a real design problem, they just didn't have a good alternative to offer at the time.
func clamp(x float64) float64 {
return max(0, min(1, x))
}
template<typename T>
T clamp(T x) {
return std::max<T>(0, std::min<T>(1, x));
}
Obviously these sample functions don't take into account all the intricacies of float min/max functions.
The proposal's real conclusion was "the decision cannot be resolved by empirical data or technical arguments."
Personally I try to avoid using floats for calculations if I can (unless it's obviously warranted), I've encountered far too many foot guns from using them, though honestly the same can be said about integers in some situations too. I wish there was a package like math/big that was more accessible, I find the current interface for it pretty abysmal.
Was that discussion pre-generics?
Most of functions and libraries introduced in Go 1.21 is stuff people already put in community libraries (lodash being probably most popular, despise utterly nonsensical name not relating to anything it does) so it is just potentially cutting extra dependencies for many projects.
Then you'd be even more surprised when you learn that the vast majority of languages do not have standard logging library in core.
Most have one or few common libraries that community developed instead, but they are not in stdlib, and if stdlib has one it's usually very simple one (Go had standard logger interface that was too simple for example)
Most languages have no logging "system" built in at all. Honestly it's really quite rare.
Also, interesting to see assembly again after many years. Haven’t touched that since college during a compilers and assembly course.
Edit: never mind, amd has implemented these “sha-ni” instructions since “Zen” [1]
This could be five years from now. Or maybe never.
This was especially important back before the days of v1/v2/v3/etc of x86_64, since Linux distros distributed binaries compiled for the lowest common denominator. So the only way you got fast SIMD instructions (beyond SSE2) was with a CPUID check and some compiler features that let you build target specific functions. (And I'm not sure what the status is of Linux distros shipping v1/v2/v3/etc binaries.)
The bug doesn't seem to discuss use cases for it either. The most I could find is: https://github.com/golang/go/issues/56351#issuecomment-13326...
Which boils down to "doing what clear(slice) does cannot be implemented efficiently today" but I'm not sure how having an efficient way to do something folks don't want is useful?
There's already a memory clearing optimization in the compiler: https://github.com/golang/go/issues/19266
So yeah I'm not sure under what situations folks will use clear(slice).
const x = min(a, b)
It helps to document intent.
Probably not so useful for min, but it can be more useful for more complex functions.
clear(f)
fmt.Println(len(f))
Maps are kind of like
type map *struct{ len int; ... }
type slice struct{ len int; ... }
I don't think any language has really given us something that is completely intuitive here. Python's semantics with the list type are a constant surprise to newcomers. C++’s semantics surprise newcomers. Rust's semantics surprise newcomers. Surprises all around. The best you can hope for is something that is internally consistent.
The slice in Go is more or less equivalent to &[] in Rust or std::span in C++. The whole idea of passing a pointer by value is key to understanding the semantics of most modern programming languages. Like, is Java pass-by-value or pass-by-reference? You can argue the point, but whatever label you decide is appropriate for Java, it’s useful to think of Java as passing pointers by value. Same with Python, Rust, Go, etc. This is not intuitive for people who are new to programming.
Not really, because they are mutable, they can mutate the underlying memory, and they can re-allocate. They are a weird mix of &mut []/Vec or std::{span,vector}.
In contrast, a Rust &[] can may the underlying storage (if it's an &mut []), but cannot spin out a new storage on its own and start a new life without a backing structure – and I'm not utterly familiar with std::span, but I would wage the semantics are close.
Go slices can, which is why they are always tricky, especially for beginners. Not only does = not really do what is intuitively expected, not only every beginner will be bitten in the ass by forgetting the `x =` in `x = append(x, y)`, but it is impossible, when calling a function expecting a slice, to know if this function only wants a view on some memory or actually expect to modify it; a capital difference that is very clear in Rust or C++ type systems.
My understanding is, to use the Rust/C++ term, slices in Go are owned, but they are not in Rust or C++. That is, they're a pointer + length in the latter two, but a pointer, length, and capacity in Go.
I don't think the implementation details matter to a user of a map or a slice (or an array for that matter) - they're language builtins (as opposed to span, vector and map in c++ which are library types).
Care to elaborate?
s := []string{"hello", "world", "foo", "bar"}
fmt.Println(s) // [hello world foo bar]
s = s[:0]
fmt.Println(s) // []
s = append(s, "XXX")
s = s[:2]
fmt.Println(s) // [XXX world]
This is why "delete(slice, n)" doesn't work and it only operates on maps.
I suppose clear(slice) could allocate a new array, but that's not the same behaviour as clear(map) either, and doesn't really represent the common understanding of "clearing a slice". The only behaviour I can think of that vaguely matches what "clearing a slice" means is what it does now.
I think I understand now why `clear` can't work on slices the way I think it should, but only because slices themselves don't work the way I feel even stronger that they should.
Dev: Can we have a function to clear a map?
Go: No, it's easy enough to write the 5 lines of code to just do it yourself every time.
Dev: Okay, I don't see why I should have to write those 5 lines every time but fine. Isn't looping over everything going to be slower than just… having a function that can empty the internals?
Go: We've implemented a compiler optimization to detect this and rewrite it to the faster code it would have been if it we were to implement it.
Dev: Isn't that… way harder than just writing the method? Anyway, I noticed this solution doesn't actually always work because of this edge case.
Go: Just handle the edge case every time then.
Dev: That's the point. I can't.
Anyhow, this explains it in detail, if you're not already familiar with it: https://go.dev/blog/slices-intro
the context stores request-scoped data, whether or not the logger is a request-scoped value is a grey area
and to reply to sibling comment, opentelemetry is basically a house of antipatterns, definitely do not look to it for guidance
"Look on My Works Ye Mighty and Despair!"
https://github.com/open-telemetry/opentelemetry-collector/tr... -> https://github.com/open-telemetry/opentelemetry-collector-re... ... and then a reasonable person trying to load that mess into their head may ask 'err, what's the difference between go.opentelemetry.io/collector and github.com/open-telemetry/opentelemetry-collector-contrib?'
$ curl -fsS go.opentelemetry.io/collector | grep go-import
<meta name="go-import" content="go.opentelemetry.io/collector git https://github.com/open-telemetry/opentelemetry-collector">
I think the distinction is useful specifically because it explains why Go slices work differently than in at least those two languages.
I have a particular axe to grind when it comes to the word “ownership” of objects in programming. In C++ and Rust there is a very natural sense of ownership in that the owner of an object is who may deallocate the object, and that ownership may be shared with std::shared_ptr<T> in C++ or Rc<T> / Arc<T> in Rust. Ownership is such a useful concept in these languages because it is generally true that somebody must deallocate the object, and it must happen safely.
As a very natural consequence, people who spend long hours working in C++, Rust, C, or other similar languages start to associate, very closely, the notions of ownership and correctness. And indeed, ownership is broadly useful outside C++, Rust, and C. Even in a garbage-collected language like Java or Go, it is generally useful to have clear ownership. You don't modify objects that you don't own, or use objects outside their scope.
But occasionally, you come across a piece of code where ownership gets in the way. Perhaps some garbage-collected algorithm that transforms data with pointers going all over the place. It probably sounds like a mess, but that is not necessarily true either—it can be perfectly good, correct, readable code.
So while ownership is a useful concept for talking about specific pieces of Go code, or specific pieces of Java code, it is not applicable to all Go or Java code, and that’s fine. It’s kind of like talking about code in terms of functions—nearly every language on the planet makes heavy use of functions (or some equivalent), but it’s also true that code does not have to be organized in functions, and you will occasionally see code that does not use functions.
If you talk about how Go slices are tricky for beginners, but you cite C++ as some kind of gold standard against which Go should be compared, then I think you’ve lost the plot—C++’s type system is a complete and utter trash fire for people who are new to programming. Rust, as well, is very difficult for people to get into. Even the Python semantics for lists get people tripped up all the time.
a = [[]] * 5
b = [[] for _ in range(5)]
The nuances of how references and values work is something that you have to work through, and then you have to come to terms with the conventions for the particular language you are using. IMO, Go’s slices are fine… you really just have to be careful about aliasing a slice you don’t own, but then again, that’s true for languages like C++, Python, Java, and C# as well. Rust is the only one that’s really different here.
Being able to change the underlying data is a pretty big difference. Technically, their only solid common point is that they address contiguous spaces in memory.
> you cite C++ as some kind of gold standard
I never did; I highlighted the difference between immutable views vs. whatever Go slices are.
> I think you’ve lost the plot
No need for the snark there.
That’s a very major thing to have in common. Definitely not a minor detail, for sure.
2. Clear
Are the two already covered not enough?
Case in point, clear. Or "typed nils". Or accidentally swallowing errors because you had to handle them manually. Or reimplementing higher-level job control on top of channels every single time.
Can you please explain this?
But as an example, if you wanted to have any sort of higher-level management of goroutines (for example, a bounded number of background workers) you get to rewrite or copy-paste that code every place you want to accomplish that. A library couldn't exist to abstract away the idea of a pool of background workers because it can't know in advance what types you want to send over your channels.
Again, I wouldn't be surprised if post-generics there's a library now to do this for you. But for years if you wanted anything higher level than raw channels, you're basically on your own.
If you see a different context, then you misinterpreted what I wrote.
It is easy—trivial, even—to imagine scenarios where a particular “X is like Y” does not make sense. What you should do, as a reader, is try and understand what the writer means, rather than try to figure out some way to interpret a comment so that it is wrong, in your view.
The easy way out—saying “X is not like Y because of difference Z”—does not meaningfully contribute to the discussion.
a context is created with each request, and destroyed at the end of it
and values stored in a context are accessible only through un-typed, runtime-fallible methods -- not something you want to lean on, if you can avoid it
Are pitfalls ever actually encountered ?
if you pass a logger to a foo as a parameter in the request context, then missing a logger is a run-time error
A pretty apt write up.
But I accept there’s probably not an amount of evidence to change your belief on Go’s dogmatism. And that’s okay! You like a language. That’s great!
Unsurprising conclusion. Enjoy the day!
But what is the SOP / Best Practice here ? Do many libraries have some sort of SetLogger(..) initialization call, so that loggers don't clutter the API ? Or are error returns info-(over-)loaded ?