The Lobster Programming Language(strlen.com) |
The Lobster Programming Language(strlen.com) |
Not a great time for HN exposure, as I just added a lot of new functionality, but none of that is documented yet :) But hey, all publicity is good publicity?
For those interested, some recent things that have been happening:
- Compile time reference counting. This does an analysis similar to the Rust borrow checker to infer lifetimes, but unlike Rust doesn't make the programmer jump through hoops.
- "Inline" structs: structs are allocated in their parent, and come at zero overhead compared to individual variables, which is particularly useful for Lobster's heavy use of 2d & 3d vector types.
- Multi-threading that uses separate memory spaces, meaning individual threads can run at max speed with no GIL overhead etc.
- A WebAssembly backend is being worked on.
- A lot of optimization work to make the language run faster.
For reference, it has been on HN before, as early as 2013 (https://news.ycombinator.com/item?id=5904430), but it has changed a TON since then (e.g. it was a dynamically typed language, now it is statically typed, with flow sensitive type inference).
Is there a comparison of different versions of Lobster somewhere? I'd like to be able to make statements like "My new language XYZ, unlike Lobster, is dynamically typed, resulting in difficulties such as plicplocpluc in developing the IDE" but different versions of Lobster now cover such a large part of the language design space that this is becoming difficult. For example, that statement doesn't make sense with respect to old versions of Lobster, which were also dynamically typed.
Yes, Lobster is meant to be a much more pragmatic language, for people like myself to actually get the job of making a game done :) Other languages of mine are much more research-y. Though I will say I think the type system and lifetime analysis are pretty novel actually.
I'd like to think there's really only one Lobster, while older versions are still recoverable, they for practical purposes don't exist anymore :) Have you seen https://htmlpreview.github.io/?https://raw.githubusercontent... ? That actually contains some history.
So imagine the current implementation does a lifetime analysis somewhat similar to Rust, but then where Rust would error out, this implementation simply inserts a runtime refc increase. This gets rid of most runtime refc overhead without the programmer needing to be clever about it.
Then, I intend to add an optional type annotation that gets you the Rust behavior, for those cases where you want maximum control. I don't think its the right default though.
The algorithm is more lenient than Rust, since I don't have any shared memory concurrency for example.
I intend to write up more thorough details about the algorithm, but again, haven't gotten to it.
It does not handle cycles. If you create cycles that you do not manually break, you'll get a cycle report at program exit that prints the kind of values that weren't reclaimed. You're then expected to fix your code accordingly :)
You could have multiline lambdas with Python's signficant whitespace, the then-BDFL just didn't think it was important enough to accept any of the syntax options proposed for it or to bother finding another option, believing named functions were sufficient for the multiline case.
- Multi-threading that uses separate memory spaces, meaning individual threads can run at max speed with no GIL overhead etc.
- Compile time reference counting. This does an analysis similar to the Rust borrow checker to infer lifetimes, but unlike Rust doesn't make the programmer jump through hoops.
As for the lifetime analysis, see some sibling comments: I need to do a good write-up of the algorithm. I wasn't anticipating this HN post :)
Seems to nail most of the desires of mobile/indie scale development.
You want it shorter? Something like "Get Python-like ease of use together with static typing and speed and built-in game programming functionality" ? That really doesn't do it justice. In the end, like most languages, it is a particular cocktail of trade-offs, and to see whether this suits you, you need see the full list of features.
Maybe you want a list of features with an example for each? It's hard to make it short though, especially the type system.
I wanted to share your language with a game programmer I know but the links to documentation from your homepage look broken.
Essentially https://htmlpreview.github.io/ is dysfunctional. Try accessing the sample links in that project's Github page.
I guess I'll have to change them, though not sure what to.. annoying github doesn't have a way to view markdown pages.
I guess for now, to read the docs online you could read the raw markdown here: https://github.com/aardappel/lobster/tree/master/lobster/doc... Better yet, just download the repo and read the html files in docs/
Flow sensitive type inference looks like exactly what I need.
But yes, I should really write up a more technical description some day.
How does it compare to Dyon?
... unless it's an obituary :)
(an old joke)
Some difference off the top of my head:
- Nim so far has been a GC-ed language (I know manual allocation is possible, and an ownership system is planned, but this was the default for most code so far). Lobster has been reference counted (and now compile time reference counted) from the start. Nims proposed ownership system is pretty cool, but still requires a fair bit of programmer cooperation, in that sense it will sit somewhere between Rust and Lobster in terms of programmer friendlyness.
- Lobster has a type inference algorithm that is able to go further than most languages in terms of allowing code without types to "just work". Nim has nothing even remotely like this.
- Nim is faster. Nim was designed for near native speeds from the start, Lobster started life a very high level language that slowly become more low level. There's nothing that prevents Lobster being as fast as Nim, it just will take a bit more time.
- Nim is more mature, in the sense that it has been around longer, has more libraries and tooling etc, and a larger community.
- Lobster has great support for programming with higher order functions: terse syntax, best possible type inference, zero overhead (compared to manual loops), non-local returns etc.
So in summary I'd choose Lobster over Nim if you care about Lobsters type and memory management or higher order function features, or if you like the idea of a language that makes games/graphics out of the box easier. If you just want to "get the job done", Nim probably is ahead of Lobster at the moment :)
Beyond that and what sibling comment said, the author is known for creating various languages, so he clearly enjoys doing it and is making this language for himself.
I have never understood why there's always somebody complaining that new languages are unnecessary when a new language is shown on HN. I personally love seeing new languages. I also recently went on a language hunt for something that met a specific need for a side project I'm working on and I came away empty handed because I couldn't find something that met my criteria[1], so I'm always on the lookout for something that might.
[1] In case somebody has any suggestions for me, I want something with Clojure-like immutable data structures as a default (or something similar, at least), is native compiled to reasonably efficient and low footprint binaries (but is not as complex as Rust or Haskell), is a functional language (but with prettier syntax than OCaml, which was almost my choice, but when I started reading example code, its a complex mess of symbols <insert perl joke>). And... it needs reasonable library support... (at a minimum I need to be able to respond to HTTP requests, make HTTP requests, parse and generate JSON and talk to postgres, ideally using SQL and not an ORM). Haskell is probably my best bet, but I don't have the energy to learn all of its complex features.
And in this case, this is someone with a 20+ year background of experimenting with new languages, ranging from production-ready (e.g. I used AmigaE a lot back in the 90's) to crazily experimental ones.
Also doing savegames by just serializing the entire scripting VM may work in some cases, but can also be problematic, since you're mixing ephemeral data with gameplay state. Last U3 engine game I worked on had explicit savegame serialization :)
Haxe is an open source toolkit based on a modern, high level, strictly typed programming language, a cross-compiler, a complete cross-platform standard library and ways to access each platform's native capabilities.
what would be the benefit for using Lobster since I can work directly with each OS with a great scripting language built in?
Haxe and Lobster are very different programming languages however, so you may prefer one over the other purely for the language design.
If you like portability, Lobster already runs on 3 desktop OSes, 2 mobile ones, and the web, using either VM, C++ or soon WebAssembly modes. While not as extensive as Haxe it is pretty good :)
There are probably a 1000+ language implementations out there that are faster than Python, most of which are single person efforts.
Lobster is a much more static language, and does inlining of functions, inlining of structs, specialization etc that allow it to remove a lot of runtime cost, such that even the VM is fast.
It also has been in development for about 8 years now. I've been working on compilers in one way or another for almost 30 years.
The cases where it differs from Python are actually for good reason: unlike Python that only has built-in control structures, here any function can look like if/for/while if it takes a lambda body.
That's absolutely fantastic. I've had to do a lot more Python programming for work, and I've been thoroughly disappointed by Python's lack of support for custom blocks/structures like that (contrast with Ruby or Elixir, where any function can take a do ... end, or with Tcl where "blocks" are just strings).
I don't think it's using Python syntax, I think it and Python share some similarities. Thinking about it in Python terms is probably why the changes appear random/different/useless. But in general not being constrained by Python or another language has value.
Are the generators implemented entirely in user space using continuation passing style? And the coroutine function turns it into something pausable?
Edit: I can't wrap my head around how return returns out of all the generators
Now the co-routine functionality takes an existing higher order function, and compiles it such that the call to the function argument instead does a yield. These are really regular co-routines that are implemented by copying stack frames off and onto the stack upon yield and resume. I just thought being able to share code between HOFs and co-routines was cute, since they often express the same patterns. Why would you write an iteration twice?
I tried running the sierpinski example with a pre-compiled Windows .exe that I placed in the lobster root directory, and it gives me this error: "can't open file: stdtype.lobster".
I placed lobster and lobster/modules in my path, and it still throws the same error. I am assuming I need to build the .exe myself due to some incompatibility with the master branch and this older .exe?
I like a self-contained game PL. I use Raylib for that type of experience right now, but I like the concepts here. Thanks!
EDIT: copied modules to root of lobster, and now I get:
"stdtype.lobster(8): error: end of file expected, found: <"
I tried an included test file and a copy/paste of the sierpinski example, and both throw this error.
Not sure about the error involving "<".. are the executable and the data from the same version of Lobster? Open an issue on github or email me for more help :)
var i = find [ 1, 2, 3 ]: _ > r
var i = find ([ 1, 2, 3 ]) a: a > r
Would multiple functions arguments be done just with parentheses, like this?:
var i = find [ 1, 2, 3 ] (: _ > r), (: _ < r)
PS: small detail, but that's got to be the best for-loop I've ever seen.
Multiple function arguments require a keyword in the caller, much like "else" introduces the second function argument in an if-then-else. So if "find" would take two functions, one for even and one for odd list elements, it would look like: "var i = find_even_odd [ 1, 2, 3 ]: _ > r odd: _ > r"
There are no features that prevent it from "approaching" C's speed, merely a question of implementation work. The language started out as very high level, so code generation needs to be improved to not do as many things at runtime as it currently does. The language semantics are totally able to be within 1.5x of C.
The many uses of : syntax are intriguing - lambdas, functions, control, python-style structure, data, globals. I reckon you've established it works in all cases, but I'm not yet familiar enough to see that for myself.
For initial adoption, I wonder if more regular sample code, without the special-case abbreviations, might be more effective? (Followed by the short version.) OTOH, maybe at this early stage it's best to select for developers interested/capable enough to handle it!
EDIT if : is used for both blocks and code structure, could everything be one-line (or is \n significant?) Not great style, but helps in understanding the syntax. Maybe returns and vars need their own lines?
def find(xs, fun): for(xs) x, i: if fun(x): return i return -1
var r = 2 var i = find [ 1, 2, 3 ]: _ > r map(my_array)element:element*2 //lobster
my_array.map{|element|element*2} //ruby
Arrays.stream(my_array).map(element -> element*2).collect(something list something) //java?
Note how if you write this example with multiple statements instead of "elements * 2", suddenly the Lobster syntax looks a lot more consistent, and the Ruby example doesn't look that great as it looks completely different from the built-in for loop.
That said, the Ruby syntax can be a bit bulky. Your syntax feels pretty clean TBH.
I'd highly recommend you build it yourself though, which can be done rather easily with VS (2019) on Windows or CMake on Linux. On Windows just press build in release mode and you're done. More notes here: https://htmlpreview.github.io/?https://raw.githubusercontent...
Then cd lobster && lobster samples/pythtree.lobster for example should run something. Or see here how to use from command line or from your fav editor: http://htmlpreview.github.io/?https://github.com/aardappel/l...
Also, have you actually programmed in Vulkan? Even a simple primitive can be hundreds of lines of setup code. In Lobster they're often a single line. A direct mapping would not make sense.
google+?
You currently run it either as a bytecode VM (which is rather fast and runs on any platform), or you can translate it to C++ if you want an additional speed boost.
I found it a bit strange that the coroutines run until the first yield at the time they are created, but otherwise I quite like the system.
The language could of course be adopted to these use cases, it simply hasn't, so far.
Doing simulation on a separate thread from main/rendering is something it is already set up to do (multi-threading uses separate memory spaces so it is easy to ensure there's no data races between the two).
That I've noticed: Twice on that page, once in the readme, again in dev/readme and lobster/readme.
Without large teams working for years, its very hard for a new languages to break in to general use. There are ofcourse outliers and you can get lucky - but then you have to be terribly lucky. So the end result is that 99.9% of languages just die or become hobby project for a person regardless of their features, benefits, beauty or aesthetics. Again, nothing wrong with becoming personal hobby project but if my goal was to create game engines that becomes force in the industry, I would probably not start by creating entirely new language.
I'm not experienced as much as the author, so this is my personal opinion and likely not very well informed. I writing this just because I see several people spending years in creating new languages while their goals have been something else. They dream about repeating success of Java while not realizing how rare it is. After they finish their labor of love, I wonder if they find it depressing to become just line item in the list of dead programming languages. What if they would have actually focused on their original goal instead with emphasis on adding new powerful features that integrates and plays well with existing stuff? It's sure not as sexy and respectable as being father (or mother) of new language but has much higher chance of generating desired impact.
Didn't know how to code back then, and probably can't rocket jump anymore now. Mostly just surprised at having that old memory surface out of nowhere.
Thanks for making that scene cooler way back when.
Smalltalker here. Being able to pass in lambdas very casually is a great way of producing an ad hoc DSL. Imagine a custom control structure which does a database transaction. Just implement a function!
Benefits show an increase in something good or a decrease in something bad.
Better speed. Less code. Easier distribution. Lower cost.
Does this particular kind of type system help you in your software engineering? Very hard to quantify, and depends on circumstances and even personal preference.
I don't think I have any hard-number benefits that are meaningful.
What would be more regular sample code? Do you mean just writing "find([ 1, 2, 3 ]) x: x > r" with explicit () and explicit variables? I agree that is easier to read, though the extreme terseness is also a feature..
Yes, everything can be one line, but in this case it would look a bit ugly:
def find(xs, fun): (for(xs) x, i: if fun(x): return i); return -1
Yes, amd yes it depends on the purpose. For learning, it can be impossible to parse; redundancy helps distinguish parts and provides a check you got it right.
OTOH for showcasing features, IMHO, the generality of : is a more impressive feature to demo.
Anyway, it's certainly intriguing! And maybe that's the true purpose of showcase code...
I've seen Swift's intentions to add lifetime analysis to ARC, but from what I could see it is much more complicated in Swift, requires annotations, and with significant runtime overhead remaining.
'including' stuff entails a whole host of ugly things including multiple definitions, circular dependencies etc. etc..
Most languages use some kind of module resolution, which when specified clearly is usually better. Of course sometimes they forget to define things, and leave us in a lurch ... but hey ...
Cool language though. I love everything except for the name, sorry :)
And this style of static analysis has been taken even further by Luc Blaeser: http://concurrency.ch/Content/publications/Blaeser_Component...
The cycle checker I'll have to have a look at - I'd like to crib parts of it for an OS I'm trying to cobble together. Your project looks very interesting, thanks!
What I do is a bit more involved though, as for every language construct I indicate if it wants to own or borrow its children (and wether it wants the parent to own or borrow), to remove maximum refcounts, also for cases which are not simply variables. Again, I will describe this in detail.
The second paper I don't see how it relates to what I am doing.
https://github.com/apple/swift/blob/master/docs/OwnershipMan...
I think Rust should allow this kind of relaxed ownership, with ARC behind the scenes, as lots of user level code should not require fussing over ownership details. A "get shit done" mode.
You can of course use Rc types in Rust, but that is verbose, and throws all ownership out the window, unlike Lobster.
Feel free to email or msg me if you want to be notified.
The name.. I think people will quickly forget about the original meaning. Think of all the languages you know, and how you associate the name with the language, not some snake, a musical note, degraded iron, a french philosopher, etc. Some of these names are actually pretty silly.
As for associating the name of the language, the only one I can't get past on that is 'Pony', it just has too much MLP fandom association for me to take it seriously.
[0] https://www.beliebte-vornamen.de/wp-content/uploads/adolf-1....
If Sartre isn't already an esolang built around the concept that hell is other programmers, then someone should make it.
Anyway, psyched to see more people experimenting with this kind of thing!
This seems to scale out very well - it's kind of static GC analysis with RAII and erlang-style messaging.
