RBS, Ruby’s new type signature language(developer.squareup.com) |
RBS, Ruby’s new type signature language(developer.squareup.com) |
When a statically typed, compiled language is an option I tend to choose Rust lately just for novelty and curiosity, but it's rare that I have those options. When I don't, I find these tools are a godsend. You don't really lose flexibility at all.
Citation needed. In my experience this is a fallacy; types take no time at all to write and significantly reduces bugs (Typescript).
I’m not super familiar with ruby outside of my work so I’m not sure if this reliance on method_missing is more widespread than rails.
Considering how prevalent Rails is for Ruby, we can assume that the majority of Ruby codebases are web apps. There are tons of way to scale Ruby web apps for high traffic, and a tiny minority of companies end up reaching a scale (like Twitter) where Ruby becomes infeasible.
Separate files for types with no inline annotations possible? What an embarrassing compromise. This is all because Matz explicitly won't allow type signatures in .rb files. I wonder how long it'll be until a hostile fork if he doesn't change his mind.
I've found that if all you can say is "I hate this", you usually don't actually understand the trade-offs.
In the meantime, you have Sorbet available. What's the problem?
Both sorbet and RBS have put huge amounts of effort into bolting type systems onto ruby in ways that don't run afoul of Matz's categorical and in-principle rejection of adding type-annotation syntax to the core language. Both or either of these projects would have much better ergonomics without having to bend to this constraint.
As ruby's user base skews further and further away from the hobbyist market and toward the startups that began in the period when ruby was 'cool' that have now grown up into enterprises, this pressure will continue. If Matz doesn't recant, the two possible futures are:
1. Deep compromise (see Sorbet, RBS) to keep types out of the core language; or
2. A hard fork, or a one-level-up language like typescript.
I was even hoping to use ruby as a main language having used it before but I'm about to lose any interest in the language when its reality is a bit decoupled from the rest of the world.
It does seem useful to have a _standard_ for type definitions - RBS as the equivalent to a .d.ts file - as that allows for different type checking implementations to use the same system under the hood. This was a big problem for Flow, and why it lost the fight as soon as TypeScript's definitely-typed repository started gaining momentum - users wanted to use the type-checker that they knew had definitions for the libraries they used.
On the other hand, RBS as hand-written seems rather dangerous, to me. Nothing wrong with using them to define previously-untyped external code, as long as you know the caveats, but I think you really want to have definitions generated from your code. Sorbet cleverly (and unsurprisingly, given it's Ruby) used a DSL for definitions in code, which had the (excellent) additional boost of runtime checking, so you actually could know whether your types were accurate - by far the biggest pain-point of erased-type systems like TypeScript.
Given that Ruby 3 was supposed to "support type checking," I'm surprised that it does not seem to have syntax for type definitions in code, and instead will focus on external type checking. I might be missing a piece of the full puzzle not covered in the blog post, however.
This is a big disappointment to me, one of the main advantages of static typing is that it can make code much easier to understand when types are added to non-obvious method parameters.
Isn't the point that you run the type checker on your own code and it checks that it implements the signature correctly? Having a mismatch between the code and the signature will give a type error. How is this different from how Sorbet works?
Flow lost because the compiler was in really bad shape, slow and frequently crashing. Also their equivalent repository to DefinitelyTyped would ignore PRs for months and years and afaik still does.
It's like it was somebody's toy project and its author eventually lost interest.
It's a pitty because TypeScript still has unsound generics. But Microsoft know how to make dev tools and maintain them.
That’s sounds like what type-profiler, mentioned in the article, is for; it's an experimental project which,if successful, seems destined to be part of Ruby’s bundled command line tooling, for generating type signatures from code.
If you mean you want type signatures embedded in code source files rather than in separate files, they seem to be taken a documentation-annotation approach, with YARD documentation format expressly called out as a mechanism to bed typing in source files. That's probably cleaner than further cluttering Ruby’s syntax with annotations.
> Given that Ruby 3 was supposed to "support type checking," I'm surprised that it does not seem to have syntax for type definitions in code
The support seems to be that, at a minimum, that it will have a standard for type definitions and provide them for Core and Stdlib and have command line tooling for working with type definitions. Which is, I would say, significante support.
Updating Ruby’s already notoriously complex syntax to support type annotations while keeping existing Ruby code valid with it's existing semantics is...not a very easy step, I suspect.
Annotations in documentation is a more viable way of integrating type definitions into program source files.
`.h` files are not something to emulate! External interfaces should be generated by tools where needed.
It's not clear to me whether Soutaro is a member of the Ruby core team, so it feels a bit odd that the post is written like an announcement from the Ruby maintainers.
He was going to keynote on this at RubyKaigi this year until it was cancelled, and had a talk at RubyConf as well on this.
Python 3 incorporated types into the language itself, in a similar way (though non-reified) to PHP. This seems much easier to deal with than requiring two files (.rb and .rbs) to describe a single data structure.
I don't like the comparison with TypeScript `.d.ts` files however, because TS still lets you do types inline in the code. I haven't seen it mentioned anywhere that this won't be supported by Ruby 3.
Does anybody know if Ruby 3 will also support inline type information or will the header RBS files be required?
Better IDE integration: Parsing RBS files gives IDEs better understanding of the Ruby code. Method name completions run faster. On-the-fly error reporting detects more problems. Refactoring can be more reliable!
IDE support (autocomplete, refactoring and quick documentation) is the most important reason to annotate argument and return types.
Jetbrains is a wonderful company.
I much prefer the Python 3+ approach of type annotations in source code.
I can't imagine having to look at a separate file just to figure out what the type of something is. You may say "tooling will fix this" but it's just far less overhead for everyone at the end of the day to just make annotations in source.
My more existential question is, is there really an advantage to doing static type checking in Ruby?
When I was doing Ruby, the way you can change objects on the fly, add methods on the fly, the vast amounts of metaprogramming, are types at "compile" (I know, not really) time really the same as types at runtime?
Like, it might be nice to get some autocomplete, but AFAIK tools already do that (RubyMine, others).
(edit: I should have explained that I'm talking about the type checking features I'm developing in Solargraph: https://solargraph.org/guides/type-checking)
It's sad though. Since poor design of Refinements, C transpiling for 3x project, and now this, I am less and less inclined to continue using Ruby. I miss some of the dynamics but I find myself using Crystal instead.
(Honestly, if any one figured out a way to supplement Crystal with dynamic behavior for those features that a static language can't offer, Ruby would be done.)
I say this as someone who has written Ruby for almost twenty years. I will _never_ use a tool that depends on YARD document formatting, because I will never use YARD document formatting.
The Ruby and Python languages do have the concept of type, it's just that they're dynamically typed, not statically typed. They check types at runtime.
Also, Python now has (in its stdlib!) things like typing.Protocol, which is almost exclusively checked at type checking time. So if such a thing exists, and you still say "types are checked at runtime", isn't that confusing?
Being able to define an interface instead of pure un-specified "duck type" is great.
With all due respect, but IMHO this is too little and much too late.
https://sorbet.org/docs/faq search for “RBS” (on mobile, no deep links)
But other type checkers could be implemented relying on the same information.
Very little of this will need to be written by hand. The underlying tech is pretty decent at guessing types, the idea is that if it's not quite specific enough you adjust it, but it should otherwise be transparent.
The comparison to .d.ts files then seems bizarre because that is helpful for language servers¹ to consume types, but there's no proof that say, the implementation matches the type specification.
TypeScript declaration files declare what the types of module exports are. For the most part, a .d.ts file informs the typechecker "the type of module Foo export bar is the interface named Quux". This is not checked, this is simply an assertion. The language server for TypeScript will pick these definitions up, assume they are correct, and provide code completions for those types as if they were correct.
On the other hand, a .ts file, combining types and codes, enables type checking. If the type declarations are incompatible with the code, an error is thrown by TypeScript. While .d.ts files declare types, .ts files verify that the code and the types declare are compatible.
Since .rbs files simply describe the external interface of types and modules, and cannot describe internal variables, I'm not sure how it's doing any type checking.
For example, if I have this code:
module Foo
class Bar
def trivial: () -> String
end
end
class Bar
def trivial
return 42
end
end
class Bar
def trivial
x = some_function_that_might_not_be_declared_in_an_rbs_file()
return x
end
end
¹ Language servers are implementations of an IDE protocol for code completion. The trend in programming language tooling is to use Microsoft's Language Server Protocol (https://microsoft.github.io/language-server-protocol/) to provide code completion, semantic navigation, refactoring, etc.
What ruby 2.7/3.0 do is rationalize some really weird counter-intuitive and ambiguous edge cases related to passing a Hash arg expecting it to be invoked as if it were keyword args. But it's a change in semantics, not syntax. The keyword argument syntax, keyword arguments with colons, in both method definitions and invocations, has been around for years, unchanged.
This is about Static Typing and Type enforcement, in the context of a language that holds flexibility and meta-programming as high values.
How about checking that a given string is non-blank?
That tends to fully leverage Ruby's dynamic nature.
But then again people are overly fixated with compile-time, Java-like signatures.
See clojure.spec for a success story.
Well some of us disagree with the statement that untyped is not suitable for large teams. And that's why we use Ruby. There is a lot of very good typed languages out there if you do want typed. I feel Square and Stripe are pushing their own codebase issues onto general Ruby - as it's our problem to solve - which is not cool.
As they mention in the post, they followed typescript's approach, here. The benefit is it allows you to layer in typing into an existing codebase in a non-disruptive way.
They didn't, though! That's what's confusing me. TypeScript has inline types. .d.ts files are typically for JavaScript files that don't have types embedded.
Hopefully, anyway!
https://sorbet.org/docs/faq#when-ruby-3-gets-types-what-will...
> Ruby 3 has no plans to change Ruby’s syntax. To have type annotations for methods live in the same place as the method definition, the only option will be to continue using Sorbet’s method signatures.
I'm pretty sure the merits of typed vs untyped has been going on since the 1950s at least. 30 years is such a specific period of time that it makes me wonder what happened in the early 90s that the author is referring to.
For the time being I think this kind of type checking is only worthwhile in big projects, for smaller projects I have found Sorbet never finds an error, so it's just extra work to generate the files on a big change.
Since we already have a specification tool (MiniTest) in the StdLib it would interesting if we could combine the rbs files with spec unit tests.
I already have my matching test file open anyway. Having the typing information in the same place would encourage the use of types, tests and documentation.
I had the good fortune to hear him talk about it at length at a conference a while ago and there's all types of fun stuff on the way.
Here's a full example, complete with a typo, based on the example in the blog post: https://bit.ly/3hMEMSp
Here's a truncated excerpt to get the basic idea across:
# typed: true
class Merchant
extend T::Sig
sig {returns(String)}
attr_reader :name
sig {returns(T::Array[Employee])}
attr_reader :employees
sig {params(token: String, name: String).void}
def initialize(token, name)
@token = token
@name = name
end
end
class Merchant
attr_reader token: String
attr_reader name: String
attr_reader employees: Array[Employee]
def initialize(token: String, name: String) -> void
# actual method body
end
def each_employee: () { (Employee) -> void } -> void
| () -> Enumerator[Employee, void]
# actual implementation body
end
end
Disclaimer: Working at Square, have friends at Stripe, enjoy both type checkers.
The reasoning is here: https://www.artima.com/forums/flat.jsp?forum=106&thread=1559...
def foo
username = T.let("heavenlyblue", String)
end
However, more important to have in the body of a program is tools for casting and asserting types, like these:
T.assert_type(foo, String)
T.cast(foo, String)
T.must(foo) # assures the compiler foo is not nil
T.unsafe(foo) # the equivalent of a TS `any` cast
I'm not sure how tools that use RBS without inline syntax will handle these situations, but to be honest I expect the community to adopt Sorbet in practice anyway. It's very fast and battle-hardened in production at Stripe and several other large companies.
Disclaimer, again: former Stripe employee.
Edit: Like, seriously. Either the local var is populated by something coming in externally (which is then typable) or, unless your code is too complex / large, it should be easy to see everywhere it's used, and then why would you need that additional typing info?
If the author thinks that's the biggest benefit, I'm inclined to think the ruby community doesn't seem to have enough eyes these days in the core development.
My guess is the latter is vanishingly small – that it's pretty much only done for libraries that were written before TS was a thing – so I wonder how things will go in Ruby.
Maybe everybody will just standardize on third-party tools like Sorbet which allow inline typedefs, or use types a lot less, or hook up a "regenerate inferred .rbs on save" workflow in their editor, or just switch between files a lot.
RBS and type files on the side were really hotly debated for a while and the core team settled on this as a way to not break the existing parser among other reasons.
While I don't 100% agree with them I have faith that Matz and the team make the decisions they do based on impact and what they see in the community.
Check out the OCaml community, interface files have been use there since basically day one, and are generally well-liked for how clean they allow the implementations to be.
> Does anybody know if Ruby 3 will also support inline type information or will the header RBS files be required?
Wait, what are we talking about? I thought this was the decision you said you completely understood, that the type information is in separate .rbs files. Isn't ruby 3 what we're talking about?
You won't need to use the header RBS files at all (types are optional in any case) but you'll likely want to use Sorbet or Steep to generate them if you're sharing your code more widely, since community tooling like YARD will probably use those for code navigation.
Steep and Sorbet are second-level, they build off of RBS. Matz has mentioned offhandedly in conversations I'd had with him in the past that there's a ton more in store with RBS beyond just type checking, so we'll see where they go with it.
As far as YARDoc I've been eyeing that one for a while now since I first heard about Steep at a Braintree Ruby meetup before Soutaro was at Square. We're still talking about what and how as far as that one.
I don't see how having to switch files to know that `input` is a `User` increases readability, though. It seems like straight-forward impl-simplicity trade-off, not one of user ergonomics.
Do you mean for Ruby specifically or in general? I've found that it's much easier to (safely, accurately) read, use, and extend e.g. a TypeScript file than its JavaScript counterpart, even when provided with a .d.ts file.
I don't disagree, but I think it's a very minor issue given that it's trivial to use color to highlight code these days. By comparison having to switch between two files (and keep them in sync!) when making changes is a far bigger usability concern.
TypeScript has this functionality (in addition to being able to write actually TypeScript files with inline annotations. The big advantage is being able to provide 3rd party type definitions for libraries that don't provide them and aren't interested in using them. This allowed TypeScript to bootstrap decent library support well before it was popular enough that the mainstream was considering adopting it, and this in turn enabled widespread adoption.
> My more existential question is, is there really an advantage to doing static type checking in Ruby? When I was doing Ruby, the way you can change objects on the fly, add methods on the fly, the vast amounts of metaprogramming, are types at "compile" (I know, not really) time really the same as types at runtime?
Again, I think TypeScript shows that there is. Sure, there are times when you want to do super-dyanamic stuff. And you can opt out of type checking using the "any" type in those cases. But a lot of the time you're not doing anything complicated, and you just want a compile-type check that ensures you're passing the correct type to the function you're calling.
There have been attempts to have types outside the main source or in comments for many dynamically typed languages. They seem to fail due bad programming ergonomics, as maintaining separate "header" files is cumbersome (hello C my old friend).
But you're absolutely right about the downsides of stuffing types into a different file. I get why Matz did it (he wants to keep Ruby beautiful and types are crufty) but I don't like them in the first place.
To answer this (as someone who basically only ever writes in Python):
There are a few cases where it's really nice to be able to add type annotations to methods or functions. The most obvious example is API calls; it's nice to be able to say "this needs to be a list, give me a list", and not have to do
if not isinstance(var, list): var = list(var)
or
if not isinstance(var, list): raise ValueError("I know I didn't tell you I needed specifically a list, but I need specifically a list in this case")
Over and over and over again all over your module. Look, give me a list, I need a list. I need the APIs that list has, I need the interface it uses. I don't want a generator that I'm going to be iterating over forever, I don't want a string that's going to get split into individual characters.
Duck typing is all well and good, but just because strings, lists, sets, and os.walk are iterable doesn't mean I'm able or willing to handle those.
It can also help a lot in IDEs; for example, if I type-annotate a method to accept "name" as a Str, then my editor can assume that "name" is a string, even without any other evidence to that being the case. Likewise for things like warning about return types.
Lastly, it lets you do automated testing as well. Hey, you're passing a FooNode to this function, but that function accepts a list. I know this because NodeCollection.find() returns a FooNode. Makes it easy for the dev to look at the report and think "Oh, I meant to use NodeCollection.findall(), oops!"
I certainly don't want a statically typed language, but there are a lot of cases where my internal logic is fixed and I don't want my method to have to know how to deal with int, str, none, bytes, etc. Type annotations can solve this problem for me and for other people using my code.
Probably using the languages for the ecosystem (e.g. Python for scientific computing or ML and ruby for ruby on rails) but still wanting to benefit from type checking
For instance, I can imagine adding something like comment blocks to Ruby code that RBS tooling can find and treat like the RBS files.
Steep: https://github.com/soutaro/steep
Sorbet: https://sorbet.org
In your example you would get a type checker error.
Because when you try to do this with some object that doesn't have a "length" or "empty?" method, your application crashes.
irb(main):013:0> a = 1
=> 1
irb(main):014:0> b = "1"
=> "1"
irb(main):015:0> b.length
=> 1
irb(main):016:0> a.length
Traceback (most recent call last):
4: from /usr/bin/irb:23:in `<main>'
3: from /usr/bin/irb:23:in `load'
2: from /Library/Ruby/Gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
1: from (irb):16
NoMethodError (undefined method `length' for 1:Integer)
irb(main):017:0> b.empty?
=> false
irb(main):018:0> a.empty?
Traceback (most recent call last):
4: from /usr/bin/irb:23:in `<main>'
3: from /usr/bin/irb:23:in `load'
2: from /Library/Ruby/Gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
1: from (irb):18
NoMethodError (undefined method `empty?' for 1:Integer)
I think it's rude to dismiss people asking for this as "fixated," and furthermore it could be no less fairly used against people who show up to these debates beating their own drum against it.
There are two different libraries that do it:
https://github.com/plumatic/schema/blob/ddb54c87dea6926c6d73...
https://github.com/clojure/spec.alpha/blob/eb49d429e85b6878a...
Honestly I highly suspect that many, many "typing" solutions out there are plain ignorant of the whole spectrum of choices one can make, are tend to lean towards Java-like APIs out of that ignorance.
This is not limited to Ruby, I also see it in TypeScript which is very much a contrived system for real-world usages while making little use of JS's dynamism.
And then you get stuck in very opaque error loops with the typing where it's expecting eg. a Number. no, not that number, a different type of number. no, not that type either. no, not that type. Most the code i've written has been typecasting.
Crystal does have some significant benefits over Ruby (which is why i'm using it - better memory use, better for scaling for my purpose) but I just spent time rewriting the non-user facing, non-scaling part of the bot in Ruby so I could actually get stuff done instead of fighting the language.
Of course a lot of this might be my inexperience with Crystal, but as a Ruby dev for 13 years, it's not as easy as just switching over from Ruby to Crystal. I've had this bot running 4 years and it hasn't got any easier for me.
Have you tried adding a `.crystal-version` file as described in the buildpack's README? https://github.com/crystal-lang/heroku-buildpack-crystal#cry...
This had never been my experience, I have a server written fully in crystal running in production serving millions upon millions of requests on heroku and crystal doesn't break a sweat.
Quite happy with it so far.
Node has had 10 years and its still not there.
And never will be.
The thing is that much of it is perfectly valid Ruby code with wildly different semantics already, so, no, without breaking a lot, you can't unify it with Ruby syntax.
Seems reasonable for someone to want types but not have to raise their own entire TypeRuby language + ecosystem + re-tooling.
Also how do you type something in an inline function?
IDEs will likely be able to seamlessly peek/go to RBS type definition on any Ruby identifier in any case.
You can configure things like this globally and/or for each method call.
Eg;
# turn off all runtime checks
T::Configuration.default_checked_level = :never
# turn off runtime checks for one method
sig {returns(String).checked(:never)}
def foo; :wont-raise; end
Personally if I were authoring a gem I'd leave the runtime checks on except in hot paths, so my users get quick feedback when they pass the wrong thing.
In any case, the library author can get the benefits of static and runtime typing, and their users will get nice static typing if they use sorbet. Users also get nice runtime typing for the library if the author chooses to leave it on for them. The overhead is usually small.
module T
module Sig
def sig *args
end
end
# You'd need to stub out a few more things here.
end
begin
require 'sorbet-runtime'
rescue LoadError
end
The Ruby syntax is too complicated to allow for changes like this to be backwards-compatible.
For example, `attr_reader token: String` is valid ruby today – that's the same as `attr_reader(:token => String)` which somebody might be doing in the wild, since you can override `def self.attr_reader`.
Similarly, `def initialize(token: String` clashes with the definition of keyword arguments.
I am not able to spin that into "And besides it's better to force it to be in two files anyway!", I don't think it is, but I guess it's not so easy to do different.
Mind you, if we could write tests in .rbs then I guess .rbs could form the basis of a new ruby syntax without breaking compatibility with old code in .rb files.
A classic example of where I might have an inline type annotation in Rust is when I'm doing a non-trivial chain of Future/Result combinators in the middle of a function. It doesn't take much code for your understanding to desync from reality. Annotating "Result<String, IOError>" inline both documents to others what this intermediate value is but also creates better, local errors as the chain is modified.
Complex stuff does generally get factored out into functions, but at the same time, it's nice when you're the one who decides when it makes sense to extract code rather than a limitation of the typing syntax. Those things don't always line up.
Like `foo=open_database("mysql://...")`.
Do you know what solves the problem of checking every type of every thing you want to use before you use it, where the checks don't incur any performance penalty at runtime? Static type checkers!
Modern compilers usually don't even require any explicit type declarations, as they can infer at assignment. So they provide more safety for less code than your examples.
Agreed. But don't read too much in RBS details yet. RBI is currently very early and will need to change substantially learning from experience of actually typechecking real codebases. Stripe and Shopify are helping with this.
RBS has better syntax, but has features that don't have clear semantics or feasible implementation. And doesn't support inline annotations that are necessary in practice.
RBI is limited by ruby syntax and thus isn't as nice, but has good semantics and support inline annotations. And has been tried on hundreds of real codebases including those with dozens of millions lines of code.
We'll need to gather benefits of both on our way forward.
The best path forward will be a lot of discussion, especially around learnings with production codebases where possible.
Personally I prefer inline annotations, but want to explore the possibilities of both and see what comes of it. I've written on Sorbet in the past and have used it on a few toy projects.
Also working on some analysis documentation comparing the two if you'd be interested in chatting later. Feel free to DM @keystonelemur on Twitter.
Soutaro has been great to work with over here (Square), and he has a ton of really amazing things coming soon that we're working on OSS'ing later.
Just to clarify, I left Stripe several months ago (and did not work on ruby infra), so we both get to cheer from the sidelines!
One incident that stands out is that certain Postgres support was only available in the latest version of a shard, which required the latest version of Crystal, which wasn't compatible with 3 of the other shards I was using.
For example, I used to use Intellij for Scala but recently switched to Emacs+Metals and haven't really missed anything. In fact, it's probably an even better editing experience.
Intellij still has better refactoring (though I don't use it much), and the integrated debugger and database viewer are really nice. I've found myself using Emacs and only switching to Intellij for the aforementioned specialized tasks.
5 years ago you would have been crazy not using an IDE for JVM work but this is no longer the case. LSP is such a wonderful technology and has empowered the creation of new programming languages like never before. It's truly remarkable.
If I was Intellij, I would be a little worried about my future market share. They simply can't provide the same value as before, and I'm not sure how they intend to change that.
However, my experience with rls/rust-analyzer with VSCode hasn't been great either. Red squiggly lines that persist even if the syntax is correct - I have to erase and type the same thing to trigger refreshed checking which is very annoying and counter-productive. Though it is possible that this is more of an integration issue considering I haven't had similar problems with Typescript on VSCode - the checking was flawless (I think typescript checking also uses a language server).
Ruby on Rails seems like a kneejerk response, but then again it doesn't exist because nobody really wants it, not for technical reasons.
For example, Python has all the ML/math stuff. Nothing comes to mind for Ruby.
Clearly not true, as there's two separate fullstack frameworks being actively developed[0][1], not mentioning the ones in the past (sails.js, meteor.js).
[0] https://github.com/blitz-js/blitz [1] https://github.com/redwoodjs/redwood
I guess a language on the opposite side of the spectrum would be Lisp-likes, which are brain-dead simple to come up with a generative grammar for, but a little hard on the eyes.
For short chunks of program text, we can probably rely on our natural language abilities to some extent. Those capabilities allow us to deal with transformational syntax, and ambiguities. So that is to say, we have a kind of general parsing algorithm that is actually way too powerful for programming language syntax, but which only works over small peepholes. Most speakers will not understand (let alone be able to produce) a correctly formed sentence that is too long or too nested. It's as if the brain has a fixed-size pattern space where a sentence has to fit; and if it fits, then a powerful pattern matching network sorts it out. Whereas a programming language parser is unfazed by a single construct spanning thousands of lines, going into hundreds of levels of nesting; it's just a matter of resources: enough stack depth and so on. As long as the grammar rules are followed, and there are resources, size makes no difference to comprehension.
When reading code, people rely on clues like indentation, and trust in adherence to conventions, particularly for larger structures. Even relatively uncomplicated constructs have to be broken into multiple lines and indented; the level of syntactic complexity that the brain can handle in a single line of code is quite tiny.
We also rely on trust in the code being mostly right: we look toward understanding or intuiting the intent of the code and then trust that it's implementing that intent, or mostly so. If something looks ambiguous, so that it has a correct interpretation matching what we think we understand to be the apparent intent, and also has one or more other interpretations, we tend to brush that aside because, "Surely the code must have been tested to be doing the right thing, right? Furthermore, if that wrong interpretation is not actually right, the program would misbehave in certain ways (I guess), and in my experience with the program, it does no such thing. And anyway, this particularly code isn't even remotely near the problem I'm looking for ..."
The grammar is notoriously complex in ways that most users of the language thankfully do not have to worry about. But it does make extending the syntax quite hard.
Why would it be less meaningful to say types are checked at runtime with ducktyping? The nature of ducktyping is that the specific class of an object does not matter relative to behaviour, but a class is not entirely equivalent to a type.
If I need an object that implements method `foo`, and don't care about class, then "objects that implements foo" is in itself a type, that can potentially be inferred and checked be it at runtime or before.
> The majority of functions written in Python, even the ones that have type annotations, do not effectively have "assert isinstance(...)" in the program text below their signature, which is what I'd expect after reading "check types at runtime".
You're thinking his "checks types" too narrowly. Every time I try to call a method on an object in a strongly typed language, typing is involved. It doesn't so much "check" it as look up the method to see whether this method applies to this specific object at this point in time and decide whether or not to throw exceptions.
But the point remains that it is a typed. And strongly so - in both Ruby and Python objects has a type associated with the object itself, unlike e.g. C or C++ which are weakly typed because it is the variables that are typed, not the values.
[0] https://learn.adacore.com/courses/intro-to-ada/chapters/stro...
E.g. Liskov and Zilles [1] defined it this way for example:
"whenever an object is passed from a calling function to a called function, its type must be compatible with the type declared in the called function."
Under this definition C and C++ fails hard, since you can statically cast a pointer to an object and pass it to a function expecting a totally incompatible type.
Note that the system described relied at least partly on dynamic/runtime type checks (in case it reads as if the quote above suggests they used "strong" to refer to static typing):
"It is desirable to do compile-time type checking, since type errors are detected as early as possible. Because of the freedom with which types can be used in the language, however, it is not clear how complete the compile time type checking can be. Therefore, the design of the language is based on a runtime type checking mechanism which is augmented by as much compile-time checking as is possible. "
[1] https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.13...
It's true that types aren't checked in the act of passing a value as an argument, but at bottom, Python still has a concept of types, and they are still checked at runtime. Try the following and you'll see Python check your types and determine that there's an error:
"Hello" + 42
> Python now has (in its stdlib!) things like typing.Protocol, which is almost exclusively checked at type checking time
Yes, Python is now adding optional static typing, and of course, JavaScript has TypeScript. I'm afraid I don't know a lot about these new systems but presumably the end result is that type errors can occur either at compile-time (or static type-checking time, or whatever we call it) or at runtime. This isn't exactly anything new, it's always been possible in Java for instance, which has always permitted downcasts, and has always had covariant array types, checking both at runtime. [0] This is despite being a statically typed language where all variables must have a fixed type, the type they are declared with. (Remember that a variable's type is distinct from the precise class of a pointed-to object.)
We can draw a distinction between statically typed languages like Java where there's still a need for runtime type checks, and statically typed languages like Haskell where there's no need for runtime type checks. Type theorists use the term soundness for this property: in a language with a sound type system, a program that is successfully validated by the static type system can never have a type-related error at runtime. In engineering terms then, a sound type system means you don't need to check types at runtime, as 100% of type errors are caught by the static type system and there's no way for type errors to ever arise at runtime.
I used Haskell as an example, rather than C, because although C doesn't give us runtime type-checks, C programs can still go haywire if you make a mistake in your program (termed undefined behaviour). C has an unsound type system, and it lacks runtime checks. This is one reason C is so famously 'unsafe'.
So anyway, we have the situation where Python code can encounter type errors at compile time or at runtime, and Java can encounter type errors at compile time or at runtime, but we call Python dynamically typed and we call Java statically typed. The difference is that in Java, a variable must be declared with a fixed type, unlike in Python.
Things can get messy in the middle-ground: in C#, the dynamic keyword allows for true dynamic typing, where a variable's type is determined at runtime. [1] So you could write a C# program in traditional Python style, where there's very little compile-time type-checking. And you could write a modern Python3 program using lots of static type assertions, minimising the opportunity for runtime type errors. We'll still call the C# language 'statically typed' as it's typically true of C# code, and we'll probably still call Python 'dynamically typed', as that will probably remain typically true of Python code.
Disclaimer: I'm not a type theorist or a programming language researcher, corrections welcome if I've got anything wrong.
[0] https://news.ycombinator.com/item?id=13050491
[1] https://docs.microsoft.com/en-us/dotnet/csharp/programming-g...
Sure they did. It's the 5th paragraph in the post:
"We defined a new language called RBS for type signatures for Ruby 3. The signatures are written in .rbs files which is different from Ruby code. You can consider the .rbs files are similar to .d.ts files in TypeScript or .h files in C/C++/ObjC. The benefit of having different files is it doesn't require changing Ruby code to start type checking. You can opt-in type checking safely without changing any part of your workflow."
.rbs files: have types
.rb files: no types
.d.ts files: have types
.ts files: have types
It's a pretty significant difference. So they didn't "follow typescript's approach" here.
But that applies to making it so old code does not work in the new version of the language. Nobody expects all new code to work in the old version of the language. Ruby adds new features including syntax that won't properly parse in old interpreters all the time. It's not clear to me why inline type definitions couldn't be such.
Probably because Ruby had a much narrower area where it was heavily used, with fewer “finished” but critical libraries.
If you take a look at his keynote video he says quite a bit on this too.
I hope people keep the type annotations sparse, and allow the tools to infer it unless they're prepared to link long and hard about the minimal restrictions that are reasonable.
I could go on about what's wrong with the language but:
It's not stable, API change all the time, breaking change all the time, cryptic errors, lot of missing basic features, IDE integration etc ...
That is changing, earlier this year, the Crystal team are working on getting the language stable for 1.0 [0]
Companies are already now using Crystal in production, most recently Nikola Motor Company [1].
Surely they wouldn't choose crystal if it was a 'half baked' language.
[0] https://crystal-lang.org/2020/03/03/towards-crystal-1.0.html
[1] https://manas.tech/blog/2020/02/11/nikola-motor-company/
I don't think average programmers even know about it.
So while I agree with you, I still look forward to see support for Windows eventually landing.
Then for instance most languages get away with inline optional typing by using “:” , for instance “ping_user(name: String)“. In ruby it’s of course already taken, in no small part because there are 3 or 4 different ways to declare hash parameters.
I’d imagine most decent syntax candidates had similar issues, due to ruby’s syntax versatility.
I don't like the separate file thing, but it does seem more challenging than I'd have thought to avoid.
I guess on a tangent Ruby code historically cares a lot more for duck typing so strong typing will be a headache for a lot of stuff.
Is the definition you use not so broad as to admit all invariants being described as types?
If a method requires a dictionary arguments with a certain key, is that a type to you? If you extend the term "type" to cover all invariants, I don't think that is the way that the term is commonly used, even though many invariants can be proven in e.g. dependently typed languages.
All these terms are so wonky because a C++ type is not equal to a Haskell type. So I feel we can't ever have solid definitions of terms like "strong", since it depends what you compare it to, and it also depends what you compare from. So while X is strong in comparison to Y, that doesn't say much about X's relation to Z.
I dunno. Massive breakages of backward compatibility in an established language may not be better than that.
Python3 didn't offer much over python2, so people just saw the downsides, while ruby pushed people to upgrade with the promise that their efforts would gain them better performance and/or save money.
I meant literally that Ruby is easier for a human to read for comprehension than say, x86 assembly, which it is. Ruby requires (however) substantially more complex parsing logic to machine parse (translate to machine instructions), because Ruby syntax tolerates an almost absurd amount of ambiguity. This distinction holds when you compare Ruby to many common programming languages. Lisp is an excellent example of a high-level language that can be parsed with minimal complexity. I can teach an undergraduate to build a Lisp parser in a day, but it would take weeks to get someone up to speed on a Ruby parser.
This was not posited as an essential tradeoff in programming languages (if I came off that way, my apologies). Ease of human readability is probably orthogonal to ease of machine parsing.
Humans don't have a magic algorithmic shortcut. If I give you scrambled word decks of various sizes to sort manually, the best time performance you will be able to show will appear as an N log N curve. Maybe you can instantly sort seven objects just by looking at them, but not 17.
The conflict here is that object oriented philosophies aren't actually about objects. They're about communication between objects. The messaging between objects. As per Alan Kay himself:
> I'm sorry that I long ago coined the term "objects" for this topic because it gets many people to focus on the lesser idea. The big idea is "messaging".
The goal of object oriented design is to focus on the communication between objects, not the objects themselves. Part of that is that the type of object receiving the message doesn't matter so long as it understands the message and knows how to respond. If the object looks like a duck, swims like a duck, and quacks like a duck, that's good enough--even if the duck turns out to be a chicken with an identity crises. It understood the message and responded, and that's all we want in object oriented programming, objects that can communicate with each other.
Adding type checking flies in the face of this philosophy. Instead of type being irrelevant as long as the receiver of a message can understand that message, suddenly it's front and center. The code will accept or reject objects based on their type even if they're fully capable of upholding their end of the conversation.
Type-less-ness is core to Ruby. But some people may still prefer to include typing. We all want to use the tools and practices that best enable us to deliver, so that's a fair want. But since Ruby as a philosophy doesn't care about type, it's important to maintain type checking as an accessory to the language, not a feature of it. Something that can be layered on top of the Ruby language for those that want it, but that can be ignored by those don't.
That's why the approaches being used keep the type annotations out of the source files themselves.
Ruby is not the first or the second or even the third dynamic language that has added static type checking support, has this _ever_ happened?
TypeScript hasn't ever done anything for me than give me 3rd party dependency integration headaches. I love strongly typed languages and compile time checking, but TypeScript has never seemed worth the trade off due to its broken interoperability with normal JavaScript and the terrible state of crowd sourced typedefs. I'm either fighting some badly defined third party typedef, spending a lot of time creating typedefs myself or dealing with a version issue because the typedef isn't compatible with the version of the library I'm using.
When I use JavaScript I hardly ever run into issues that static typing would have prevented and I have zero TypeScript issues.
Honestly how has it improved the speed at which you get things done? Were you just constantly running into JavaScript bugs due to the lack of typing?
Then at the end of the day, it was still JavaScript (an interesting word for "not ruby"), but with types slapped on top.
I ended up switching to crystal, which is basically ruby + types (infered when possible, but I actually wanted the types) with the performance of golang.
Honestly, I rarely refer to documentation for these things because every project is a snowflake and the documentation gradient goes from no documentation to perfect documentation. By that, I don't just mean the words, I mean the website or the framework used to document, as well as the style of documentation (more like flavor?) Typescript is the great equalizer that makes a project with no documentation (but decent comments or method/var names) just as documented as one that that does.
I can also ctrl-space easy to get a list of methods in case I forgot which method I needed, or if I want to discover what's available. That's enormous in my style of programming. Sure beats going to someone else's documentation page, trying to read it.
Some of the improvement is not necessarily that I have javascript bugs due to lack of typing but rather that with typescript I don't get those bugs which means I don't have to reason about avoiding those bugs anymore like I did with javascript. Sort of a reduced cognitive load.
Also, I have a few coworkers that are not javascript/typescrpt savvy that I was able to get up to speed with typescript fairly easily due to the easy of using the types. There are, of course, hard things such as partials or understanding tsconfig.json or even generating types that I don't cover with them and just have them come and get me when they're ready.
For most things without types I just do the declare module in a d.ts - however, I will first try to find another package that does the same thing with types. Most popular packages these days do include types, some better than others.
After I re-read above, I realized that a lot of it depends on the IDE. If I were still using vim or kate/gedit, it probably wouldn't be a huge timesaver. Fortunately, I settled on one of the intellij editors.
It's about having things auto completed and you get to see errors before you run.
And what kind of library are you using to complain that third party typings are the source of your concern?
I'm having a really hard time understanding this "I need types forced down my throat" and "I like typing 3x as much as I would otherwise need to" and "yes, I want half my screen obscured by the types of everything I'm doing, not the actual code" and the "adding types now means bugs are impossible" mass cult hysteria that's running so rampant. Typing very occasionally prevents bugs that are generally easy to catch/fix or show up straight away when running an app. It's mostly a documentation system. And it slows development down.
Especially in Ruby which is such an elegant "programmer's language" I think it would just be silly.
Static types do make certain classes of bugs impossible, like missing method bugs, typos, and the like. You can eliminate a large group of defensive programming techniques and trivial unit tests that you would need in a dynamic language to ensure a similar level of confidence in a program. Obviously they don't make all bugs impossible, there will be bugs as long as there are programs, because we write programs without perfect knowledge of the requirements, and this is an unavoidable pitfall of software.
This can depend really heavily on what you mean by "development." If it's just getting the first version banged out, sure. If it includes coming back to code a couple years later in order to incorporate a new business requirement, having that documentation present can be a really big deal. 2 seconds spent typing out a type hint now might, down the line, save several minutes on average. Even in a recent Python project I did over the course of just a couple weeks, when I got to the "clean the code up and get it ready to put on the shelf for now" phase of the project, I ended up wishing that I had bothered to use type hints just a wee bit more when I was banging it out in the first place. It would have been a net time saver.
I don't like static typing super a lot in all cases because it makes it hard to do data-level programming. Which I find to be the true productivity booster in dynamic languages. But optional typing seems to hit the sweet spot for a great many purposes.
This is not true. You could paint almost every language feature aimed at producing correct software in this way: "writing tests makes me type more, and they catch very few bugs that would have been shown when running my app anyway". (Or, as an ex coworker once told me, "I don't need to write tests because I never have any bugs").
And what are types if not a kind of test/proof that the computer writes for you?
> And it slows development down.
There's a software development adage that goes like this: "I don't like writing tests, because they make me waste time I need to fix bugs on production that weren't caught because I don't write tests."
Well, I guess this is also a matter of perspective.
From where I'm standing, I'd rather you slow down and "document" your code. Code written at the speed of thought makes for an awesome MVP and for an awful legacy for your co-workers.
In the course of my job I write Swift for iOS and Ruby for server APIs and our web-based UIs.
Type issues are about 0% of my Ruby bugs, but dealing with all the damn type requirements in Swift regularly takes dozens of minutes to track down when some weird esoteric error message pops up. And God help you if you try to use generics.
If you want strong typing, then good for you. Just pick a language that fits that mold.
So much of what I love about Ruby is what it doesn't make me do.
Doubt.
In my experience at least 70% of bugs are ones that you'd catch by using types - things like x instead of y, possibly-empty list instead of known-nonempty list, user ID instead of group ID. Logic errors that couldn't be caught by typing do exist, but they're very much the minority.
3x? Even on languages that do not support type inference I would say that this is at most 1.1x. Even then, type inference exists.
> adding types now means bugs are impossible
I usually see that as a mis-representation of what type advocates say. Rather, it seems that people just support that types reduce the amount of bugs.
> or show up straight away when running an app
Or that show up after you had said app running for a while, and then you get a run-time type error which appears only after doing certain actions. This is the main reason that I am avoiding languages like lua and python.
(In addition languages with more advanced type-systems allow you to catch bugs such as buffer overflows or division by 0 at compile time)
The actual bugs I have to fix are nearly always business logic issues. Edge cases around 3rd party integrations, incomplete implementations, unintended side effects, etc.
70%+ of bugs I deal with are business logic issues that no type system could solve.
Sure, as I code I run into an occasional nil object or NoMethod error, but those last as long in Ruby as they do in Swift (about 2-5 minutes while working on that specific part of the code).
A calculation that involves 21 parameters (in a particular insurance industry underwriting) yields a number. A threshold is read from the database. This threshold could change every month.
Suppose that the current value of the threshold is 0.78. The calculation above can yield an `x` with the following cases: (i) x <= 0.78, (ii) x > 0.78.
We have hundreds of test cases for the combinations of the 21 parameters, leading to hundreds of values for `x`. It is a bug for `x` to be > 0.78 when it should be the other way.
Is there a way this can be encoded in types? That would be very interesting.
Thanks.
I'm sure there are developer/projects that both enjoy and benefit from static typing and strict type systems of various kinds. I just want Ruby to remain a place for those of us who aren't in those positions.
However, even with TypeScript ascendant, the vast majority of people programming JavaScript write vanilla dynamic JS. I don't think dynamically typed Ruby is ever going to die. Whether large enterprise codebases will standardize on requiring type signatures is a different matter, because the benefits always outweigh what downsides you see in static typing once you surpass a certain scale.
> "Whether large enterprise codebases will standardize on requiring type signatures is a different matter"
Totally agree that there will always be people who value this tradeoff. That's fine, I just want the Ruby I know and love to keep existing.
I do hear a lot of complaints about Swift's type system. I wonder what the specific problems are, because I do not hear similar complaints about Rust. I wonder if it's the combination of subtyping with a lot of type inference and also a full-on trait system with protocols and extensions and such.
For example, JSON describes a logical structure of nested lists and dictionaries. If you were doing data-level programming, you would just map the JSON into actual nested lists of dictionaries and get on about your business.
The alternative, which is more common in static languages like Java, is to transform it all into some set of domain model objects, and probably validate it up-front, too. Even the bits you don't actually need to look at in order to accomplish the job at hand. IMO, that approach tends to mean creating a lot of unnecessary work for oneself. It also makes it harder to obey Postel's law.
(The corollary to that last bit is that it is also possible for static typing to create bugs.)
A personal example of this was Httpd used to accept standard headers with spaces instead of dashes, this leads to strange behavior if you accidentally include both. So they decided to stop doing that in a major version. This major version was opaquely included by ops accidentally into our base images. This lead to a very long day of debugging on our end.
Point is, by being liberal with what you accept you create ambiguity, which you may not totally understand at the time. By putting that out into the wild you basically are forced to keep this ambiguous, undocumented spec alive or you no doubt will end up breaking some client.
I'm talking about situations where the JSON is formatted fine, it's just that some field wasn't specified, so then the entire input gets rejected. Even though there was zero need to read the contents of that field in the first place. It just happened to be included in some domain object that gets re-used everywhere, including some other places where the field's contents do matter.
Keep in mind that, when we're dealing with anything that might be transmitted in JSON, thinking that there might be a published spec, and that it manages to accurately cover all these details, is really optimistic. I've honestly never seen it happen in the wild. Oftentimes, any validation rules you might try to impose are guesswork as much as they are anything else. So complaining that a piece of data didn't conform to the spec might not even be a valid thing to do. All you can say for sure is that the data didn't meet the needs of some piece of business logic.
It's not perfect, but it's life. This tension, for example, is at the heart of why proto2 got replaced with proto3, and why using proto3 is strongly encouraged if you're looking to build a robust infrastructure.
One way is to treat the JSON as a generic JSON structure, and traverse it manually. Of course, you will have to be explicit about what should happen when children are of different types from what you expect, though this explicitness could just be throwing an exception or ignoring it. Haskell's Aeson and Rust's serde_json both support this, as does .NET's JsonElement type.
Unfortunately, this means you're passing around a lot of objects called something like "JSON" without any information about what they contain at the type level, and as an alternative between that approach and creating domain objects, there are row polymorphic records, which allow you to write functions that accept any record that has certain fields, and also specify that they may also contain other fields which you do not handle. This allows you to program to what you know about the types you're ingesting without having to write a lot of new types.
They're orthogonal concerns. C is statically and weakly typed. Clojure is dynamically and strongly typed. PHP is dynamically and weakly typed. Haskell is statically and strongly typed. Java, as the most design-by-committe language ever, manages to be a mix of all four.
Beyond that I think you’re operating from a misconception about JSON parsing in static languages. There’s no requirement to convert to domain objects and reject data that doesn’t fit on a triviality, you’re just required to specify explicitly what happens when you encounter unexpected structure or data.
Whereas the ergonomics of popular dynamic languages tend to favor an approach that I find, for this specific purpose, to be both less verbose and more robust.
> For example, suppose we have JSON that represents a set of metric data (this isn't our real JSON, this is just a thought experiment) that should look like this, with "tags" being optional attribute: { "id": "1", "timestamp":"12:30pm", "value":"999", "tags": [ "myapp" ] }
> Suppose a python client sends tags but calls the attribute "tag" rather than "tags" (its missing the "s"). Its an optional attribute, so the server won't consider it an error if the "tags" attribute is missing. But it also won't fail due to this unknown attribute called "tag" - it will just silently ignore it now. The Python developer is wondering why his tags aren't being stored - he is getting no errors but they are just silently being ignored. He would need to figure out he is sending in the wrong attribute name, with no error messages to help him out.
> That's the use-case I'm asking about - the "silent error" that will occur due to malformed JSON messages.
Static typing — types checked at compile time. Dynamic typing — types checked at runtime.
I'd consider Python to be more strongly typed than JavaScript. It doesn't do quite so many automatic conversions. For example, in Python, `1 + "foo"` is a TypeError. In JavaScript, it's "1foo". Sadly, `1 == True` in Python, so it certainly doesn't get full marks.
{-# LANGUAGE MultiParamTypeClasses, TypeSynonymInstances, FlexibleInstances #-}
import Prelude (String, (++), show, Int, (==))
import qualified Prelude
class Add x y where
(+) :: x -> y -> y
instance Add Int String where
(+) x y = show x ++ y
instance Add Int Int where
(+) x y = x Prelude.+ y
instance Add String String where
(+) x y = x ++ y
a = ((1 :: Int) + (1 :: Int)) == 2
b = ((1 :: Int) + "aa") == "1aa"
c = ("a" + "aa") == "aaa"I think a more interesting question is typecasts, like happens in languages like Java and C#. These languages are nominally statically typed, but they retains some type information at run-time, so that you can perform run-time type conversions, which requires run-time type checking. Which is the defining feature of dynamic typing.
C# is a little bit more straightforward about being a hybrid static/dynamic language, with its reified generics and dynamic references. But teasing out the details of where, how, and the extent to which Java is statically or dynamically typed would make a decent topic for a master's thesis.
It also hints at a deeper thing that one must be mindful of: static/dynamic and strong/weak are not binary categories. They're not even the extremes of two binary scales. They are somewhat vague descriptions that are meant to serve as useful shorthands for certain sets of choices that one must make when designing a language's type discipline.
But the fact that they're not cut-and-dry terms does not mean that they're meaningless. It just means that one must disabuse oneself of the notion that they're cut-and-dry before one can have a conversation about type discipline that goes beyond a certain level of detail.
Java is a statically typed language with late binding implemented through subtype polymorphism and its type system has been explored pretty extensively in the literature.