TypeSpec: A new language for API-centric development(typespec.io) |
TypeSpec: A new language for API-centric development(typespec.io) |
(My $0.02 as someone who works on TypeSpec)
But I've also been kind of holding my breath for typescript making it's breakthrough as a schema language. More specifically its surprisingly big non-imperative, non-functional subset, obviously. And at first glance this seems to be exactly this, "what if we removed javascript from typescript so that only typing for JSON remains and added some endpoint metadata description to make it a viable successor to openapi and wsdl.
Challenge accepted!
https://github.com/bufbuild/protovalidate/blob/main/examples...
it's fairly concise, the method and the request/response types are well separated and readable
the only thing I could argue with is mixing validation and type defs, as it looks like one of these things that quickly evolve over time and you end up duplicating both in the schema and the business logic
If for some reason your problem does involve tapping the depth of typescript type expressivity (the elaborate rule systems expressed in maplibre style JSON come to mind?), you'd better have the closest approximation you can get on the other end of the line.
One would think, unfortunely that is not how it looks like in the zoo of native Windows desktop development, or Xamarin/MAUI adoption in their mobile apps.
I've used Microsoft Graph to manage emails, and I'd be very surprised if they use if for Outlook...
I wouldn't say that TypeSpec is like GraphQL, so it would be hard for TypeSpec to become that on its own. GraphQL has a lot of opinions that are required in order to build a concrete application (protocols, error handling, query semantics, etc.), whereas TypeSpec at its core is just an unopinionated DSL for describing APIs. Bindings for particular protocols are added via libraries, and a GraphQL library is something we have long considered.
So in short, if Microsoft invented a set of opinions that solved similar scenarios to GraphQL, it might use TypeSpec as the API description language in that context, but it wouldn't be fair to equate those opinions with TypeSpec itself.
https://en.wikipedia.org/wiki/Web_Services_Description_Langu...
Perhaps it will last longer than WSDL did?
1. A more exact analogy would be WSDL+SOAP.
2. WSDL and SOAP are defined in XML, and SOAP describes XML.
3. The popularity of these technologies followed the popularity (both rise and decline) of XML generally.
4. TypeSpec describes JSON and protobuf, and will likely also lose popularity if those formats do.
Nowadays big companies rarely work together and prefer to throw their own solutions to the market, hoping to capture it. That results in a higher quality approaches, because it's developed by a single team and focused on a single goal, rather than trying to please 10 vendors with their own agendas.
WSDL 1.0's list of editors reads [2]:
> Erik Christensen, Microsoft; Francisco Curbera, IBM; Greg Meredith, Microsoft; Sanjiva Weerawarana, IBM
IOW, TypeScript is by the same company as SOAP and WSDL.
> Nowadays big companies rarely work together [...] That results in a higher quality approaches
[Citation needed]
Codegen is annoying and error prone.
The "annoying" part I do get -- it's obviously nicer to have instant feedback than being forced to rebuild things -- so for things where you iterate a lot, that does weigh in the other direction.
A subset of typescript could work, but I imagine it would be fairly confusing to support some features here and other features there.
I think they are going for a minimum common denominator approach and eventually add other targets besides OpenAPI.
I'd be delighted to have a high-level IDL that gave the same sort of thing that CORBA IDL gave us 25 years ago -- schema and stub generation for multiple languages.
They provide an API to convert to openapi documents so it was pretty painless (https://github.com/mnahkies/openapi-code-generator/pull/158)
My focus is on doing both client sdk and server stub generation, though only typescript so far - will hopefully add other languages eventually, when I'm satisfied with the completeness of the typescript templates.
1. People don't want to learn bespoke languages.
2. You have to build all the ecosystem tools for a language (compiler, docs, language-server, IDE integrations, dependency management)
Similar endeavors are WaspLang and DarkLang, which I have yet to see in the wild or (meaningfully) on HN. Better to use an existing language and focus on the value add.
I personally built something with similar value add (source of truth -> all the things). I've been through some of these pain points myself.
https://github.com/hofstadter-io/hof
The idea is CUE + text/template = <all the things> (not limited to APIs)
I think this will become more common and not really a barrier
I've been working on some AI related stuff lately, part of the reason for the slowdown. And actually using hof myself for real work
Code gen is pretty stable. I've been meaning to fix the TUI keybindings on mac before releasing the current beta. Was also hoping the the evaluator improvements would land upstream, but that hasn't happened yet...
I'll take a stab at releasing a new version this weekend, per your inspiration
I guess it's NIH for Microsoft. Oh well, at least it's TypeScript.
1. Read a .pkl file from disk and generate (for example) a Person struct with a first, last name and an age value.
2. Let's say that according to some TypeSpec, the HTTP endpoint /greet accepts a POST request with a JSON payload containing a first and a last name. You convert your Person struct into a JSON literal (and drop the age field in the process) and send it to the HTTP endpoint
3. You should receive a greeting from the HTTP endpoint as a response. The TypeSpec may define a greeting as a structure that contains the fields "message" and "language".
4. You can then use pkl to write that greeting structure to disk.
Sidenote: pkl also supports methods[1] but in almost all use cases you only use pkl to define which fields your data has. TypeSpec cares most about the methods/endpoints. Of course, you still need to define the shape of the arguments to these methods and that is where you have a bit of overlap between these two technologies. I can imagine that you would generate pkl definitons from TypeSpec definitions.
[1] https://pkl-lang.org/main/current/language-reference/index.h...
I might be wrong, but I suspect that the crazy hype-driven-development has started to move on from frontend to backend.
I wasn't alive in the 1970s, but I'm guessing that those who were would say it was just as faddish then as well.
Since CUE is written in Go, you can output a .so that is then used like your C based desire, if I understand you correctly
I have a project in mind and was looking for something like this. Closest I found was CueLang.
Now just need to find the time...
Any plans to add code generatio to this project?
Feels like it's going backwards - there's really no reason why it has to be a .tsp, instead of a .ts with actual api code. It's even using @annotations. In fact the annotations i see in the screenshot (@route, @query, @path) are practically the same in NestJS.
I feel that we should be focusing on enhancing that paradigm instead. In fact I already have a working POC of NestJS -> OpenAPI -> Client libraries so I see no place for this. The spec itself is simply a vehicle for code generation, and serves little purpose otherwise and I'd be happy to be rid of it.
Is this something that helps if you, say, are building a new API and will need to create both the server implementation as well as the client implementations in multiple languages? And so, it can automatically do all of that for you based on an API spec? Or is it something different.
Funnily enough, I developed a Python library recently that allows you to build API clients in a way that very closely resembles the TypeSpec example. But I'm pretty sure they are very different things.
Not to mention, how else do you see what complex logic might happen in an endpoint?
It seems typespec deals only with extremely simple CRUD APIs, for which again just reading the code would be good enough.
In scenarios where you want to offer the API consuming team some mock, I'd argue time would be better spent providing them with a a json-server implementation (see: https://www.npmjs.com/package/json-server).
Looking at https://smithy.io/2.0/index.html which can already generate much more than TypeSpec based on the docs and awesome list.
If it does become a long lived artifact, CI/CD must also be a nightmare, having to figure out which commit matches which version of the specification, since the spec is now a distinct artifact from it's documented target, and similar which version of the client. A literal "3 body problem".
On the other hand, if you already have a project template (granted, you do need to fight through all of the various build-time configuration required to get a typescript project up and running) you could probably achieve the same by simply stubbing the API endpoints in code to generate the spec.
If there was an advantage to a spec first model, it would be that any change to the api would be a conscious change, and highly visible. I've also encountered situations where a innocuous refactor (changing a class name or method name) broke the previous builds. But one could potentially integrate a gate into CI/CD by diffing the outputs of generated specs.
Much of my opinion on this subject is based on my own experience using Postman as a pre-implementation spec. But conceptually I see the same problems arising from any spec-first approaches.
You don’t always go server -> spec -> client.
Even so, a e2e test suite would surely serve far more utility over a spec that simply stubs out endpoints with no functionality.
At the moment, OpenAPI with YAML is the only way to go but you can't easily split the spec into separate files as you would do any program with packages, modules and what not.
There are third party tools[0] which are archived and the libraries they depend upon are up for adoption.
In that space, either you can use something like cue language 1] or something like TypeSpec which is purpose built for this so yet, this seems like a great tool although I have not tried it yet myself.
[0]. https://github.com/APIDevTools/swagger-cli
[1]. https://cuelang.org/
EDIT: formating
My question is probably a more general one around the use case for writing an API spec (in a format like OpenAPI or TypeSpec), and then translating and writing the server implementation to match that. As opposed to being able to create the API spec automatically based on server implementation (and being able to easily refresh it).
Understand that writing the spec and then the server implementation seems to have some benefits. I'm curious to hear about the common use cases for it, as in my mind I could quickly stub a server implementation (and automatically generate a spec) rather than try to create the spec by hand and then write the server implementation again. But I'm sure there's some other things I'm missing.
Eg: https://github.com/mnahkies/openapi-code-generator/blob/main...
I believe recent versions of OpenAPI are "compatible" with JSON Schema (at least they "wanted to be" last I checked as I was implementing some schema converters).
Even TypeScript is not enough to represent all of JSON Schema! But it gets close (perhaps if you remove validation rules and stuff like that it's a full match).
But even something like Java can represent most of it pretty well, specially since sealed interfaces were added. I know because I've done it :).
See my comment below to another question [0]
That's the upside that instead of generating specs from comments of methods, you actually generate the methods from the formally verified/linted and crafted spec.
And then one proceeds to spend days trying to mash that JSON mess into a protobuff and debugging segfaults, rather than just getting the job done with HTTP.
For example, the problem of matching commits with specs doesn’t even exist in environments without continuous deployment (which is rarely a real necessity and often is even undesirable). You just tag your releases in VCS (can be easily automated) and track their scope in documentation (job of responsible product and engineering managers which know what goes live and when).
How would you handle a typical case where based on request data an endpoint fetches some additional information from various sources and depending on that performs several different actions?
This is the most common scenario I've encountered outside of extremely trivial CRUD endpoints.
EDIT: don't get me wrong, I'm not being purposefully obtuse - I was a big swagger advocate back in the day, however over time I came to realize that effort was much better invested in writing clear API code and possibly a mock impl like json-server.
It leaves the middleware library selection for the user, and with middleware you can do whatever more complex operations you need.
TypeScript server overview: https://smithy.io/2.0/ts-ssdk/introduction.html
> This execution flow allows the service developer to choose not only their endpoint, but the programming model of their service. For instance, all of the shim conversion and handler invocation can be refactored into a convenience method, or the service developer could choose to incorporate their favorite open source middleware library, of which the server SDK would simply be one layer. It also allows open-ended request preprocessing and response postprocessing to happen independent of Smithy. For instance, a developer could add support for request or response compression, or a custom authentication and authorization framework could be plugged into the application before the server SDK is invoked, without having to fight against a more heavyweight abstraction.
The Service type itself also seems to make it possible to define quite complex logic: https://smithy.io/2.0/spec/service-types.html
Coral is used by every AWS service -- every AWS service defines their APIs using Coral, and if you want to interact with another service you use a generated client.
The client can generally be used as-is, though sometimes you might want some custom client code. In the case of a generated client you can just grab the API definition of the API you want to call, generate the client in the language of your choice, and... that's it!
For the server, you still have to implement the logic of how to form the responses, but what the request/responses look like it enforced by the framework.
Here's an example: https://gist.github.com/shepherdjerred/cb041ccc2b8864276e9b1...
I'm leaving out a _lot_ of details. Coral is incredibly powerful and can do a lot of things for you for free. Smithy is, from what I can see, the same idea.
https://github.com/RicoSuter/NSwag
There is no need to be facetious, solutions like these exist for many platforms and ecosystems out there. I can't possibly entertain all of them.
By all means, if such similar solution does not exist for your platform, then TypeSpec could indeed be a solution for you.
With best regards.
Code-first:
- No extra steps between code and running application
- No mismatch between schema and code
- Requires tooling for every language used in your stack. This tooling is usually more complex than schema-first codegen, but it is almost always built-into and core of the backend framework you are using so it tends to be better supported.
- Requires teams to know each service's backend language to propose changes
- Harder to build a coherent central API documentation if you have multiple services. Requires complex tooling for merging the different schemas from the different services
Schema first:
- Schema first means contract-first design, scales better to products with multiple separate teams using multiple languages. It is easier to learn the schema DSL than poke around backend language unknown to the developer
- Any change in the contract requires changes in two places (schema and code)
- API consumers can easily suggest changes to the schema that are implemented by the relevant team
- Usually requires some codegen step for each backend and client languages (with the usual codegen problems). Runtime-only schema validation can be done but it is usually a bad idea to rely only on it.
- Easier mocking. Clients can start implementing before backend is ready, tests can be written against mocks.
There are definitely more pros and cons that I am missing, but it is a tradeoff. I would say if you have multiple backend services and supporting multiple backend languages I would definitely go for schema-first.
You can work with OpenAPI in a schema-first way but as many people have pointed out over the years OpenAPI yaml files are, to be polite, not very human-friendly. TypeSpec seems to be a more sensible way to work with HTTP apis in a schema first way while keep interoperability with existing OpenAPI tooling at the cost of extra codegen step (typespec .tps -> openapi .yaml)