Since then the situation hasn't gotten any better, but sort of stayed the same. Two people on our team have been solving bugs in this DSL for the last few months, rather minor ones. Tomorrow in fact I have a meeting with one of them about adding a sort of limited pass-by-reference semantics into the language (which currently doesn't allow pass-by-reference). I'm very torn about it, as I feel this is a waste of time, and we should ditch this DSL. Users complain rightly that it isn't a "real" programming language. Our "standard library" is very very lacking, for example there's no I/O.

We just added FFI 2 months ago. My team lead who was supposed to do this years ago didn't, and in fact effectively stopped developing it 2 years ago, feeling overwhelmed, and ended up leaving a couple of months ago.

What this language or rather ecosystem DOES do well is having an internal build system that is completely transparent to the users (unless there are bugs). Well, I designed this build system so I'm partial to it.

I had actually written the C# API which I wrote about at the end of [0], that was supposed to be a replacement of this DSL with a "real" language. However I got burned out a couple of months ago after my team lead left, and halted development. There was also a long series of heated discussions with some of my users, who wanted me to remove some of the (rather simple) guardrails I made, and give them the ability to build complex hierarchies above this API, an API which I designed for simplicity. I warned them those hierarchies would end up creating DLL hell and would make it necessary to build devops tooling and maybe even infrastructure as code [1], which IMO is silly for most of the internal projects, which contain at most 5-6 people. From 2 different organizational groups, but still. My users aren't developers per se, and have a more or less support role.

I'm still torn as to what the correct direction here. Long term our small team (only 2 full time developers and 3 part time) cannot support 2 half-assed non-standard DSLs, plus that C# API for our many internal users. Something gotta give.

[0] https://news.ycombinator.com/item?id=31205265

[1] There, I tied this response to the OP!

Tools like terraform offer simple modules to make your devops life easy.

But simple looking modules abstract a complex thing by hiding details. Details which are critical once your infrastructure reaches a critical complexity threshold.

At this point the tool is a burden. It is actively getting in your way to understand and get at the the actual infrastructure you are maintaining.

This is why imo alternative like CDK got popular. It doesn’t try to over abstract for you. It just gives you an api you write custom code over. The complexity of your code goes up with the intrinsic complex of the infrastructure.

By compare every dsl requires learning some new language. The languages seem to grow and grow. I know plenty of people with a little bit of Terraform experience, but almost no one with extensive knowledge. It's great that surfing along the top works as well as it does, and having such a confined language keeps a normativeness, but in general I so strongly feel it's just better to use available known well supported tools.

True, but it's not an either/or situation. Maybe it's not optimal, but a lot of people do both. I consider myself a Python and Terraform expert and I'd say I reached that stage in Terraform in less than 10% of the time it took for Python.

The example given is pretty good at demonstrating how Terraform itself has some unique problems that they themselves created: using counts for enabling/disabling resources, using locals as proxies for data comprehension, writing long-ass resource descriptions, often with built-in functions and ternaries to get it to do what you want in various scenarios. Trying to be DRY in Terraform can often be a chore. I think Terraform's MO is actually that some duplicate code isn't the worst thing, so you end up with piles of duplicate lines that are extremely difficult to read, and where writing a function would be super helpful. We had an engineer do a bunch of Terragrunt work for a project and I found that code nigh-incomprehensible.

So, I get it as well, I get where they're coming from.

I also think working with DSLs isn't a huge waste of time, since many of the things we work on are abstractions of abstractions. It seems like everything I'm doing today is probably going to go away in due time, but I know how to program, read docs, and pivot quickly. I do feel like as long as you're gaining an understanding of the underlying elements, your time spent in a DSL (assuming you're getting paid or you end up using it for several years), is still well spent.

They mention Puppet, and I remember getting a job offer once because I was able to read the Puppet documentation and write out a little script that could do the ask in Puppet, even though I had never used it in a professional capacity before. It was easy to pick up and I would've happily worked in it if I accepted the offer. The knowledge I gain around deploying to AWS or other cloud providers is often transferrable in that manner too.

The clouds themselves are a sort of DSL often, but I can't say that my time spent with Azure was a waste just because I'm working in AWS now. No, we delivered stuff on that platform, I learned its pros and cons, I gained some wider understanding of what cloud platforms could do, and I got paid along the way :)

  budget: $23000
  pricing: cheapest
  cpu:
    preferred: xeon
    cores: 8
    mem: 64gb
  disk:
    preferred: ssd

However, this is not at all an argument against DSLs. Using a "real" language to specify the cloud infrastructure, you'd still want your API to be declarative and ergonomic, i.e. you'd want a DSL.

A lot of DSLs are a wall to understanding what each "figure of speech" actually does. You need to navigate a parser before you get to the underlying execution of the language

Really, if OpenTF happens, you’ll be back to square one?

“ When the maintainers of OpenTF inevitably decide that they want to add enhancements that require language changes, you as the end user end up in a situation where you’re effectively back to square 1, learning (potentially) a new DSL.”

Define your abbreviations, people, especially when its not the first hit when googling it. It's not that hard and makes your writing much more accessible.

  [for x in x.addresses: x if length(regexall(".*[:].*", x))>0]

Throwing a full programming language in place of a DSL means now you deal with everyone and their mom inventing their own way of doing things.

I have no dog in this fight. But I do have opinions about DSLs.

The declarative vs imperative, or some hybrid, debate is ongoing with build systems.

James Duncan Davidson, author of Ant build system for Java, famously wrote a post-mortem. Mostly wrt to choice of XML for Ant's syntax and semantics. TLDR: Wheels fall off a declarative model once imperative logic is added.

FWIW, Here's how I think about declarative DSLs. Such as a scenegraph.

You have an exquisite mental model for a scene. It's probably hierarchical.

You attempt to capture that mental model in code. The representation is probably a grove data structure. aka Directed Acyclic Graph where nodes can have key/value pairs for metadata.

You need to serialize that grove. So you pick a suitable syntax. Hopefully something that looks like VRML-97. Basically s-expressions (JSON without the syntactic vinegar), with some way to represent object prototyping (define/use), and don't forget identifiers ("id" field). Bonus points if you include path expressions in your syntax.

That (human readable) serialized grove is the declarative DSL.

In an future perfect world, your mental model, runtime representation, serialization format, AND the real world will be isomorphic.

Not isomorphic? Of course they're isomorphic. Why even mention it?

Funny you should ask.

In the Java world, dom4j for representing XML documents, and most of its progeny, is not isomorphic. Only JDOM2's class hierarchy design actually matches the serialization format. (As of the last time I checked, a few years ago.)

dom4j's mismatch, leaky abstraction, poor design, what have you, begat untold additional labor and heartache. Which is all completely mooted by JDOM2's correct design.

Ensuring correct isomorphism avoids errors in the same way that strong typing and garbage collection do.

To wrap this up...

In my experience, isomorphism is the exception.

I have no idea if Terraform is a good declarative DSL. But I do know that it should be isomorphic with the runtime representation AND the real world. And that should be settled before weaving in imperative functionality.

> local tool... to sanity check

Yes, exactly. Some open-source project as a layer between the cloud providers, and budget and other elements are just knobs to twiddle.

If this "open pricing" project could swarm various known constraints between money and virtual hardware, would it be a kind of constraint satisfaction problem?

If the environment hosting and managing the queries (e.g. psql) is open source, maybe you can hack in support. Often that is not the case. Even when you are able, then your work becomes non-standard and thus not easily shared with other analysts, which kind of defeats the whole sharing of modules.

If you are writing a full-fledged application, using "real" programming languages, then a lot of people build their own query languages on top of SQL, treating SQL as just an "assembly language" that their query language compiles to, to add what SQL lacks. That's all well and good, but totally overboard when you just want to answer some questions about your data.

I do think you make a point that SQL tries to be too many things to too many people. No question, it is useful being able to embed SQL strings into other code without needing to worry about complex dependencies... Except you still have that worry as we already tried adding that composition through stored procedures, views, etc. which are dependent on a particular database state being true. It's there, just not very well thought out from a developer ergonomic perspective. Indeed, SQL is confused.

They are also not application specific. Language design is hard. DSLs are made up by people who's main project isn't the language.

Sometimes the motive for a DSL is something like "I don't like how this takes 5 lines of code in JS, if I made my own language it would only take 3"

Regardless the author getting 7+ years of runway out of a specific technology is hardly a waste! My average technology switching time is probably closer to 5-6 years.

Both are (or at least were originally) “crippled” compared to “real” languages.

Both grew immensely each eventually escaping their single-vendor origins.

Exactly - Unless you are doing something really unusual (or wrong), the vendors of these engines have almost certainly encountered and optimized for some shape approximating your scenario.

If you want to get into some extreme ends of the practice, "ancient" engines like DB2 are some of the most capable. Many have been around longer than most developers today have been alive (myself included). That is a lot of optimization legacy to argue against. The halloween problem & iceberg meme comes quickly to mind. Why wouldn't you want to stick with something that has already dealt with all of that bullshit?

If possible I'd rather examine why people need so many config options, and design from the start so that things don't need to be manually configured as much.

If a turning complete config really is the way to go, I'd rather just use JS or Python, not something metaprogramming focused that just invites and almost requires building more layers on top to do real programming.

so i am favoring the two products approach, or maybe a language with a well designed syntax for data, where the relationship between the two is like that of JSON and javascript.

The engine doesn't compile to a declarative template. It's a common misconception, however because the user's program runs concurrently with the Pulumi engine, fully dynamic resource graphs are possible, including those inexpressible using templates.

This is Pulumi pseudocode:

    const group = new ldap.DirectoryGroup("$saas-users);

    // or any other runtime query:
    for (const member of group.getMembers()) {
      new saas.User(member.id, {
          email: member.email,
          name: member.fullName,
      });

      // could also do some other dynamic action, such as 
      // calling a webhook to complete user setup

(n.b.: this is a simplified, illustrative example that the program drives the graph using its own using loops or other control flow.)

I'd prefer it to lisp, and to JSON if I have to hand read or write it.

What's the alternative, besides specifying your declarative stuff inside something like Python?

I won't need to, you're already giving yourself carpal tunnel with all the :wQ! INSERT-MODE :wmkGDonzn zz CWSFD, these hands are already marked for death.

Prolog for Pulumi is an absolutely doable thing - and indeed sounds sufficiently diabolical that someone should do it ;-)

i think at least half of those who keep up with the emacs vs vim war are doing it just to confuse newbies. the other half really just prefers the emacs way or the vi way.

either way though you are right. they are both very distinct, and it's not one being absolutely better than the other.

> Regular expressions have a deep mathematical background. An ad-hoc DSL from a company that specifies infrustructure does not.

You very specifically went far beyond making any statement about ad-hoc DSLs from companies that make infrastructure. You said "DSLs are, universally, awful".

I'm also interested why you left out makefiles in your original rebuttal. I think makefiles are a good example a simple and useful DSL for accomplishing what it sets out to do. That's not to say it can't turn into a mess (what language, domain specific or not, is free from that concern?), not that it solves all needs to all people, but makefiles are simple, straightforward, solve a useful problem, are well known by many people (and easy to explain and teach), and I don't think most people would consider them a mathematical concept core to computer science. Are they awful? If so, why?

A Regex, SQL, TLA, CSS and so on are all better DSLs than allowing a general purpose language because they are meant to restrict inputs of lower priority than other goals, where priorities are largely fixed.

A corporate DSL is meant to expand to whatever grandiose plans the company has for accepting impractical input from platinum customers.

As I’ve quite clearly said, regex is a good DSL - so I obviously do think, and have argued above, that there is a split between “good” and “bad” DSLs, but examples of good ones are vanishingly scarce.

I think, given that it’s your position that DSLs are generally good; or at least that there are many examples that you find fine, then it’s upon you to provide examples. So far you named Makefiles, which are hilariously bad, but your response to my comments on it was basically some lazy ad-hominem, so I don’t hold out hope for any actual intelligent analysis.

But as I have outlined above there is a giant chasm of difference between mathematically-backed deep and broad DSLs, and… some ad-hoc commercialware that iterates towards being a full language but with bad, ad-hoc design and worse tooling.