Ignition: A new CoreOS machine provisioning utility

Ignition: A new CoreOS machine provisioning utility(coreos.com)

67 points by polvi 10 years ago | 55 comments

Apologies for the somewhat off-topic remark, but ... Can we please stop creating new JSON-based config files? Sane config file formats must support comments.

Sanddancer 10 years ago | |

I wouldn't apologize for that, you're addressing the elephant in the room here. JSON has turned into the new XML. There seems to be a pervasive undercurrent that if it looks website-ish, it's got to be good. So it seems to be used quite a bit in lieu of looking carefully at the problem and figuring out the best structured format. Configuration files tend to have a lot of ambiguity in them, going to a format that makes it all that much harder to pull out the why part of settings does nothing to improve on that.

benley 10 years ago | | |

Honestly, I don't think JSON config files are a big problem. What's shitty is when humans are expected to manually create and edit plain JSON, but what's stopping us from using whatever tools we want to generate those JSON configs?

There are some very nice tools out there for this; I have taken to using jsonnet (http://jsonnet.org/), which is excellent as a configuration language. It's pretty comparable to Hashicorp's HCL, but it's a standalone thing.

Summary: Write your config files in a language of your choice. Render them to JSON for your apps to consume. Be thankful that every app doesn't have its own special snowflake config format like they used to.

geofft 10 years ago | | |

> There seems to be a pervasive undercurrent that if it looks website-ish, it's got to be good.

I don't get that from the article. There's a stated reason, that it's a stricter format with much less risk of type confusion, which sounds plausible enough to me. There's also the benefit that tool support is good. Why does the web need to be relevant at all?

kkirsche 10 years ago | |

100% agreed. This is the reason I personally find YAML to be a great config alternative to JSON due to their ease of converting between the two

brandonbloom 10 years ago | | |

YAML isn't a particularly good config format either, since it has a lot of unnecessary complexity. For example, bare words are strings, but some bare words, such as "true" are booleans. Which wouldn't be so bad, but did you know that "yes" and "no" are also booleans? I've gotten YAML files from an i18n-system that screwed this up in a pretty bad way.

jonaf 10 years ago | | |

In fact, YAML is a superset of JSON, and any valid JSON is also valid YAML. This becomes more obvious when you realize you can put any JSON objects inside of your YAML document and any YAML parser will correctly deserialize it.

otterley 10 years ago | |

At long as an app isn't strictly validating the JSON, you should be able to add keys like { "comment": "this does xxx" } to a JSON configuration object.

e12e 10 years ago | | |

There's that, or:

"Comments in JSON

I removed comments from JSON because I saw people were using them to hold parsing directives, a practice which would have destroyed interoperability. I know that the lack of comments makes some people sad, but it shouldn't.

Suppose you are using JSON to keep configuration files, which you would like to annotate. Go ahead and insert all the comments you like. Then pipe it through JSMin before handing it to your JSON parser."

https://plus.google.com/+DouglasCrockfordEsq/posts/RK8qyGVaG...

pjmlp 10 years ago | |

I just keep using XML, when the decision falls on my side.

- It supports comments;

- A very flexible format;

- I don't care it is verbose, it should be managed by tools anyway

- Can be automatically validated;

- Plenty of tools to choose from

pm 10 years ago | |

Any suggestions for a format that is sane?

mintplant 10 years ago | | |

TOML.

https://github.com/toml-lang/toml

cookrn 10 years ago | | |

Hashicorp has created HCL https://github.com/hashicorp/hcl

benley 10 years ago | | |

Jsonnet is a really excellent one, imo: http://jsonnet.org/

There is also Flabbergast, which is comparable but a little less intuitive for those accustomed to Python-like languages: http://flabbergast.org/

The flabbergast homepage has a comparison table that covers most of the things mentioned in this thread: http://flabbergast.org/#comparison

ddispaltro 10 years ago | | |

My personal favorite is HOCON from Typesafe, https://github.com/typesafehub/config#examples-of-hocon

benley 10 years ago | | |

This paper also pretty interesting: http://www.masella.name/~andre/2015-srecon-andre_masella.pdf

pjmlp 10 years ago | | |

I am going to be downvoted, but for me XML is sane and I don't bother to use anything else when given the option.

darren0 10 years ago |

The transition away from cloud-init and YAML is really quite odd to me. Nobody enjoys editing JSON files. Forgetting commas, no multi-line strings, no comments, escaping characters in strings, etc. Just reading the documentation of ignition should be enough to illustrate the pain it is to manage multiline content for unit files in an JSON string.

But why abandon the cloud-init format in general? Again, why would somebody want to learn a new configuration syntax. Using CoreOS already requires you to know and use systemd units (most other distros this really isn't required knowledge), so that adds two steps to users learning/using CoreOS.

chrissnell 10 years ago | |

I'm totally with you. We're big users of CoreOS and while there are aspects of cloud-init that I find janky, it's at least mostly easy to read and understand. The use of JSON is nuts. Just looking at the one-shots consolidated to a single line with \n's gives me a headache. This is moving forwards?

What I would prefer is a compiler. You feed it a directory of unit file drop-ins and app config templates and it builds a single artifact that can be served over HTTP and pulled by the server booting up. This could allow for dynamic configuration and automation but still makes it easy for the admin to piece the config together.

robszumski 10 years ago | | |

Check out bootcfg, a server that sends out customized cloud config or ignition files based on mac address, IP, serial number, etc

https://github.com/coreos/coreos-baremetal#bootcfg

seanp2k2 10 years ago | | |

A compiler. So now you have another language which compiles into JSON. So now there at least 2 problems, more if we want to be able to edit these configs on the systems directly, since we'll need some kind of tool chain.

Why do we keep re-inventing this wheel over and over again? Very plain text configs have worked for decades.

nikolay 10 years ago | |

YAML is good for simple configs. As the article says, JSON is much easier to programmatically generate than non-standard YAML. You still can store files in YAML, but compose and convert them into JSON in user data.

seanp2k2 10 years ago | | |

Again here, now you have 2 problems. Why not just go with a simpler config language and skip converting it and all the complexity that adds? It's possible to make this extremely complex, to the point where the config is in a database with ACLs and requires a DB server system (which itself requires config) to host (e.g. WebLogic) , but what do you gain?

_0w8t 10 years ago |

JSON is not language that "makes it very easy to write tools to generate new configs or manipulate existing ones". One cannot realistically consume JSON in a shell script, and even with tools that have native JSON support manipulating the config require the knowledge of document semantics as the language itself does not define how to combine or merge 2 documents into one.

For example, consider an application that comes with a complex JSON (or YAML as that language also share this shortcomings) that essentially describes default settings. I cannot just define an extra file that specifies a couple settings that I alter/delete/add, I have to have my own copy of the original file with my changes resulting in painful merge when the original is updated.

What I have found is that things like .ini files or config files in style of ssh_config work much better in practice. It is easy to generate and process them in any language with a notion of text IO including shell scripts and the merging functionality can be provided independent of semantics so I can keep config fragment outside the main file/files.

HorizonXP 10 years ago |

Alex this looks great and likely addresses a lot of the problems I had with cloud-config. I was one of those with a Bash script that I used to generate multiple files, so this is great.

My biggest issue so far is CoreOS' naming of Ethernet interfaces on VMWare ESXi. It always uses some eno* name for each interface. I have a unique case where each VM I spin up has up to 10 interfaces.

I've solved this by adding net.ifnames=0 to my grub.cfg. It requires that I reboot the machine at least once to get it to take.

If I could have predictable interface names using Ignition, then I'm set!

jsmthrowaway 10 years ago | |

> My biggest issue so far is CoreOS' naming of Ethernet interfaces on VMWare ESXi. It always uses some eno* name for each interface. I have a unique case where each VM I spin up has up to 10 interfaces.

That's a systemd decision, not CoreOS, and also impacts the 7.x series of RHEL derivatives and anything that uses systemd > v197, really. Your way is one of three to revert it.

https://www.freedesktop.org/wiki/Software/systemd/Predictabl...

dghubble 10 years ago |

Some useful Ignition configs can be found in the https://github.com/coreos/coreos-baremetal examples. For those wondering about the format, they're converted to JSON and served to machines.

songgao 10 years ago |

I'm curious about how it works. I must be missing something, but if it runs before the network and file system are setup, how does it pull configure from a http:// URL (say on bare metal) or oem:// which is on FS? Does it run with a default setup for network/fs and let systemd redo everything later?

rdtsc 10 years ago |

> Running before systemd helps ensure that all services established by Ignition are known to and can be managed by systemd when it subsequently starts. This allows systemd to do what it does best: concurrently start services as quickly as possible.

I think this looks like a slight jab at systemd ;-)

Systemd thinks it does a lot more other things best

wmf 10 years ago | |

I think that was poorly explained on their part. Digging deeper into the docs, it looks like Ignition writes unit files that systemd runs; e.g. networkd is still used to configure networking.

thebeardisred 10 years ago | | |

Affirmative. This is just a mechanism to make sure that interfaces can be properly plumbed, disks formatted with the desired states & filesystems, etc.