In my mind if you want to build a competitive library that will attract users for reasons other than "It uses language x" doing some in depth research of the existing solutions and their shortfalls is probably a good idea.
For instance, the reference to "copy/paste" you mention, and which is easy to misinterpret. Recipe reuse in chef is very much at the source code level - checkout a recipe and edit it for your specifics. Pallet places much more emphasis on using recipes (or crates as we call them) as library functions, in cleanly versioned packages (jar files). This allows you version your recipes, add them as dependencies to your projects, and cleanly separates the configuration data from the recipe code.
On a perhaps more fundamental level, the push model seems to be much more suited to automating configuration across nodes - eg. getting your haproxy pointing to your web frontends. While this is somewhat possible in Chef, it is left to the user to achieve. In Pallet there is first class support for this sort of cross node configuration. It makes it easy to have mixins, that say ensure a node is monitored, or it's log files collected.
A couple of additional advantages of the push model are that it leaves you free to use API's without having to push credentials for those api's to all your nodes and allows you to use Pallet for command and control.
Finally Chef and Puppet are both great frameworks. Pallet is a library.
Hopefully this outlines some of the differences in approach between Pallet and Chef/Puppet. The use of clojure was certainly not the driving force, and I hope that wasn't the main message that came across.
Recipe reuse in chef is very much at the source code level
- checkout a recipe and edit it for your specifics. Pallet
places much more emphasis on using recipes (or crates as
we call them) as library functions, in cleanly
versioned packages
This is something that I can certainly appreciate and something that I've discussed at length with colleagues. In general dsl's require that you perform some impressive acrobatics when you want to extend them and this becomes particularly frustrating when you consider that a programming language like Ruby already has many well thought out facilities for said extensions (mostly directed at chef here). I'd like to think there's some happy middleground between a dsl that's easy to dive into and making use of the language constructs we've come to depend on everywhere else.
There is also a new way of managing cookbooks more like the Ruby Bundler called Librarian https://github.com/applicationsonline/librarian
It's also super fun to use, partly because Clojure is a great language.
Currently this comes with the price of a steeper learning curve, but the benefit is well worth it, as far as I'm concerned.
Each community has its own requirements and those of the Chef cookbook authors in general are different than those of Ruby library authors.
Lets say that I right a cookbook for apache2 on Ubuntu. What should I name the gem? Many other people will write this cookbook, particularly since mine only works for Ubuntu. Perhaps 'chef-apache2-ubuntu-btm'? Where do you store such metadata so a user can find, use, and contribute to a cookbook?
This isn't a simple solution and is a bit unique, which is why Opscode has been crafting a solution at http://community.opscode.com. It is easy to get a copy of a cookbook from the repository using 'knife cookbook site install COOKBOOK_NAME' and you can search the site through the web interface and using 'knife cookbook site search' to find the cookbook you're looking for.
The community site development has been slow, but continuous. In December we added source browsing so you can review cookbooks without downloading them.
http://www.opscode.com/blog/2011/12/14/chef-community-site-c...
There's still much to do, but I think we're (I work for Opscode) headed in the right direction for the project and community.
That said, perhaps none of that really matters; I wonder if all this comes down to hidden assumptions re: constituencies and heterogeneous skillsets, which I think is what you were pointing at in the beginning of your comment. i.e. "you need to learn the Chef toolchain" may be a more sale-able/scalable message than "you need to learn the Ruby/gem toolchain".
There are many views out there for what the answer should be for Chef cookbook workflow. What you are "baffled" by, another person uses daily and argues for.
When using knife to download a cookbook into your git managed chef repository, it creates a 'vendor' branch to store the upstream version. You can utilize this to create diffs and patches to contribute back to that cookbook project. It is up to the author exactly how to do that.
Some really like git submodules, others prefer one cookbook per git repository, some just have one company-wide repository and ask you to fork and apply branches against.
I think you're right about the 'Chef toolchain.' There are [large enterprise] users of Chef who don't care that Chef is written in Ruby nor particularly want to learn Ruby. They want to use Chef. Opscode continues to strive to provide tools to make the workflow easier for Chef users as a whole over preference to a particular toolchain, especially one tied to and solving problems for a single language.