Maybe stop using Grunt?

58 points by NTH 11 years ago | 49 comments

falcolas 11 years ago |

I keep seeing people mention "a mess of shell scripts", and then use replacements which are dozens if not hundreds of lines of custom-configuration files.

Shell scripts, when you're not dealing with autotools generated script, aren't really that big, or even that complicated.

Not to mention that the ability to chain disparate tools and subshells can be incredibly elegant and powerful.

jeswin 11 years ago | |

I agree mostly. But when you're in the node ecosystem, often times you need to run JS as part of the build. For example, you might need to create some JSON data as part of the build.

I ended up writing my own build library (npm install crankshaft). Docs aren't up yet, but here's what build scripts look like - https://github.com/jeswin/hitchslap/blob/master/build.js (simple example) and https://github.com/jeswin/fora/blob/master/www-client/build-... (more complex, with dependent tasks)

tieTYT 11 years ago | |

> Shell scripts, when you're not dealing with autotools generated script, aren't really that big, or even that complicated.

They're not always portable. Some people develop on windows.

falcolas 11 years ago | | |

Windows offers shell scripts as well. Powershell, instead of Bash/Zsh/..., but they still offer many of the same opportunities.

Considering that teams frequently homogenize on one type of machine - bet it windows or Mac or Linux, shell scripts are typically portable enough for these use cases.

serve_yay 11 years ago | |

Indeed. For JS projects these days I just use npm as the command-line tool. npm scripts gives you an easy way to create commands for building, watching, etc.

jessaustin 11 years ago |

...I would like the task system to provide an abstraction for pipelining tasks. (Perhaps gulp helps with this?)

Hmmm, perhaps someone needs to look at any gulpfile ever written, anywhere.

   gulp.src('src')
     .pipe(this())
     .pipe(that());

Maybe it's frustrating for anyone who spent "18 months" hacking grunt, but gulp really is better. Both tools are built on node for Pete's sake; async really shouldn't be that hard.

nostrademons 11 years ago | |

There is also the 'scripts' section of your package.json, 'npm run', and UNIX pipes:

    scripts: {
      "do_something": "this | that"
    }

   $ npm run do_something

jessaustin 11 years ago | | |

Yes while gulp > grunt, in many cases npm by itself is what you want. gulp.watch(), in particular, seems more difficult to use than the watch facility built into many (but not all) node modules. Also something like Browserify must be somewhat shoehorned into gulp. It might be worth it to look at what gulp has that npm run is missing? Maybe it doesn't need to worry about streaming files, but the notion of some tasks depending on arbitrary sets of other tasks (a generalization of npm's "pre-" and "post-") seems pretty handy.

NTH 11 years ago | |

I am excited to look into gulp!

asadlionpk 11 years ago | |

yes I found gulp to be more easy to implement than grunt. Maybe I haven't mastered grunt but I am happy with gulp.

MichaelGG 11 years ago |

How are all of you using any npm-based system combined with a continuous integration or clean build environment? It seems that a simple yeoman/grunt setup that just preprocesses some HTML and JS takes... a very long time. A minute or two just for the actual work. Plus 5-15 minutes for npm to make a thousand requests and write 10K+ files to the filesystem.

I hack around it by keeping node_modules around as a special resource on a custom build server and symlinking it in, but that wouldn't work with a proper "start from zero" build system.

doublerebel 11 years ago | |

1) We have our own private npm that can optionally include a caching layer. This has been invaluable since the beginning of our company to manage internal dependencies.

2) We use StriderCD, the self-hosted CD/CI, and it too supports caching node_modules. 2 minutes is a 'long' time for us from git push to autodeploy.

3) We use gulp and npm always. Grunt and bower are a mess.

EDIT: Also, we npm shinkwrap projects. It avoids numerous issues and makes module caching a natural part of the process.

jessaustin 11 years ago | | |

Grunt and bower are a mess.

grunt has its issues, but you are correct that bower is truly a mess. "Let's do one of the many jobs that npm (the tool we use to install bower) already does, but not as well!" No thanks.

Kudos 11 years ago | |

Both Travis CI and Circle CI cache your node_modules directory between builds. If you don't change your package.json, it just uses that cache instead.

tieTYT 11 years ago | |

> It seems that a simple yeoman/grunt setup that just preprocesses some HTML and JS takes... a very long time.

I have not experienced this. I built a 15k+ line CoffeeScript app. Every time a file changed it would compile it to JS and run thousands of unit tests in ~1 second.

As for the clean NPM... I never deleted that directory. I believe I read an article somewhere that you should actually commit your module dir. Maybe that's no longer in vogue though?

lucaspiller 11 years ago | |

Why does it matter if your build system takes a while? You shouldn't be pushing it to there until you are happy it works locally, at which point you can move on to the next task and forget about it.

andrewchambers 11 years ago |

Maybe I am missing something, but it seems like most of theses tools are working around the fact the windows doesn't come with make by default.

They are essentially reinventing the wheel for web developers who have never worked from the command line before.

falcolas 11 years ago | |

There's a lot of distaste for make as well, typically centered on the limitations of the dependency and .PHONY targets.

I rarely find them to be limitations for my projects, but there's certainly a reason so many different build systems have been built, even for Linux.

brandonmenc 11 years ago | |

I've written both makefiles and gulpfiles.

Yeah, I guess you can get make to do the exact same things as gulp, but gulp already does what a web developer needs, in a language they already know, with a massive plugin ecosystem you don't have to re-implement in shell.

andrewchambers 11 years ago | | |

A plug in system is nice. I didn't say it was totally bad, I just think its interesting.

badthingfactory 11 years ago |

This article should be titled "Maybe you should stop using Grunt for things it was never meant to do to begin with"

NTH 11 years ago | |

Yes, I acknowledge in the article that Grunt is not designed for this, and if you use Grunt for what it is designed for, it works pretty well. However, it wasn't obvious to me the using Grunt to build an SDK was outside of its scope when I started the project, so I figured it may not be obvious to other people as well.

latchkey 11 years ago |

I started with grunt, then moved to gulp because I like the plugin support more and it felt less magical to me.

Then, I found that working with many different projects, gulp wasn't DRY enough for me. I kept copying any pasting my tasks across projects and forgetting to go back and improve them as I went. I also disliked the chain of require statements and devDependencies in my package.json. I was spending too much time coding my build.

So, I started a project [1] to move gulp tasks into re-usable functions and hide the devDependencies. Now I'm pretty happy with the result, my gulpfile [2] is now mostly configuration and very little code.

[1] https://github.com/lookfirst/gulp-helpers/

[2] https://github.com/lookfirst/systemjs-seed/blob/master/gulpf...

nmalaguti 11 years ago |

Check out Laravel Elixer[0]. It's built on top of gulp and exposes a very simple SDK for compiling the various parts of your app.

[0]: http://laravel.com/docs/5.0/elixir

mike_hearn 11 years ago |

I feel I must be missing something here, but ...

The inability to specify config asynchronously turned into a disaster. Our primary need for async config was specifying temp directories for tasks to operate in. We used tmp, which does not have a synchronous api ...... I would like the task system to provide an abstraction for pipelining tasks

Is this abstraction not normally just called a program? Where you can do something like:

val tmp = makeTempDir()

generateFoo(outputDir = tmp)

generateBar(inputDir = tmp)

etc?

How much of this complexity is fundamental and how much is people attempting to work around Node's inherent limitations and fully asynchronous APIs? Perhaps the author should check out something like Gradle instead?

woah 11 years ago | |

Gulp has no temporary directories.

greggman 11 years ago |

I probably just don't have enough experience. I like the idea that Grunt and Gulp are written in JS so using JS to build JS seems like a win.

But...., aren't build systems supposed to do dependency checking and just built the minimal amount of stuff? Maybe few JS projects are big enough for it to matter or maybe I missed where the dependencies are checked but AFAICT the default is to build everything always.

jdlshore 11 years ago | |

Grunt doesn't do dirty-file dependency checking. Gulp does, sort of, but it's awkward.

There are many, many build tools in JS-land. Grunt and Gulp happen to be the most popular. (This isn't really due to technical merit. They're popular mostly because they're popular.) My personal favorite is Jake, which is a classic Rake-like (or Make-like) tool.

acdlite 11 years ago | | |

Or instead of a Make-like tool you could just use Make itself :) Works really well.

digitalzombie 11 years ago | |

You can do dependency check via Bower + NPM.

Your grunt or glup file can run a command to check for those dependency and update it via bower and npm.

Bower are for front end dependency check and NPM are for the backend dependency check.

Grunt and Glup are more of a task management system where you run tasks. Such as transpiling your jade/haml/sass, minifying your js, running your test cases. You can run these tasks in specific orders.

There's also gruntfile too but it's been a while since I've use these things. I think gruntfile are for grunt specific module for tasks though, so it'll dependency check grunt modules?

I'm a recovering frontend guy, well at least I did front end in a few gigs that were suppose to be fullstack.

Arnavion 11 years ago | | |

greggman is talking about dependencies between build targets (such as files or directories or projects), eg: "lib.js depends on lib.coffee" + "lib.coffee's last modified time > lib.js's last modified time" = "must run task that regenerates lib.js"

skybrian 11 years ago | |

First the build tool needs to load the build files that declare the dependencies, so it knows which dependencies to check. For big, distributed projects, this step can be slow.

cdnsteve 11 years ago | | |

Could this not be cached? If deps aren't changing, why do them all from scratch?

gadr90 11 years ago |

I have had similar problems with my team, and I went with the "meta-project" approach. I created an external project with all default configs[1]. It works well-ish. Main complaint is speed, grunt really is dumb to not lazy load...

[1]: http://github.com/vtex/grunt-vtex

tinco 11 years ago |

I'm a bit confused, what are they using grunt for? We're using gulp, but it's basically the same software. It just compiles our coffeescript, tags some assets and moves some files around, perhaps run a minifier somewhere.

In what situation do you need your build toolchain to do more than that?

geetee 11 years ago | |

I think I found myself in a similar situation as the OP. We have multiple different, but architecturally similar, products which each get their own Gruntfile. If we stop here, we're solid, perfect. Unfortunately, each product has multiple flavors which each need to build into their own separate output directories. For us, that's where the Grunt patterns break down and get awkward. It's actually been good enough for almost 2 years now, but I can definitely see the cracks.

NTH 11 years ago | | |

Yes, that sounds very much like the scenario we see ourselves in. For us, we have too many projects for copy/pasting the Gruntfile between them to be a viable option, and the Gruntfiles themselves would have been too complex.

I'm not convinced the shell scripts are a good alternative, though. (Maybe they are for your scenario.) I would like to write a follow-up post talking about this.

mikegioia 11 years ago | | |

Why don't you just write shell scripts?

pothibo 11 years ago |

I couldn't refrain myself for posting something I just wrote http://pothibo.com/2015/3/churn

I think JavaScript people might need to read it.

jakswa 11 years ago |

I switched to skimming the article at:

  [If] you just want to set up your single project with a task workflow, 
  then go ahead and use Grunt (or gulp). They will serve you just fine.

NTH 11 years ago | |

Yep, my article is aimed at people with a very specific use case, and if that's not your use case, the article will not be as relevant to you.

jakswa 11 years ago | | |

Maybe the title should reflect that?

mcx 11 years ago |

I switched to webpack (the config is like ~100 loc) and it has replaced the need for a grunt or gulp file entirely.

mikejholly 11 years ago |

grunt < gulp < webpack < make

jessaustin 11 years ago | |

I'm willing to believe the first two comparisons even though I haven't used webpack, but I have tried to use make on multiple occasions over the course of decades. (That is, I've attempted to write new Makefiles and to modify existing ones to fit a purpose; I'm not talking about just using a Makefile someone else already wrote.) make is not nice. I admire anyone who can use it!

To be specific, what advantages does webpack have over gulp that webpack doesn't also have over make? (I would also accept advantages that gulp has over grunt that gulp doesn't also have over make.)

pluma 11 years ago | |

* < npm scripts