A Roadmap to Merging 'dep' into the Go Toolchain

A Roadmap to Merging 'dep' into the Go Toolchain(github.com)

138 points by numo16 9 years ago | 44 comments

joaodlf 9 years ago |

This can't happens fast enough.

I have been having a fantastic time with Go... Up until the point I have to add a new package to my projects and go through CI and deployment.

Glide works mostly OK, but it's not the final solution, that's for sure. Even big packages on github fail to produce releases, so you have to depend on master branches... One day your builds just decide to break. It's a nightmare.

garfij 9 years ago | |

I haven't used Glide (only govendor) so I'm not sure exactly how it works.

Are you checking your dependencies into your repository (we do this, it works very well).

If not, are you pinning specific commit hashes? Is that even possible in Glide?

AFAIK those are the only two ways to get even close to reproducible builds.

joaodlf 9 years ago | | |

Commit hashes seem to be the way to go, for sure, at least to be safe. I really don't want to put our vendor content in the repos.

It's just a tad absurd when you're used to mature package managers in other languages.

camus2 9 years ago | | |

> Are you checking your dependencies into your repository (we do this, it works very well).

it work if you are working on a project, it does not if you are library author who rely on other dependencies.

zeveb 9 years ago | |

What I don't get is why more folks don't just use per-project GOPATHs.

I have all my stuff in $PROJECT/src/$PROJECT, and all the dependencies are git submodules in $PROJECT/src/, and it all Just Works. Perfectly. git submodule is a bit of an annoyance, but it works. Anyone fetching my project just checks everything out, and it all Just Works™.

It's a lot better than using a dependency-management tool and shared GOPATH, at least in my experience.

camus2 9 years ago | | |

> What I don't get is why more folks don't just use per-project GOPATHs.

because that's not how go works. You need extra work to make it work. That's not up to debate anymore anyway, dep will be the official package manager and will be merged with the rest of the official ecosystem.

lobster_johnson 9 years ago | |

Glide lets you pin dependencies to specific commits (the "version" key takes a semver version constraint, a branch name or a commit hash), and transitive dependencies will be pinned via the lockfile, so it works even if the implicitly included dependencies don't use versioning.

The main problem with Glide is that it's buggy. It also lacks some important features. You still can't do "glide update" on a single dependency, for example. It's all or nothing.

sdboyer 9 years ago | | |

yeah, this is one of the main pain points there. the reason it's so difficult to solve in glide is intrinsically tied to the engine glide uses for version selection - otherwise we'd have dealt with it long ago.

OhSoHumble 9 years ago | |

I agree. I have a friend who likes the concept of Go but refuses to try it out unless it has a good dependency management solution.

ernsheong 9 years ago |

This podcast (iOS) is really worth listening. Straight from the horse's mouth:

"Dependency Management, Semver, and Community Consensus" https://itunes.apple.com/my/podcast/go-time/id1120964487?mt=...

It explains how go dep fundamentally works alongside the other tools.

OrangeTux 9 years ago | |

Web version of same podcast https://changelog.com/gotime/36

zellyn 9 years ago |

If you have any skills or interest in this area, the dep team could use your help. Even just trying it out, running into trouble, and filing bugs would be helpful. Join #vendor on the Golang Slack.

geodel 9 years ago |

Great initiative. If it goes well we hopefully will have an integrated dependency management solution with go command by next year this time.

devty 9 years ago |

The `dep` repository: https://github.com/golang/dep

ernsheong 9 years ago |

This podcast (iOS) is really worth listening. Straight from the horse's mouth:

"Dependency Management, Semver, and Community Consensus with Sam Boyer" https://itunes.apple.com/my/podcast/go-time/id1120964487?mt=...

vkat 9 years ago |

Started with dep recently on a real go project at work. It worked initially fine until I started adding dependencies and I had to hunt down the right commit hash by looking at the dependencies. I am probably going to stick with it and maybe contributing upstream as well.

politician 9 years ago |

Anyone know how dep's approach differs from the two-step process in `gb`?

(gb vendor fetch github.com/some/pkg, gb build, git commit -am "all of your vendor'd source")

sdboyer 9 years ago | |

sadly, gb is one of the extant systems i haven't had a ton of time to explore. but...

i guess the analogue to what you're describing would be

1. <add an import path> 2. `dep ensure` 3. `git commit -am "all of your vendor'd source`

a bit more detail, starting at a high level: gb is a replacement toolchain. dep is focused strictly on dependency management. there is no notion of a `dep build`.

with dep, there's no explicit command to actually fetch a dep; the import graph is still queen, as is customary in go. so there's no direct analogue to `gb vendor fetch github.com/some/pkg`. if you want to add a dep, import it in your source code, and run `dep ensure`. (there's some flux in exactly how that works right now, but what i'm describing is the state we're moving towards)

`dep ensure` is really the workhorse command, and pretty much the only thing you'll ever need to run. (in fact, the only three subcommands we currently plan on having are `init`, `ensure`, and `status`).

mappu 9 years ago | | |

`gb` has a package-management-only version in `gvt` by @FiloSottile .

shadowmint 9 years ago | |

In summary, via gb:

    git add vendor/ && git commit

In dep:

    echo vendor >> .gitignore

Ie. Dep is designed to easily restore specific versions of packages online, like other package managers. Gb expects you just commit them.

Practically speaking I guess the manifest that dep uses makes it easier to see at a high level exactly what versions of what packages are installed and from where.

cybernytrix 9 years ago |

Our team runs _the_ largest Go middleware. Checking in external dependencies into your repo scales - you don't want 200+ users hitting Github every 10 minutes, when you can hit the local Git repo on a 10Gbps n/w. Also, using GOPATH with a simple Makefile is pretty much all you need -- unless you don't grok Makefiles. But whatever floats your boat and as usual the latest hotness sells.

poofyleek 9 years ago |

I use gopkg.in. Personally I would like to see the community avoid adding more to the 'go' tool. It is already doing too much. I think the solution provided by gopkg.in is better than other more conventional methods (e.g. npm or maven like). I love Go. But it is ironic that 'go fmt' was designed in from the beginning. And now we are still dealing with semver and package dependencies. I went with gopkg.in because it made the dependencies a non issue from the tooling point of view. It is made possible by having 'import' direct from the repositories, which archive different versions and tags. Perfect for something like gopkg.in. I am probably in the minority.

lobster_johnson 9 years ago | |

You say you don't want the go tool to be more complex -- but gopkg.in just moves dependency management elsewhere. You have to have it somewhere.

gopkg.in only works properly if everyone uses it. The second you have a non-gopkg.in package, you have no way to manage that dependency.

Does gopkg.in work for non-public libraries?

michael-go 9 years ago | | |

another problem with gopkg.in it uses only the major version number and you can't import a specific minor version. what if version 2.5 works great but 2.6 breaks something for you? you can't use pkg.in until this issue is resolved in 2.7 .. and most package authors don't want to bump the major version on every change ..