Show HN: A sample project for Golang

57 points by shivylp 7 years ago | 17 comments

I have been using golang at work and personally for more than 2 years now. A problem i faced when I started and the problem new gophers even today is how to structure the project. Most (if not all) golang tutorials seem to be very simple and do not describe how to structure a large application. Following project is an attempt to Showcase a manageable project layout for services in golang.

https://github.com/spy16/droplets

This project is far from complete. Would like to hear some feedback from the community before continuing on this.

divan 7 years ago |

I really appreciate the work done to help newcomers to answer the question "how to structure Go project propertly", but unless you're building N-th microservice in the company with already established project structures, you don't really need to ask that question.

Every single Go program starts with a simple layout - empty directory and main.go file.

Once you start adding things, you create more .go files in the same folder, and only when it grows bigger you start thinking of moving some abstractions into separate packages.

The key difference with most other languages is that in Go folder means "package", not "namespace for bunch or files". It's a crucial difference that important to keep in mind with Go. And what is (sub)package? When do you need new subpackage? It's just a higher level way to abstract logic or system component - make it more isolated than just concrete type, change the naming and usage API to be more clear and self-sustained.

Before that, you don't have to create new packages (=folders) in Go.

The vast majority of all Go projects will never need `internal` or `cmd` folder, not to mention `pkg` or `web`.

Just start with main.go and let it grow naturally. The simpler and easier your package is, the better.

dullgiulio 7 years ago | |

Absolutely. And never group files in a package by type ("models".) A package should have a function ("auth", "health", "reporting", ...)

shivylp 7 years ago | |

Thanks for the feedback. The layout used here is not something I came up with. It's an emerged pattern from popular open source projects. Like you said, most probably they all started with simple package and as the project grew abstracted out things finally leading to a similar layout. The layout itself is another community effort (https://github.com/golang-standards/project-layout).

This is not to say that the layout is a standard or the best possible one. It has it's advantages and disadvantages (a whole git repository and a blog article is dedicated to describe both aspects. Hence, I will not get into that discussion here). I chose the layout simply because it's already a well documented one. That being said, I will be more than happy to see other layout and even refactor the structure.

Open source projects generally are not typical web apps or REST API services. They are platforms or tools like docker, kubernetes, Hugo etc. While these projects are good usecases for studying golang code, they feel very complex due to their specific non-simple (if not complex) domain. My idea was to have a web app and API service project which has dead simple domain but highlights the coding practices so that newcomers can read and understand without having to understand what containers are etc.

Few other clarifications I would like to make:

- every directory is not essentially a package in Go. Only directories that have at least one go file are packages. Internal and PKG are not supposed to be packages in that sense. They are just a superficial boundaries between internal and public code.

- web directory is not a package in this project.. it contains static asset files.

- simplicity is something i absolutely love. That is the main reason I like golang. If you look past the outer structure of the project and dig into the code and packages, I think you will see the simplistic design inside. My other projects (e.g. https://github.com/spy16/radium) follow the simplistic method you explained.

emourujarvi 7 years ago | |

When using a single folder structure, how should one go about creating multiple functions with a same name?

I'm creating a web service with Gin, and I have moved each http request handler into its own file. I would like to be able to pass these request handlers to Gin's router with pageA.Handler, pageB.Handler, pageC.Handler, etc, but compiler won't let me because function names have to be unique. Creating unique names for each handler function works fine, but personally it somehow feels messy.

Am I thinking anti-golang way, if I want to namespace these request handlers?

dullgiulio 7 years ago | | |

Sort of yes. I have never tried Gin and I actually suggest you to use something else, like gorilla/mux, but that said what you describe sounds like several types implementing an interface...

aryamaan 7 years ago | |

Could you please elaborate more. I always have to scratch my head to really think what should go where. Most of the time I am following MVC paradigm so it dictates about but would like to know more about it.

gtrubetskoy 7 years ago |

You might like to read my BP from 2017: https://grisha.org/blog/2017/04/27/simplistic-go-web-app-par... I have been using this layout in many projects and it worked out really well. When having multiple packages under the same github repo, you sooner or later will run into the problem of what happens if it's forked, I have work-around here: https://grisha.org/blog/2018/10/18/relative-imports-hack-in-...

tuyguntn 7 years ago |

Thanks for the work done.

Current project structure felt like non-Go way to me, let me explain why, I always thought go packages should be independent (not sure about sub-packages though)

`encoding/json`, `encoding/gob` they do share concept of encoding/decoding, but contain totally different implementations, types, models, logic inside. whenever you change something in `encoding/gob` package, you don't need to touch `encoding/json`

but in proposed structure. whenever you want to add some logic, you would end up adding stuff in multiple places, starting with `domain`, then `stores`, then `delivery` (but this is exception, since this would be changed anyway as this is view layer), then other places.

Wouldn't it make sense if you update only specific package (except view layer)

let's say, you have `user` package with all necessary models, storage, business logic related to user and dependency from `core/db` package. Now you want to add avatar to user, change only in `user` package and that's all.

same with auth mechanism, call your package as `auth` and have a dependency on `cookies`/`session` and `user` package, if you want to add OAuth2 only change auth, no need to add yet another model to `domain`, want to add token based auth, again just change auth package set/create necessary tokens on top of `session` package using `user.Entity` or `user.GetID(authedEntity)`

shivylp 7 years ago | |

You are totally right. It would be ideal to have a package called `users` which does everything related to user. But if proper care is not taken while laying out the packages and types that go inside, you might quickly run into cyclic-dependency issues (for the lack of better example: post requiring User type and User requiring post type). This does not happen in encoding/json or enconding/gob because of the fact that they are completely different domains and are simple enough. But in this project for example, they belong to the same domain and user and post entities have a relation between them.

I am not saying it's not possible to move towards the pattern you are suggesting. But since i decided to demonstrate Clean Architecture in this, i went with the current pattern which shows clear boundary between layers. In other words, what is used and what you are suggesting are just 2 patterns having their own advantages and disadvantages. It is probably not appropriate to call one or the other `non-Go way`.

But now that we have discussed this, i am interesting in trying that as well. Not sure if i should refactor or create a new project.

somada141 7 years ago |

I personally love the idea of template projects especially when they follow best practices. I’m a long-time user of https://github.com/audreyr/cookiecutter and https://github.com/audreyr/cookiecutter-pypackage which I’ve modified for my purposes. Has anyone used the golang equivalent under https://github.com/lacion/cookiecutter-golang ? Any thoughts as to whether it’s on par with commonly used practices?

mneves 7 years ago |

This is a useful repo for how to structure a Go project: https://github.com/golang-standards/project-layout

emourujarvi 7 years ago |

Thanks, I started learning golang few weeks ago, and had a problem with how to structure my code.

shivylp 7 years ago | |

Welcome to Golang!. I'm glad you found this useful. But i advise you to look at other patterns as well. This is just one of the patterns that can be used and may not be suitable for all usecases. For example, for smaller projects a much simpler layout with main or importable package at the root would work really well. Just to give an idea, my other project https://github.com/spy16/radium follows a very simple straightforward layout.

rsneha 7 years ago |

Nice work i really like it

shivylp 7 years ago | |

Thanks!