BDD testing for golang: Ginkgo and Gomega(pivotallabs.com) |
BDD testing for golang: Ginkgo and Gomega(pivotallabs.com) |
What I would like is a tool that allowed testers and QA people to define tests using BDD for RESTful APIs.
Something along the lines of this syntax:
Using https://example.com/myapi
With access_token [accessToken]
With content_type application/json
When I create:
{
"somejson": true
"foo": "bar"
}
Expect status 200
Expect returnData.somejson = true
Expect returnData.foo = "bar"
Store returnData.id as myid
And
Using https://example.com/myapi/{{myid}}
When I get
Expect status 200
Expect returnData.somejson = true
Expect returnData.foo = "bar"
And
With access_token [accessToken]
With content_type application/json
When I update:
{
"somejson": true
"foo": "foo"
}
Expect status 200
Expect returnData.somejson = true
Expect returnData.foo = "foo"
Store returnData.id as myid
And
With access_token [accessToken]
When I delete
Expect status 200
And
When I get
Expect status 404
And I'd like it to be in Go as I'd like to just give a single binary to the testers (rather than an "install node, get project, fetch dependencies, resolve any issues, run the project").
When I read the headline, this is roughly what I hoped for. If I can find the time (unlikely), I'd build this. But then... maybe someone reading this knows something very close to this.
This feels weird. I have a gut reaction to this, and it's conflicted. Perhaps the same feeling `redbad` has, perhaps not.
On the one hand, I feel like this is a million miles away from the Go I know and love. I've used Ruby, Python, C#, Java, C, Obj-C, JavaScript etc in the past. In terms of mental processes, the way I write go is closest the way I would write C or Java.
I write tests that do the same thing as this, that have pre and post conditions and assertions and explanations in comments. The whole point of static type systems, surely, is that you represent things (constraints, state, possible values, etc) in code rather than comments and rely on the language to enforce them rather than the next developer to read your comments, so the explosions happen at compile time rather than production. So, by that token, this kind of DSL seems like a step in the direction of static typing, rather than the other way round.
I've not looked at the source, but the line `Expect(scoreKeeper.Stats["Manning"]["Touchdowns"]).To(Equal(1))` might use all kinds of clever closure stuff but ultimately wouldn't compile if the right interfaces weren't satisfied. Unless there are `interface{}`s all over the place, which would be regrettable.
On the surface, this seems like a step toward Ruby with its endless DSLs which feels like the antithesis of static typing. That naturally feels a bit weird and a bit of a culture lurch. But perhaps the DSLs are closer to those of Scala than Ruby.
Both feelings I get are fuzzy and not particularly arguable, but I think peoples' relationships with languages are very personal and that's such a fundamental quality of the PL landscape that it's worth not discounting.
Those things aren't all bad, and _some_ of their lessons can be successfully "ported" to languages and ecosystems that don't suffer the same fundamental shortcomings as e.g. Ruby or Python. But when I see developers take e.g. the BDD ethos as axiomatic and just run with it, it makes me feel like they don't really understand what BDD is designed to address. Likewise with hyper-expressive testing DSLs, or the concept of "mocking" as it's normally used.
Forgive the loaded language, but bringing BDD et. al. to languages like Go feels, to me, like cargo-cult development.
I prefer to use Gherkin syntax for by bdd testing, but the matchers are nice. The bootstrapper and helpers seem to fail hopelessly on windows, and the console writer outputs a lot of control characters which do not work in windows command prompt.
1. Mao: https://github.com/azer/mao
2. Zen: https://github.com/pranavraja/zen
3. GoConvey: https://github.com/smartystreets/goconvey
Are you planning to add selenium support?
- Godoc documentation is coming soon. Godoc is great for API-style documentation but not particularly appropriate for tutorial/structured/narrative-style documentation. Think golang.org/pkg vs golang.org/doc. I wanted to write the /doc first as I believe it to be more valuable for beginners and is often overlooked by the go community. /pkg is coming soon (a few days) and is much easier to write.
- @buro9: Golang is BDD-style in the same way that Cedar and Jasmine are BDD-style. This is not Cucumber (way too much DSL for me!) With that said, asynchronous testing support is baked right into Ginkgo and Gomega so the API-testing that you'd like would be easily expressed in Ginkgo and Gomega.
- Assertions that I have fundamentally misunderstood Golang or the distinction between dynamic and static typed languages are interesting and I'd like to address some of them.
BDD-style testing need not be limited to dynamically-typed language. Yes dynamically-typed languages need far more comprehensive test coverage to make up for the missing compiler. That's why I tend to prefer statically-typed languages. To me BDD-style (vs the XUnit style) isn't primarily about addressing these deficiencies in dynamic languages -- it's about expressiveness.
To that end, I think that BDD and Golang go hand in hand. Let me explain. BDD is exceptionally good at describing the behavior of branching code. Golang is filled with branching code that needs to be described in test. Here's a classic example in pseudo-go:
func DoSomethingAwesome(dependencyA DepA, dependencyB DepB) awesome Awesome, err error {
stuffA, err := dependencyA.Gimme()
if err != nil {
return Awesome{}, err
}
stuffB, err := dependencyB.Gimme()
if err != nil {
return Awesome{}, err
}
....
return awesome, nil
}
Describe("Doing something Awesome", func() {
BeforeEach(...) //common happy case setup
Context("When all is well", func() {
It("should be awesome", ...)
})
Context("When dependencyA fails", func() {
BeforeEach(...) //set dependencA up for failure
It("should return a zero Awesome", func() {
...
Expect(awesome).To(BeZero())
})
It("should error", func() {
...
Expect(err).To(HaveOccured())
})
})
Context("When dependencyB fails", func() {
//etc...
})
})
func TestDoingSomethingAwesomeWhenAllIsWell() {
//setup
//some sort of assertion
}
func TestDoingSomethingAwesomeWhenDependencyAFails() {
//setup + tweak
//some sort of assertion that awesome is zero
//some sort of assertion about err
}
func TestDoingSomethingAwesomeWhenDependencyBFails() {
//setup + tweak
//some sort of assertion that awesome is zero
//some sort of assertion about err
}
- Concerns about using interface{} and reflection seem odd to me. First off, the Go authors provide both of these things to precisely solve the sort of problem that Gomega is trying to solve. Nobody wants to write and maintain matchers that look like:
Expect(foo).To(EqualInt64(3))
Expect(bar).To(EqualString("blarp"))
Expect(baz).To(EqualMyCustomAwesomeType(awesome))
Most importantly of all: we're all using interface{} all the time: `fmt.Sprintf("%d %f %s %v", ...)`!
And thanks for seeing the point @karma_fountain: Gomega's matchers have excellent reporting about what precisely went wrong when a matcher fails. Bare assertions lack this and a look through the Go tests shows a lot of reinventing-the-wheel to provide what probably amounts to inconsistent error output to the developer. Why not put that all in one place, call it a matcher library, and make it dead easy to write custom matchers?
- And @shoo: yes, thanks for pointing it out: the example is somewhat fabricated and isn't a compelling argument for BDD vs XUnit. It's hard to cook up a good BDD example in very short space!
> Nobody wants to write and maintain matchers that
> look like: Expect(foo).To(EqualInt64(3))
if foo != 3 {
t.Errorf("foo: expected 3, got %d", foo)
}
t.Errorf("foo: expected 3, got %d", foo)
gets old quickly.
Say you wanted to write a web application in Go, and you wanted to change your login system... Now it may be just me, but having a test suite to determine where you'd have breakages would DEFINITELY be handy in overhauling your login. Doesn't matter that it's statically typed.
Notice that I said "ensure", and not "describe" or "define". Because I think that's the disconnect. When you come from a dynamically-typed language background, I think you're more apt to believe your testing needs to describe/define your contracts, because you don't have a type system to leverage. But if you _do_ have a type system to leverage, the pathological context that justifies e.g. BDD is no longer valid, and tools like Ginkgo -- enabling "descriptive tests that can act as effective documentation" -- therefore don't make much sense. To me.
When testing a statically typed language, you have to jump through a lot of hoops to create mock objects, shims and things like that to satisfy the compiler, especially with legacy code that wasn't written in an easy way to test.
Haskell, for instance, is statically typed and does DSLs very very well. In fact, you use the type system as an augmentor of the DSL: it sets up the rules for how to embed the DSL in the code base itself. And it disallows a class of programs which would not work at all.
The right way to view types is that they allow a far richer description of what your code is doing because you can discriminate values on their types.
And before you lynch me: my professional work job is in Erlang which is a dynamically typed language.
EDIT: And when I said 'the antithesis of static typing' I meant purely in the Ruby context, which is famous for its DSLs.
My initial reaction to the example was that it looked like something that could be easily unit-tested, without the added layer of BDD over the top. But that's probably just a symptom of the example appearing as something that would be simple to build and test.
E.g. in python i'd translate `Expect(scoreKeeper.Stats["Manning"]["Touchdowns"]).To(Equal(1))` to `assert scoreKeeper.Stats["Manning"]["Touchdowns"] == 1` which is essentially the same, but only uses one bit of machinery (assert) rather than three.
That said, the more ways to test things the merrier. I think if you are interested in testing larger chunks of functionality / user-facing bits of behaviour rather than invariants of the building blocks of your program, then perhaps BDD starts to look more appealing, so the two approaches seems complementary.
Having an expressive static type system is again complementary, as that prevents you from making many trivial errors, and you just have to test for the remaining ones that the type system cannot express. (i miss that in python)
after a little dig through the code, it looks like "EXPECT foo TO EQUAL barr" translates as
https://github.com/onsi/gomega/blob/master/gomega.go#L15 https://github.com/onsi/gomega/blob/master/actual.go#L27 https://github.com/onsi/gomega/blob/master/actual.go#L48 https://github.com/onsi/gomega/blob/master/matchers.go#L7 https://github.com/onsi/gomega/blob/master/matchers/equal_ma...
so there's lots of `interface{}`s and reflection and so on
http://pytest.org/latest/assert.html#advanced-assertion-intr...
The equivalent in Python would actually be building up a whole AST-style object tree full of expressions, and then executing it.