How I write unit tests in Go(blog.verygoodsoftwarenotvirus.ru) |
How I write unit tests in Go(blog.verygoodsoftwarenotvirus.ru) |
Also, the shadowing of t in t.Run is intentional. You should not try to work around it. There's no risk of confusion either because you will always use the right one in the right place.
As that applies to software, a mock is a fully-fledged implementation of an interface, but not the implementation you use under normal use. For example, a mock might be an in-memory database where you normally use a persisted database. Not the real implementation, but does not try to deceive – it is equally functional.
Mockery appears to be an assertion library that, bizarrely, moves the assertions into interface implementation. What purpose does it serve? You are going to end up testing implementation details by using that, which is a horrible road to go down.
I wouldn't describe swapping a persisted DB for an in-memory DB mocking personally.
I'd add using testing.T.Cleanup for tearing down the testcontainer (or use a TestMain and a deferred if the container is slow and concurrency-safe.)
- I prefer state-based TDD as opposed to interaction-based (see https://martinfowler.com/articles/mocksArentStubs.html).
- I've used both testcontainers and dockertest, and from my experience dockertest is more robust.
- The capital T for the outer argument comes across as being hypercorrect. Why would one consider the shadowing of this argument bad?
It's perfectly ok to take what you like from them and leave the things you don't.
Is something that has 7 useful things and 3 things you disagree with merited to be buried from public view because of the 3 things you personally disagree with?
I've never really understood this perspective.
Consider a function with a key/value database dependency:
type DB interface {
Get(key string) string
Set(key string, value string)
}
func ContrivedFunction(db DB) string {
db.Set("key", "value")
return db.Get("key")
}
func TestContrivedFunction(t *testing.T) {
db := mocks.NewDB(t)
if ContrivedFunction(db) != "value" {
t.Error("unexpected result")
}
}
But, okay. It is still something. We will play its game:
func TestContrivedFunction(t *testing.T) {
db := mocks.NewDB(t)
db.Mock.On("Set", "key", "value").Return()
db.Mock.On("Get", "key").Return("value")
if ContrivedFunction(db) != "value" {
t.Error("unexpected result")
}
}
But now, the contrived project manager just called and, for contrived reasons, would like to change the organization of record keys:
func ContrivedFunction(db DB) string {
db.Set("ns:key", "value")
return db.Get("ns:key")
}
But not only is it broken, it doesn't seem to serve a purpose. Why would you even use it?
I like to say that normal tests where you test by passing in params and assert the result is outside in testing.
Mocking is inside out test. Follows naturally from dependency inversion of di. You want to assert your function is making the right calls/params to it's dependencies interfaces from inside.
In your contrived example there is not much value in knowing whether "ns:key" or "key" was used. But if you those are params to some external RPC suddenly this actually becomes pretty useful.
Providing working fakes for every dependency isn't always realistic.
Should I really spend time building a fake of Stripe. Or should I just assert the request I am making to it are the expected ones.
TLDR: I only saw the value of mocks when testing DI server code with many external service dependencies.
I don't know what a fake is. Is that what mockery gives you? Per the dictionary, fake is defined similar to mock, but without the no deception condition, so I suppose that adds up.
There are also stubs, which is defined as something that is truncated or a part of. Which, as it pertains to software, is an implementation that implements some kind of bare minimum to satisfy the interface – often returning canned responses, for example. Mockery arguably also fits here, except the assertion part, which is something else. But I guess that's where fake comes in to draw that differentiation?
> But if you those are params to some external RPC suddenly this actually becomes pretty useful.
Sure, a stub might check the inputs and return an error if some condition is not met, without needing to implement the service in full. This remains true to what the real service would also do.
But that's not what mockery does. It just blows up spectacularly if something wasn't right. That doesn't really make any sense. That is now how the real implementation works. Not only that, but in the case of mockery, its documentation advises that you put the dependency logic in the test. How silly is that? Now when you replace Stripe with Line all your tests are broken. If you used a stub, you merely change the stub to match and you're good to go. This way the tests remain pure, as they need to as they are the contract you make with your users. Changing the contract is unacceptable.
And for all that, it doesn't seem to serve any purpose. But we did ask the other guy for a concrete example (i.e. code) to show where one would want to use it. Looking forward to it.
The above example, of course, isn't a good one as you can inject an in-memory database (i.e. a hash map) with far less effort than running the mockery tool. But, you are quite right that not all situations are as simple.
If I recall correctly, rsc says to just buckle up and provide a working implementation no matter how hard or how much work it requires, but that's easy for someone who works for Google, in a prestigious position at that, to say. We don't all live in the same lap of luxury. I grant some pragmatism here.
What have you got?
type Server struct {
Payment services.PaymentService
Storage services.StorageService
Quota services.QuotaService
Auth services.AuthService
Notification services.NotificationService
}
// TransactionRequest is the request structure for a purchase
type TransactionRequest struct {
UserID string `json:"user_id"`
AuthToken string `json:"auth_token"`
ProductID string `json:"product_id"`
Price float64 `json:"price"`
Currency string `json:"currency"`
File []byte `json:"file"`
Key string `json:"key"`
}
// TransactionResponse is the response structure after processing a transaction
type TransactionResponse struct {
Status string `json:"status"`
Message string `json:"message"`
}
// HandleTransaction handles the incoming transaction request
func (s *Server) HandleTransaction(w http.ResponseWriter, r *http.Request) {
var req TransactionRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
http.Error(w, "invalid request", http.StatusBadRequest)
return
}
// Authenticate user
userID, err := s.Auth.Authenticate(req.AuthToken)
if err != nil {
http.Error(w, "authentication failed", http.StatusUnauthorized)
return
}
// Check and update quota
allowed, err := s.Quota.CheckQuota(userID)
if err != nil || !allowed {
http.Error(w, "quota exceeded", http.StatusForbidden)
return
}
s.Quota.UpdateQuota(userID, 1) // Assume updating quota by 1 unit per transaction
// Process payment
transactionID, err := s.Payment.Charge(req.Price, req.Currency, "payment-token")
if err != nil {
http.Error(w, "payment failed", http.StatusInternalServerError)
return
}
// Upload file to S3
fileURL, err := s.Storage.Upload(req.File, userID + "-files", req.Key)
if err != nil {
http.Error(w, "file upload failed", http.StatusInternalServerError)
return
}
// Send notification
notificationMessage := "Your purchase has been processed successfully."
s.Notification.SendNotification(userID, notificationMessage)
response := TransactionResponse{
Status: "success",
Message: "Transaction processed successfully: " + transactionID + " File uploaded to: " + fileURL,
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(response)
}
In this example that would be the userId, the bucket name, the key.
Mock allows you inject faults in the test setup. Lets say I want to test the error handling logic. I could do it with multiple types of stubs but if you squint its starting to look the same.
On the topic of Fake vs Stub vs Mock. Real or Fake is the ideal solution here but not always viable due to time/practical constraints. Stubs can be useful but don't give you fine grained control.
I don't follow. Under no circumstance would the test pass under invalid inputs, be it whether you use mocks, stubs, or even mockery (fakes?).
- Obviously it cannot pass while using mocks – a mock matches the real thing to every last detail. If it will fail while using the real thing, it will fail here as well.
- Obviously it cannot pass while using stubs – a stub matches the real thing, partially, to at least the minimum amount necessary for the sake of testing. If it will fail while using the real thing, it will fail here as well.
- Obviously it cannot pass while using mockery – mockery does not match the real thing in any way, but does require you to specify matching inputs with failure when they don't match. If it will fail while using the real thing, it will fail here as well.
I don't know what you are imagining, but I'm not convinced it is a thing. I mean, you could make it a thing - you can do anything your little heart desires, but there would be no reason to ever make it a thing.
Maybe we need some concrete tests to better illustrate what you are talking about?
https://stackoverflow.com/questions/3459287/whats-the-differ...
Stubs do not react based on their input parameters. However, even fakes have the issue of fault injection.
I don't want to just test valid inputs. I want to make sure I pass the correct inputs for the request parameters through to the dependencies. If pass 100.00 in the request but in my impl I pass 0.00 to the stub. The test will pass because the input is valid. But the implementation is not correct.
Also, mockery is just an alternative mocking api. There is nothing special here.
How would you test the case where the quota update fails with a stub? Here is mockery example again generated with chatgpt.
func TestHandleTransaction_QuotaUpdateFails(t *testing.T) {
// Create the mock services
mockAuth := new(mocks.AuthService)
mockPayment := new(mocks.PaymentService)
mockStorage := new(mocks.StorageService)
mockQuota := new(mocks.QuotaService)
mockNotification := new(mocks.NotificationService)
// Setup expectations
mockAuth.On("Authenticate", mock.Anything).Return("12345", nil)
mockQuota.On("CheckQuota", "12345").Return(true, nil) // Quota check passes
mockQuota.On("UpdateQuota", "12345", mock.AnythingOfType("int64")).Return(errors.New("quota update failed")) // Quota update fails
// No need to mock payment and storage as they should not be called if quota update fails
// Create the server with mock services
server := &Server{
Auth: mockAuth,
Payment: mockPayment,
Storage: mockStorage,
Quota: mockQuota,
Notification: mockNotification,
}
// Create a test request
transaction := TransactionRequest{
UserID: "12345",
AuthToken: "valid-token",
ProductID: "product123",
Price: 100.0,
Currency: "USD",
File: []byte("file data"),
}
requestBody, _ := json.Marshal(transaction)
request := httptest.NewRequest(http.MethodPost, "/transaction", bytes.NewReader(requestBody))
responseRecorder := httptest.NewRecorder()
// Call the endpoint
server.HandleTransaction(responseRecorder, request)
// Check the results
assert.Equal(t, http.StatusInternalServerError, responseRecorder.Code)
response := TransactionResponse{}
json.NewDecoder(responseRecorder.Body).Decode(&response)
assert.Equal(t, "error", response.Status)
assert.Contains(t, response.Message, "quota update failed")
// Assert that all expectations were met
mockAuth.AssertExpectations(t)
mockQuota.AssertExpectations(t)
// Assert no calls to payment and storage
mockPayment.AssertNotCalled(t, "Charge", mock.AnythingOfType("float64"), mock.AnythingOfType("string"), mock.AnythingOfType("string"))
mockStorage.AssertNotCalled(t, "Upload", mock.AnythingOfType("[]uint8"), "bucket-name", "key-name")
// No notification should be sent
mockNotification.AssertNotCalled(t, "SendNotification", "12345", mock.AnythingOfType("string"))
}I willingly accept your unconventional usage if it greases discussion, but the dictionary is right there. It records how most people use words. What mockery gives appears to be what most people consider a fake. It clearly does not fit the definition of mock. It arguably matches the definition of stub and I think you could reasonably call it that, but I posit that doesn't tell the whole story, which is no doubt why fake emerged.
That said, I don't know what your definitions are. You seem to flail around with them.
> Stubs do not react based on their input parameters.
All of mocks, stubs, and fakes react based on input...
> How would you test the case where the quota update fails with a stub?
... Although this seems like a poor example. It is not clear if the failure state is due to invalid input or due to some other issue.
This, I must say, makes for a really bad test. Tests are, first and foremost, the contract and documentation for your users. They are written so that other people can learn about the system and know what they can depend on during use. The self-validation offered by testing is there merely to ensure that what is written in the contract is true. –– Not that we were expecting anything more from ChatGPT, but this does, interestingly, indicate yet another problem with mockery. I hadn't even picked up on that one earlier. Glad we were able to keep the discussion going to learn more!
But, importantly, we lack necessary information to respond to your question. Perhaps you can clarify the intent here?
My suggestion is for you to provide code how you would test the same handler code and scenario using your methodology.
Show me a test that checks the error handling of quota check. Pretend the quota service is experiencing an outage on the second request you make to it.
type quotaNetworkFailure struct{ quotaSuccess }
func (quotaNetworkFailure) UpdateQuota() (bool, error) { return false, quota.ErrNetworkFailure }
func TestTransaction_QuotaUpdateNetworkFailure(t *testing.T) {
tx := &Transaction{
Auth: authSuccess{},
Payment: paymentSuccess{},
Storage: storageSuccess{},
Quota: quotaNetworkFailure{},
Notification: notificationSuccess{},
}
_, err := tx.Request("12345", "valid-token", "product123", 100.0, "USD", []byte("file data"))
if !errors.Is(quota.ErrNetworkFailure, err) {
t.Errorf("unexpected error: %v", err)
}
}
Yes, okay, you got me. That isn't the HTTP service anymore. But it should have never been. The transaction processing is not an HTTP concern. In fixing that, now you can make the HTTP end of things far less obtuse:
var errGeneralError = errors.New("general error")
type transactionFailure struct{}
func (transactionFailure) Request(id, token, productID string, amount float64 /* yikes */, currency string, file []byte) {
return false, errGeneralError
}
func TestBuy_Failure(t *testing.T) {
srv := httptest.NewServer(&Server{
Transaction: transactionFailure{},
})
defer srv.Close()
c := client.New(srv.URL)
_, err := c.Buy("product123")
if !errors.Is(transaction.ErrFailed, err) {
t.Errorf("unexpected error: %v", err)
}
}
type transactionNoProduct struct{}
func (transactionNoProduct) Request(id, token, productID string, amount float64 /* yikes */, currency string, file []byte) (bool, error) {
if productID != "product123" {
return true, nil
}
return false, transaction.ErrProductNotFound
}
func TestBuy_ProductNotFound(t *testing.T) {
srv := httptest.NewServer(&Server{
Transaction: transactionNoProduct{},
})
defer srv.Close()
c := client.New(srv.URL)
_, err := c.Buy("product123")
if !errors.Is(transaction.ErrProductNotFound, err) {
t.Errorf("unexpected error: %v", err)
}
}
func (transactionNoProduct) Request(id, token, productID string, amount float64 /* yikes */, currency string, file []byte) (bool, error) {
if productID != "product123" {
return true, nil
}
return false, transaction.ErrProductNotFound
}
mockValidator.On("Request", "12345", "valid-token", "product123", 100.0, "USD", mock.AnythingOfType("[]uint8")).Return(false, services.ErrProductNotFound)
Essentially if you take your concept further and try build a general purpose library for building out those stub functions you'll get something that looks like the mock library. They just give you all the utilities out of the box and you can apply to the granularity you see fit.
Indeed. Hence why I said that they can both be considered stubs (or whatever you want to call them). We already went over this. But one comes without all the devastating baggage of mockery, which is a pretty big deal.
> The nice thing about these mocker libraries is they integrate with the testing framework to show nicely formatted error messages about what exactly went wrong
Except you also need all of that same information when the same thing happens in production – for your logs, the end user, or even plain functionality – so you haven't gained anything. You're just duplicating efforts for no reason.
That or, knowing the average programmer, they'll just haphazardly throw in that information and not test it since they think mockery gives them the information anyway, and then the future programmer won't be able to figure out when it does some particular thing, which would have been avoided if properly documented in test. Indeed, programmers by and large seem to hate other programmers for some reason. That doesn't seem like a good justification, though.
> Essentially if you take your concept further and try build a general purpose library for building out those stub functions you'll get something that looks like the mock library.
It may be that mockery is just a poor implementation of the concept. There are some obvious improvements that could be made to mockery to at least begin to help with its grievous problems, so we know it is not as good as it could be. Is there a tool that does it better?
Or is this a classic case of someone taking DRY much too literally? As the Go proverb goes: A little copying is better than a little dependency. Is an abstraction truly warranted in the first place?