Rails Testing with Factory Girl(hiringthing.com) |
Rails Testing with Factory Girl(hiringthing.com) |
First example there: test_new_customer_defaults. Really, it needs 3 saved models to test this? FactoryGirl isn't solution here. It's just hiding the problem.
If you need a complex network of collaborators to test a method, the answer is not to obscure the creation of those collaborators, it's to refactor so that each object talks only to its immediate neighbours, and then stub the heck out of them. If it absolutely has to be something that acts like an ActiveModel, then use mock_model().
The Rails Tutorial (railstutorial.org) uses Factory Girl and integration tests for just about everything, which is a fast enough approach for its toy application, but I've found that when your application gets more complex your test suite takes forever to run, and setting up the tests gets more and more painful (especially when you start dealing with external APIs).
Does anyone have any opinions on the best resource for teaching effective unit testing in Rails? I'd love to speed up my test suite while not losing coverage.
[1]: http://objectsonrails.com/
I'm not sure if it's the best resource, though. I'd love to hear other opinions.
FactoryGirl.define do
factory :user do
association :account, :strategy => :build
end
end
Even if you do this, though, you're still spinning up a whole network of real objects in what is supposed to be a unit test.
Edited to add: Using FactoryGirl to generate a cache only addresses the performance issues without addressing the complexity-hiding problem.
Factories are slow. Yes, they really are. Persisting everything along a chain of associations will always be slower than properly isolating the dependencies of the unit under test, SSD or not. It's easier to enjoy Ruby when not constantly waiting minutes for tests to run.
Factories increase the over-all complexity of your app. Are you kidding me? Yes, they do. Expertise and huge, easy gains have nothing to do with reducing complexity and more often (but not always) increase it. A test suite where every test depends on state in a database is more complex than one that tests units in isolation.
None of these points are arguing against Factory Girl, which is a great tool for doing what it does; they are arguing against integrating with the database in "unit" tests.
"Watch out everybody, he's hitting the database!!!"
Ruby is a great language that I love, but this is one of those cases where Ruby's conciseness and readability hide a TON of complexity. And complexity in tests is death.
I've seen it for myself -- there's a lot of technical debt interest in those tests. It looks manageable at the beginning, especially within the scope of a blog post, but when the app grows and grows and grows, the tests fail.
However I more and more agree with the general notion that your unit test should not hit the DB. However, Rails works against that. Avdi Grimm's "Object's on Rails" (http://objectsonrails.com/) has some interesting suggestions on how to get some good test isolation out of Rails.
You can use build instead of create to avoid the db, as per previous comments. You don't have to use it at all if writing a simple unit test.
But for teeing up complex scenarios prior to a fat integration test, it rocks.
factory :user # sets only user fields, no relations
factory :account # sets only account fields, no relations
factory :user_account {creates both}
If you stick to the above strategy, things are much faster, but more importantly, there are less unintended surprises, like associations doing a create instead of a build, etc.
I agree that excessive association building can get out of hand (and the latest version of FG even allows you to subscribe to factory events just so you can work out what the hell your test suite is building - a sure sign that you've lost the ability to reason about your tests if ever there was one), but completely abstaining from building associations seems to remove more or less all of the attraction of using factories as a fixture replacement in the first place.
All of the above applies only to integration/acceptance tests, though, where factories-as-fixture-replacements are at least arguably justifiable; my recommendation for unit tests is "don't use factories at all."
Complexity is a moot point for those extraneous objects. You want that complexity out of the tests since it's not the focus of the test itself, and makes it more readable.
But I'll take this one step further. To each their own. There is no one exact correct way. Do what's right for your code, domain, and tests. I wouldn't test an API codebase the same I would a site codebase or a gem codebase anyways.
If you're using your unit tests to inform your design, then the complexity in the extraneous objects is relevant, because they tell you how coupled your unit under test is to its dependencies. This is what FactoryGirl hides from you.
I tend to use fake objects for the collaborators. Usually just a vanilla test double will do, but sometimes I use RSpec's stub_model if the collaborator has to behave more "model-like".
I'd recommend http://pyvideo.org/video/631/fast-test-slow-test and http://www.confreaks.com/videos/641-gogaruco2011-fast-rails-... if you haven't seen them already.
Tests and code don't always fall into these neat cult buckets we've drawn up for ourselves. Good doesn't mean just fast. It can also mean readable, to wit FG usage addresses quite nicely. Just another tool in the toolbox.
Just like ruby, I can write crap tests with or without FG and great tests with or without FG. Same goes for stubs, doubles, and collaborators.
I think we need the same thing in programming.
If it's slow, make it faster. If it's unreadable, make it readable. If it's coupled, decouple it.
Then just stop worrying if we fall into official definitions of unit, integration, collaborator, mock, stub, double, etc. I swear we spend more time as an industry arguing about that stuff than actually writing code, myself included.
If performance wasn't an issue, the convenience of auto-created objects would mean more 'unit' tests (edit: usually closer to integration tests) that wouldn't exist otherwise. Sure it's best to de-couple things, but there's room for something between no [unit] tests at all and perfect unit tests.
IMO, DHH (and others) argue too hard against core OO principles (like SRP and LoD) that have the possibility of helping out when the domain model becomes more complex. "Just use Rails" is fine when spinning up, but as the app grows, it becomes increasingly less pleasant. Coupling increases in the name of less code.
Ideas like hexagonal architecture look nice, but I'm left wondering how to implement them when the framework insists on infecting core domain classes (AR::Base).
I say "higher-level" tests to refer to integration, system, and acceptance tests, because I don't have an opinion of where to draw the line for FactoryGirl. Is there a reason you would consider FactoryGirl for acceptance tests, but not integration or system tests? Just curious, because I might learn something.
You do raise a valid point: we're arguing about a definition. I don't think that misses the point though, because this thread is specifically discussing the use of FactoryGirl in unit tests. Where I disagree with you is on the importance of definitions. I think it's important to understand the definition of a unit test vs an integration test vs a system test, because without agreeing on those definitions, we don't stand a chance of having a reasonable discussion. If we have different definitions for a word, then our arguments will be talking past each other.
> Tests and code don't always fall into these neat cult buckets we've drawn up for ourselves.
(I'll ignore the loaded language, because that's not conducive to rational discussion.) From this point on, you use the word "tests" instead of "unit tests," and your points are valid because you're talking about the broader category of tests. I'm not. I'm specifically discussing unit tests, which has been the topic of discussion for this entire thread, going all the way up to the top-level comment by indrekju.
> because every single codebase is different for a variety of reasons
I agree with you here. Every codebase is different, and so are their testing requirements. For small gems, I tend to write all integration tests with few unit tests. For larger Rails projects, I write mostly unit tests with some integration tests to catch the cases where an interface changed but I forgot to update a fake object. If you're saying that different codebases differ in their relative requirements of different types of tests (unit vs integration, etc), then I agree. In fact, I would take that a step further, and say the testing requirements also depend on the team. But if you're arguing that the definition of a unit test changes depending on the codebase's testing requirements, then I can't agree.
I'm not refuting the usefulness of FactoryGirl in integration, system, or acceptance tests. Just unit tests.
Having definitions for jargon gives us tools to learn and discuss things at a more abstract level. That doesn't necessitate applying those ideas dogmatically. If anything, I think it helps you understand the trade-offs so that you can be pragmatic, because pragmatism depends on understanding.