...
| Table of Contents |
|---|
Best Practices
Be Test-Driven
Brendan Hofmann - I'm not sure I agree with this one. A lot of development is iterative in nature, and TDD can make that more expensive. We should have best-practices beyond TDD as well.
Steve Lombardi (Deactivated) - This seems OK but I wouldn't want developers to loose the flexibility to work how they work best.
Comment:
- Make it clear this is just a recommendation and doesn't always apply.
- Add "when appropriate" or "for new code" in title or description?
Follow the Test-Driven Development Test Cycle.
First, come up with an initial list of end to end, component and/or unit tests that will be needed to show that the user story, requirement or improvement works as expected or issue has been fixed.
Then for each one:
- Add a test
- Run all tests and see if the new test fails
- Write the code
- Run the tests until they pass
- Refactor the code as needed
- Move on to the next test
Be Behavior-Driven
Test behaviors, not methods or lines of code.
Comment:
- May need to clarify that "exceptional" behaviors can be included and how
- Mention that test names should also be from a behavior perspective
Write each test following the Behavior-Driven Development structure, i.e., Given/When/Then. In other words make sure each test:
- First provides a clear context in which it will be run (given)
- Performs the operation to be tested (when)
- Asserts that the expected outcomes have been met (then)
Use Clear Names
Tests should have names that clearly indicate the behavior and state being tested, and should make it easy to determine what failed without having to look at the test's implementation. For instance, catalogReturnsMetacardIdWhenIngestSucceeds or exceptionThrownWhenInvalidUserNameProvided.
Joseph North I think test names should include the state under test as well as the expected behavior.
Question:
...
| Note |
|---|
JavaDoc comments should be used when capturing a test's state and behavior in the test method name would make it too long. JUnit 5 |
Test Positive and Negative Scenarios
Always remember to tests all positive and negative scenarios and ensure that the proper results (error code, exception, message, etc.) are returned.
| Warningnote |
|---|
Some error scenarios are extremely difficult to test using end-to-end or component tests, which makes testing all exception scenarios at the unit test level even more critical. |
Keep Related Tests Together
...
Try to keep tests that relate to a similar behavior together inside the test class. This makes it easier for other people to determine what's already been tested and what needs more testing.
Use Proper Assertions and Validations
...
- Assert that the expected value is returned
- Assert that the object under test is in the expected state
- Verify that the mocked dependencies were called/not called as expected
- Verify that all the mocked dependencies that were called were called with the right parameters
- Have just enough assertions and verifications to prove that the test passes, but no less
Never Assertions and verifications should be repeated in different tests only when they are required to verify the behavior being validated. In other words, separate tests should never assert or verify the same behavior in more than one test.
Brendan Hofmann This should be reworded so it doesn't unintentionally exclude interacting permutations of otherwise identical behaviors.
Tracy Batie (Deactivated) Clarify that this is not across test layers, e.g., unit tests and component tests.
| Note |
|---|
This best practice applies mostly to unit and component tests. Since End to End tests tend to be more complex and time consuming, it sometimes makes sense to verify related behavior in a single tests. This should however be avoided if possible as doing so makes it more difficult to determine what the issue is. |
Mock Dependencies
When writing unit or component tests, mock all dependencies that may fail. This allows the tests to guarantee that the unit or component under test behaves as excepted when one of its dependencies fails.
Steve Lombardi (Deactivated) - Clarification: Are there are cases where it's acceptable to use a mock-less test? Can I setup a temp folder if it's easier?
Comment:
...
.
This is especially important in unit tests as these may be the only tests where some of those exception scenarios can be tested.
| Note |
|---|
Simple dependencies (e.g., POJOs) or dependencies that are difficult to mock out (e.g., file system, static utility classes) can be used directly without being mocked, as long as doing so doesn't go against the basic testing best practices. |
Clean Up After Yourself
Always make sure that tests clean up after themselves to ensure that a test failure doesn't impact other tests.
...
Use Existing Test Tools and Frameworks
Do not re-invent the wheel. Every homegrown mock, test tool or framework needs to itself be tested and maintained. There are tons of good testing frameworks and tools out there, use those instead.
Write Clean and Re-Usable Test Code
Maintain testing code as you would production code. This means keep the test code clean and extract re-usable code when it makes sense to do so.
Tests Before Changes
Before changing, fixing a bug or refactoring existing code, always make sure that tests exist and pass first.
If not, write unit test tests for the code that is about to be changed or refactored.
...
| Info |
|---|
Refactorings automatically performed by an IDE are usually safe and may not require adding tests first. This is especially true for simple structural refactorings that do not affect the logic or flow of the code, such as extract class, extract method, etc. If such refactorings make writing the tests easier, then applying them without existing tests can be considered. |
Be Test-Driven
Try tollow the Test-Driven Development Test Cycle when it makes sense and feel comfortable with it.
First, come up with an initial list of end to end, component and/or unit tests that will be needed to show that the user story, requirement or improvement works as expected or issue has been fixed.
Then for each one:
- Add a test
- Run all tests and see if the new test fails
- Write the code
- Run the tests until they pass
- Refactor the code as needed
- Move on to the next test
Be Behavior-Driven
Test behaviors, not methods or lines of code. This includes exceptional (e.g., error or unexpected) behaviors.
Write each test following the Behavior-Driven Development structure, i.e., Given/When/Then. In other words make sure each test:
- First provides a clear context in which it will be run (given)
- Performs the operation to be tested (when)
- Asserts that the expected outcomes have been met (then)
Smells
Tests Difficult to Name
...
A test that requires many mocks usually indicates that the class under test is breaking the Single Responsibility Principle or the Law of Demeter and may need to be refactored.
Comments:
...
...
Dependencies Difficult to Mock
Classes that make it difficult to mock their dependencies usually need to be refactored to follow the Dependency Inversion Principle and use dependency injection. For cases where the class to be mocked is instantiated inside the class multiple times, consider creating and injecting a factory class or a Supplier, or use the Factory Method design pattern.
A Lot of Repeated Test Code
See Write Clean and Re-Usable Test Code
Creating Mocks Manually
See Use Existing Test Tools and Frameworks
Test Multiple Behaviors in a Single Test
...
Repeated Given, When or Then sections in a BDD test, or difficult to name tests are usually two good indicators that a test is doing too much.
Comment:
...
| Note |
|---|
When testing behaviors that build on top of each other |
...
(e.g., test step A, then test step B after step A has been done, etc.), consider extracting the common steps into private methods or external support class. This will reduce code duplication while ensuring that each test method verifies a specific behavior. |
Anti-Patterns
Sleeps
Sleeps in tests should be avoided as they open the door to timing issues and race conditions, slow tests down and are a major cause of test flakiness.
...
Undoing Setups in Specific Tests
Common test fixtures that are run before all or each tests should remain common to all tests and should not be undone by individual tests. Doing so introduces a lot of confusion when reviewing, changing or maintaining the tests.
Instead, code that is common to many but not all tests should be extracted to one or more private methods or external support class to eliminate duplication.
Tests Need to be Run in a Specific Order
...