Java

Standards and Guidelines

jUnit

Test Classes

Test Methods


The above guidelines do not all apply when writing Spock tests.

For more information please see the Spock Primer

Spock

Test Classes

Test Methods

Best Practices

General

  1. Test fixtures that are common to all the tests in a test class and need to be initialized only once should be put inside the @BeforeClass method.
  2. Test fixtures that are common to all the tests in a test class and need to be initialized before every test is run should be put inside the @Before method.
  3. Clean up code that needs to be run after every test method should be put inside the @After method.
  4. Clean up code that needs to be run only once after all tests have been run should be put inside the @AfterClass method.
  5. Minimize the use of private inner-classes (anonymous or named) as they usually encapsulate a separate responsibility and could be extracted and unit tested on their own.
  6. Unit tests should test all exception scenarios, including checked and unchecked exceptions thrown by themselves or their dependencies.

See Testing Best Practices for other general best practices.

Assertions

  1. Favor jUnit's assertThat()and hamcrest Matchers or AssertJ over plain jUnit assert methods such as assertTrue, assertEquals, etc. (see http://junit.sourceforge.net/doc/ReleaseNotes4.4.html for the rationale).
  2. If you use jUnit's assert methods, consider providing a message to make the assertion failure clearer, e.g., assertEquals("Unexpected multiplication result", 4, multiplcator.multiply(2, 2));
  3. A test method should only test one specific behavior and only contain assertions related to that behavior.
    // Do not do this
    @Test
    public void testMultiply() {
    	assertThat(multiplicator.multiply(2, 2), equalTo(4));
    	assertThat(multiplicator.multiply(2, -2), equalTo(-4));
    	assertThat(multiplicator.multiply(-2, -2), equalTo(4));
    }
     
    // Do this instead
    @Test
    public void testMultiplyPositiveNumbers() {
    	assertThat(multiplicator.multiply(2, 2), equalTo(4));
    }
     
    @Test
    public void testMultiplyPositiveWithNegative() {
    	assertThat(multiplicator.multiply(2, -2), equalTo(-4));
    }
     
    @Test
    public void testMultiplyNegativeNumbers() {
    	assertThat(multiplicator.multiply(-2, -2), equalTo(4));
    }


  4. When testing that an exception should be thrown,  use  @Test(expected=<ExceptionClass.class>) or the ExpectedException rule and let the jUnit runner fail the test. Do not use a try/catch block in your code and call fail() if the exception isn't thrown.

    // Do not do this
    @Test
    public void testMultiplyOverflowWrongWay() {
    	try {
    		multiplicator.multiply(Integer.MAX_VALUE, Integer.MAX_VALUE);
    		fail("Expected multiplication overflow");
    	}
    	catch (MultiplicationOverflowException e) {
    		// Success
    	}
    }
     
    // Do this instead
    @Test(expected=MultiplicationOverflowException.class)
    public void testMultiplyOverflowWrongWayRightWay() throws MultiplicationOverflowException {
    	multiplicator.multiply(Integer.MAX_VALUE, Integer.MAX_VALUE);
    }


API documentation and unit tests

Documentation (JavaDoc) and unit tests should always be in-sync and support each other.

  1. Unit test every constraint or exception documented in the class' API (public and protected methods).
  2. Document every constraint and exception covered by the unit test.

    // Class
     
    public class TextParser {
    	/**
    	 * Counts the number of times a {@code token} appears in a string.
    	 *
    	 * @param text
    	 *		text to search. Cannot be {@code null}.
    	 * @param token
    	 *		token to look for. Cannot be empty or {@code null}.
    	 * @throws IllegalArgumentException
    	 *		thrown if {@code text} is {@code null}, or if {@code token} is {@code null} or empty.
    	 */
    	public int countOccurences(String text, String token) throw IllegalArgumentException {
    		Validate.notNull(text, "Text cannot be null");
    		Validate.notNull(token, "Token cannot be null");
    		Validate.notEmpty(token, "Token cannot be empty");
    
    		// Implementation
    	}
    }
     
    // Unit Test
     
    public class TextParserTest {
    	private TextParser textParser;
     
    	@Before
    	public void setup() {
    		textParser = new TextParser();
    	}
     
    	@Test
    	public void testCountOccurencesFindsOneOccurence() {
    		// Test code and assertions
    	}
     
    	@Test
    	public void testCountOccurencesFindsNoOccurences() {
    		// Test code and assertions
    	}
    
    	// More happy path test cases...
    	@Test(expected=IllegalArgumentException.class)
    	public void testCountOccurencesFailsWithNullText() {
    		textParser.countOccurences(null, "A");
    	}
     
    	@Test(expected=IllegalArgumentException.class)
    	public void testCountOccurencesFailsWithNullToken() {
    		textParser.countOccurences("Some text", null);
    	}
    
    
    	@Test(expected=IllegalArgumentException.class)
    	public void testCountOccurencesFailsWithEmptyToken() {
    		textParser.countOccurences("Some text", "");
    	}
    }


Mock Objects

  1. Production code should avoid final methods and classes as well as static methods whenever possible.
  2. Mock objects should always be created using a mocking framework, never written by hand.
  3. All the dependencies of a class should be replaced with mock objects before the test is run.
  4. When verifying a mock object's expectations, expect specific argumentsIn other words, avoid argument matchers such as any()anyList(), etc.
  5. Favor argThat() or ArgumentCaptor over custom Matcher to validate argument values
  6. Avoid the use of Mockito's thenAnswer() and doAnswer() to validate input parameters, use ArgumentCaptor instead
  7. Only verify conditions that are related to the behavior being tested. Over-verification makes tests brittle and break for no good reason.
  8. Mocking tools such as PowerMock should only be used to unit test complex legacy code that cannot be easily refactored.
    1. Exception: An acceptable use of PowerMock outside of legacy code testing is when a class needs to use a system or third-party class' static method. This is especially useful when testing exception scenarios that would otherwise be difficult or impossible to test. It can also help prevent dependencies on the environment itself, one of the basic qualities of a good unit test.

Dependency Injection

Leveraging dependency injection provided by tools such as Blueprint is a great way to make writing unit tests easier (see Mock Objects section above).

  1. Production classes should be written in such a way that their dependencies are injected through their constructors or setter methods and not created or looked-up from within the classes themselves.
  2. Prefer dependency injection over static methods in production classes.
  3. When a dependency cannot be injected (e.g., multiple instances of the same dependency are needed within the class), consider creating and injecting a factory class or a Supplier, or use the Factory Method design pattern.

  4. When testing concurrent classes that use a thread Executor, use MoreExecutors.directExecutor() to inject an Executor that runs in the testing thread and eliminate any issues associated with tested concurrent code.
  5. Similarly, when testing concurrent classes that use an ExecutorService, replace it with MoreExecutors.newDirectExecutorService().

Code Coverage

  1. Getters and setters should not be tested unless they contain some logic.
  2. Aim for 100% code coverage and always set or update the thresholds in the module's pom file to match the actual code coverage (minus 2% to account for rounding errors).
  3. Coverage must have at least 75% at the bundle level for instruction, line and path coverage as well as cyclomatic complexity.
  4. Coverage thresholds should always go up, unless the new code added only contains getters and setters or logging statements
  5. Write tests that cover exceptions thrown by a class' dependencies.
  6. Ensure that all paths in a stream or Optional are covered, including all filternonNull, etc.


    By default, JaCoCo will consider a stream as a single path and mark all the lines as covered as long as the line has been executed once. This means that actual path coverage may not be complete.

    Same comment applies to Optional.orElse methods.



Smells

  1. Slow unit tests

Anti-Patterns

General

  1. The visibility of private methods should not be changed only to allow their internal logic to be tested.

See Testing Anti-Patterns for other general testing anti-patterns.