Testify Design Guide

Engine

The engine that runs the various components of Testify is split into four parts, the TestEngine Class, the FelixBuilder Class, the TestPropertiesBuilder Class, and the TestFileBuilder Class. The engine establishes logging, installs the service bundles, drills down through the test file directory, and calls the 6 main Testify component handlers.

TestEngine Class

The TestEngine contains the main method of Testify, the testifyRunner. This class calls the TestPropertiesBuilder and TestFileBuilder, and calls all the main Testify components (except the parsers) for each of the test files. The properties parser handler is called by the TestPropertiesBuilder and the test parser handler is called by the TestFileBuilder.

FelixBuilder Class

The FelixBuilder starts a felix instance and then drills down through the deploy directory for service bundles. Any .jar file found in this directory will be installed and started.

TestPropertiesBuilder Class

The TestPropertiesBuilder parses a configuration file (a file that ends with .properties) and stores the properties. If the input is a directory, it drills down, checking for all configuration files and storing properties.

TestFileBuilder Class

The TestFileBuilder drills down through the test file directory, calls the test parser handler, and sets the correct property values in order to create a list of test data.

Action Handling

Actions are general functions that are needed for certain tests but not for others. When a test requires something specific to be done at a certain point in the test engine cycle, it calls an action. Currently there are three phases in the engine at which an action can be called. To call an action during one of these phases, the user must specify the action as part of one of these blocks: Pre Test Setter Action, Pre Test Processor Action, Post Test Processor Action. Any action can be called during any of these phases.

Pre Test Setter Action

A Pre Test Setter Action runs right before a test file has its variable properties set by the TestPropertiesBuilder.

Pre Test Processor Action

A Pre Test Processor Action runs right before the processor handler is called by the engine.

Post Test Processor Action

A Post Test Processor Action is called immediately after the processor returns its response and the value is stored.

Properties Parsing

This component is organized through the PropertiesParserHandler which calls a properties parser. Currently, only one properties parser service can be used per test run, so the first properties parser service found by the handler will be used. Each of these parsers is an implementation of the PropertiesParser interface allowing for implementations to be easily swapped out for different Testify runs. A PropertiesParser implementation takes in a properties file (ex, configuration.properties) and stores the various properties and values in a TestProperties object.

Test Parsing

This component is organized through the TestParserHandler which uses the test file extension to call a matching test parser. If the extension matches to more than one test parser, the handler will attempt to parse with each test parser until a non null response is returned. Each of these parsers is an implementation of the TestParser interface. A TestParser implementation takes in a test file and breaks it up based on a defined schema yielding a type, endpoint, test body, assertion block, and the different action types. It stores the parsed data as a ParsedData object.

Processor Handling

This component is organized through the ProcessorHandler which calls the correct processor for the given test based on the processor type listed in the test file. Each of the processors is an implementation of the TestProcessor interface and contains a method that takes in a Request object, runs the test, and returns a response object.

Assertion Handling

This component is organized through the AssertionHandler which breaks the assertion block into individual components and calls the corresponding Assertion service implementation to evaluate the assertion using the response content. Each assertion that is run returns an AssertionStatus Object. If any assertions fail, the DTF will evaluate the entire test as failed.

Writing

The last component that the engine calls for a test is writing. Each writer is an implementation of the Writer Interface allowing for implementations to be easily swapped out or added for different Testify runs. A writer implementation takes in the testData, response, and result to create a specific output for the test or group of tests.

Objects

There are many custom objects created as part of Testify core that are available to the service bundles. While most of these are specific to component types, there are two objects that give general functionality.

TestifyLogger

The TestifyLogger is a class of static methods instantiated at runtime to provide centralized logging. To implement logging, import org.codice.testify.objects.TestifyLogger. There are separate logging methods for each type of message (debug, info, error, etc). To write a debug logging message, for example, write TestifyLogger.debug( message,  location_name ). The location_name will be the printed name in the log for the location of the error. To use the current class name as the location, enter this.getClass().getSimpleName(). The log level for the log system is set by the user at runtime (defaults to INFO).

AllObjects

The AllObjects is a static HashMap that stores Objects under String key names. This object is instantiated a runtime and is used to store and retrieve objects that may be needed by various components and are not provided as part of the method. To add an Object to the map, use the setObject method (AllObjects.setObject). This Object can later be retrieved using the getObject method (AllObjects.getObject). The start timestamp, test properties, test file, and response are stored in AllObjects for use by any component.

Adding Functionality

Testify is designed to allow for new features to be added without changing the systems core. Adding additional capability only requires the implementation of a specified interface. The new packaged code can then be dropped in the "bundles" directory of the Testify build. To remove a service bundle, just delete it from the "bundles" directory. For instructions on adding component functionality, visit the component design pages listed below.

• Action Design
• Assertion Design
• Processor Design
• Properties Parser Design
• Test Parser Design
• Writer Design