JSR-349 Dynamic Metacard Prototype

Initial Prototype

The initial prototype incorporates many of the ideas above in a DDF branch (DDF-1760). In an effort to not reinvent the wheel, we took the approach of basing the dynamic metacards on the Apache BeanUtils library and the validation on the Bean Validation Framework (JSR-349). The combination of these two will allow us to use existing libraries and standards while providing most of the concepts discussed above.

DynamicMetacard Implementation

The initial hope for the dynamic metacard was that it could be made to look as much like a JavaBean as possible - that would allow it to be directly validated by the Bean Validation libraries that expect to work on JavaBeans. With that in mind, the initial investigation zeroed in on the Apache BeanUtils library. It contains several different classes that provide flexibility and extensibility while still looking somewhat Bean"ish". In particular, the DynaBean and the LazyDynaBean along with their requisite DynaClass descriptors parallel much of what currently exists in our Metacard and MetaCardType classes today. They provide the ability to add properties on the fly to handle situations where we want to dynamically modify the metacard throughout its ingest life cycle as well as the ability to retrieve descriptors for each managed attribute.

The approach taken to prototype the DynamicMetacard was to create the classes that implement dynamic metacards in their own packages and modules, apply them to a sample scenario, in this case a NITF transformer, and see that they work throughout the system. That is, does the rest of the system notice any difference between existing metacards and new dynamic metacards in all aspects?

The structure of the prototype follows that of the current Metacard implementation. The following diagram shows the modules and source package structure of the Dynamic Metacard prototype:

Dynamic Metacard Prototype Structure

The following table describes each module and the classes included in each:

catalog-core-dynamic-api
	Class/Interface	Description
	DynamicMetacard	The equivalent of Metacard interface. It extends "Metacard" and "MetacardType" and adds methods: addAttribute(name, object); setAttribute(name, object); getMetacardTypes() and isType().
	MetacardPropertyDescriptor	This is roughly the equivalent of AttributeDescriptor. It includes the methods to describe an attribute - it's name, type, etc.
	MetacardFactory	Interface used for creating instances of registered metacard types. Includes the ability to combine metacard types resulting in a new metacard type. Also provides methods for retrieving MetacardPropertyDescriptors for specified types. Allows the registration of new metacard types.
catalog-core-dynamic-impl
	DynamicMetacardImpl	Implements the DynamicMetacard interface (as well as the Metacard and MetacardType interfaces). Utilizes LazyDynaBean instances as an extensible map for attributes.
	MetacardAttributeDescriptor	Implements the existing AttributeDescriptor and AttributeType interfaces. Used to bridge dynamic metacards with the existing system that is expecting standard Metacards. Used in implementations such as getAttributeDescriptors() and getAttributeDescriptor(String s). Eventually this could be removed if/when the entire system is updated to deal with DynamicMetacard instances directly.
	MetacardPropertyDescriptorImpl	Extends the DynaProperty class to include fields used by the catalog: indexedBySource, tokenized, and store.
	MetacardFactoryImpl	Manages the registered metacard types and generates instances of requested types.
catalog-core-dynamic-registry
	MetacardReader	Reads metacard definitions from an input stream and creates and registers a new metacard type. Coupled with a camel directory monitor, this allows administrators to drop a new metacard definition into a directory and that metacard with the defined attributes is immediately available in the system.
	MetacardAttribute	Corresponds to a MetacardPropertyDescriptor - but handles translation between String representation of types and actual java objects.
	MetacardClassBuilder	As metacard and attributes are read, this class takes them one component at a time and then generates the underlying LazyDynaClass that is registered with the factory and used to create new instances of DynamicMetacard.

The MetacardReader class reads a metacard description xml file like the one above, generates and registers a new metacard type with the MetacardFactory. This is accomplished by establishing a camel route that monitors a directory, initially /etc/metadata, and as a file is dropped into the directory, it invokes the parseMetacardDefinition method on the MetacardReader instance. Currently, this is configured in a blueprint file.

The DDF-1760 branch provides an example of a metacard description file for the NITF transformer that is currently being worked. When the sample nitf metacard descriptor is dropped into the /etc/metadata directory, it is read, parsed, and the new nitf metacard type is registered with the factory allowing new instances of NITF metacards to be created from that point forward.

Using Dynamic Metacards

For an example of using the dynamic metacards, a prototype NITF input transformer has been modified to create instances of Dynamic Metacards instead of the traditional metacards. The key aspects of the usage are shown in the code snippet below:

Creating a NITF metacard

// Create a metacard with all the base metacard attributes plus nitf-specific attributes
Metacard metacard = metacardFactory.newInstance("nitf");
 
metacard.setAttribute(Metacard.ID, id);
metacard.setAttribute(Metacard.CONTENT_TYPE, MIME_TYPE);
 
...
 
metacard.setAttribute("complexityLevel", nitfFile.getComplexityLevel());

Metacard Validation

The Bean Validation framework libraries validate JavaBeans and return a set of constraint violations that occurred. It is also possible to validate individual fields as well. There are currently a couple of different implementations of the validation framework. The two most notable are the Hibernate Validator which is the JSR-349 reference implementation, and the Apache BVal implementation. Both have been updated recently. Camel includes the Hibernate validator as one of its components and the Hibernate validator is OSGi-friendly so that was selected for the initial validation work.

Currently the validation framework expects to be operating on proper JavaBean classes - meaning that it can retrieve properties and fields from the class definition in order to check for annotations, etc. Our goal is to extend this capability to not just JavaBeans but DynaClass instances that have methods of the form get(name) and set(name, value) rather than proper getName() and setName(value) methods. There are a couple of different approaches to make this work and we are currently looking at implementing a prototype of the validation capabilities.