...
This document describes at a high level the overall use of metacards, metacard types, metacard attributes, and their interaction with the rest of the system.
Metacard Usages
The primary use of metacards is to store metadata that describes an item or product ingested into the catalog. When processing and storing metacards in the catalog, the catalog framework processes metacards in multiple ways, including:
- Validating that metacard data is complete and compliant on ingest
- Indexing metacard attributes for optimized searching
- Validating results from other systems before caching/aggregating
- Displaying metacards for editing
- Discovering which fields of the metacard should be displayed
- Validating individual attributes of a metacard upon user entry
- Transforming metacards into requested response formats
Metacard Validation
DDF validates metacards at various points throughout the system. Metacards can be validated as a whole – i.e. every attribute in the metacard is validated successfully, or individual attributes of the metacard may be validated – i.e. from the UI as the user is entering data into a metacard editor.
...
Data Type | Potential constraints |
|---|---|
Integer or Long | Allowable range (0..100, -180..180) |
Float | Allowable range (0.0-1.0) |
XML | Schema validation (xml schema definition) Schematron ruleset validation (ruleset file) |
Enumerations | List of allowable values (“Red,Green,Blue”, “SiteA,SiteB,SiteC”) |
String | Length ( <= 80 characters) Pattern matching (email address, telephone numbers, IDs) |
Normalized Metacards
In order to be truly useful in the enterprise, where users search for information across multiple systems, metacards from each system need to represent common information in a common manner. Data representing a common data type, e.g. file size, should all be represented in the same manner (bytes vs. kilobytes vs. megabytes, etc.). In other words, the data in each metacard should be represented in a normalized fashion.
...
One key enabling flexibility at the metacard level is allowing metacard types to be assigned to an existing Java object. The metacard type in Java includes a MetacardType attribute. That attribute defines the type of metacard represented by this Java object. The definition of that metacard type includes a list of attributes that are unique to that type. Since the metacard type can be created dynamically by passing in a string defining the type name and a set of attributes that describe that type, metacard types can be created either through reading XML descriptions of those metacard types (as happens on system startup with the core metacard types), or dynamically using a description returned by a remote source.
Metacard Types and Attributes Overview
The following diagram provides an overview of how metacard types and attributes are used in the system:
...
The MetacardType Registry holds the definitions of each metacard type that has been created. The definitions for each metacard type are created by reading XML descriptors or by dynamic API invocations. Input transformers can ask for a metacard of a given type and it will be generated with all of the attributes populated (no values assigned yet) based on that type’s definition.
Defining New Metacard Types
New metacard types can be defined either programmatically or by declaratively using an XML metacard description file. Metacard types are defined by providing the name of the metacard type, referencing any existing metacard type(s) that are to be included, and a set of attribute declarations identifying unique attributes that make up this new metacard type. The constraints for the provided attributes are handled separately from the metacard type definition.
...
| Name | Description |
|---|---|
| multiValued | The attribute can contain more than one value |
| indexed | Indicates whether or not this attribute should be indexed by the catalog and participate in query evaluations. Some attributes may only want to be stored and not indexed. |
| stored | Indicates whether or not the catalog must store the value of this attribute. Some attributes may only need to be indexed and not stored. |
| tokenized | Indicates whether or not this attribute should be tokenized, i.e. remove stopwords, before storing or indexing the resulting values. |
Example Metacard Definition
The following xml snippet shows what the format of a metacard definition looks like:
...
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
<metacard>
<name>nitf</name>
<include>ddf</include>
<attributes>
<!-- nitf-specific fields-->
<attribute>
<name>nitf.version</name>
<type>String</type>
<indexed>true</indexed>
<stored>true</stored>
<tokenized>false</tokenized>
<multi-valued>false</multi-valued>
</attribute>
<attribute>
<name>complexityLevel</name>
<type>String</type>
<indexed>true</indexed>
<stored>true</stored>
<tokenized>false</tokenized>
<multi-valued>true</multi-valued>
</attribute>
<attribute>
<name>originatingStationId</name>
<type>String</type>
<indexed>true</indexed>
<stored>true</stored>
<tokenized>false</tokenized>
<multi-valued>true</multi-valued>
</attribute><metacard>
<!-- additional nitf attributes here -->
</attributes>
</metacard> |
Adding new metacard types to the system
New metacard type definitions can be added to the running system by dropping them in to metadata folder (default is <DDF_HOME>/etc/metadata). Once a file is detected in that directory, the system will read it, parse it, and register the corresponding metacard type with the Metacard Type Registry. Once registered, the new metacard type can be used to generate new metacards of that type, or it can be used as the basis for new metacard type definitions.