Overview

Role

The

DDF

Catalog provides a framework for storing, searching, processing, and transforming information. Clients typically perform query, create, read, update, and delete (QCRUD) operations against the Catalog. At the core of the Catalog functionality is the Catalog Framework, which routes all requests and responses through the system, invoking additional processing per the system configuration.

Design

The Catalog is composed of several components and an API that connects them together. The Catalog API is central to

DDF

's architectural qualities of extensibility and flexibility. The Catalog API consists of Java interfaces that define Catalog functionality and specify interactions between components. These interfaces provide the ability for components to interact without a dependency on a particular underlying implementation, thus allowing the possibility of alternate implementations that can maintain interoperability and share developed components. As such, new capabilities can be developed independently, in a modular fashion, using the Catalog API interfaces and reused by other

DDF

installations.

Ensuring Compatibility

The Catalog API will evolve, but great care is taken to retain backwards compatibility with developed components. Compatibility is reflected in version numbers. For more information, see the Software Versioning section in the Integrator's Guide Appendix.

Catalog Architecture

Unable to render {include} The included page could not be found.

DDF Camel Components
- Catalog Framework Camel Component
Sources — connect Catalog components to data sources, both local and remote
- Catalog Provider — writable data stores that support query, create, update, and delete operations to manage data
- Remote Sources — read-only data Sources
  - Federated Source — a Remote Source that can be included in federated queries by request or as part of an enterprise query
  - Connected Source — a Source that is included in all local and federated queries, but is hidden from external clients
Developing at the Framework Level — Help with framework concepts and development.
- Developing Complementary Frameworks
- Developing Console Commands
Federation — provides the capability to extend the DDF enterprise to include Remote Sources, which can include other instances of DDF
- Federation Strategy — federates a query to all of the Remote Sources in the query's list, processes the results in a unique way, and then returns the results to the client
DDF Catalog OpenSearch
- OpenSearch Source — allows the DDF Catalog to federate queries via OpenSearch to a CDR-compliant Search Service
- OpenSearch Endpoint — provides a RESTful endpoint that a client accesses to send OpenSearch formatted queries
Catalog Transformers — transform data to and from various formats
- XSLT Transformer — allows developers to create light-weight Query Response Transformers and Metacard Transformers using only a bundle header and XSLT files.
- Included Input Transformers — transforms raw data (text/binary) into a Metacard
  - Tika Input Transformer — transforms a Microsoft Office, PDF, and OpenOffice documents into a Catalog Metacard
  - GeoJSON Input Transformer — transforms GeoJSON into a Catalog Metacard
- Included Metacard Transformers — transforms a Metacard into a text/binary format
  - HTML Metacard Transformer — transforms a Metacard into an HTML formatted document
  - XML Metacard Transformer — transforms a Metacard into an XML formatted document
  - GeoJSON Metacard Transformer — transforms a Metacard into GeoJSON text
  - Thumbnail Metacard Transformer — retrieves the thumbnail bytes of a Metacard
  - Metadata Metacard Transformer — returns the Metacard.METADATA attribute value when given a Metacard.
  - Resource Metacard Transformer — retrieves the resource bytes of a Metacard product
- Included Query Response Transformers — transforms a query response containing multiple Metacards into a text/binary format
  - XML Query Response Transformer — transforms a Query Response into an XML formatted document
  - Atom Query Response Transformer — transforms a Query Response into an Atom 1.0 http://tools.ietf.org/html/rfc4287 feed
  - SearchUI — a QueryResponseTransformer that not only provides results in a html format but also provides a convenient, simple querying user interface
Catalog Fanout Framework App — provides an implementation of the Catalog Framework that acts as a proxy, federating requests to all available sources
DDF Catalog REST
- RESTful CRUD Endpoint — allows clients to perform CRUD operations on DDF using REST, a simple architectural style that performs communication using HTTP.
Operations — represent all transactions that occur in the Catalog, including requests and responses
Eventing — allows endpoints (and thus external users) to create a "standing query" and be notified when a matching Metacard is created, updated, or deleted
Catalog Framework — the core of the Catalog application, routes requests and responses between all Catalog Components
DDF Catalog Application Install/Uninstall
DDF Catalog Core
- Catalog API — The Catalog API is an OSGi bundle that contains the Java interfaces and classes for the DDF Catalog components.
- Standard Catalog Framework — provides the reference implementation of a Catalog Framework
- Standard Event Processor — creates and deletes subscriptions, applies the criteria for each subscription to ingest events, and sends events to registered consumers when the criteria is matched
- URL Resource Reader — obtains a resource given an http, https, or file-based URL
- Sorted Federation Strategy — the default Federation Strategy which returns results sorted by the sorting parameter specified in the federated query
- Fanout Event Processor — Event Processor for fanout configuration
- Metacard Groomer — The Metacard Groomer Pre-Ingest Plugin makes modifications to CreateRequest and UpdateRequest metacards.
Resource Components — used to work with Resources, i.e., the data represented by the cataloged metadata
- Resource Reader — retrieves resources associated with Metacards via URIs
- Resource Writer — stores a resource and produces a URI that can be used to retrieve the resource later
Catalog Plugins — process Catalog operations, generally before and after they are executed
- Pre-Query Plugin — invoked before a query operation is sent to a Source
- Post-Query Plugin — invoked after a query has been executed successfully, but before the response is returned
- Pre-Ingest Plugin — invoked before an ingest operation is sent to a Source
- Post-Ingest Plugin — invoked after data has been created, updated, or deleted
- Pre-Get Resource Plugin — invoked before a request to retrieve a resource is sent to a Source
- Post-Get Resource Plugin — invoked after a resource has been retrieved, but before it is returned
- Pre-Subscription Plugin — invoked before a Subscription is activated by an Event Processor
- Pre-Delivery Plugin — invoked before a Delivery Method is invoked on a Subscription
Developing Catalog Components — Describes how to create Catalog components. Used in conjunction with the Javadoc to begin extending the DDF Catalog.
- Catalog Development Fundamentals — Introduces the fundamentals of working with the Catalog API, including Metacards and use of the OGC Filter for Queries
  - Simple Catalog API Implementations — Describes the simple implementations that are included out-of-the-box with DDF.
  - Use of the Whiteboard Design Pattern — Introduces the Whiteboard Design Pattern and describes its use in OSGi.
  - Working with the Catalog Framework — Provides an overview and usage of the Catalog Framework
  - Working with Metacards — Overview of how Metacards are used, created, and processed
  - Working with Queries — Overview of Query objects such as what they are, how they are created, and how are they processed.
  - Working with Subscriptions — Description of how to create and use subscriptions with ApplicationName Eventing.
  - Working with Filters
  - Working with Transformers — Description and usage of what transformers are and how they are used.
  - Working with Resources — Description of retrieval and storage of resources.
  - Commons-DDF Utilities — Explains this bundle's classes and usage.
  - Working with Settings
- Developing an Endpoint — Describes how to create an Endpoint and the description of methods an Endpoint can invoke.
- Developing a Catalog Plugin — Describes the different plugins that can be created.
- Developing a Source — Describes the different types of sources and how to create them.
  - Developing a Filter Delegate
- Developing Transformers — Describes the different types of transformers and how to create them.
- Developing a Resource Reader — Describes how to create a ResourceReader.
- Developing a Resource Writer
- Developing a Registry Client — Briefly describes how Registry Clients create Sources within the framework.
Endpoints — components that accept external requests and interface with internal components, normalizing the request and denormalizing the response
Data Components — representations of data in the Catalog, primarily metadata represented as a Metacard
DDF Catalog Schematron
- Schematron Pre-Ingest Plugin Framework — provides a pre-ingest interceptor that validates the incoming request against a Schematron ruleset (or rule sets)

Whitelist

The following packages have been exported by the DDF Catalog Application and are approved for use by third parties:

ddf.catalog
ddf.catalog.util
ddf.catalog.event
ddf.catalog
ddf.catalog.validation
ddf.catalog.source
ddf.catalog.filter
ddf.catalog.federation
ddf.catalog.plugin
ddf.catalog.operation
ddf.catalog.transform
ddf.catalog.data
ddf.catalog.resource
ddf.measure
ddf.catalog.filter.delegate
ddf.catalog.impl.filter
ddf.services.schematron
ddf.geo.formatter

Browser not supported