Prerequisites

Understand the desired component for development as described in the DDF Catalog section.
- Review existing implementations in the source code and application documentation.
Have an IDE and the ability to create OSGi bundles.
Understand the Use of the Whiteboard Design Pattern section and how to publish services to the OSGi service registry.

Overview

Sources are components that enable DDF to talk to back-end services. They let DDF do query and ingest operations on catalog stores, and query operations on federated sources. Sources reside in the Sources area of the DDF Overview.

Creating a New Source

Implement a Source Interface

There are three types of sources that can be created. All of these types of sources can perform a Query operation. Operating on queries is the foundation for all sources. All of these sources must also be able to return their availability and the list of content types currently stored in their back-end data stores.

Catalog Provider - ddf.catalog.source.CatalogProvider
Used to communicate with back-end storage. Allows for Query and Create/Update/Delete operations.
Federated Source - ddf.catalog.source.FederatedSource
Used to communicate with remote systems. Only allows query operations.
Connected Source - ddf.catalog.source.ConnectedSource
Similar to a Federated Source with the following exceptions:
- Queried on all local queries
- SiteName is hidden (masked with the DDF sourceId) in query results
- SiteService does not show this Source's information separate from DDF's.

Implementing any of the source types follows a similar format:

Create a new class that implements the specified Source interface.
Implement the required methods.
Create OSGi descriptor file to communicate with the OSGi registry (described in the Working with OSGi section).
1. Import DDF packages
2. Register source class as service to OSGi registry
Deploy to DDF
- Check the Working with OSGi - Bundles section.

Catalog Provider

Create java class that implements CatalogProvider

public class TestCatalogProvider implements ddf.catalog.source.CatalogProvider

Implement the required methods from the ddf.catalog.source.CatalogProvider interface

public CreateResponse create(CreateRequest createRequest) throws IngestException;

public UpdateResponset update(UpdateRequest updateRequest) throws IngestException;

public DeleteResponse delete(DeleteRequest deleteRequest) throws IngestException;

Import the DDF interface packages to the bundle manifest (in addition to any other required packages):
```
Import-Package: ddf.catalog, ddf.catalog.source
```

Export the service to the OSGi registry:

Blueprint example

<service ref="[[TestCatalogProvider]]" interface="ddf.catalog.source.CatalogProvider" />

The DDF Integrator's Guide provides details on the following Catalog Providers that come with DDF out of the box:

Dummy Catalog Provider

A code example of a Catalog Provider delivered with DDF is the Catalog Solr Embedded Provider.

Federated Source

Create Java class that implements FederatedSource

public class TestFederatedSource implements ddf.catalog.source.FederatedSource

Implement the required methods of the ddf.catalog.source.FederatedSource interface.
Import the DDF interface packages to the bundle manifest (in addition to any other required packages):
```
Import-Package: ddf.catalog, ddf.catalog.source
```

Export the service to the OSGi registry:

Blueprint example

<service ref="[[TestFederatedSource]]" interface="ddf.catalog.source.FederatedSource" />

The DDF Integrator's Guide provides details on the following Federated Sources that come with DDF out of the box:

OpenSearch Source

A code example of a Federated Source delivered with DDF can be found in ddf.catalog.source.solr

Connected Source

Create Java class that implements ConnectedSource

public class TestConnectedSource implements ddf.catalog.source.ConnectedSource

Implement the required methods of the ddf.catalog.source.ConnectedSource interface.
Import the DDF interface packages to the bundle manifest (in addition to any other required packages):
```
Import-Package: ddf.catalog, ddf.catalog.source
```

Export the service to the OSGi registry:

Blueprint example

<service ref="[[TestConnectedSource]]" interface="ddf.catalog.source.ConnectedSource" />

Please Note

In some Providers that are created, there is a need to make Web Service calls through JAXB clients. It is best NOT to create your JAXB client as a global variable. We have seen intermittent failures with the creation of Providers and federated sources when clients are created in this manner. Create your JAXB clients every single time within the methods that require it in order to avoid this issue.

Exception Handling

In general, sources should only send information back related to the call, not implementation details.

Examples:

"Site XYZ not found" message rather than the full stack trace with the original site not found exception.
The caller issues a malformed search request. Return an error describing the right form, or specifically what was not recognized in the request. Do not return the exception and stack trace where the parsing broke.
The caller leaves something out. Do not return the null pointer exception with a stack trace, rather return a generic exception with the message "xyz was missing".

External Resources

Three Rules for Effective Exception Handling