/
DDF Catalog Solr External Provider

DDF Catalog Solr External Provider

Usage

The Solr Catalog Provider is used in conjunction with an Apache Solr Server data store and acts as the client for an external Solr Server. It is meant to be used only with the Standalone Solr Server (catalog-solr-server).

Solr Catalog External Provider Pros/Cons Chart

FeatureProCon
Scalability
  • Allows the middle-tier to be scaled by pointing various middle-tier instances to one server facade.
  • Possible data tier scalability with Solr Cloud. Solr Cloud allows for "high scale, fault tolerant, distributed indexing and search capabilities."
  • Solr Cloud Catalog Provider not implemented yet
Flexibility
  • REST-like HTTP/XML and JSON APIs that make it easy to use from virtually any programming language
  • Ability to run in separate or same JVM of middle tier.
 
(Administrative) Tools
  • Contains Solr Admin GUI, which allows admins to query, check health, see metrics, see configuration files and preferences, etc
  • External open source tools like Luke will work to allow admins to check index contents
  • JMX metrics reporting is enabled
 
Security
  • Inherits app server security
  • Admin console must be secured and is openly accessible
  • REST-like HTTP/XML and JSON APIs must be secured
  • Current Catalog Provider implementation requires sending unsecured messages to Solr. Without a coded solution, requires network or firewall restrictions in order to secure.
Performance
  • If scaled, high performance.
  • Near Real-time indexing
  • Possible network latency impact
  • Extra overhead when sent over HTTP. Extra parsing for XML, JSON, or other interface formats
  • Possible limitations upon requests and queries dependent on HTTP server settings
Backup/Recovery
  • Built-in Recovery tools which allow in-place backups (does not require server shutdown).
  • Backup of Solr indexes can be scripted
  • Recovery is done as a HTTP request

When to Use

Use the Solr External Provider when the Standalone Solr Server is being used on a separate machine. See the Standalone Solr Server Recommended Configuration.

Installation/Uninstall

Prerequisites

Before the DDF Solr Catalog Application can be installed:

  • the DDF Kernel must be running
  • the DDF Platform Application must be installed
  • the DDF Catalog Application must be installed

Installation

By default the DDF Solr Catalog Application installs the Embedded Solr Provider.

A user will need to uninstall the Solr Catalog Embedded Provider by uninstalling the feature catalog-solr-embedded-provider, and then install the external Solr Catalog Provider by installing the feature catalog-solr-external-provider. This will not install any Solr Servers. Installing this feature will provide a user an "unconfigured" Solr Catalog Provider. See the Configuration section for how to configure this external Solr Catalog Provider to connect to an external Solr Server.

Verification

To verify if the DDF Solr Catalog Application was successfully installed for the External Solr Catalog Provider configuration, verify the appropriate features have been successfully installed. The features catalog-solr-embedded-provider and catalog-solr-server should be uninstalled.

DDF Solr Catalog Application features for External Solr Catalog Provider configuration
ddf@local>features:list | grep catalog-solr-app
[uninstalled] [2.3.0] catalog-solr-embedded-provider       catalog-solr-app-2.3.0    Catalog Provider with locally Embedded Solr Server, implemented using Solr 4.1.0.
[installed  ] [2.3.0] catalog-solr-external-provider       catalog-solr-app-2.3.0    Catalog Provider to interface with an external Solr 4.1.0 Server
[uninstalled] [2.3.0] catalog-solr-server                  catalog-solr-app-2.3.0    Deploys and starts a preconfigured Solr War into this container

Verify the DDF Solr Catalog Application bundles are Active for the External Solr Catalog Provider configuration.

DDF Solr Catalog Application active bundles for External Solr Catalog Provider configuration
ddf@local>list | grep -i solr
[ 271] [Active     ] [            ] [       ] [   80] Apache ServiceMix :: Bundles :: SolrJ (0.3.0.2)
[ 273] [Active     ] [Created     ] [       ] [   80] DDF :: Catalog :: Solr :: External :: Provider (2.3.0)

Uninstall

Reverting the Uninstall

Upgrade

 

Configuration

In order for the external Solr Catalog Provider to work, it must be pointed at the external Solr Server. When the catalog-solr-external-provider feature is installed, it is in an unconfigured state until the user provides an HTTP url to the external Solr Server. The configurable properties for this SCP are accessed from the Catalog External Solr Catalog Provider Configurations in the Admin Console.

Configurable Properties
TitlePropertyTypeDescriptionDefault
Value
Required
HTTP URLurlString

HTTP URL of the standalone, preconfigured Solr 4.x Server.

http://localhost:8181/solrYes
Force Auto CommitforceAutoCommitBoolean / CheckboxWARNING: Performance Impact. Only in special cases should auto-commit be forced. Forcing auto-commit makes the search results visible immediately.Unchecked/FalseNo

Implementation Details

Indexing Text

When storing fields, the Solr Catalog Provider will analyze and tokenize the text values of STRING_TYPE and XML_TYPE AttributeTypes. These types of fields are indexed in at least three ways: in raw form, analyzed with case sensitivity, and then analyzed without concern to case sensitivity. Concerning XML, the Solr Catalog Provider will analyze and tokenize XML CDATA sections, XML Element text values, and XML Attribute values.