How to Configure and Use a Data Distribution Engine

Using a Data Distribution engine can simplify your data management configuration and make it easier to deliver high availability and scalability.

Data Master and Data Slaves

The Data Distribution Master engine is a small-footprint engine whose sole job is to manage to all changeset and dimension changes. You can configure one or more Data Distribution Slave engines so that their changeset data and dimensions is automatically synchronized to the Data Distribution Master. The Data Distribution Slave engines generally are dedicated to serving queries.

Failover

The Data Distribution Master needs no more than 250 MB of available RAM. It should be running on a machine that can be segregated from your Data Distribution Slave hardware for maximum availability. You can also choose to have a backup Master engine as well; however, keep in mind that a new Data Distribution Master engine can be brought up in a few minutes time -- a window that may be acceptable for your business.

Usage for Dimensions Synchronization

If you are synchronizing your dimensions.xml file from the Data Distribution Master, make certain that any changes you make to your dimensions.xml file are imported to the Data Distribution Master engine. Do not import dimensions or changesets directly to the Data Distribution Slave engines.

Data Master Configuration

To configure a Data Distribution Master engine, the discovery.properties file needs to specify that the engine will not handle queries nor create any indices. The following discovery.properties file excerpt shows the relevant settings to enable a Data Distribution Master engine.

port = 8090
jvm.memory = 256
total-partitions = 0

For purposes of illustration, the Data Distribution Master engine is located at

http://localhost:8090

Data Distribution Slave Configuration

To configure a Data Distribution Slave engine, use that engine's Admin Tool Feeds tab to specify pulling changeset data and dimensions from the Data Distribution Master engine.

For purposes of illustration, the Data Distribution Slave engine is located at

http://localhost:8091

View Feeds Tab of Admin Tool

To Synchronize Changeset Data

  1. Use the Admin Tool's Feeds Page Changesets section to specify the URL of the Data Distribution Master engine in the Sync field. The URL to the Master engine is http://localhost:8090
  2. Confirm that the Sync radio button is selected.
  3. Specify a interval to be used to poll the Data Distribution Master for changes.
  4. Apply the changes.

To Synchronize Dimensions

  1. Use the Admin Tool's Feeds Page Dimensions section to specify the URL of the Data Distribution Master engine in the Sync field. The URL to the Master engine is http://localhost:8090
  2. Specify a interval to be used to poll the Data Distribution Master for changes.
  3. Apply the changes.