Pinot

0: Prerequisites

Run through the Quick Start guide and shut down the frontend server process.

1: Update the data sources configuration

Insert the connector configuration for Pinot in thirdeye-pinot/config/data-sources/data-sources-config.yml. Your config should look like this:

dataSourceConfigs:
  - className: com.linkedin.thirdeye.datasource.pinot.PinotThirdEyeDataSource
    properties:
        zookeeperUrl: 'myZkCluster.myDomain:12913/pinot-cluster'
        clusterName: 'myDemoCluster'
        controllerConnectionScheme: 'https'
        controllerHost: 'myPinotController.myDomain'
        controllerPort: 10611
        cacheLoaderClassName: com.linkedin.thirdeye.datasource.pinot.PinotControllerResponseCacheLoader
    metadataSourceConfigs:
      - className: com.linkedin.thirdeye.auto.onboard.AutoOnboardPinotMetadataSource

  - className: com.linkedin.thirdeye.datasource.mock.MockThirdEyeDataSource
    ...

Note: You’ll have to change the host names and port numbers according to your setup

2: Enable Pinot auto-onboarding

Update the thirdeye-pinot/config/detector.yml file to enable auto onboarding of pinot data sets.

autoload: true

3: Run the backend worker to load all supported Pinot data sets

./run-backend.sh

Note: This process may take some time. The worker process will print log messages for each data set schema being processed. Schemas must contain a timeFieldSpec or a dateTimeFieldSpec in order for ThirdEye to onboard it automatically

4: Stop the backend worker

By pressing Ctrl-C in the terminal

5: Run ThirdEye frontend

./run-frontend.sh