Installing Druid

Apache Druid is a real-time analytics database used in HCL CDP. The following steps detail its installation using MySQL for metadata storage and S3 for segment storage.

1. Install MySQL and create metadata database.

   yum install mysql-server
    systemctl start mysqld
   
    mysql -u root -p
    CREATE USER 'druid'@'localhost' IDENTIFIED BY '<password>';
    CREATE DATABASE druid
           DEFAULT CHARACTER SET utf8mb4;
    GRANT ALL PRIVILEGES ON druid.* TO
           'druid'@'localhost';
    FLUSH PRIVILEGES;

2. Download and set up Druid.

   wget https://archive.apache.org/dist/druid/25.0.0/apache-druid-25.0.0-bin.tar.gz
    tar -xzf apache-druid-25.0.0-bin.tar.gz
    mv apache-druid-25.0.0 /sdd/
   
    wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/5.1.49/mysql-connector-java-5.1.49.jar
    cp mysql-connector-java-5.1.49.jar /sdd/apache-druid-25.0.0/extensions/mysql-metadata-storage/

3. Configure metadata and segment storage in Druid.

   vi /sdd/apache-druid-25.0.0/conf/druid/single-server/small/_common/common.runtime.properties
   
    # Load extensions
    druid.extensions.loadList=["druid-hdfs-storage", "druid-kafka-indexing-service", "druid-datasketches", "druid-multi-stage-query","mysql-metadata-storage", "druid-s3-extensions"]
   

4. Comment Derby configurations and update the following properties in.

   
    common.runtime.properties
   # Metadata storage
    druid.metadata.storage.type=mysql
   
           druid.metadata.storage.connector.connectURI=jdbc:mysql://<host>/druid?allowPublicKeyRetrieval=true&useSSL=false
   
           druid.metadata.storage.connector.user=druid
    druid.metadata.storage.connector.password=<password>
   

5. Update the following in S3 to store segments.

   
    # Deep storage
    druid.storage.type=s3
    druid.storage.bucket=druid-bucket
    druid.storage.baseKey=druid/segments
    

6. Add the following at the end of the file.

   druid.storage.disableAcl=true

7. Start Druid.

   cd /sdd/apache-druid-25.0.0
    nohup ./bin/start-single-server-small &
    ps -eaf | grep supervise