Druid Uses BOS as Deep Storage

Updated at：2025-11-03

Druid

Apache Druid is a high-performance real-time analytics database. Designed specifically for workflows that require rapid data querying and ingestion, it excels in instant data visibility, ad-hoc queries, operational analysis, and high concurrency workloads.

The BOS-HDFS plugin allows BOS to serve as deep storage for storing data.

Install

1. Prepare environment

Download and install Druid (installation steps omitted).

Download the required jar packages from BOS-HDFS and copy them to the extensions/druid-hdfs-storage directory under the Druid installation path.

2. Modify configuration

Modify the conf/druid/_common/common.runtime.properties file to enable access to BOS via HDFS.

                Bash
                
                druid.extensions.loadList=["druid-hdfs-storage"]
druid.storage.type=hdfs
druid.storage.storageDirectory=bos://{bucket}/druid-path

Create a new HDFS configuration file named hdfs-site.xml in the conf/druid/_common/ directory, and configure the BOS access parameters.

                Bash
                
            

                <?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
    <property>
      <name>fs.bos.access.key</name>
      <value>{Your AK}</value>
    </property>
    <property>
      <name>fs.bos.secret.access.key</name>
      <value>{Your SK}</value>
    </property>
    <property>
      <name>fs.bos.endpoint</name>
      <value>http://bj.bcebos.com</value>
    </property>
    <property>
      <name>fs.bos.impl</name>
      <value>org.apache.hadoop.fs.bos.BaiduBosFileSystem</value>
    </property>
    <property>
      <name>fs.AbstractFileSystem.bos.impl</name>
      <value>org.apache.hadoop.fs.bos.BOS</value>
    </property>
</configuration>
            

For more configurable properties, see BOS-HDFS.

Use

Activate Druid-related services to leverage BOS as deep storage for data storage needs.

Elasticsearch Snapshot Data Storage to BOS

BOS HDFS tool