#How to Interface with External ES Storage Clusters

You can interface with external Elasticsearch or Kafka clusters by writing YAML configurations. Depending on your business requirements, you can choose to interface with only the external Elasticsearch cluster (while installing Kafka in the current cluster), or you can interface with both the external Elasticsearch and Kafka clusters simultaneously.

#Resource Preparation

Before interfacing, you need to prepare the required credential information.

1. In the left navigation bar, click on Cluster Management > Resource Management, then switch to the cluster that needs the plugin installation.

2. Click on Create Resource Object, and fill in the code box after modifying the parameters according to the code comments.

• Credentials required for interfacing with external Elasticsearch:

  apiVersion: v1
  type: Opaque
  data:
    password: dEdWQVduSX5kUW1mc21acg==  # Must be base64 encoded. Reference command: echo -n <password_value>| base64
    username: YWRtaW4=                  # Must be base64 encoded. Reference command: echo -n <username_value>| base64
  kind: Secret
  metadata:
    name: elasticsearch-basic-auth      # Credential name. Ensure that the value of elasticsearch.basicAuthSecretName in the log storage YAML matches this parameter.
    namespace: cpaas-system             # The namespace where the Elasticsearch component is located, generally cpaas-system.

• If you need to use an external Kafka cluster, you will also need to create credentials for interfacing with the external Kafka cluster:

  apiVersion: v1
  type: Opaque
  data:
    password: dEdWQVduSX5kUW1mc21acg==  # Must be base64 encoded. Reference command: echo -n <password_value>| base64
    username: YWRtaW4=                  # Must be base64 encoded. Reference command: echo -n <username_value>| base64
  kind: Secret
  metadata:
    name: kafka-basic-auth              # Credential name. Ensure that the value of kafka.basicAuthSecretName in the log storage YAML matches this parameter.
    namespace: cpaas-system             # The namespace where the Kafka component is located, generally cpaas-system.

3. Click on Create.

#Operating Procedures

1. In the left navigation bar, click Marketplace > Cluster Plugins.

2. In the top navigation, select the Cluster Name where you want to install Alauda Container Platform Log Storage for Elasticsearch.

3. Click the action button on the right side of Alauda Container Platform Log Storage for Elasticsearch and select Install.

4. Enable the External Elasticsearch switch, then configure the YAML file using the following examples and parameter descriptions:

• Interfacing with the external Elasticsearch cluster while installing Kafka in the current cluster:

  elasticsearch:
    install: false
    address: http://fake:9200                # External ES access address, e.g., http://192.168.143.252:11780/es_proxy
    basicAuthSecretName: elasticsearch-basic-auth # Credentials required for interfacing with external Elasticsearch created in the prerequisites.
  storageClassConfig:
    type: "LocalVolume"   # Default is LocalVolume. Options are "LocalVolume" or "StorageClass".
  kafka:
    auth: true                               # Whether to enable authentication.
    k8sNodes:
    - log1                                   # Node name, obtained from kubectl get nodes.
    - log2
    - log3
    storageSize: 10                          # Storage size, in Gi, default is 10 Gi.

• Interfacing with both the external Elasticsearch cluster and the external Kafka cluster:

  elasticsearch:
    install: false
    address: http://fake:9200                # External ES access address, e.g., http://192.168.143.252:11780/es_proxy
    basicAuthSecretName: elasticsearch-basic-auth # Credentials required for interfacing with external Elasticsearch created in the prerequisites.
  kafka:
    auth: true                               # Whether to enable authentication.
    install: false
    basicAuthSecretName: kafka-basic-auth  # Credentials required for interfacing with external Kafka created in the prerequisites.
    address: 192.168.130.169:9092,192.168.130.187:9092,192.168.130.193:9092     # Kafka access addresses, separated by commas.