Installation

This chapter installs the logging stack on a fresh platform. If you are moving an existing ACP 4.3 install to 4.4 instead, see Upgrade from ACP 4.3 to 4.4.

The stack has three parts and installs in this order:

  1. Log storage — deploy either ClickHouse or OpenSearch, plus Kafka, then install the Log Storage operator and hand the endpoints to it. This runs on every storage cluster.
  2. Log Collector — install the collector plugin on every cluster whose logs, events, or audit records must be collected, and point it at the storage cluster.
  3. Log Essentials — install once on the global cluster; publishes the query APIs used by the console.

At the end, the ACP console can query the collected data in Operations Center > Logs.

What you install

The stack ships as three cluster components — two plugins (Log Essentials and Log Collector) and one operator (Log Storage Management). Install only the components that match your topology.

ComponentKindWhere to installPurpose
Alauda Container Platform Log Storage ManagementOperatorEvery cluster that stores logsDeploys log-storage-operator, which watches PlatformLogForward and builds the storage-side data path (aggregate-vector, Razor, schema-migrations) against either ClickHouse or OpenSearch. The backend choice lives in the PLF, not in the operator install form.
Alauda Container Platform Log CollectorPluginEvery cluster whose data must be collectedDeploys the node-level collector (nevermore) and forwards to a nominated storage cluster.
Alauda Container Platform Log EssentialsPluginglobal cluster only, once per platformPublishes the log, event, and audit query APIs used by the console.

A single storage cluster picks ClickHouse or OpenSearch as its backend and serves many collector clusters; a single collector cluster ships to exactly one storage cluster.

For a side-by-side comparison of ClickHouse and OpenSearch, see Log Component Selection Guide.

WARNING
  • Upgrading from ACP 4.2 or 4.3 that uses Log Storage for Elasticsearch? Do not follow this chapter. Follow Upgrade from Log Storage for Elasticsearch and Historical Data Migration instead. Applying a fresh PlatformLogForward on top of a chart-managed installation collides with the operator's auto-adoption path.
  • Alauda Container Platform Log Storage for Elasticsearch is not offered on ACP 4.4. Fresh installs pick ClickHouse or OpenSearch only; there is no Elasticsearch install form to fall back to.

1. Install log storage

Do this on every cluster that will store logs.

1.1 Prepare the external components

The storage side depends on two external systems: a Kafka cluster (mandatory) and a ClickHouse or OpenSearch cluster (choose one). Prepare them before installing the Log Storage operator.

1.1.1 Kafka

Kafka buffers records between the collector and the storage backend. Both ClickHouse and OpenSearch backends require it.

ItemRequirement
VersionKafka 3.7 or later (baseline: 4.2.0).
AuthenticationSASL/SCRAM-SHA-512 (recommended), SASL/SCRAM-SHA-256, or SASL/PLAIN.
TLSOptional. When enabled, the SASL listener runs over TLS.
TopicsThree topics must exist before the storage operator runs: ALAUDA_LOG_TOPIC, ALAUDA_EVENT_TOPIC, ALAUDA_AUDIT_TOPIC.
Account ACLsOne SASL account with Read, Write, Create, Delete, Describe on all three topics, and Read, Describe on the three consumer groups alauda_log, alauda_event, alauda_audit.
Broker configmessage.max.bytes and replica.fetch.max.bytes at least 10 MiB so audit batches that occasionally exceed 1 MiB are not rejected.

If you already run a Kafka cluster, provision the account, topics, and ACLs on it and continue to Section 1.1.2.

Using Alauda Container Platform Kafka. ACP ships its own Kafka product (Alauda Kafka Operator, delivered as a separate cluster plugin). For the full install, tuning, and operations guide, see the Alauda Container Platform Kafka documentation.

A minimal set of CRs to bring a Kafka cluster up for logging looks like the following. The kinds shown use the Alauda strimzi fork (middleware.alauda.io/v1, RdsKafka / RdsKafkaUser / RdsTopic), which is the recommended path on ACP. Upstream strimzi (kafka.strimzi.io/v1beta2, Kafka / KafkaUser / KafkaTopic) also works — the operator does not care which fork you use, only that the mq-conn Secret and the broker-side ACL / topics line up.

The authoritative reference for this section is log-storage-operator/config/samples/quickstart_external_ch_kafka.yaml; the snippets below are trimmed for the doc.

Edit before you apply — these fields do NOT have safe defaults

The manifest below carries placeholders and example values that you MUST customise for your cluster. Do not kubectl apply it as-is.

  • Password (Secret platform-logging-password.stringData.password): replace <generate a strong password of at least 32 characters> with a real secret generated by one of the openssl commands shown. On FIPS hosts this is enforced: < 32 chars is silently rejected by the broker.
  • StorageClass (RdsKafka.spec.storage.class AND RdsKafka.spec.controller.storage.class): fill in your cluster's StorageClass. Leaving it as "" only works when your cluster has a default StorageClass; otherwise brokers Pending on PVC.
  • Broker storage size (RdsKafka.spec.storage.size): default 100Gi fits an evaluation cluster. For production, size it by retention.ms × daily volume × replication factor.
  • Replicas (RdsKafka.spec.replicas, RdsKafka.spec.controller.replicas, RdsTopic.spec.replicas): the manifest shows production values (3 / 3 / 3). For a single-node evaluation cluster, set every one of them to 1. Mixing production topic replicas with a 1-broker cluster blocks topic creation.
  • Topic partitions (RdsTopic.spec.partitions): 30 is a production starting point. Lower for small clusters; must match (or divide into) the aggregate-vector parallelism you expect downstream.
  • Replication-factor config (RdsKafka.spec.config.default.replication.factor / min.insync.replicas / offsets.topic.replication.factor / transaction.state.log.replication.factor / transaction.state.log.min.isr): manifest ships 3 / 2 / 3 / 3 / 2 for a 3-broker production cluster. For a single-broker evaluation cluster set every 3 to 1 and every 2 to 1; a 3-broker config on a 1-broker cluster leaves __consumer_offsets under-replicated and no consumer can commit.
  • Resources (RdsKafka.spec.resources, controller.resources, entityOperator.topicOperator.resources / userOperator.resources, kafkaExporter.resources): the example limits/requests are a reasonable per-broker baseline. Tune for your traffic — brokers scale with throughput, entity-operator stays small.
  • Tolerations (controller.template.pod.tolerations, kafka.template.pod.tolerations, entityOperator.template.pod.tolerations, kafkaExporter.template.pod.tolerations): the manifest tolerates everything so single-node evaluation clusters (where control-plane tainted nodes host workloads) can schedule. Tighten or remove for production clusters where you want Kafka only on worker nodes.
  • upgradeOption.autoUpgrade (RdsKafka.spec.upgradeOption.autoUpgrade): false pins the version — SRE drives upgrades. Setting true lets a CSV bump roll your brokers unattended.

Apply only after every placeholder above is substituted.

# 1. SASL password Secret. On Alauda OS / FIPS hosts the password MUST be
# at least 32 characters; below that, the broker rejects the KafkaUser with
# UnacceptableCredential. Generate with one of:
#   openssl rand -base64 24 | tr -d '\n'   # 32 chars base64
#   openssl rand -hex 16                   # 32 chars hex
apiVersion: v1
kind: Secret
metadata:
  name: platform-logging-password
  namespace: cpaas-system
type: Opaque
stringData:
  password: "<generate a strong password of at least 32 characters>"
---
# 2. The Kafka cluster. Alauda's `RdsKafka` is a strimzi fork whose
# schema differs from upstream Kafka: `version` / `config` / `replicas`
# / `mode` / `controller` / `storage` sit at `spec.*` top level (NOT
# under `spec.kafka.*`), and `spec.kafka.listeners` is a MAP keyed by
# `plain` / `tls` / `external` (NOT a list). Listener authentication is
# SCRAM-SHA-512. The five `config` values below are MANDATORY for
# logging — the broker defaults are too small for the audit-ingest
# batch size and will silently drop audit batches otherwise.
apiVersion: middleware.alauda.io/v1
kind: RdsKafka
metadata:
  name: cpaas-kafka
  namespace: cpaas-system
spec:
  version: 4.2.0
  mode: KRaft                       # MANDATORY: v4.4 kafka runs in KRaft mode (no ZooKeeper)
  replicas: 3                       # broker count — 1 for single-node evaluation, 3 for production
  upgradeOption:
    autoUpgrade: false              # SRE pins the version; set true only if you accept rolling upgrades on CSV bumps
  storage:
    class: ""                       # fill in your StorageClass; leave "" to use the cluster default
    size: 100Gi
  resources:                        # broker pod sizing — tune to your traffic
    limits:
      cpu: "2"
      memory: 4Gi
    requests:
      cpu: 200m
      memory: 1Gi
  config:
    # MANDATORY. Broker default 1 MiB is too small for K8s audit batches
    # (~1.1-1.5 MiB decompressed) and rejects all audit ingest. Value
    # aligns with razor producer max.request.size (10 MiB).
    message.max.bytes: "10485760"
    # MANDATORY. Must be >= message.max.bytes or replica sync stalls.
    replica.fetch.max.bytes: "10485760"
    # MANDATORY. Supports an 8 MiB batch envelope plus SASL overhead.
    socket.request.max.bytes: "104857600"
    # MANDATORY. Prevents ghost topics when razor / aggregate-vector are
    # temporarily misconfigured.
    auto.create.topics.enable: "false"
    # MANDATORY. Prevents silent data loss on broker crash.
    unclean.leader.election.enable: "false"
    # Production replication defaults for a 3-broker cluster. For a
    # single-broker evaluation cluster set every "3" below to "1" and
    # min.insync.replicas / transaction.state.log.min.isr to "1".
    default.replication.factor: "3"
    min.insync.replicas: "2"
    offsets.topic.replication.factor: "3"
    transaction.state.log.replication.factor: "3"
    transaction.state.log.min.isr: "2"
    # Retention sizing — razor consumes within seconds, 48h window is
    # the usual production SLO for replay / incident forensics.
    log.retention.ms: "172800000"             # 48 hours
    log.retention.bytes: "4928307200"         # ~4.6 GiB / partition
    num.partitions: "30"                      # default when auto.create is on; topics below pin this explicitly
  controller:                       # MANDATORY in KRaft mode: dedicated controller quorum
    replicas: 3                     # 1 for single-node evaluation, 3 for production quorum
    roles: [controller]
    storage:
      class: ""                     # same as broker; leave "" for cluster default SC
      size: 5Gi                     # controller logs are small, 5Gi is enough
    resources:
      limits:
        cpu: "1"
        memory: 2Gi
      requests:
        cpu: 200m
        memory: 512Mi
    template:
      pod:
        tolerations:                # tolerate control-plane taint so single-node evaluation clusters schedule
          - operator: Exists
  kafka:
    authorization:
      type: simple
    listeners:                      # MAP keyed by `plain` / `tls` / `external`, NOT a list
      plain:                        # SASL/SCRAM over plain TCP on :9092
        authentication:
          type: scram-sha-512
      # To enable SASL-over-TLS on :9093, add a `tls:` entry here (same
      # `authentication` block as `plain:`) and set `tls.ca` + bootstrap
      # port 9093 in the mq-conn Secret below.
    template:
      pod:
        tolerations:
          - operator: Exists
  entityOperator:
    topicOperator:
      resources:
        limits:
          cpu: 500m
          memory: 1Gi
        requests:
          cpu: 100m
          memory: 256Mi
    userOperator:
      resources:
        limits:
          cpu: 500m
          memory: 1Gi
        requests:
          cpu: 100m
          memory: 256Mi
    template:
      pod:
        tolerations:
          - operator: Exists
  kafkaExporter:                    # exposes broker metrics for the aggregate-vector lag watchdog and dashboards
    resources:
      limits:
        cpu: 200m
        memory: 256Mi
      requests:
        cpu: 100m
        memory: 128Mi
    template:
      pod:
        tolerations:
          - operator: Exists
---
# 3. The SASL account. The nine ACL rules below are ALL required — the
# storage operator's preflight check reports each missing rule by name
# and refuses to roll out until they are in place.
apiVersion: middleware.alauda.io/v1
kind: RdsKafkaUser
metadata:
  name: platform-logging
  namespace: cpaas-system
  labels:
    middleware.alauda.io/cluster: cpaas-kafka
spec:
  authentication:
    type: scram-sha-512
    password:
      valueFrom:
        secretKeyRef:
          name: platform-logging-password
          key: password
  authorization:
    type: simple
    acls:
      # (a) All permissions on the three topics.
      - resource: {type: topic, name: ALAUDA_LOG_TOPIC,   patternType: literal}
        operation: All
        host: "*"
      - resource: {type: topic, name: ALAUDA_EVENT_TOPIC, patternType: literal}
        operation: All
        host: "*"
      - resource: {type: topic, name: ALAUDA_AUDIT_TOPIC, patternType: literal}
        operation: All
        host: "*"
      # (b) Read on the three aggregate-vector consumer groups.
      - resource: {type: group, name: alauda_log,   patternType: literal}
        operation: Read
        host: "*"
      - resource: {type: group, name: alauda_event, patternType: literal}
        operation: Read
        host: "*"
      - resource: {type: group, name: alauda_audit, patternType: literal}
        operation: Read
        host: "*"
      # (c) Read on the razor and LogForward consumer-group prefixes.
      # Without these, razor and LogForward tenants get silent
      # GroupAuthorizationFailed and drop to 0 throughput.
      - resource: {type: group, name: razor-,      patternType: prefix}
        operation: Read
        host: "*"
      - resource: {type: group, name: logforward-, patternType: prefix}
        operation: Read
        host: "*"
      # (d) DescribeConfigs on the cluster (used by the preflight check).
      - resource: {type: cluster, name: kafka-cluster, patternType: literal}
        operation: DescribeConfigs
        host: "*"
---
# 4. The three topics. `spec.topicName` must match the values you put in
# the Kafka connection Secret in Section 1.3 and the ACL literal names
# in step 3. Set `replicas` to `1` for a single-node evaluation cluster
# and `3` for a production three-broker cluster.
apiVersion: middleware.alauda.io/v1
kind: RdsTopic
metadata:
  name: alauda-log-topic                 # DNS-safe CR name
  namespace: cpaas-system                # must match the RdsKafka cluster's namespace
  labels:
    middleware.alauda.io/cluster: cpaas-kafka
spec:
  topicName: ALAUDA_LOG_TOPIC            # broker-side topic name (uppercase)
  partitions: 30
  replicas: 3
  config:
    retention.ms: "172800000"            # 48 hours
    max.message.bytes: "10485760"        # MANDATORY, must match broker
---
apiVersion: middleware.alauda.io/v1
kind: RdsTopic
metadata:
  name: alauda-event-topic
  namespace: cpaas-system
  labels:
    middleware.alauda.io/cluster: cpaas-kafka
spec:
  topicName: ALAUDA_EVENT_TOPIC
  partitions: 30
  replicas: 3
  config:
    retention.ms: "172800000"
    max.message.bytes: "10485760"
---
apiVersion: middleware.alauda.io/v1
kind: RdsTopic
metadata:
  name: alauda-audit-topic
  namespace: cpaas-system
  labels:
    middleware.alauda.io/cluster: cpaas-kafka
spec:
  topicName: ALAUDA_AUDIT_TOPIC
  partitions: 30
  replicas: 3
  config:
    retention.ms: "172800000"
    max.message.bytes: "10485760"
Two names, one topic

The CR's metadata.name is a DNS-safe kebab-case name (alauda-log-topic) and spec.topicName is the actual name the broker exposes (ALAUDA_LOG_TOPIC, uppercase with underscores). Alauda Container Platform Log Collector and the storage operator always read the broker-side name, so the values you paste into the Kafka connection Secret in Section 1.3 must match spec.topicName, not metadata.name.

1.1.2 ClickHouse or OpenSearch

Pick one backend per storage cluster. See Log Component Selection Guide if you have not decided.

ClickHouse
ItemRequirement
VersionClickHouse 24.x or later (baseline: 25.8).
OperatorClickHouse Operator (altinity/clickhouse-operator v0.20 or later) installed cluster-wide.
EndpointHTTP on :8123 or HTTPS on :8443.
Cluster nameMust match spec.configuration.clusters[0].name on the ClickHouseInstallation and the name the platform uses with ON CLUSTER. ACP baseline default: replicated.
DatabaseACP baseline default: observability. Must exist before installing the storage operator.
AccountOne account with full DDL/DML permission on the target database. ACP baseline default: platform-logging.

Cluster topology. Choose one of the profiles below and use the same shards / replicas values when you fill in PlatformLogForward in Section 1.4. Setting the PlatformLogForward values lower silently leaves shards or replicas unused; setting them higher makes the data path try to reach nodes that do not exist.

ProfileShardsReplicasTotal nodes
Single-node evaluation111
Three-node HA133
Six-node scale-out236
Nine-node scale-out339

For component sizing at each profile, see Log Component Capacity Planning.

If you already run a ClickHouse cluster, ensure the database and the logging account are provisioned on it. To build a cluster from scratch, see the Alauda Container Platform ClickHouse documentation.

OpenSearch
ItemRequirement
VersionOpenSearch 3.7.0 (tested version).
Operatoropensearch-project/opensearch-operator installed cluster-wide.
EndpointHTTPS on :9200 (HTTP is supported but not recommended).
Analyzer pluginanalysis-ik (an OpenSearch 3.7.0-compatible build) installed on every OpenSearch node. Required. The query path defaults to the ik_smart analyzer; without the plugin, every keyword or Chinese-content query returns HTTP 400 and the console log page is unusable.
AccountOne account with cluster-manage and log-index manage permission. ACP baseline default: platform-logging.

If you already run an OpenSearch cluster, ensure analysis-ik is installed on every node and the logging account exists. To build a cluster from scratch, follow the Alauda Container Platform OpenSearch product documentation.

1.2 Install the Log Storage operator

Alauda Container Platform Log Storage Management installs log-storage-operator on the storage cluster. One operator covers both ClickHouse and OpenSearch backends; the backend choice is made in the PlatformLogForward you create in Section 1.4, not in the operator install form.

Console

  1. Navigate to Marketplace > Cluster Plugins and select the storage cluster.
  2. Find Alauda Container Platform Log Storage Management and click Install in its row.
  3. Accept the defaults. Optionally set the node selector and tolerations to pin log-storage-operator to infra nodes.
  4. Click Install.

Verify

kubectl -n cpaas-system get deployment log-storage-operator

The READY column must show 1/1.

1.3 Create the connection Secrets

Two Secrets in cpaas-system carry the endpoint and credentials for the external stack. log-storage-operator reads them by their default names (platform-default-*); do not rename them.

Every command in this section runs on the storage cluster.

1.3.1 Kafka Secret

Required for both backends.

Edit before you apply
  • bootstrap: the FQDN:port of your Kafka's bootstrap Service. The example cpaas-kafka-kafka-bootstrap.cpaas-system.svc:9092 matches the plain SASL listener on the RdsKafka from Section 1.1.1; change the hostname if your RdsKafka has a different name, and change the port to :9093 only if you additionally enabled a tls: listener in the RdsKafka.
  • kafkaClusterName: the metadata.name of your RdsKafka CR.
  • username: the SASL account you created with RdsKafkaUser (Section 1.1.1 uses platform-logging).
  • password: resolve from KafkaUser.spec.authentication.password.valueFrom (plaintext — operator reads this Secret, not the K8s SASL Secret the broker reads). The retrieval command is below the manifest.
  • TLS is off by default in this example (plain SASL on :9092). If you enabled the tls: listener in the RdsKafka, add a tls.ca: key with the broker CA PEM (readable from Secret <RdsKafka name>-cluster-ca-cert, key ca.crt) and point bootstrap at :9093.

Apply only after every placeholder above is substituted.

apiVersion: v1
kind: Secret
metadata:
  name: platform-default-mq-conn
  namespace: cpaas-system
type: Opaque
stringData:
  # Required. Comma-separated host:port. The bootstrap Service name is
  # `<RdsKafka name>-kafka-bootstrap` and the plain SASL listener runs on
  # :9092. Defaults in this example match the RdsKafka in Section 1.1.
  bootstrap: "cpaas-kafka-kafka-bootstrap.cpaas-system.svc:9092"

  # Required. Name of the Kafka cluster CR.
  kafkaClusterName: "cpaas-kafka"

  # Required. SASL username.
  username: "platform-logging"

  # Required. SASL password.
  password: "<kafka-password>"

  # Optional. Defaults to SCRAM-SHA-512.
  # Allowed: SCRAM-SHA-512, SCRAM-SHA-256, PLAIN.
  sasl_mechanism: "SCRAM-SHA-512"

  # Optional. Topic names. Defaults shown. Must match the topics you created.
  topics.log: "ALAUDA_LOG_TOPIC"
  topics.event: "ALAUDA_EVENT_TOPIC"
  topics.audit: "ALAUDA_AUDIT_TOPIC"

  # SASL-over-TLS is optional. If you enabled the `tls:` listener in the
  # RdsKafka above, change `bootstrap` to port :9093 and add a `tls.ca` key
  # here with the broker CA PEM — read it from the strimzi-maintained
  # Secret `<RdsKafka name>-cluster-ca-cert` (key `ca.crt`). Omit `tls.ca`
  # entirely for the plain :9092 listener shown above.

Read the values you do not have at hand:

# Bootstrap Service and its port
kubectl -n cpaas-system get svc cpaas-kafka-kafka-bootstrap

# Kafka SASL password (from the Secret you created in Section 1.1.1)
kubectl -n cpaas-system get secret platform-logging-password \
  -o jsonpath='{.data.password}' | base64 -d

# Kafka CA — ONLY needed if you enabled a `tls:` listener on the RdsKafka.
# Omit this step for the plain :9092 listener used by the default example.
#   kubectl -n cpaas-system get secret cpaas-kafka-cluster-ca-cert \
#     -o jsonpath='{.data.ca\.crt}' | base64 -d

Save the manifest as platform-default-mq-conn.yaml and apply it:

kubectl apply -f platform-default-mq-conn.yaml
WARNING

Do not add tls.insecure_skip_verify: "true". When the listener uses TLS, provide tls.ca; when it uses plain text, omit tls.ca and connect on the plain port.

1.3.2 ClickHouse Secret

Skip this section if you use OpenSearch.

Edit before you apply
  • endpoint: HTTPS or HTTP URL of your ClickHouse — the native HTTP interface (default port 8123 / TLS 8443), not the native TCP port 9000.
  • username / password: the ClickHouse user the operator will connect as. The example <clickhouse-password> is a placeholder — put the real password here. The user must have CREATE DATABASE, CREATE TABLE, INSERT, SELECT on the logging database (razor's schema-migrations job creates it on first run). Write the plain-text password here, not a hash. Some ClickHouse deployment guides (for example the ecosystem team's deploy_as_logging_storage how-to) ask you to put a sha256_hex digest into the ClickHouseInstallation manifest; that digest is for the ClickHouse server itself. The platform-default-ch-conn Secret is read by razor-api / aggregate-vector to actually connect to ClickHouse, so it needs the matching plain-text value. Mixing the two produces a PLF that goes Ready but a schema-migrations Job that fails every CREATE TABLE ... ON CLUSTER with 516 Authentication failed.
  • tls.ca: only when ClickHouse runs HTTPS with a certificate your K8s pods do not already trust. Replace <clickhouse-ca-certificate> with the PEM block, or remove it for HTTP / system-trusted CA.

Apply only after every placeholder above is substituted.

apiVersion: v1
kind: Secret
metadata:
  name: platform-default-ch-conn
  namespace: cpaas-system
type: Opaque
stringData:
  # Required. HTTP endpoint, scheme and port included.
  #   http://…:8123     (plain HTTP)
  #   https://…:8443    (HTTPS)
  endpoint: "http://clickhouse-cpaas-clickhouse.cpaas-system.svc:8123"

  # Required. Must match spec.configuration.clusters[0].name on the CHI.
  # ACP baseline default: replicated
  cluster: "replicated"

  # Required. Target database. Must exist. ACP baseline default: observability
  database: "observability"

  # Required. Account with full permission on the target database.
  # ACP baseline default: platform-logging
  username: "platform-logging"
  password: "<clickhouse-password>"

  # Optional. TLS CA for HTTPS with a private CA. Omit for plain HTTP or a
  # trusted CA.
  tls.ca: |-
    -----BEGIN CERTIFICATE-----
    <clickhouse-ca-certificate>
    -----END CERTIFICATE-----

Read values:

# ClickHouse Service and port
kubectl -n cpaas-system get svc -l clickhouse.altinity.com/chi=cpaas-clickhouse

# Cluster name, database, shards, replicas
kubectl -n cpaas-system get chi cpaas-clickhouse \
  -o jsonpath='{"cluster="}{.spec.configuration.clusters[0].name}{"  database="}{.spec.configuration.settings.default_database}{"  shards="}{.spec.configuration.clusters[0].layout.shardsCount}{"  replicas="}{.spec.configuration.clusters[0].layout.replicasCount}{"\n"}'

Apply:

kubectl apply -f platform-default-ch-conn.yaml
WARNING

Do not add tls.insecure_skip_verify: "true". Provide tls.ca for HTTPS endpoints; omit it for plain HTTP endpoints.

1.3.3 OpenSearch Secret

Skip this section if you use ClickHouse.

Edit before you apply
  • endpoints: HTTPS or HTTP URL of your OpenSearch (default port 9200). Comma-separated for multi-node coordination.
  • username / password: the OpenSearch account the operator will connect as. The example <opensearch-password> is a placeholder — put the real password here. The user must be able to manage index templates and ILM policies on the logging index pattern.
  • tls.ca: only when OpenSearch runs HTTPS with a certificate your K8s pods do not already trust. Remove the whole tls.ca key for HTTP / system-trusted CA.

Apply only after every placeholder above is substituted.

apiVersion: v1
kind: Secret
metadata:
  name: platform-default-os-conn
  namespace: cpaas-system
type: Opaque
stringData:
  # Required. Comma-separated HTTP(S) URLs. Put the highly-available
  # coordinator or load balancer first; some data paths use only the first
  # entry.
  endpoints: "https://cpaas-opensearch.cpaas-system.svc:9200"

  # Required unless the OpenSearch cluster allows anonymous access.
  # ACP baseline default: platform-logging
  username: "platform-logging"
  password: "<opensearch-password>"

Apply:

kubectl apply -f platform-default-os-conn.yaml
INFO

The OpenSearch Secret does not carry a tls.ca field. The platform connects to OpenSearch with certificate verification disabled, so a private-CA endpoint needs no CA material here. Use https:// in endpoints and the connection succeeds regardless of the issuer.

1.4 Create the PlatformLogForward

PlatformLogForward (PLF) is the cluster-scoped CR that log-storage-operator watches to build the data path. It is always named platform-default, and there is exactly one per storage cluster.

Edit before you apply

Three sections are environment-specific — pick values that match your cluster size before kubectl apply:

  • externalStorage.shards / .replicas must match the ClickHouse profile you picked in Section 1.1.2. Setting these lower than the ClickHouse topology silently leaves nodes unused; setting them higher makes the data path reach for nodes that do not exist. Single-node evaluation = 1 / 1; three-node HA = 1 / 3; six-node scale-out = 2 / 3; nine-node scale-out = 3 / 3.
  • aggregateVector.replicas scales the Kafka→ClickHouse ingest tier. Evaluation = 1; production = 3 (operator default) and up for high ingest throughput.
  • razor.replicas and razor.resources scale the query-plane serving razor-api. Example values below are a reasonable per-pod baseline; production sites with heavy query load should raise replicas (3+) and tune the limits for the expected concurrent-query budget.

Apply only after the three sizing choices match your target topology.

apiVersion: log.alauda.io/v1alpha1
kind: PlatformLogForward
metadata:
  name: platform-default                            # fixed singleton name; do not change
spec:
  output:
    type: clickhouse                                # required for ClickHouse
  externalStorage:
    type: clickhouse
    shards: 1                                       # match the profile from Section 1.1.2
    replicas: 1                                     # match the profile from Section 1.1.2
    secretRef:
      name: platform-default-ch-conn
      namespace: cpaas-system
  externalMessageQueue:
    type: kafka
    secretRef:
      name: platform-default-mq-conn
      namespace: cpaas-system
  aggregateVector:
    replicas: 1                                     # Kafka→CH ingest tier; raise to 3+ for production throughput
  razor:
    replicas: 1                                     # query-plane serving tier; raise for heavier query load
    resources:
      limits:
        cpu: "4"
        memory: 2Gi
      requests:
        cpu: "1"
        memory: 512Mi

For OpenSearch:

Edit before you apply

externalStorage.shards / .replicas are still accepted on the OpenSearch branch for consistency with the ClickHouse schema, even though OpenSearch manages its own shard/replica allocation on each index. Setting them to 1 / 1 is safe for every topology — the fields are informational for the operator, not propagated to the OpenSearch cluster. Scale aggregateVector.replicas and razor.replicas the same way as the ClickHouse branch above.

apiVersion: log.alauda.io/v1alpha1
kind: PlatformLogForward
metadata:
  name: platform-default                            # fixed singleton name; do not change
spec:
  # No `output.type` on OpenSearch — `externalStorage.type` is the source of
  # truth for backend routing (added in pt L, 2026-08). The ClickHouse branch
  # predates that refactor and its admission + provisioning paths still read
  # `output.type: clickhouse`; the OpenSearch branch was added after and never
  # inherited that legacy field. Setting `output.type: clickhouse` on an
  # OpenSearch PLF is accepted but ignored — the scenario is still routed
  # by `externalStorage.type`.
  externalStorage:
    type: opensearch
    shards: 1                                       # informational on the OS branch; keep as 1
    replicas: 1                                     # informational on the OS branch; keep as 1
    secretRef:
      name: platform-default-os-conn
      namespace: cpaas-system
  externalMessageQueue:
    type: kafka
    secretRef:
      name: platform-default-mq-conn
      namespace: cpaas-system
  aggregateVector:
    replicas: 1                                     # Kafka→OS ingest tier; raise to 3+ for production throughput
  razor:
    replicas: 1                                     # query-plane serving tier; raise for heavier query load
    resources:
      limits:
        cpu: "4"
        memory: 2Gi
      requests:
        cpu: "1"
        memory: 512Mi

Save the file as platform-log-forward.yaml and apply:

kubectl apply -f platform-log-forward.yaml

Watch the resource until Phase becomes Ready and Ready becomes True, then stop with Ctrl-C:

kubectl get platformlogforward platform-default -w

If it does not go Ready within about ten minutes, read the status conditions:

kubectl get platformlogforward platform-default \
  -o jsonpath='{range .status.conditions[*]}{.type}{"\t"}{.status}{"\t"}{.reason}{"\t"}{.message}{"\n"}{end}'

Optional fields worth setting only when needed:

FieldDefaultWhen to set
spec.aggregateVector.replicas3Lower to 1 on a single-node evaluation cluster.
spec.razor.replicas2Same guidance.
spec.razor.resources.limits.cpu1Raise to 2–4 if you observe collector-side buffer growth combined with Razor CPU saturation.
spec.clickhouse.ttl.*See the plugin docsChange per-log-type retention.
spec.clickhouse.tieredPolicy + spec.clickhouse.coldTTL + spec.clickhouse.s3ColdSecretRef(disabled)Enable ClickHouse hot-cold tiered storage on S3. See ClickHouse S3 Storage.
Adopt mode is not for fresh installs

When log-storage-operator starts on a cluster that already contains chart-managed ClickHouse or Kafka resources from a legacy ACP 4.2 or 4.3 installation, it creates PlatformLogForward/platform-default on its own with installMode: Adopt so that migration can proceed non-destructively.

On a fresh install this should not happen. If kubectl get platformlogforward platform-default returns a resource before you apply your manifest and its spec.installMode is Adopt, something on this cluster was mistaken for a legacy installation. Recover by deleting the auto-created object and re-applying yours in one step:

kubectl delete platformlogforward platform-default && kubectl apply -f platform-log-forward.yaml

kubectl -n cpaas-system annotate secret platform-default-mq-conn platform-default-ch-conn \
  log.alauda.io/derived-by=sre-managed --overwrite

For OpenSearch, replace platform-default-ch-conn with platform-default-os-conn.

If the cluster genuinely holds legacy logging data (AppRelease/logclickhouse exists, or PVCs from a previous ACP release are still bound), stop here and follow Upgrade from Log Storage for Elasticsearch and Historical Data Migration instead.

2. Install Log Collector

Install this plugin on every cluster whose logs, events, or audit records must be collected. That includes the global cluster if its own data must be collected.

Console

  1. Navigate to Marketplace > Cluster Plugins and select the target cluster.
  2. Find Alauda Container Platform Log Collector and click Install in its row.
  3. Configure the parameters:
ParameterValue
Log Center PluginSet to Standard. This is the current data path managed by log-storage-operator; it applies to both ClickHouse and OpenSearch backends. The ClickHouse and ElasticSearch options in the same dropdown belong to the chart-installed stacks from earlier releases; on ACP 4.4 they exist only for upgrading customers whose collector still points at a chart-installed storage cluster. Fresh installs must pick Standard.
Storage Cluster NameEnter the name of the cluster where PlatformLogForward/platform-default is Ready. When the collector and the storage are the same cluster, enter that cluster's name. This field is a plain text input in the Standard branch.
Storage PathAbsolute path for the collector's local working data. Default /cpaas/data/nevermore. On Alauda OS nodes use /var/cpaas/data/nevermore. Cannot be changed after installation.
Mount PathsOptional. Absolute host paths containing log files the collector must read.
Audit, Event, Kubernetes, Platform, System, WorkloadLog types to collect. Defaults enable Audit, Event, System, and Workload.
  1. Optional. Expand the advanced form to pin the collector to specific nodes with a node selector and tolerations.
  2. Click Install.

YAML

Edit before you apply

Every <...> is a placeholder and the version string is example-only. Apply as-is will fail — ACP's cluster-transformer rejects the ModuleInfo if the name/label does not resolve or the version is not registered.

  • Replace both <collector-cluster> occurrences (in metadata.labels.cpaas.io/cluster-name and in metadata.name) with the name of the cluster where the collector runs. Both must match, and the K8s name constraint rejects < / > / uppercase.
  • Replace <storage-cluster> (in spec.config.crossClusterDependency.logstorage) with the cluster where PlatformLogForward/platform-default is Ready.
  • Replace spec.version: v4.4.0 with the actual version your cluster has: kubectl get moduleconfigs | grep logagent and copy a logagent-v4.4.x name. The ModuleConfig CR must exist for the version you set here, otherwise the webhook rejects with moduleconfig <name> not found.
  • Review spec.config.dataSource.* and spec.config.components.nevermore.storageSpec.dataDir: the defaults are fine for a stock Alauda host, but Alauda OS uses /var/cpaas/data/nevermore and air-gapped sites may need a different volume.

Apply only after every placeholder above is substituted.

apiVersion: cluster.alauda.io/v1alpha1
kind: ModuleInfo
metadata:
  annotations:
    cpaas.io/display-name: logagent
    cpaas.io/module-name: '{"en": "Alauda Container Platform Log Collector", "zh": "Alauda Container Platform Log Collector"}'
  labels:
    cpaas.io/cluster-name: <collector-cluster>
    cpaas.io/module-name: logagent
    cpaas.io/module-type: plugin
    cpaas.io/product: Platform-Center
  name: <collector-cluster>-logagent
spec:
  version: v4.4.0                                   # substitute the version reported by ModuleConfig
  config:
    crossClusterDependency:
      logstorage: <storage-cluster>                 # cluster where PlatformLogForward is Ready
    dataSource:
      audit: true
      event: true
      kubernetes: false
      platform: false
      system: true
      workload: true
    components:
      nevermore:
        storageSpec:
          dataDir: /cpaas/data/nevermore            # on Alauda OS: /var/cpaas/data/nevermore
    storage:
      type: Standard                                # v4.4 data path via log-storage-operator

Key fields:

FieldDescription
spec.config.crossClusterDependency.logstorageName of the storage cluster where PlatformLogForward/platform-default is Ready. The v4.4 Standard branch uses this single field for both ClickHouse and OpenSearch. Do not set the legacy logcenter or logclickhouse keys.
spec.config.storage.typeMust be Standard on a fresh 4.4 install. ClickHouse and ElasticSearch remain for upgrading customers whose collector still points at a chart-installed storage cluster; do not use them on new deployments.
spec.config.dataSource.*Log types to collect.
spec.config.components.nevermore.storageSpec.dataDirLocal working-data path on the node. Cannot be changed after installation.

Verify

Run from the global cluster:

kubectl get moduleinfo -l cpaas.io/module-name=logagent

Every collector cluster's row must show STATUS=Running.

3. Install Log Essentials

Run on the global cluster. Install once per platform. The install flow is unchanged from earlier releases.

Console

  1. Navigate to Marketplace > Cluster Plugins and select the global cluster.
  2. Find Alauda Container Platform Log Essentials and click Install in its row.
  3. Accept the defaults and click Install.

YAML

Edit before you apply

Replace spec.version: v4.4.0 with the actual version registered in your cluster: kubectl get moduleconfigs | grep log-api and copy a log-api-v4.4.x name. If the version is not registered, cluster-transformer rejects the ModuleInfo with moduleconfig <name> not found. The metadata.name (global-log-api) and metadata.labels.cpaas.io/cluster-name (global) are fixed — the log-api plugin is a global-cluster-only component.

apiVersion: cluster.alauda.io/v1alpha1
kind: ModuleInfo
metadata:
  annotations:
    cpaas.io/display-name: log-api
    cpaas.io/module-name: '{"en": "Alauda Container Platform Log Essentials", "zh": "Alauda Container Platform Log Essentials"}'
  labels:
    cpaas.io/cluster-name: global
    cpaas.io/module-name: log-api
    cpaas.io/module-type: plugin
    cpaas.io/product: Platform-Center
  name: global-log-api
spec:
  version: v4.4.0                                   # substitute the version reported by ModuleConfig

Verify

kubectl get moduleinfo -l cpaas.io/module-name=log-api

The STATUS column must be Running.

4. Verify end-to-end

After every plugin is Running, verify that new records actually flow from a collector cluster into the storage backend.

  1. On every storage cluster:

    kubectl get platformlogforward platform-default \
      -o jsonpath='{"phase="}{.status.phase}{"  ready="}{.status.conditions[?(@.type=="Ready")].status}{"\n"}'

    Must read Ready / True.

  2. On every storage cluster:

    kubectl -n cpaas-system get pods -l app.kubernetes.io/managed-by=log-storage-operator

    aggregate-vector-* and razor-* pods must be Running; the schema-migrations-* Job must have STATUS=Completed.

  3. Log in to the ACP console, open Operations Center > Logs > Log Query Analysis, pick the storage cluster in the top-of-page cluster picker, and query for Workload Logs from a namespace that is actively producing logs. Recent records must appear.

If no records arrive within ten minutes, before changing configuration, work through the layers in order: collector ModuleInfo status → PlatformLogForward conditions → Kafka topic activity → backend ingest.

Next steps

Once every plugin above reports installed: