Planning Infra Nodes for Logging Storage
This guide explains the planning considerations for running Logging storage components on dedicated Kubernetes infra nodes.
TOC
ObjectivesWhere to Configure PlacementBefore You Configure PlacementCheck Local PVs and nodeAffinityHistorical Kafka and Zookeeper NodesTroubleshootingObjectives
- Isolate resources: Prevent contention with business workloads.
- Enforce stability: Reduce evictions and scheduling conflicts.
- Simplify management: Centralize infra components with consistent scheduling rules.
Where to Configure Placement
The storage-side workloads (Razor, aggregate-vector, and the one-shot schema-migrations Job) are derived by log-storage-operator from PlatformLogForward. Set their placement through the plugin install form or through the underlying ModuleInfo; do not patch the generated Deployments directly.
- For Alauda Container Platform Log Storage Management (ClickHouse or OpenSearch backend), use the Advanced Configuration section of the operator install form to set the node selector and tolerations. See Installation.
- For Alauda Container Platform Log Collector (a DaemonSet), use the plugin install form on the collector cluster; the same Advanced Configuration section applies.
- For Alauda Container Platform Log Storage for Elasticsearch (legacy), configure placement through
spec.valuesOverride.ait/chart-alauda-log-center.global.nodeSelectorandspec.valuesOverride.ait/chart-alauda-log-center.global.tolerationsin the ACP 4.3 documentation for that plugin.
Do not patch the generated StatefulSets, Deployments, or ClickHouseInstallation resources as the standard way to place Logging storage workloads on infra nodes.
Before You Configure Placement
- Plan the infra nodes according to Cluster Node Planning.
- Confirm whether your storage uses LocalVolume or other PVs with
spec.nodeAffinity. - Make sure the selected infra nodes can satisfy both the scheduling rules and the storage placement constraints.
On Alauda OS nodes, local storage paths are under /var/cpaas, for example /var/cpaas/data/clickhouse or /var/cpaas/data/nevermore. These nodes are immutable and /cpaas, the layout of the traditional operating system, is not writable there. Plan the infra nodes so that the /var/cpaas path is backed by a persistent disk that survives node re-provisioning, and expect pv.spec.local.path on those nodes to reference the /var/cpaas path instead of the /cpaas paths shown in the examples below. For the per-plugin parameters, see Installation.
Check Local PVs and nodeAffinity
If your components use local storage (for example TopoLVM, local PV), confirm whether PVs have spec.nodeAffinity. If so, either:
- Add all nodes referenced by
pv.spec.nodeAffinityto the infra node group, or - Redeploy components using a storage class without node affinity (for example Ceph/RBD).
Example (Elasticsearch):
If the PV shows:
Then Elasticsearch data is pinned to node 192.168.135.243. Ensure that node is part of the infra node group, or migrate storage.
The same principle applies to any Logging storage component that uses node-bound local storage.
Historical Kafka and Zookeeper Nodes
Kafka runs in KRaft mode and does not deploy Zookeeper. On clusters upgraded from an earlier release, however, historical Kafka and Zookeeper node labels may still be in use. Ensure any historically labeled Kafka or Zookeeper nodes are also included in the infra node group:
Newly provisioned clusters only need the kafka label; the zk label is not used.
Troubleshooting
Common issues and fixes:
Example error:
Fix: add matching tolerations to the plugin configuration and make sure the selected infra nodes also satisfy the required storage placement constraints.