Log Component Capacity Planning
The log storage components hold logs, events, and audit records collected from one or more clusters. Estimate your log scale before install and size the storage cluster using the profiles below.
WARNING
- The tables in this chapter are laboratory reference numbers. Provision more resources than the reference numbers, and do not push the collected volume above the profile's throughput.
- Disk baseline for the tables:
6000 iops,250 MB/ssequential read and write, SSD, independently mounted. Weaker disks require a larger profile than the table suggests and additional CPU + memory headroom. - Kafka runs in KRaft mode. No Zookeeper is deployed; the tables in earlier releases had a separate Zookeeper row that no longer applies.
- Older ACP releases ran the Elasticsearch backend with a separate
Lanayaconsumer between Kafka and Elasticsearch. The recommended backends (ClickHouse and OpenSearch) do not use Lanaya; the ingest role belongs toaggregate-vector. - In ACP 4.4 the storage backend (ClickHouse or OpenSearch) and the Kafka cluster are user-owned external endpoints; the sizes below assume the storage cluster and the Kafka cluster are sized against the same profile. If your Kafka is shared with other workloads, size the shared cluster to the sum of every consumer's
Kafka brokerrow.
TOC
Common componentsClickHouse profilesSingle node — evaluation only, up to ~18,000 logs/sThree nodes — up to ~20,000 logs/sSix nodes — up to ~40,000 logs/sNine nodes — up to ~69,000 logs/sOpenSearch profilesSmall — three data nodes, up to ~9,000 logs/sMedium — three master, five data nodes, up to ~25,000 logs/sLarge — three master, seven data nodes, up to ~30,000 logs/sElasticsearch profiles (legacy)Small — three nodes, up to ~6,300 logs/sSmall — five nodes, up to ~9,900 logs/sLarge — three master, five data nodes, up to ~25,000 logs/sLarge — three master, seven data nodes, up to ~30,000 logs/sSizing notesCommon components
All backends share these components:
ClickHouse profiles
The ClickHouse backend is the recommended choice for high throughput and cost-sensitive retention.
Single node — evaluation only, up to ~18,000 logs/s
Three nodes — up to ~20,000 logs/s
Six nodes — up to ~40,000 logs/s
Nine nodes — up to ~69,000 logs/s
OpenSearch profiles
The OpenSearch backend is the recommended choice when Lucene-style search experience matters more than raw ingest throughput.
Small — three data nodes, up to ~9,000 logs/s
Medium — three master, five data nodes, up to ~25,000 logs/s
Large — three master, seven data nodes, up to ~30,000 logs/s
Elasticsearch profiles (legacy)
The Elasticsearch backend remains available for existing customers. The classic ACP 4.3 pipeline (Filebeat → Kafka → Lanaya → Elasticsearch) is unchanged; the sizing tables below are reproduced for reference.
Small — three nodes, up to ~6,300 logs/s
Small — five nodes, up to ~9,900 logs/s
Large — three master, five data nodes, up to ~25,000 logs/s
Large — three master, seven data nodes, up to ~30,000 logs/s
Sizing notes
- Set
PlatformLogForward.spec.aggregateVector.replicasandspec.razor.replicasto match the profile you deploy. The CRD defaults (3and2) suit the medium and large profiles; drop them to1on a single-node evaluation cluster. - Raise
spec.razor.resources.limits.cpuabove the profile default when the collector-side buffer grows during traffic bursts and Razor CPU sits at its limit. See How to size Razor for burst traffic once that page is published. - Kafka broker sizing is separate from the storage backend sizing. Follow the Alauda Container Platform Kafka capacity guide for broker-side numbers.