Alauda Streaming Service for Kafka
  • 简体中文

    • 创建实例

    您可以创建 Kafka 实例，构建高吞吐、低延迟的实时数据管道，以支持流式数据处理、服务解耦等场景下业务系统的多样化需求。

    目录

    创建 Kafka 实例操作步骤创建单节点 Kafka 实例操作步骤

    创建 Kafka 实例

    操作步骤

    CLI
    Web 控制台

    通过 CLI 创建 Kafka 实例：

    cat << EOF | kubectl -n default create -f -
apiVersion: middleware.alauda.io/v1
kind: RdsKafka
metadata:
  name: my-cluster
spec:
  entityOperator:
    topicOperator:
      resources:
        limits:
          cpu: 1
          memory: 2Gi
        requests:
          cpu: 1
          memory: 2Gi
    userOperator:
      resources:
        limits:
          cpu: 1
          memory: 2Gi
        requests:
          cpu: 1
          memory: 2Gi
    tlsSidecar:
      resources:
        limits:
          cpu: 200m
          memory: 128Mi
        requests:
          cpu: 200m
          memory: 128Mi
  version: 4.2
  replicas: 3
  config:
    auto.create.topics.enable: "false"
    auto.leader.rebalance.enable: "true"
    background.threads: "10"
    compression.type: producer
    default.replication.factor: "3"
    delete.topic.enable: "true"
    log.retention.hours: "168"
    log.roll.hours: "168"
    log.segment.bytes: "1073741824"
    message.max.bytes: "1048588"
    min.insync.replicas: "1"
    num.io.threads: "8"
    num.network.threads: "3"
    num.recovery.threads.per.data.dir: "1"
    num.replica.fetchers: "1"
    unclean.leader.election.enable: "false"
  resources:
    limits:
      cpu: 2
      memory: 4Gi
    requests:
      cpu: 2
      memory: 4Gi
  storage:
    size: 1Gi
    # Replace with available storage class
    class: sc-topolvm
    deleteClaim: false
  kafka:
    listeners:
      plain: {}
      external:
        type: nodeport
        tls: false
  mode: KRaft
  controller:
    replicas: 3
    resources:
  	limits:
  	  cpu: 1
  	  memory: 2Gi
  	requests:
  	  cpu: 1
  	  memory: 2Gi
    roles:
  	  - controller
    storage:
  	  size: 1Gi
  	  class: sc-topolvm
  	  deleteClaim: false
EOF

    创建实例后，您可以使用以下命令查看实例状态：

    $ kubectl get rdskafka -n <namespace> -o=custom-columns=NAME:.metadata.name,VERSION:.spec.version,STATUS:.status.phase,MESSAGE:.status.reason,CreationTimestamp:.metadata.creationTimestamp
NAME         VERSION   STATUS   MESSAGE                                   CreationTimestamp
my-cluster   4.2       Active   <none>                                    2025-03-06T08:46:57Z
test42       4.2       Failed   Pod is unschedulable or is not starting   2025-03-06T08:46:36Z

    输出表字段含义如下：

    字段描述
    NAME实例名称
    VERSION当前仅支持版本：4.2
    STATUS实例当前状态，可能包含以下状态：
    • Creating：正在创建实例
    • Updating：正在更新实例
    • Failed：实例发生了不可恢复的错误
    • Paused：实例已被手动暂停
    • Restarting：实例正在重启
    • Active：实例已可使用
    MESSAGE实例当前状态的原因
    CreationTimestamp实例创建时间戳

    创建单节点 Kafka 实例

    重要提示

    建议在创建实例时将 Broker 节点数设置为 3。如果您的 Broker 节点数小于建议值 3，则需要在创建过程中修改某些参数。

    操作步骤

    CLI
    Web 控制台

    通过 CLI 创建单节点 Kafka 实例：

    cat << EOF | kubectl -n default create -f -
apiVersion: middleware.alauda.io/v1
kind: RdsKafka
metadata:
  name: my-cluster
spec:
  entityOperator:
    topicOperator:
      resources:
        limits:
          cpu: 1
          memory: 2Gi
        requests:
          cpu: 1
          memory: 2Gi
    userOperator:
      resources:
        limits:
          cpu: 1
          memory: 2Gi
        requests:
          cpu: 1
          memory: 2Gi
    tlsSidecar:
      resources:
        limits:
          cpu: 200m
          memory: 128Mi
        requests:
          cpu: 200m
          memory: 128Mi
  version: 4.2
  replicas: 1
  config:
    auto.create.topics.enable: "false"
    auto.leader.rebalance.enable: "true"
    background.threads: "10"
    compression.type: producer
    delete.topic.enable: "true"
    log.retention.hours: "168"
    log.roll.hours: "168"
    log.segment.bytes: "1073741824"
    message.max.bytes: "1048588"
    min.insync.replicas: "1"
    num.io.threads: "8"
    num.network.threads: "3"
    num.recovery.threads.per.data.dir: "1"
    num.replica.fetchers: "1"
    unclean.leader.election.enable: "false"
    ## Ensure that the following parameter configurations are correct
    default.replication.factor: "1"
    offsets.topic.replication.factor: 1
    transaction.state.log.replication.factor: 1
    transaction.state.log.min.isr: 1
  resources:
    limits:
      cpu: 2
      memory: 4Gi
    requests:
      cpu: 2
      memory: 4Gi
  storage:
    size: 1Gi
    # Replace with available storage class
    class: local-path
    deleteClaim: false
  kafka:
    listeners:
      plain: {}
      external:
        type: nodeport
        tls: false
  mode: KRaft
  controller:
    roles:
      - controller
    # Currently the same as Kafka broker
    replicas: 1
    resources:
      limits:
        cpu: 1
        memory: 2Gi
      requests:
        cpu: 1
        memory: 2Gi
    # Storage maintained the same as Kafka broker
    storage:
      size: 1Gi
      # Replace with available storage class
      class: local-path
      deleteClaim: false
EOF