#Installation

#Introduction

The Connectors system has a modular architecture with the following components:

• Connectors Operator: The central management component that handles the deployment and lifecycle of other connector components
• ConnectorsCore: Required core component that provides the foundation for all connector types
• ConnectorsGit: Optional component that adds support for Git services (GitHub, GitLab, etc.)
• ConnectorsGitLab: Optional component that adds support for GitLab-specific features (GitLab CLI, enhanced authentication)
• ConnectorsOCI: Optional component that adds support for container registries (Harbor, Distribution, etc.)
• ConnectorsK8S: Optional component that adds support for Kubernetes clusters
• ConnectorsMaven: Optional component that adds support for Maven registries (e.g., Maven Central, or Maven repositories hosted on Sonatype Nexus).
• ConnectorsPyPI: Optional component that adds support for Python package registries (e.g., PyPI, or Python repositories hosted on Sonatype Nexus).
• ConnectorsNPM: Optional component that adds support for Node.js package registries (e.g., npm, or Node.js repositories hosted on Sonatype Nexus).
• ConnectorsHarbor: Optional component that adds support for Harbor container registries. This document provides instructions for installing and configuring the Connectors system.

#Prerequisites

Before installing, ensure you have:

• A kubernetes cluster
• A kubectl cli configured to communicate with your cluster
• Admin permissions on the cluster
• Connectors Operator is Ready on ACP Operator Hub

#Pod Security Requirements for Installation

Kubernetes enforces Pod Security Standards (PSS) at the namespace level. The Connectors system consists of components with different privilege requirements:

Note: If the namespace is configured with an insufficient policy (e.g., restricted or baseline for CSI components), the CSI driver will fail to start due to blocked privileged operations. Conversely, applying privileged where not needed broadens the namespace's attack surface.

#Install Connectors Operator

First, install the Connectors Operator which manages the lifecycle of all other components.

1. Create a namespace for the operator:

   kubectl create namespace connectors-operator

2. Apply the operator subscription YAML:

   cat <<EOF | kubectl apply -f -
   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     annotations:
       cpaas.io/target-namespaces: ""
     labels:
       catalog: platform
     name: connectors-operator
     namespace: connectors-operator
   spec:
     channel: alpha
     installPlanApproval: Manual
     name: connectors-operator
     source: platform
     sourceNamespace: cpaas-system
   EOF
   
   kubectl wait --for=condition=InstallPlanPending subscription.operators.coreos.com/connectors-operator -n connectors-operator
   
   installplanname=$(kubectl get subscription.operators.coreos.com -n connectors-operator connectors-operator -ojsonpath='{.status.installPlanRef.name}')
   kubectl patch installplan -n connectors-operator ${installplanname} --type='merge' -p='{"spec":{"approved":true}}'

3. Verify the operator is running:

   kubectl get pods -n connectors-operator

   You should see the connectors-operator pod running:

   NAME                                                  READY   STATUS    RESTARTS   AGE
   connectors-operator-controller-manager-xxxxxx-xxxxx   2/2     Running   0          1m

4. Verify that the Custom Resource Definitions (CRDs) have been created:

   kubectl get crds | grep connectors

   You should see CRDs including:

   connectorscore.operator.connectors.alauda.io
   connectorsgit.operator.connectors.alauda.io
   connectorsoci.operator.connectors.alauda.io

#Install ConnectorsCore

After the operator is running, install the required ConnectorsCore component:

1. Create a namespace for connector components (if not already created):

   kubectl create namespace connectors-system

2. Create the ConnectorsCore custom resource:

   cat <<EOF | kubectl apply -f -
   apiVersion: operator.connectors.alauda.io/v1alpha1
   kind: ConnectorsCore
   metadata:
     name: connectors-core
     namespace: connectors-system
   spec: {}
   EOF

3. Monitor the deployment progress:

   kubectl get connectorscore -n connectors-system

4. Wait until the status shows that ConnectorsCore is ready:

   kubectl wait --for=condition=Ready connectorscore/connectors-core -n connectors-system --timeout=300s

5. Verify that the core pods are running:

   kubectl get pods -n connectors-system

   You should see core components including:

   NAME                                              READY   STATUS    RESTARTS   AGE
   connectors-api-xxxxxx                             1/1     Running   0          2m
   connectors-controller-manager-xxxxxx              1/1     Running   0          2m
   connectors-proxy-xxxxxx                           1/1     Running   0          2m

6. Verify that the CRDs required for connector functionality are installed:

   kubectl get crds | grep connectors.alauda.io

   You should see:

   connectorclasses.connectors.alauda.io
   connectors.connectors.alauda.io

#Install ConnectorsGit (Optional)

To add support for Git services like GitHub, GitLab, etc., install the ConnectorsGit component:

1. Create the ConnectorsGit custom resource:

   cat <<EOF | kubectl apply -f -
   apiVersion: operator.connectors.alauda.io/v1alpha1
   kind: ConnectorsGit
   metadata:
     name: connectors-git
     namespace: connectors-system
   spec: {}
   EOF

2. Monitor the deployment progress:

   kubectl get connectorsgit -n connectors-system

3. Wait until the status shows that ConnectorsGit is ready:

   kubectl wait --for=condition=Ready connectorsgit/connectors-git -n connectors-system --timeout=300s

4. Verify that the Git plugin is running:

   kubectl get pods -n connectors-system | grep git

   You should see:

   NAME                                   READY   STATUS    RESTARTS   AGE
   connectors-git-plugin-xxxxxx           1/1     Running   0          1m

5. Verify that the Git ConnectorClass has been created:

   kubectl get connectorclass git

   You should see:

   NAME  READY  AGE
   git   True       1m

#Install ConnectorsGitLab (Optional)

To add support for GitLab-specific features (GitLab CLI, enhanced authentication), install the ConnectorsGitLab component:

1. Create the ConnectorsGitLab custom resource:

   cat <<EOF | kubectl apply -f -
   apiVersion: operator.connectors.alauda.io/v1alpha1
   kind: ConnectorsGitLab
   metadata:
     name: connectors-gitlab
     namespace: connectors-system
   spec: {}
   EOF

2. Monitor the deployment progress:

   kubectl get connectorsgitlab -n connectors-system

3. Wait until the status shows that ConnectorsGitLab is ready:

   kubectl wait --for=condition=Ready connectorsgitlab/connectors-gitlab -n connectors-system --timeout=300s

4. Verify that the GitLab ConnectorClass has been created:

   kubectl get connectorclass gitlab

   You should see:

   NAME     READY  AGE
   gitlab   True   1m

#Install ConnectorsOCI (Optional)

To add support for container registries, like Harbor, Distribution, etc., install the ConnectorsOCI component:

1. Create the ConnectorsOCI custom resource:

   ClusterIP Expose:

   cat <<EOF | kubectl apply -f -
   apiVersion: operator.connectors.alauda.io/v1alpha1
   kind: ConnectorsOCI
   metadata:
     name: connectors-oci
     namespace: connectors-system
   spec: {}
   EOF

   NodePort Expose:

   cat <<EOF | kubectl apply -f -
   apiVersion: operator.connectors.alauda.io/v1alpha1
   kind: ConnectorsOCI
   metadata:
     name: connectors-oci
     namespace: connectors-system
   spec:
    expose:
      type: NodePort
      domain: 192.168.1.123
      nodePort:
        port: 30000
   EOF

   Ingress Expose:

   cat <<EOF | kubectl apply -f -
   apiVersion: operator.connectors.alauda.io/v1alpha1
   kind: ConnectorsOCI
   metadata:
     name: connectors-oci
     namespace: connectors-system
   spec:
    expose:
      type: Ingress
      domain: connectors.proxy.com

2. Monitor the deployment progress:

   kubectl get connectorsoci -n connectors-system

3. Wait until the status shows that ConnectorsOCI is ready:

   kubectl wait --for=condition=Ready connectorsoci/connectors-oci -n connectors-system --timeout=300s

4. Verify that the OCI plugin is running:

   kubectl get pods -n connectors-system | grep oci

5. Verify that the OCI ConnectorClass has been created:

   kubectl get connectorclass oci

#Install ConnectorsK8S (Optional)

To add support for integration with Kubernetes clusters, install the ConnectorsK8S component:

1. Create the ConnectorsK8S custom resource:

   cat <<EOF | kubectl apply -f -
   apiVersion: operator.connectors.alauda.io/v1alpha1
   kind: ConnectorsK8S
   metadata:
     name: connectors-k8s
     namespace: connectors-system
   spec: {}
   EOF

2. Monitor the deployment progress:

   kubectl get connectorsk8s -n connectors-system

3. Wait until the status shows that ConnectorsOCI is ready:

   kubectl wait --for=condition=Ready connectorsk8s/connectors-k8s -n connectors-system --timeout=300s

4. Verify that the Kubernetes ConnectorClass is ready:

   kubectl get connectorclass k8s

#Install ConnectorsMaven (Optional)

To add support for integration with Maven registries, install the ConnectorsMaven component:

1. Create the ConnectorsMaven custom resource:

   cat <<EOF | kubectl apply -f -
   apiVersion: operator.connectors.alauda.io/v1alpha1
   kind: ConnectorsMaven
   metadata:
     name: connectors-maven
     namespace: connectors-system
   spec: {}
   EOF

2. Monitor the deployment progress:

   kubectl get connectorsmaven -n connectors-system

3. Wait until the status shows that ConnectorsMaven is ready:

   kubectl wait --for=condition=Ready connectorsmaven/connectors-maven -n connectors-system --timeout=300s

4. Verify that the Kubernetes ConnectorClass is ready:

   kubectl get connectorclass maven

#Install ConnectorsPyPI (Optional)

To add support for integration with PyPI registries, install the ConnectorsPyPI component:

1. Create the ConnectorsPyPI custom resource:

   cat <<EOF | kubectl apply -f -
   apiVersion: operator.connectors.alauda.io/v1alpha1
   kind: ConnectorsPyPI
   metadata:
     name: connectors-pypi
     namespace: connectors-system
   spec: {}
   EOF

2. Monitor the deployment progress:

   kubectl get connectorspypi -n connectors-system

3. Verify that the Kubernetes ConnectorClass is ready:

   kubectl get connectorclass pypi

#Install ConnectorsNPM (Optional)

To add support for integration with NPM registries, install the ConnectorsNPM component:

1. Create the ConnectorsNPM custom resource:

   cat <<EOF | kubectl apply -f -
   apiVersion: operator.connectors.alauda.io/v1alpha1
   kind: ConnectorsNPM
   metadata:
     name: connectors-npm
     namespace: connectors-system
   spec: {}
   EOF

2. Monitor the deployment progress:

   kubectl get connectorsnpm -n connectors-system

3. Verify that the NPM ConnectorClass is ready:

   kubectl get connectorclass npm

#Install ConnectorsHarbor (Optional)

To add support for integration with Harbor registries, install the ConnectorsHarbor component:

1. Create the ConnectorsHarbor custom resource:

   cat <<EOF | kubectl apply -f -
   apiVersion: operator.connectors.alauda.io/v1alpha1
   kind: ConnectorsHarbor
   metadata:
     name: connectors-harbor
     namespace: connectors-system
   spec: {}
   EOF

2. Monitor the deployment progress:

   kubectl get connectorsharbor -n connectors-system

3. Verify that the Harbor ConnectorClass is ready:

   kubectl get connectorclass harbor

#Uninstall Connectors

To uninstall the Connectors system, remove components in the reverse order of installation.

1. Delete the optional components first (if installed):

   # Delete ConnectorsOCI
   kubectl delete connectorsoci --all -n connectors-system
   
   # Delete ConnectorsGit
   kubectl delete connectorsgit --all -n connectors-system
   
   # Delete ConnectorsGitLab
   kubectl delete connectorsgitlab --all -n connectors-system
   
   # Delete ConnectorsK8S
   kubectl delete connectorsk8s --all -n connectors-system
   
   # Delete ConnectorsMaven
   kubectl delete connectorsmaven --all -n connectors-system
   
   # Delete ConnectorsPyPI
   kubectl delete connectorspypi --all -n connectors-system
   
   # Delete ConnectorsNPM
   kubectl delete connectorsnpm --all -n connectors-system
   
   # Delete ConnectorsHarbor
   kubectl delete connectorsharbor --all -n connectors-system

2. Delete the core component:

   kubectl delete connectorscore --all -n connectors-system

3. Delete the operator:

   kubectl delete -n connectors-operator subscription.operators.coreos.com/connectors-operator

4. Delete the namespaces:

   kubectl delete namespace connectors-system
   kubectl delete namespace connectors-operator

#Custom Configuration

You can customize the deployment of connector components to better suit your environment. All connector components share a similar configuration structure.

#ConnectorsCore Configuration

When creating the ConnectorsCore resource, you can specify custom configuration:

apiVersion: operator.connectors.alauda.io/v1alpha1
kind: ConnectorsCore
metadata:
  name: connectors-core
  namespace: connectors-system
spec:
  # Configure specific workloads
  workloads:
  - name: connectors-api
    replicas: 2
    template:
      spec:
        containers:
        - name: api
          imagePullPolicy: Always
          resources:
            limits:
              cpu: 500m
              memory: 512Mi
            requests:
              cpu: 200m
              memory: 256Mi
          securityContext:
            readOnlyRootFilesystem: true
        nodeSelector:
          kubernetes.io/os: linux

  - name: connectors-controller-manager
    replicas: 1
    template:
      spec:
        containers:
        - name: manager
          resources:
            limits:
              cpu: 300m
              memory: 512Mi

  - name: connectors-proxy
    replicas: 2
    template:
      spec:
        containers:
        - name: proxy
          resources:
            limits:
              cpu: 200m
              memory: 256Mi

#ConnectorsGit Configuration

Custom configuration for the Git plugin:

apiVersion: operator.connectors.alauda.io/v1alpha1
kind: ConnectorsGit
metadata:
  name: connectors-git
  namespace: connectors-system
spec:
  # Configure workloads
  workloads:
  - name: connectors-git-plugin
    replicas: 2
    template:
      spec:
        containers:
        - name: plugin
          resources:
            limits:
              cpu: 300m
              memory: 256Mi
            requests:
              cpu: 100m
              memory: 128Mi

#ConnectorsOCI Configuration

Custom configuration for the OCI plugin:

apiVersion: operator.connectors.alauda.io/v1alpha1
kind: ConnectorsOCI
metadata:
  name: connectors-oci
  namespace: connectors-system
spec:
  # Configure workloads
  workloads:
  - name: connectors-oci-plugin
    replicas: 2
    template:
      spec:
        containers:
        - name: plugin
          resources:
            limits:
              cpu: 300m
              memory: 256Mi
            requests:
              cpu: 100m
              memory: 128Mi

#Additional Configurations

For advanced deployments, you can also specify:

apiVersion: operator.connectors.alauda.io/v1alpha1
kind: ConnectorsCore
metadata:
  name: connectors-core
  namespace: connectors-system
spec:
  # Specify additional manifests to install
  additionalManifests: "<additional manifests>"

  # Other configurations as needed

#Troubleshooting

#connectors-csi is not ready

If daemonset/connectors-csi is not ready, check the events of the connectors-csi pod. A common error looks like:

Error creating: pods "connectors-csi-d4r6r" is forbidden: violates PodSecurity "baseline:latest": non-default capabilities (container "driver" must not include "SYS_ADMIN" in securityContext.capabilities.add), host namespaces (hostNetwork=true), hostPath volumes (volumes "socket-dir", "mountpoint-dir", "registration-dir") . . .

This means the namespace's Pod Security level is too restrictive for the CSI driver.

Fix

• Ensure the namespace is configured with the privileged Pod Security level.
• Update the namespace with the correct labels.
• Restart the connectors-csi DaemonSet.

For details, see Pod Security Requirements for Installation.