#Adding Custom Catalogs to Tekton Hub

#Overview

This guide explains how to configure a Tekton Hub instance to include custom catalogs from existing Git repositories. If you need to create a new catalog repository first, see Creating a Custom Catalog.

Custom catalogs allow organizations to share their Tekton resources through Tekton Hub, making them discoverable via the Hub resolver and web interface.

Important: If you deployed Tekton Hub through TektonConfig, modify the catalog configuration in the TektonConfig resource instead of directly editing the ConfigMap. Direct ConfigMap modifications will be overridden by the Tekton Operator controller.

Namespace Note: Throughout this guide, <tekton-pipelines> is used as a placeholder for your Tekton Hub namespace. Replace it with your actual namespace name. The default installation uses tekton-pipelines namespace.

#Prerequisites

Before you begin, ensure you have:

• A Kubernetes cluster with Tekton Pipelines installed
• A running Tekton Hub instance
• Administrator access to the cluster
• A Git repository containing your custom Tekton resources following the Tekton Catalog structure

#Adding Custom Catalogs

#Step 1: Prepare Your Catalog Repository

Ensure your catalog repository follows the Tekton Catalog Organization TEP structure (see the tutorials section for details).

#Step 2: Configure the API ConfigMap

Add your custom catalog to the tekton-hub-api ConfigMap in the CATALOGS section:

apiVersion: v1
kind: ConfigMap
metadata:
  name: tekton-hub-api
  namespace: <tekton-pipelines>
data:
  CATALOGS: |
    # Default local catalog
    - name: catalog
      org: tektoncd
      type: community
      provider: github
      url: file:////mnt/git/catalog.git
      revision: master

    # Official Tekton catalog (public repository)
    - name: catalog-official
      org: tektoncd
      type: community
      provider: github
      url: https://github.com/tektoncd/catalog
      revision: main

    # Custom public repository
    - name: my-public-catalog
      org: myorganization
      type: community
      provider: github
      url: https://github.com/myorganization/my-public-catalog
      revision: main

    # Custom private repository (requires SSH authentication)
    - name: my-private-catalog
      org: myorganization
      type: community
      provider: github
      url: https://github.com/myorganization/my-private-catalog
      sshurl: git@github.com:myorganization/my-private-catalog.git
      revision: main

Catalog Configuration Fields:

Field Usage Details:

• org: Used for organizing and grouping catalogs in the Hub UI. Can be any meaningful string representing the catalog owner.
• type: Determines how the catalog is classified and displayed in the Hub interface:
  • official: Official Tekton catalogs (typically maintained by Tekton team)
  • community: Community-maintained catalogs (most common choice)
  • enterprise: Enterprise/company-specific catalogs
  • private: Private catalogs (internal use)
• provider: Specifies the Git platform for proper API integration and authentication:
  • github: GitHub and GitHub Enterprise
  • gitlab: GitLab and GitLab self-hosted

URL Configuration Priority:

• sshurl has higher priority than url - when both are provided, sshurl will be used for git operations

• For Public Repositories: Use only url with HTTPS format

  url: https://github.com/org/public-repo

• For Private Repositories: Must provide both url and sshurl - url is required for database storage, sshurl is used for actual git operations

  url: https://github.com/org/private-repo        # Required field (database constraint)
  sshurl: git@github.com:org/private-repo.git     # Used for git operations (takes precedence)

Important: url is always required due to database constraints, even when using sshurl for private repositories.

#Step 3: Apply the Configuration

Apply the updated ConfigMap to your cluster:

$ kubectl apply -f tekton-hub-api-configmap.yaml

#Step 4: Apply Changes

After updating the ConfigMap, apply the changes:

Option 1: Restart the API Pod

$ kubectl delete pod app=tekton-hub-api -n <tekton-pipelines>

Option 2: Wait for Automatic Refresh

The catalog will be automatically refreshed based on the configured CATALOG_REFRESH_INTERVAL.

#Prerequisites for Custom Catalogs

Before adding a custom catalog to Tekton Hub, your repository must meet these requirements:

• Repository Structure: Follow the Tekton Catalog Organization standard
• Resource Validation: All Tasks/Pipelines must include required metadata
• Documentation: Each resource must have proper README and examples

For detailed instructions on creating compliant catalog repositories, see Creating a Custom Catalog.

#Managing Multiple Catalogs

You can configure multiple catalogs in the same Tekton Hub instance:

data:
  CATALOGS: |
    - name: tekton
      org: tektoncd
      type: community
      provider: github
      url: https://github.com/tektoncd/catalog
      revision: main
    - name: company-catalog
      org: mycompany
      type: community
      provider: github
      url: https://github.com/mycompany/tekton-catalog
      revision: main
    - name: team-catalog
      org: mycompany
      type: community
      provider: github
      url: https://github.com/mycompany/team-tekton-catalog
      revision: develop

Best Practices:

• Use descriptive, unique catalog names
• Keep catalog names consistent across environments
• Use appropriate Git revisions (main/master for stable, develop for testing)

#Private Repository Authentication

#SSH Key Authentication

For accessing private Git repositories or repositories from private Git instances, you need to create SSH credentials as a Kubernetes secret.

#Prerequisites

• SSH keys should exist in your ~/.ssh directory
• The SSH public key should be added to your Git repository's deploy keys or your user account

#Creating SSH Secret

Create a Kubernetes secret named tekton-hub-api-ssh-crds with your SSH credentials:

$ kubectl create secret generic tekton-hub-api-ssh-crds \
  --from-file=id_rsa="/path/to/id_rsa" \
  --from-file=id_rsa.pub="/path/to/id_rsa.pub" \
  --from-file=known_hosts="/path/to/known_hosts" \
  --namespace=<tekton-pipelines>

Important Notes:

• The secret must be named exactly tekton-hub-api-ssh-crds
• The secret must be created in the same namespace as your Tekton Hub installation
• Include all three files: private key, public key, and known_hosts

#Generating known_hosts Entries

Important: You must generate the known_hosts file by actually connecting to the Git server to capture its SSH fingerprint. Simply copying example entries may not work due to server key changes or different Git providers.

Recommended Approach:

1. Test SSH connection locally to capture the server's fingerprint:

   # This will add the server's fingerprint to your local known_hosts
   $ ssh -T git@github.com
   $ ssh -T git@your-git-server.com

2. Alternative: Use ssh-keyscan to generate known_hosts entries:

   # For GitHub
   $ ssh-keyscan github.com >> ~/.ssh/known_hosts
   
   # For custom Git servers
   $ ssh-keyscan your-git-server.com >> ~/.ssh/known_hosts

3. Verify by cloning your repositories locally using both URLs:

   # Test the SSH URL that will be used by Tekton Hub
   $ git clone ssh://git@github.com:org/private-repo.git
   
   # Also test the HTTPS URL for verification
   $ git clone https://github.com/org/private-repo.git

Example known_hosts Entry (generated after successful SSH connection):

github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndN...

#Using SSH URLs in Catalog Configuration

When configuring catalogs that require SSH authentication, provide both url and sshurl fields:

data:
  CATALOGS: |
    - name: private-catalog
      org: myorg
      type: community
      provider: github
      url: https://github.com/myorg/private-catalog        # Required for database
      sshurl: git@github.com:myorg/private-catalog.git     # Used for git operations
      revision: main

Note: Both url (HTTPS) and sshurl (SSH) are required for private repositories. The sshurl will be used for actual git operations.

#Verification and Testing

#Verify Catalog Configuration

After adding a custom catalog, verify it's properly configured:

Tips: The default Tekton Hub API endpoint is tekton-hub-api.tekton-pipelines:8000, which is only accessible within the cluster.

For the <your-tekton-hub-api> placeholder in the commands below:

• Inside the tekton-hub-api pod: use localhost:8000
• From other pods in the cluster: use tekton-hub-api.tekton-pipelines:8000
• From outside the cluster: create a NodePort service or Ingress (not covered in this guide)

For testing purposes, you can run these commands inside the tekton-hub-api pod using wget.

# Execute inside the tekton-hub-api pod for testing
$ kubectl exec -it deployment/tekton-hub-api -n <tekton-pipelines> -- /bin/sh

# Check if the catalog appears in the API
$ wget -qO- http://<your-tekton-hub-api>/v1/catalogs

{"data":[{"id":1,"name":"catalog","type":"community","url":"file:////mnt/git/catalog.git","provider":"github"},{"id":2,"name":"<new-catalog>","type":"private","url":"<url>","provider":"<provider>"}]}

# Check catalog resources
$ wget -qO- "http://<your-tekton-hub-api>/v1/query?catalogs=<my-custom-catalog>"

{"data":[{"id":1,"name":"<name>","catalog":{"id":1,"name":"<my-custom-catalog>","type":"community"},"categories":[{"id":9,"name":"<name>"}],"kind":"Task","hubURLPath":"","hubRawURLPath":""}]}

# Verify specific resource
$ wget -qO- http://<your-tekton-hub-api>/v1/resource/<my-custom-catalog>/task/<my-task>

{"data":{"id":4,"name":"<my-task>","catalog":{}}}

#Test Resource Resolution

Test that resources can be resolved via Hub resolver:

apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
  name: test-catalog-resolution
spec:
  taskRef:
    resolver: hub
    params:
    - name: catalog
      value: my-custom-catalog
    - name: type
      value: tekton
    - name: kind
      value: task
    - name: name
      value: test-task
    - name: version
      value: "0.1"

#Troubleshooting

#Catalog Not Appearing

Issue: Catalog configured but not showing in API

Solutions:

1. Check ConfigMap was applied: kubectl get cm tekton-hub-api -o yaml
2. Restart API pod: kubectl delete pod app=tekton-hub-api -n <tekton-pipelines>
3. Check API logs: kubectl logs deployment/tekton-hub-api -n <tekton-pipelines>

#Resources Not Loading

Issue: Catalog appears but resources are missing

Possible Causes:

• Repository structure doesn't follow standard
• Required metadata missing (see Creating a Custom Catalog)
• Git repository authentication issues

Solutions:

1. Check API logs for parsing errors
2. Validate resource YAML syntax with kubectl apply --dry-run

#SSH Authentication Failures

Issue: Private repository access denied

Solutions:

1. Verify secret exists: kubectl get secret tekton-hub-api-ssh-crds
2. Verify known_hosts includes your Git server
3. Test SSH connection manually from a pod

#Resource Resolution Failures

Issue: Hub resolver cannot find resources

Common Causes:

• Catalog name mismatch between ConfigMap and resolver parameters
• Resource name/version doesn't match directory structure
• Resource failed validation during catalog parsing

Debug Steps:

# Check available catalogs
$ wget -qO- http://<your-tekton-hub-api>/v1/catalogs

# Verify resource exists
$ wget -qO- http://<your-tekton-hub-api>/v1/resource/<my-custom-catalog>/task/<my-task>/<version>

# Check catalog sync status
$ kubectl logs deployment/tekton-hub-api -n <tekton-pipelines>

#FAQ

#Q: I configured according to the documentation, but still can't see the new catalog

A: First, check the tekton-hub-api logs for authentication-related errors such as "Host key verification failed":

$ kubectl logs deployment/tekton-hub-api -n <tekton-pipelines>

Common causes:

• SSH key lacks sufficient permissions for the repository
• known_hosts file missing the Git server's fingerprint
• SSH key not properly added to repository deploy keys

Solution:

1. Test locally first: Ensure you can successfully clone the repository locally using the same SSH credentials:

   # Test with the exact same SSH key and known_hosts
   $ git clone ssh://git@github.com:org/private-repo.git

2. Only after local clone succeeds, update your Kubernetes environment:

   • Refer to the Generating known_hosts Entries section to properly generate the known_hosts file
   • Recreate the tekton-hub-api-ssh-crds secret with the working credentials
   • Restart the tekton-hub-api pod and check the logs:

     $ kubectl delete pod -l app=tekton-hub-api -n <tekton-pipelines>
     $ kubectl logs deployment/tekton-hub-api -n <tekton-pipelines>