#Retention Policy Configuration

Tekton Results retention policy provides automated cleanup for both database records and archived logs in S3-compatible storage systems. The retention agent periodically removes expired results and their associated log payloads according to configurable time-based policies.

#Configuration

Retention policies are configured through the TektonConfig custom resource using the spec.result.options field to specify ConfigMap overrides:

apiVersion: operator.tekton.dev/v1alpha1
kind: TektonConfig
metadata:
  name: config
spec:
  result:
    logs_api: true
    logs_type: S3
    secret_name: my-s3-secret
    is_external_db: true
    db_host: your-postgres-host.example.com
    db_port: 5432
    db_name: tekton_results
    db_sslmode: require
    db_secret_name: tekton-results-postgres
    options:
      configMaps:
        tekton-results-config-results-retention-policy:
          data:
            defaultRetention: "30"  # Days to retain results (30 days)
            runAt: "0 2 * * *"      # Cron schedule: daily at 2 AM

#Configuration Parameters

• defaultRetention: Specifies the number of days to retain results before automatic deletion. After this period elapses, both the database records and associated log payloads in S3 storage will be cleaned up.

• runAt: Cron expression defining when the retention agent runs.

  • "0 2 * * *" - Daily at 2:00 AM
  • "0 2 * * 0" - Weekly on Sundays at 2:00 AM
  • "0 2 1 * *" - Monthly on the 1st at 2:00 AM

#Apply Configuration

Apply the TektonConfig configuration:

kubectl apply -f tekton-config-retention-config.yaml

Example Output:

tektonconfig.operator.tekton.dev/config configured

Wait for the retention agent to restart:

kubectl rollout status -n tekton-pipelines deployment/tekton-results-retention-policy-agent

Example Output:

deployment "tekton-results-retention-policy-agent" successfully rolled out

Verify the configuration is active:

kubectl get pods -n tekton-pipelines -l app.kubernetes.io/name=tekton-results-retention-policy-agent

Example Output:

NAME                                                   READY   STATUS    RESTARTS   AGE
tekton-results-retention-policy-agent-5c9g2            1/1     Running   0          5m

#Important Note

Important: The unified cleanup functionality (deleting both database records and S3 log payloads together) is a enhancement (patch-based) and is not currently available in community Tekton Results. The community version only removes database records but leaves log payloads in S3 storage, which can lead to unbounded storage growth.

Check TektonConfig retention configuration:

kubectl get tektonconfig config -o yaml

Example Output (abbreviated):

apiVersion: operator.tekton.dev/v1alpha1
kind: TektonConfig
metadata:
  name: config
spec:
  result:
    logs_api: true
    logs_type: S3
    secret_name: my-s3-secret
    is_external_db: true
    db_host: your-postgres-host.example.com
    options:
      configMaps:
        tekton-results-config-results-retention-policy:
          data:
            defaultRetention: "30"
            runAt: "0 2 * * *"

Check retention agent logs:

kubectl logs -n tekton-pipelines deployment/tekton-results-retention-policy-agent

Example Output:

{"level":"info","ts":"2023-05-15T10:30:45.123Z","caller":"retention/main.go:89","msg":"Starting Tekton Results retention policy agent"}
{"level":"info","ts":"2023-05-15T10:30:45.124Z","caller":"retention/agent.go:45","msg":"Retention policy configured: 30 days, schedule: 0 2 * * *"}
{"level":"info","ts":"2023-05-15T10:30:45.125Z","caller":"retention/scheduler.go:123","msg":"Next execution scheduled for: 2023-05-16T02:00:00Z"}

#Verification

To quickly verify that the retention policy is functioning as expected, follow these steps:

1. Create test results/logs: Generate some Tekton PipelineRuns or TaskRuns that will produce results and logs in your Tekton Results database.

   # Create a sample PipelineRun to generate test data
   kubectl apply -f sample-pipelinerun.yaml

2. Verify initial state: Check that the results and logs are present in the database and S3 storage (if applicable).

   # Check Tekton Results
   tkn results list
   
   # Or query directly from the database if accessible
   kubectl exec -it -n tekton-pipelines deployment/tekton-results-read-api -- grpcurl -plaintext -d '{}' localhost:8080 api.v1alpha2.Results.ListResults

3. Modify retention policy for testing: Temporarily set a very short retention period and a specific cron schedule for testing purposes (e.g., execute once at current time + 10 minutes).

   options:
     configMaps:
       tekton-results-config-results-retention-policy:
         data:
           defaultRetention: "0"  # Retain for 0 days (immediate cleanup)
           # Execute once at a specific time (current time + 10 minutes)
           # Example: If current time is 14:30, set to "40 14 * * *" for 14:40
           # Modify the minute value to current minute + 10 (with day/hour adjusted if needed)
           runAt: "40 14 * * *"  # Example: run at 14:40 (current time + 10 mins approx)

4. Apply modified configuration: Apply the temporary test configuration and wait for the retention agent to restart.

   kubectl apply -f tekton-config-test-config.yaml
   kubectl rollout restart -n tekton-pipelines deployment/tekton-results-retention-policy-agent

5. Monitor retention agent logs: Watch the retention agent logs to confirm cleanup operations are occurring.

   kubectl logs -n tekton-pipelines deployment/tekton-results-retention-policy-agent -f

6. Verify cleanup occurred: After the retention schedule executes, verify that the older results and logs have been removed.

   # Check if results have been cleaned up
   tkn results list
   
   # Also check S3 storage to verify log files have been cleaned up
   # Use your S3 client to list objects in the bucket
   aws s3 ls s3://your-bucket-name/path/to/tekton-logs/ --recursive

7. Revert to production settings: Once verified, revert the retention policy to your intended production values.

Warning: Be careful when setting very short retention periods as this will delete results immediately. Ensure you are using a test environment for verification.