Troubleshooting

The following problems can occur while a Kiali server observes a multi-cluster mesh. The cluster names and environment variables are the ones used in Installing Kiali in a multi-cluster mesh.

The remote cluster is reported as accessible, but its namespaces are missing from the console

The Kiali server log shows the following entries:

INF Discovered cluster: Name=[cluster2], Accessible=true, IsKialiHome=false, ...
INF Error fetching Namespaces for cluster [cluster2]: Unauthorized
WRN Error checking Istio API configuration: the server has asked for the client to provide credentials

The remote cluster secret addresses the cluster API server directly, so the service account token of Kiali works but the platform token of the logged-in user is rejected. Change the server field of the kubeconfig in the secret to the platform endpoint, <platform-url>/kubernetes/<cluster-name>, together with the matching certificate authority, and trigger a reconciliation. See step 4 of the installation procedure.

Generating the remote cluster secret never completes

The script repeats the following message and then fails:

INFO: Waiting for the remote cluster SA secret token to be created...
ERROR: There is no token assigned yet to the remote cluster SA secret [kiali-service-account] found in remote cluster namespace [istio-system].

The long-lived service account token secret is missing on the remote cluster. Complete step 3 of the installation procedure, then run the script again.

The Kiali server pod stays in ContainerCreating after a cluster was removed

The pod events report a missing secret:

MountVolume.SetUp failed for volume "cluster2" : secret "kiali-remote-cluster-secret-cluster2" not found

The deployment still mounts the remote cluster secret that was deleted, and reconciliation does not drop that volume. Recreate the deployment as described in step 4 of Removing a cluster from Kiali.

Graphs are empty although the clusters are accessible

Confirm that the metrics of all clusters carry the mesh_id label that spec.external_services.prometheus.query_scope filters on:

curl -sS -u "<monitoring-user>:<monitoring-password>" \
  --data-urlencode 'query=count(istio_requests_total{mesh_id="mesh1"}) by (source_cluster, destination_cluster)' \
  "<monitoring-address>/api/v1/query"

The result must contain one series per cluster pair that exchanges traffic. If a cluster is missing, its metrics either never reach the store or carry no matching mesh_id label: check that the cluster has the ServiceMonitor and PodMonitor objects, and that the control plane which configures it has the Telemetry resource whose mesh_id tag override matches the meshID of the Istio resource. See Configuring Monitoring with Service Mesh.