Troubleshooting
Controller is not ready
These commands use the Deployment name of the raw manifests. A Helm install names it <release>-kuvryn-sync, such as kuvryn-sync-kuvryn-sync for the release kuvryn-sync.
kubectl -n kuvryn-sync-system get pods
kubectl -n kuvryn-sync-system logs deployment/kuvryn-sync-controller-manager -c manager
kubectl -n kuvryn-sync-system describe deployment kuvryn-sync-controller-manager
Common causes:
- image pull secret missing for private registries;
- image architecture does not match cluster nodes;
- image from a different release than the chart or manifests, which exits on an unknown flag such as
--drift-resync-interval; - read-only filesystem without a writable
/tmpmount; - Pod evicted for ephemeral storage, when the source cache outgrows the node; see Source cache;
- RBAC denied for managed resources.
No traces arrive
Tracing is off unless OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_TRACES_ENDPOINT is set on the manager container; with Helm, set it through extraEnv. Kuvryn Sync exports over OTLP gRPC only, usually port 4317, not OTLP HTTP on 4318. A plain-text collector needs OTEL_EXPORTER_OTLP_INSECURE=true or an http:// endpoint. Export failures appear in the manager log as Failed to export traces. See Metrics and tracing.
Repository is Failed
kubectl describe repo <name> -n <namespace>
kubectl get repo <name> -n <namespace> -o yaml
Check:
- Git URL is reachable from the cluster;
- branch, tag, or commit exists;
- referenced Secret exists in the same namespace and is labelled
sync.kuvryn.io/git-credentials: "true"; - credentials are valid and allowed to read the repository;
- every configured
spec.applicationConfigPathsentry is repository-relative, unique, stays inside the repository, and is named.ksync.yaml; - discovered Application names are unique across all configured
.ksync.yamlfiles; - discovered Applications switch on only what
spec.applicationPolicyallows; the message names the field and the missing allowance, such asspec.applicationPolicy.allowAutomatic.
Application is Planning or AwaitingApproval
kubectl describe app <name> -n <namespace>
ksync history <name> -n <namespace>
ksync plan <name> -n <namespace>
For manual approval policies, approve the exact Revision. An ApprovalStale Event means the plan changed after approval; review ksync plan and approve again:
ksync sync <application> -n <namespace> --revision <revision-name>
Application is OutOfSync or Drifted
Check the latest plan:
ksync plan <application> -n <namespace>
If self-heal is disabled, Kuvryn Sync reports drift but does not mutate live objects. Enable spec.sync.selfHeal if automatic correction is intended.
Application is Degraded
ksync diagnose <application> -n <namespace>
kubectl describe app <application> -n <namespace>
kubectl get events -n <namespace> --sort-by=.lastTimestamp
The Application’s Ready condition is False with the reason of the last failure until a rollout completes Synced and Healthy again, when it turns True with reason Healthy. While a completed rollback holds, it is False with reason RolledBack until a new commit arrives; see Rollback for every way a hold ends.
ksync diagnose prints the root causes Kuvryn Sync recorded in status.diagnosis; see Reading a diagnosis. Health timeouts are controlled by spec.health.timeout and failure behavior by spec.strategy.failurePolicy.
Reading a diagnosis
Each cause in status.diagnosis names a root resource, a reason, a message, and a chain. Read the chain from the top: the first entry is the managed resource that is not Healthy, the last is the root cause, and the entries between are how one leads to the other, such as the ReplicaSet and Pod between a Deployment and a missing Secret. Fix the last entry; the others recover on their own.
| Reason | Root resource | What to check |
|---|---|---|
MissingSecret, MissingConfigMap | the missing object | Create it, or fix the name in the Pod template; mark the reference optional: true if it may be absent. |
MissingPersistentVolumeClaim | the missing claim | Create the claim or fix claimName. |
MissingServiceAccount | the missing account | Create the account or fix serviceAccountName. |
ImagePullBackOff, ErrImagePull | the Pod | The image name and tag, and the pull Secret; a missing pull Secret is reported as MissingSecret. |
CrashLoopBackOff | the Pod | The last exit code in the message, then kubectl logs --previous. |
CreateContainerConfigError | the Pod | A key missing from a ConfigMap or Secret that exists. |
Unschedulable | the Pod | Requests, node selectors, taints, and quotas named in the message. |
ClaimPending | the claim | The storage class and its provisioner. |
NoReadyEndpoints | the Service | Whether its selector matches ready Pods. |
JobFailed, OOMKilled, ContainerFailed | the Job or its Pod | The Job’s Pods and their logs. |
PodFailed, or the Pod’s reason such as Evicted | the Pod | The Pod’s status message: eviction, node pressure, or deadline. |
FailedCreate, ProgressDeadlineExceeded | the workload | The workload’s conditions: quotas, admission, or a rollout that stopped progressing. FailedCreate is the usual ReplicaFailure reason. |
Missing<Kind> for any other kind | the missing object | A referenced object, such as a Service behind an Ingress, that does not exist. |
A cause whose chain is only the managed resource itself means Kuvryn Sync found no deeper evidence; its reason is the resource’s health verdict, such as ReplicasUnavailable during an ordinary rollout. Kuvryn Sync then emits no Diagnosed Event unless the Application is Degraded. When a list failed, such a message ends with what was not visible, such as not visible: could not list Pods: forbidden: the evidence may be there, but the service account may not read it.
Kuvryn Sync reads the objects below managed resources as the Application’s service account. If the account may not list Pods or read Secrets, the diagnosis stops higher up the chain, and a reference it could not check is never reported as missing; see Diagnosis permissions. ksync graph <application> shows the same graph with your own credentials.
ServiceAccountRequired or Forbidden
ServiceAccountRequired means the Application sets no spec.serviceAccountName and the manager has no --default-service-account. Set one of them.
Forbidden means the Application’s service account may not read, apply, or delete a resource. The failure message names the verb and resource. Check what the account may do:
kubectl auth can-i --list -n <destination-namespace> \
--as system:serviceaccount:<application-namespace>:<service-account>
Grant the missing permission, then push a new commit or switch the Application to a service account that has it; retry limits otherwise keep the failed Revision blocked. A PruneInventoryIncomplete Warning Event names kinds the account may not list, whose managed objects Kuvryn Sync cannot prune.
Server-Side Apply conflict
Kuvryn Sync fails conflicts by default. Inspect the failing field manager with:
kubectl get <kind> <name> -n <namespace> -o yaml --show-managed-fields
Resolve ownership intentionally: update the external manager, move the field out of Kuvryn Sync’s desired state, recreate the resource under a clear owner, or, when Kuvryn Sync should take over (for example while migrating), set spec.sync.conflictPolicy: adopt and review the takeover in the plan.
Helm or Kustomize render failure
Check that the desired-state repository contains the expected path and renderer inputs:
render.type: yamlexpects Kubernetes YAML files under the path.render.type: kustomizeexpects a Kustomize root. Bases and resources must be in the same repository; a kustomization naming a URL or Git remote is refused with the reference in the error.render.type: helmexpects a chart and optional values files inside the repository. Chart dependencies must be vendored intocharts/(runhelm dependency buildand commit the result)..Release.Namespaceis the Application’s destination namespace.- A checkout fails if the commit contains a symlink, or chain of symlinks, that resolves outside it, or an entry named
.kuvryn-sync-checkout. render.helm.chart.versionmust be an exact version; ranges are refused.- Without a
revision, the remote must advertise a default branch (HEAD); otherwise set a revision.
Use ksync diagnose and controller logs for the deterministic failure reason.