[title] Onboarding [path] Getting Started/ # **Overview** After connecting you first cluster, the product onboarding guide will trigger and walk you through the main features of Kubegrade. **** # **Dashboard Overview** Here you will be able to see and organise your cluster groups, clusters, environments and related on the highest level. ![](https://archbee-image-uploads.s3.amazonaws.com/7Lzjl4NBI_sc1o8F1zL-G/nf8GYzqexcjYpJ0YvPrnJ_image.png) ![](https://archbee-image-uploads.s3.amazonaws.com/7Lzjl4NBI_sc1o8F1zL-G/FQsDMhe0H6yH1TGTi6aff_image.png) # **Cluster Table View** See a list of all clusters in a Cluster group with high level metrics. ![](https://archbee-image-uploads.s3.amazonaws.com/7Lzjl4NBI_sc1o8F1zL-G/UPkRO2oARVEKGF8NnkHla_image.png) # **Cluster Info View** This view will give you a detailed view and information about a specific cluster. ![](https://archbee-image-uploads.s3.amazonaws.com/7Lzjl4NBI_sc1o8F1zL-G/REitTqyGXnjz6UJU6MzVO_image.png) # **Visualization** See here your clusters visualized with dependencies. From there, trigger various agents and workflows for specific namespaces and objects in them. ![](https://archbee-image-uploads.s3.amazonaws.com/7Lzjl4NBI_sc1o8F1zL-G/wtjVLR7NrIV3QpgP4rqO4_image.png) # **AI Agent Side Panel** Chat with the Assistant and other agents to analyze, diagnose, or troubleshoot your selection. ![](https://archbee-image-uploads.s3.amazonaws.com/7Lzjl4NBI_sc1o8F1zL-G/7zDoGTzrKLkXNHTDAWLV7_image.png) ![](https://archbee-image-uploads.s3.amazonaws.com/7Lzjl4NBI_sc1o8F1zL-G/xsnoM_AOlEDwL5jX2J8Ch_image.png) [title] Getting Started [path] / [title] Upgrade Management [path] Product Modules/ # **Overview** Kubegrade Upgrade Management helps teams reduce manual effort and upgrade risk by turning Kubernetes upgrades into a guided, auditable workflow. ### Core outcomes - Faster upgrade planning - Preflight validation before changes - Safer execution with approvals - PR-based, GitOps-compatible remediation where config changes are required - Better auditability and rollback readiness # **Supported Kubernetes versions/providers** ## **Providers** - Amazon / AWS / EKS - GoogleCloud / GKE - Microsoft Azure / AKS - OpenShift - DigitalOcean - AlibabaCloud - Rancher/RKE (if supported) - Self-managed upstream Kubernetes ## **Version support format (recommended)** ### Provide - Minimum supported version per provider - Tested upgrade paths (N → N+1, etc.) - Unsupported/skipped paths - Provider-specific notes (managed add-ons, control plane sequencing, nodegroup specifics) # **Preflight checks (API deprecations, add-ons, constraints)** Preflight checks are run before upgrade execution to identify blockers and risk. ## **Typical checks** - Kubernetes API deprecations in manifests - Add-on compatibility (CNI, CSI, ingress controllers, metrics stack) - Version skew constraints - Pod disruption budget / capacity constraints - Node image/runtime prerequisites - Admission controller and policy conflicts - Deprecated Helm chart APIs / CRD compatibility - Cluster health baseline (unhealthy nodes, failing workloads) ## **Output** ### Preflight results should be categorized as - Blockers (must fix) - Warnings (recommended fixes) - Informational notes # **Upgrade plans (single / multi-cluster)** ## **Single-cluster upgrade plan** ### Includes - Cluster target version - Preflight status - Required remediations - Maintenance window - Rollback strategy - Approval requirements ## **Multi-cluster upgrade plan** ### Adds - Sequencing (development -> canary → staging → production) - Cluster grouping by environment/provider/team - Parallelism constraints - Success gates between waves - Standardized runbook application ## **Recommended pattern** 1. Development cluster 2. Canary cluster 3. Staging cluster(s) 4. Limited production wave 5. Broad production rollout # **Scheduling + maintenance windows** Kubegrade supports scheduling upgrades and workflows within controlled windows via the agents. ## **Scheduling capabilities** - One-time scheduled execution - Recurring maintenance windows - Timezone-aware windows - Environment-specific restrictions - Freeze windows / blackout periods ## **Operational controls** - Preflight refresh before execution - Re-approval if conditions changed - Auto-cancel on critical health regressions (if configured) # **What gets written where (PR contents, provenance)** ## **Typical PR contents** - Manifest or values updates - Version pin changes - Compatibility fixes - Config changes required by target version - Explanatory summary of why changes are required - References to preflight findings ## **Provenance metadata (recommended)** - Workflow run ID - Cluster / environment - Policy checks passed/failed - Agent/version used - Timestamp - Approver(s) (linked in audit trail) # **Runbooks: common upgrade failure modes** ## **1) Node upgrade stalls** ### Possible causes - Pod disruption budgets too strict - Capacity shortage - DaemonSet rollout blocking drain - Misconfigured eviction settings ### Response - Review draining events - Temporarily scale capacity / relax constraints (approved) - Retry wave ## **2) Workloads fail after upgrade** ### Possible causes - API deprecations missed - Admission/policy incompatibility - Ingress/controller changes - CRD/version mismatch ### Response - Review failing workloads and dependency graph impact - Generate remediation PR - Roll back workload config or cluster wave if needed ## **3) Add-on incompatibility** ### Possible causes - Unsupported add-on version for target K8s version - CSI/CNI version skew - Metrics/monitoring stack incompatibility ### Response - Pin compatible add-on version - Upgrade add-ons in required order - Re-run preflight ## **4) Control plane/node version skew issues** ### Possible causes - Validate provider sequencing rules - Reconcile managed nodegroups - Apply supported path only ### Response - Validate provider sequencing rules - Reconcile managed nodegroups - Apply supported path only [title] Product Modules [path] / [title] Alert Sorting (coming soon) [path] Product Modules/ # **Overview** Alert Sorting will help teams reduce alert noise by grouping related events, surfacing high-priority issues, and connecting alerts to Kubegrade workflows for faster action. This module is **coming soon** and details may evolve. ![](https://archbee-image-uploads.s3.amazonaws.com/7Lzjl4NBI_sc1o8F1zL-G/aXudfpMBicwYmT0dtbjlN_image.png) ![](https://archbee-image-uploads.s3.amazonaws.com/7Lzjl4NBI_sc1o8F1zL-G/KeIrk9kMm5C_vMfABfLBi_image.png) ## **Ingest sources (Prometheus/Alertmanager etc.)** Planned/common sources: - Prometheus Alertmanager - Grafana alerts - Datadog monitors - PagerDuty incidents/events - Webhook-based alert feeds - Additional observability tools (as integrations expand) ## **Dedup, grouping, suppression** Planned capabilities: - Deduplicate repeated alerts - Group by service/namespace/cluster/root-cause candidate - Suppress known-noisy or low-signal alerts - Correlate alerts with active incidents and graph state ## **Routing rules (team/service/cluster)** Planned routing based on: - Team ownership - Cluster/environment - Namespace/service labels - Severity - Time windows/on-call schedules (via integrations) ## **Suggested actions + links into workflows** Planned actions: - Open troubleshooting investigation with alert context - Jump to impacted dependency graph - Trigger drift scan on affected namespace - Draft remediation PR (where applicable) - Create/update Jira/PagerDuty ticket ## **Noise tuning playbook** Recommended future playbook structure: 1. Identify top noisy alert classes 2. Define ownership labels and routing metadata 3. Add dedup/grouping rules 4. Add suppressions with expiry 5. Track MTTA/MTTR and false-positive reduction 6. Review monthly [title] GitOps Remediation [path] Product Modules/ # **Overview** GitOps Remediation is Kubegrade’s workflow for converting operational findings (drift, incidents, upgrade blockers, optimization recommendations) into reviewable pull requests and applying changes through your existing GitOps system. # **Supported repo layouts** ## **Common layouts** - Monorepo (all environments/services) - Environment-per-folder - Service-per-folder - Repo-per-service - Platform repo + app repos split - Helm values repos - Terraform module + environment composition repos ## **Best practice** Require users to define: - Repo path mappings per cluster/environment - File ownership / CODEOWNERS expectations - Branch strategy # **PR generation rules (naming, commit strategy, reviewers)** ## **Branch naming (recommended)** :::BlockQuote kubegrade/\/\/\ ::: Examples: :::BlockQuote kubegrade/drift/prod-eu1/restore-nginx-limits ::: :::BlockQuote kubegrade/upgrade/staging/1-29-preflight-fixes ::: ## **Commit strategy** - One commit per logical remediation (preferred) - Squash option for noisy generated changes - Signed commits if required by org policy ## **Reviewers** - Auto-assign based on CODEOWNERS - Add platform approvers for prod scopes - Tag relevant service owners for workload-level changes # **Policy checks before PR** Before opening a PR, Kubegrade validates: - Scope and permissions - Policy compliance - Environment restrictions - Change-type restrictions - Required approvals - Optional dry-run validation/tests (if integrated) If checks fail: - Open recommendation without PR - Show blocking reasons and remediation steps # **Merge-to-apply mechanics** ## **Typical flow** 1. PR merged in Git provider 2. GitOps tool (Argo CD / Flux) detects commit 3. GitOps sync applies changes to cluster 4. Kubegrade watches rollout status / post-change signals 5. Audit trail links PR → sync → outcome ## **Recommended controls** - Sync windows - Manual sync for production - Health checks before marking complete # **Rollback PRs** ## **Rollback approaches** - Revert generated PR commit(s) - Generate rollback PR from known-good state - Partial rollback for specific resources (advanced / policy-controlled) ## **Rollback triggers (examples)** - Failed post-change health checks - Elevated error rates - Dependency graph degradation - SLO breach detected via integrations # **Auditability: change provenance** Every GitOps remediation should be traceable. ## **Provenance fields (recommended)** - Workflow source (drift/troubleshoot/upgrade/etc.) - Cluster/environment/namespace - Agent/workflow version - Trigger type (manual/scheduled/alert-driven) - Approvers and timestamps - PR link and commit SHA - GitOps sync result - Verification result [title] AI Agents [path] Product Modules/ # **Overview (agents are “goal-oriented workflows”)** Kubegrade AI Agents are goal-oriented operational workflows designed to help teams complete specific Kubernetes tasks with strong context and human control. # **Built-in agents** ## **Assistant Agent** Purpose: - Serve as an intelligent high level cluster assistant for users - Provide explanations of cluster metrics, components, and operational states - Answer technical questions related to configurations, upgrades, and troubleshooting processes - Assist users by contextualizing issues, linking relevant documentation, and suggesting next actions - Support other agents’ workflows by offering guidance and clarification during multi-agent operations ## **Troubleshooting Agent** Purpose: - Investigate symptoms across multiple signal layers - Identify probable causes - Suggest and optionally prepare remediations ## **Upgrade Agent** Purpose: - Analyze target version readiness - Run preflight checks - Propose upgrade sequencing and remediation steps - Create executable and specific upgrade plans - Generate data necessary to generate PRs for required config changes ## **PR Generation Agent** Purpose: - Convert approved remediation plans into repo-specific pull requests - Preserve format/structure conventions - Attach rationale and provenance metadata # **Custom agents** Custom agents let teams define workflows that combine Kubegrade context with external tools via MCP Servers. ## **Typical custom agent inputs** - Cluster metadata and object state - Dependency graph context - IaC source context (Terraform/Helm/Kustomize) - Git/GitOps state - External tools via connectors/MCPs (e.g., Terraform/ArgoCD/etc.) ## **Example custom use cases** - Policy enforcement workflows - Scheduled drift checks + PR creation - Environment readiness checks before releases - Cost/risk posture reviews across fleet # **Prompting patterns + guardrails** ## **Recommended prompting patterns** - Goal + scope + constraints (cluster/env/namespace) - Desired action mode (analyze-only / propose / PR-ready) - Risk tolerance and change restrictions - Output format requirements (summary, checklist, PR body) ## **Guardrails** - Read-only by default for new workflows - Policy checks before proposing or executing - Explicit approval for PR generation/execution - Scope limits (namespace/workspace/project) - Logging/audit of agent actions and outputs # **Approval + execution controls** Kubegrade keeps AI-assisted workflows under operator control. ## **Controls to document** - Approval requirements by severity/environment - Who can run agents vs who can approve results - Suggest-only mode vs PR generation mode - Execution via GitOps only (recommended default for prod) - Time-window restrictions # **Feedback loop ** Kubegrade can improve agent usefulness over time through operator feedback. ## **Feedback signals (recommended)** - Accepted vs rejected suggestions - Edits made before PR generation - Rollback frequency - Time to resolution after applying recommendation - “Incorrect root cause” or “missing context” tags ## **Why it matters** - Better remediation quality - Less noisy suggestions - More consistent outputs across teams [title] Troubleshooting [path] Product Modules/ # **Overview** Kubegrade Troubleshooting turns Kubernetes troubleshooting from a reactive, manual process into a guided, context-rich workflow with AI, automation and human control. # **Guided investigations (what Kubegrade inspects)** Kubegrade agents investigate issues using correlated signals across Kubernetes layers. ## **Common inspection domains** - Pod and container status - Events and recent changes - Resource pressure (CPU/memory) - Scheduling constraints - Service/endpoints/ingress routing - DNS/network indicators - ConfigMaps/Secrets references and misconfigurations - Dependency relationships across workloads/services # **Common incident playbooks ** ## **CrashLoopBackOff** Kubegrade checks: - Container logs - Exit codes - Liveness/readiness probe failures - Missing env/config/secret references - Recent config changes Typical outputs: - Root-cause candidates - Impacted dependent services - Proposed config or probe adjustments via PR ## **OOMKilled / resource exhaustion** Kubegrade checks: - Memory limits/requests - Actual usage patterns - Recent release changes - Node pressure and eviction signals Typical outputs: - Right-size recommendations - HPA/resource tuning suggestions - PR-ready config updates (if enabled) ## **5xx / ingress/service routing issues** Kubegrade checks: - Ingress/controller status - Service selectors/endpoints - Pod readiness - Upstream dependency health - DNS/network path issues ## **DNS / network connectivity issues** Kubegrade checks: - CoreDNS status - Service DNS resolution - NetworkPolicy constraints - Endpoint availability - Namespace/service name mismatches ## **Scheduling failures** Kubegrade checks: - Resource requests vs node capacity - Taints/tolerations - Node selectors / affinity - PDB interactions ## **Evidence collection + timelines** ### **Evidence types** - Events - Status transitions - Metrics snapshots - Dependency graph changes - Config diffs / Git changes - Alert triggers (if integrated) ### **Timelines** A useful incident timeline should show: - First observed symptom - Related alerts and state transitions - Changes preceding the incident - Actions taken (manual or Kubegrade-assisted) - Recovery confirmation ## **Suggested remediations → PR workflow** Kubegrade can convert troubleshooting findings into reviewable changes. ### Flow: 1. Investigation identifies root-cause candidates 2. Kubegrade proposes remediation options 3. User selects/edits preferred option 4. Kubegrade generates PR against Git/IaC source 5. User reviews and merges 6. GitOps applies 7. Kubegrade verifies recovery signals ## **Post-incident review / export** After resolution, the Kubegrade Assistant agent can support review and reporting. ### **Post-incident review contents** - Incident summary - Affected clusters/namespaces/services - Timeline of events and changes - Root cause (confirmed/probable) - Actions taken - PR links / approvals - Prevention follow-ups [title] Intelligent Dependency Graphs [path] Product Modules/ # **Overview** Kubegrade provides deep cluster visualization and dependency mapping to help teams understand relationships, blast radius, and warning states quickly. ![](https://archbee-image-uploads.s3.amazonaws.com/7Lzjl4NBI_sc1o8F1zL-G/sfJ59I3Q-dhp--SKz0ZJK-20260430-064853.gif) # **Object-level graph model** Kubegrade’s graph models relationships such as: - Namespace → workload (Deployment/StatefulSet/DaemonSet) - Workload → pods - Service → endpoints/pods - Ingress → services - ConfigMap/Secret references - NetworkPolicy relationships - Storage dependencies (PVC/PV) - Optional external dependencies (via integrations/annotations) Purpose: - Faster root cause analysis - Change impact assessment - Upgrade dependency planning # **Filters and scopes** ## **Filters** - Cluster / environment - Object kind - Cluster wide objects - Namespace ## **Scopes** Users should be able to graph: - Full cluster - Namespace only - Incident-specific objects - Nested objects # **Warning states + “what changed” views** ## **Warning states** Overlay graph nodes/edges with states such as: - Degraded - Pending / unschedulable - Failed probes - Policy violation - Drift detected - Upgrade blocker ## **“What changed” views** Compare graph snapshots over time to identify: - New/removed dependencies - Routing changes - Config-linked impact - Service endpoint shifts This is especially useful for post-incident reviews and upgrade regressions. # **Sharing and exporting views** ## **Sharing** - Link-based internal sharing (RBAC-respected) - Saved workspace/project views - Embed in incident tickets/docs ## **Exporting** - Image export (PNG/SVG) - Data export (JSON graph metadata) - Snapshot attachment to incident reports [title] Security overview [path] Security & Infrastructure/ # Overview Kubegrade is designed from the ground up to ensure that enterprise‑grade security, compliance, and scalability are built into every layer of our platform with strong emphasis on: - Least-privilege access - Human-in-the-loop approvals - GitOps-based change execution - Auditability - Support for private/on-prem/hybrid deployments This document outlines Kubegrade’s infrastructure design, security practices, and compliance posture. For deployment‑specific architecture details, refer to our Architecture section. :::CtaButton{label="Architecture" docId="XSz8TnDLVBYQPXN_vRoFn" openInNewTab="false" noFollow="false"} ::: # How we handle your data to achieve Enterprise-Grade Security ## Control & Data Plane separation Kubegrade follows a strict separation of concerns between the Control Plane and the Data Plane: - **Control Plane (Kubegrade‑managed)**: Handles authentication, user management, licensing, configuration, scheduling, agent orchestration, and alerting. - **Data Plane (deployment‑dependent)**: Connects to your Kubernetes clusters and infrastructure, executes discovery and upgrade checks, and stores operational metadata, either in a shared or isolated environment depending on the deployment model. Details for each deployment model are available in: - Multi‑tenant SaaS - Single‑tenant SaaS - Bring‑your‑own‑Cloud (BYOC) / private deployment ## Collection - Kubegrade’s connectors and agents only access cluster metadata, configuration state, logs, and upgrade‑related signals required to assess the health and posture of your environments. - Data collected is used solely to power recommendations, upgrade planning, and incident visibility that you have explicitly enabled. - All communication between Kubegrade and your infrastructure uses encrypted connections (HTTPS/TLS). - For enterprise customers, Kubegrade supports running agents and MCP‑connected services entirely within your own cloud or network perimeter, so sensitive systems do not need to be directly exposed to Kubegrade’s control plane. ## Compliance - Kubegrade can support enterprise security and compliance requirements such as SOC 2 and ISO 27001, and we are able to share details of our current certifications and roadmap under NDA. - Kubegrade will sign NDAs and/or DPAs where appropriate. - While Kubegrade collects operational metadata and logs to drive upgrade planning and monitoring, we acknowledge that personal data may be processed in some environments. Where this occurs, it is used only for the purpose of delivering Kubegrade’s Kubernetes operations capabilities. - Internal use of SaaS applications at Kubegrade is vetted to ensure confidential company and customer information is handled in line with our security and privacy policies. ## Organizational security and privacy practices Kubegrade’s team follows industry best practices to protect the platform and our customers’ data. - Kubegrade engages independent third parties to perform regular penetration tests of the platform. - Processing of collected data is conducted on secure infrastructure hosted with leading cloud providers (for example, AWS), configured according to security best practices. - Kubegrade employees receive privacy and security training during onboarding and on a recurring basis, and must acknowledge our security policies. - Access to production systems is protected with strong authentication (including multi‑factor authentication and SSO where applicable), based on least‑privilege access and periodic access review. ## Data collected by Kubegrade The following information may be processed and stored by Kubegrade: | Collected data | Details | Purpose of collection | | ------------------------ | ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------- | | Cluster metadata | Cluster names, Kubernetes versions, node group identifiers, labels | To inventory clusters and node groups and support version‑aware analysis and recommendations. | | Configuration metadata | Manifests, Helm values, add‑on and controller versions, upgrade history | To analyse configuration, detect drift, validate upgrade paths, and generate remediation plans/PRs. | | Operational metrics | Status of workloads, node health, upgrade status, high‑level performance | To track upgrade progress, detect common failure modes, and surface SLO‑relevant signals. | | Aggregated statistics | Aggregated counts, error rates, rollout outcomes | To improve recommendations, run safety checks, and support reporting on reliability over time. | | Logs / events (optional) | Selected logs and events relevant to upgrades and troubleshooting | To enable diagnostics, runbooks, and AI‑assisted incident analysis (opt‑in and scope‑controlled). | | User actions & PR data | Upgrade plans, approvals, comments, generated pull requests | To provide audit trails, provenance for AI‑generated changes, and workflow history. | | Integration credentials | Tokens and keys for Git, cloud, and Kubernetes integrations | To securely connect Kubegrade to your repositories and clusters, using least‑privilege access. | ## Data access & encryption All connections from Kubegrade to your clusters, clouds, and Git providers use least‑privilege, read‑only or narrowly scoped credentials wherever possible, and all traffic is encrypted with TLS. Kubegrade supports multiple layers of encryption: - **Content‑level encryption** for sensitive data such as credentials and secrets. - **At‑rest encryption** for all data stored within Kubegrade’s services. Encryption details - **Content encryption** Sensitive fields (for example, integration credentials and secret material) are encrypted using strong, industry‑standard algorithms (such as AES‑256‑GCM) before storage. - **At‑rest encryption** All data stored in databases and object storage is encrypted at rest using cloud‑provider‑managed encryption, adding a second layer of protection in case underlying storage is accessed. - **Retention** Kubegrade retains operational data only as long as necessary to provide the service (for example, while a cluster or integration is configured in Kubegrade) or as required by contractual agreements and applicable law. Certain transient artefacts (like detailed logs) may have shorter retention periods configurable for enterprise customers. # Scalability & reliability ## Scalable engine / worker pool Kubegrade’s analysis and automation engine scales elastically with workload, provisioning additional workers under load (for example, during large fleet scans or multi‑cluster upgrades) and reducing capacity during idle periods. This distributed architecture provides resilience against individual node failures. ## High‑performance and reliable scheduling Kubegrade’s job scheduling and orchestration layer is designed for safe, reliable execution of cluster‑level tasks: - Ensures that upgrade and analysis tasks are scheduled without duplication and with clear idempotency guarantees. - Retries failed work where appropriate to maintain coverage of critical workflows while respecting rate limits and provider‑specific constraints. [title] Security & Infrastructure [path] / [title] Data handling & what leaves the cluster [path] Security & Infrastructure/ # Overview Kubegrade uses a **minimum necessary data** model, with deployment-mode differences. # **Typical data used by Kubegrade** - Kubernetes object metadata and configuration state - Events / status conditions - Health and dependency context - Workflow execution metadata - Optional integration metadata (Git/IaC references, alert identifiers) # **What may leave the cluster (SaaS / hybrid)** Depending on configuration: - Telemetry metadata - Resource state summaries - Policy/drift findings - Workflow request/response metadata - Optional logs/diagnostic snippets (if enabled) # **What should be explicitly documented by deployment mode** - Whether payloads include object specs vs metadata only - Whether secrets are collected (recommended: no secret values) - Retention periods - Encryption in transit / at rest - Regional hosting options (if any) [title] Untitled [path] / [title] What can I do with Kubegrade? [path] Use Cases/ # Overview At its core, Kubegrade connects to your clusters, understands what is running, and then surfaces concrete recommendations around upgrades, reliability, performance, and security. The following use cases are showcasing how to apply Kubegrade in real scenarios, such as preparing for a cluster upgrade, investigating incidents, optimising resource usage, and automating repetitive operational tasks. To see all the videos, go to -> [https://www.youtube.com/@Kubegrade](https://www.youtube.com/@Kubegrade) # Use Cases ::::LinkArray{contentSource="DOCS" sourceParentDocId="vrqL9AkX6qSR3YpIdx6vk" childrenDocs="[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]"} :::LinkArrayItem{headerType="NONE" docId="pvhOZDcQBPEAAd-TnQdxQ"} [What can I do with Kubegrade?](docId\:pvhOZDcQBPEAAd-TnQdxQ) ::: :::LinkArrayItem{headerType="NONE" docId="az_106kULjRXT1Djk2nuA"} [Fixing a 502 bad gateway on kubernetes cluster namespace using Kubegrade](docId\:az_106kULjRXT1Djk2nuA) ::: :::LinkArrayItem{headerType="NONE" docId="SDJ3N46B0xKq44PUwG30p"} [Fixing Kubernetes CrashLoopBackOff using Kubegrade (Automated Root Cause & PR Fix) (cloned)](docId\:SDJ3N46B0xKq44PUwG30p) ::: :::LinkArrayItem{headerType="NONE" docId="OCGKb0YjBj4Goz_VbcaGB"} [Fixing ImagePullBackOff in Kubernetes using Kubegrade (Automated Root Cause & PR Fix)](docId\:OCGKb0YjBj4Goz_VbcaGB) ::: :::LinkArrayItem{headerType="NONE" docId="N0_s13SbpW1w4y4unj28h"} [Fixing Kubernetes OOMKilled Errors using Kubegrade (Out of Memory CrashLoop Fix)](docId\:N0_s13SbpW1w4y4unj28h) ::: :::LinkArrayItem{headerType="NONE" docId="lEblxNHzEYXrN_v2k3fVl"} [Fix Kubernetes Pending Pods Fast (CPU Limits Error) using Kubegrade](docId\:lEblxNHzEYXrN_v2k3fVl) ::: :::LinkArrayItem{headerType="NONE" docId="AgA8bZTsOC5685k1ABw4P"} [Fix Kubernetes CPU Throttling using Kubegrade (Performance Issue Troubleshooting)](docId\:AgA8bZTsOC5685k1ABw4P) ::: :::LinkArrayItem{headerType="NONE" docId="pWuK1JO06yq_B45yYrSh2"} [Fix Kubernetes Pod Restarting Issues using Kubegrade (Auto Root Cause + PR Fix)](docId\:pWuK1JO06yq_B45yYrSh2) ::: :::: [title] Fixing a 502 bad gateway on kubernetes cluster namespace using Kubegrade [path] Use Cases/ ::embed[]{url="https://www.youtube.com/embed/_3mO1gPeS7U"} # Overview This short demo shows how Kubegrade troubleshoots and fixes a real Kubernetes production issue end-to-end. Kubegrade automates Kubernetes troubleshooting by detecting issues, explaining root causes, and fixing them through GitOps pull requests. # Scenario An application returns a 502 Bad Gateway due to a misconfigured container port. # What happens in the demo - Navigate cluster and namespace visualization - Ask Kubegrade AI to diagnose the issue - AI correlates Git repo config with live cluster state - Root cause identified: incorrect container/target port - AI generates a Git pull request with the fix - Change is merged and applied via Argo CD - Application recovers successfully - No manual YAML digging - No tribal knowledge - No copy-paste debugging # This is one example. The same workflow applies to: - Networking misconfigurations - PVC and storage issues - Deployment and runtime errors - Drift between Git and cluster state [title] FAQ [path] / ## **Security FAQ** ### **Does Kubegrade execute changes directly in my cluster?** Kubegrade supports controlled execution models and is designed to keep humans in the loop. In GitOps workflows, changes are proposed as PRs and applied through your GitOps system after review/merge. (*Kubegrade*) ### **Can Kubegrade run in private or regulated environments?** Yes, Kubegrade publicly positions support for cloud, on-prem, and hybrid environments. Exact deployment architecture depends on your setup and requirements. (*Kubegrade*) ### **What permissions does the agent need?** Permissions depend on enabled modules and whether you use read-only or execution-enabled workflows. Start with least privilege and expand only as needed. ### **Does Kubegrade collect secrets?** Kubegrade should not collect secret values unless explicitly required for a configured integration. Secret handling behavior must be documented by deployment mode and integration. ## **Pricing/billing FAQ (optional)** ### **How is Kubegrade priced?** Document your actual pricing model only (e.g., by cluster, node, seat, or enterprise plan). If still evolving, say so and direct to sales. ### **Is on-prem pricing different from SaaS?** Typically yes, because support, deployment complexity, and infrastructure requirements differ. Publish only confirmed commercial policy. ### **Are there limits on clusters or users?** List plan-based limits here and link to the Limits & Quotas page. ## **Troubleshooting FAQ** ### **Why is my cluster showing as disconnected?** Common causes: - Agent not running - Network egress blocked - TLS/certificate trust issue - Invalid credentials/token - Proxy misconfiguration ### **Why can’t Kubegrade generate a PR?** Common causes: - Git repo not connected - Insufficient repo permissions - Path mapping not configured - Policy blocked PR generation - Unsupported repo layout edge case ### **Why is drift reported for fields we don’t manage?** You may need exclusions/suppression rules for provider-managed or runtime-generated fields. ### **Why is execution blocked after approval?** A second gate may still apply: - Maintenance window closed - Policy check failed on re-validation - GitOps sync restrictions - Environment freeze window [title] Troubleshooting [path] FAQs/ # **Why is my cluster showing as disconnected?** Common causes: - Agent not running - Network egress blocked - TLS/certificate trust issue - Invalid credentials/token - Proxy misconfiguration :::CtaButton{label="How to connect a cluster" docId="Rl2X2urit-J3brg7CG5UI" openInNewTab="true" noFollow="false"} ::: # **Why can’t Kubegrade generate a PR?** Common causes: - Git repo not connected - Insufficient repo permissions - Path mapping not configured - Policy blocked PR generation - Unsupported repo layout edge case :::CtaButton{label="Check your GitOps setup" openInNewTab="true" noFollow="false"} ::: # **Why is drift reported for fields we don’t manage?** You may need exclusions/suppression rules for provider-managed or runtime-generated fields. # **Why is execution blocked after approval?** A second gate may still apply: - Maintenance window closed - Policy check failed on re-validation - GitOps sync restrictions - Environment freeze window # What if my d**ashboard is loading slowly?** Ensure stable internet connectivity. Clear browser cache and try accessing the dashboard again. It can take a few minutes for the dashboard to load up initially. If the problem persists, please contact customer support via the app. # How does user and team access work in Kubegrade? Multiple users in the same organization can have their own logins while sharing access to the same clusters and repositories. A user can create an account, connect a cluster, and then invite colleagues; invited users who join the organization will see and can interact with the same clusters, subject to their assigned permissions. :::CtaButton{label="More about user management" docId="ky1DTshcIat75whIaO4eN" openInNewTab="true" noFollow="false"} ::: # How does Kubegrade create pull requests and whose identity is used? Kubegrade opens pull requests in your own IaC (infrastructure‑as‑code) repositories, using your Git provider and credentials. In the GitOps integration settings, you configure a token (for GitHub, Azure DevOps, etc.) that represents a specific account, and Kubegrade uses that token to open pull requests on your behalf. :::CtaButton{label="More about GitOps" docId="IZw3mEh0xkiTKuxLY7Y3S" openInNewTab="true" noFollow="false"} ::: # Do we need to change our workflows or infrastructure to use Kubegrade? No, Kubegrade is built to fit into existing GitOps and infrastructure‑as‑code workflows. You install a lightweight agent into the cluster using a kubectl command; once it’s running, the cluster appears in Kubegrade and the platform works against your existing IaC repositories rather than requiring a new deployment model. The design goal is to avoid forcing teams to change how they manage clusters and instead layer better visibility and automation on top. # How dependent is Kubegrade on AI providers and models like Claude, Gemini or ChatGPT? Kubegrade currently uses Anthropic’s API and the Claude model to power its AI agents, but the platform is not tightly coupled to one provider. Kubegrade is also connected to Gemini and OpenAI which can be utilized upon request. Many core capabilities, such as upgrade orchestration and GitOps integration, work independently of AI, and the architecture allows switching to the other providers (besides Gemini or ChatGPT) with minimal changes so the team can choose the best model at any time. # Is Kubegrade cost‑effective compared to hiring a dedicated DevOps engineer? Kubegrade aims to cost roughly 10–30% of a full‑time DevOps engineer while automating and accelerating much of the cluster‑management workload. Given typical DevOps service or engineer costs in 2026, many organizations can either delay hiring extra headcount or run leaner teams by using Kubegrade to augment their existing engineers. :::CtaButton{label="More on pricing" openInNewTab="true" noFollow="false"} ::: # **How do I provide feedback or suggest features for Kubegrade?** Reach out to us via our support channels: - **Application:** When facing issues in the application, please use the Intercom chat to contact us. Provide detailed descriptions and error messages to expedite the help process. - **Email**: If issues remain unresolved, contact KubeGrade support via [support@kubegrade.com](mailto\:support@kubegrade.com). Provide detailed descriptions and error messages to expedite the help process. - **Other**: If you cannot get access to the application, contact us via the contact form on the website [https://kubegrade.com/company/contact-us/](https://kubegrade.com/company/contact-us/) [title] Overview [path] / ## ** Welcome to Kubegrade** Kubegrade is an operational layer for Kubernetes teams that need to keep clusters reliable, auditable, and maintainable as environments grow. It helps platform teams and developers: - Understand what is running across clusters - Detect risk (drift, misconfigurations, upgrade issues) - Investigate incidents faster - Propose safe fixes as pull requests - Execute changes through existing GitOps workflows Kubegrade is built for real production environments and supports cloud, on-prem, and hybrid deployments. ([Kubegrade](https://kubegrade.com/)) ## **What Kubegrade is (and isn’t)** ### **What Kubegrade is** Kubegrade is a Kubernetes operations platform focused on **day-2 operations**, including: - Upgrade management - Troubleshooting - Drift detection - Visualization and dependency mapping - GitOps-based remediation - Guided/agentic workflows with human approval It combines cluster state, configuration context, and workflow automation to help teams move from detection to action faster. ([Kubegrade](https://kubegrade.com/)) ### **What Kubegrade isn’t** Kubegrade is **not**: - A replacement for your Kubernetes provider (EKS/GKE/AKS/OpenShift) - A cluster provisioner by default - A generic CI/CD platform - A pure observability tool - A “black box” auto-remediator that bypasses human review Kubegrade is designed to work **with** your existing tooling and processes, not replace them. The platform emphasizes human-in-the-loop approvals and GitOps execution. ([Kubegrade](https://kubegrade.com/)) ## **Architecture at a glance** Kubegrade typically includes the following components: ### **1) Kubegrade Control Plane** The central service/UI where users: - View cluster posture - Run investigations - Review plans and recommendations - Approve remediations - Manage policies, roles, and workflows ### **2) In-Cluster Agent(s)** Lightweight agents/controllers deployed in target clusters that: - Collect telemetry/metadata - Execute approved workflows - Respect cluster RBAC and network boundaries - Support upgrade planning and operational actions Your current docs draft already describes Kubegrade agents as lightweight controllers that execute plans and collect telemetry securely, while respecting RBAC/network isolation. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ### **3) IaC / Git / GitOps Integrations** Kubegrade connects to: - IaC/config sources (Terraform, Helm, Kustomize) - Git repositories - GitOps systems (Argo CD / Flux) to generate auditable PRs and align runtime state with declared state. ### **4) Observability / Notification Integrations** Kubegrade can connect to monitoring, alerting, and collaboration systems to enrich context and route actions. ## **Core workflow: Scan → Visualize → Propose PR → Approve → Execute (GitOps)** This is Kubegrade’s core operating loop and mirrors your current site workflow. ([Kubegrade](https://kubegrade.com/)) ### **1) Scan** Kubegrade analyzes: - Cluster resources and workload state - Events and signals - Configuration and policy posture - IaC / Git definitions (when connected) Purpose: - Identify drift, risk, failures, or upgrade blockers - Build context for troubleshooting and change recommendations ### **2) Visualize** Kubegrade presents interactive views of: - Cluster objects and dependencies - Service relationships - Warning states and impacted components - Relevant context for investigations This aligns with your current positioning around interactive dependency graphs and detailed insights. ([Kubegrade](https://kubegrade.com/)) ### **3) Propose PR** Kubegrade drafts changes as pull requests (e.g., YAML, Helm values, Terraform changes) so teams can review proposed remediations in their existing workflow. Your homepage explicitly references AI-drafted PRs and GitOps execution. ([Kubegrade](https://kubegrade.com/)) ### **4) Approve** Teams review: - Proposed changes - Safety checks and policy outcomes - Scope of impact - Execution plan / timing Kubegrade is designed to keep humans in the loop. ([Kubegrade](https://kubegrade.com/)) ### **5) Execute (GitOps)** Changes are applied through your GitOps process (e.g., Argo CD or Flux), preserving: - Auditability - Review controls - Provenance of change - Rollback paths ## **Glossary** ### **Agent** A lightweight Kubegrade component deployed in a cluster that collects metadata/telemetry and executes approved workflows within the permissions granted to it. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ### **Audit Trail** A record of user actions, system recommendations, approvals, and executed changes. ### **Cluster** A Kubernetes environment managed by a provider or self-hosted control plane. ### **Drift** A mismatch between desired configuration (Git/IaC) and actual runtime cluster state. ### **Environment** A logical stage of delivery such as dev, staging, or production. ### **GitOps Remediation** A workflow where Kubegrade proposes infrastructure/application changes as pull requests and changes are applied by GitOps tooling after merge. ### **Human-in-the-loop** A control model where Kubegrade can automate analysis and recommendations, but users review/approve before execution. ### **Policy / Guardrail** A rule that constrains or validates what changes are allowed, who can run actions, and when. ### **Project** A logical grouping of clusters, workflows, and policies tied to a team or application area. ### **Workspace** A higher-level boundary used to organize projects, clusters, users, and policies (especially for larger teams). Your draft docs already mention workspaces grouping clusters, policies, and automation workflows. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) # **Getting Started** ## **Quickstart (first cluster in 5 mins)** This quickstart gets you from account access to a connected cluster and first safe change proposal. ### **Step 1 — Sign in to Kubegrade** - Create/login to your Kubegrade account - Select or create an organization/workspace ### **Step 2 — Add a cluster** - Choose your provider (EKS / GKE / AKS / OpenShift / self-managed) - Install the Kubegrade agent into the target cluster - Confirm cluster registration in the UI ### **Step 3 — Connect GitOps and/or IaC (recommended)** - Link your Git provider/repository - Optionally connect Terraform/Helm/Kustomize sources - Map cluster/workspace to repo(s) ### **Step 4 — Run an initial scan** - Trigger a cluster scan - Review health, warnings, drift indicators, and object graph ### **Step 5 — Run a safe workflow** Example: - Select a non-production namespace - Trigger a validation/fix recommendation workflow - Review generated PR - Approve and merge via your normal Git process - Confirm rollout status in GitOps ## **Requirements** ### **Minimum platform requirements** - A Kubernetes cluster you can administer (or delegated access to install an agent) - Permission to create Kubernetes resources (agent install) - Outbound network access from the cluster to Kubegrade endpoints (for SaaS/hybrid modes) - Access to your Git/IaC repositories for PR workflows (optional but strongly recommended) ### **Recommended operational prerequisites** - One GitOps tool (Argo CD or Flux) - Namespace/environment conventions (dev/stage/prod) - Basic RBAC model for platform/dev/auditor roles - Staging environment for testing before production rollout ### **Supported targets (high level)** - Managed Kubernetes providers (EKS, GKE, AKS) - Enterprise/self-managed clusters (OpenShift, Rancher/RKE, upstream Kubernetes) - Hybrid/multi-cluster environments ## **Deployment options (SaaS / on-prem / hybrid)** ## **SaaS** Kubegrade control plane is hosted by Kubegrade. You install agents in your clusters and connect supported integrations. Best for: - Faster setup - Lower operational overhead - Centralized management ## **On-prem** Kubegrade control plane and data plane components run in your environment. Best for: - Regulated workloads - Strict data residency or network constraints - Enterprises requiring private deployment models ## **Hybrid** A mix of hosted and private components (e.g., hosted control plane with private agents/connectivity and controlled data paths). Best for: - Enterprises that want managed UX plus controlled execution/data boundaries Your homepage explicitly states cloud, on-prem, and hybrid support. ([Kubegrade](https://kubegrade.com/)) ## **Connect your first cluster** ### **1) Choose provider and cluster** From the Kubegrade UI: - Select **Add Cluster** - Choose provider type - Name the cluster and assign environment (dev/stage/prod) - Assign workspace/project ### **2) Install Kubegrade agent** Deploy the provided manifest/Helm chart to the cluster. Typical installation includes: - Namespace for Kubegrade agent components - Service account and RBAC resources - Deployment(s) for agent/controller - Optional secret/config resources (depending on integration mode) ### **3) Register and validate** Kubegrade validates: - Agent heartbeat - API connectivity - Permission scope - Basic cluster inventory access ### **4) Assign scope** Optionally restrict the cluster integration to: - Specific namespaces - Specific actions/workflows - Read-only mode for initial rollout ## **Connect IaC** Kubegrade becomes more powerful when it can compare live state against declared configuration and generate changes in the same systems your team already uses. ### **Supported IaC/config types** - Terraform - Helm - Kustomize ### **What connecting IaC enables** - Runtime vs IaC comparison - Drift detection with source context - PR generation against your actual config repo - Safer remediation with reviewable changes ### **Typical setup flow** 1. Connect Git provider/repository 2. Select repo path(s) 3. Identify config format (Terraform/Helm/Kustomize) 4. Map repo to cluster/workspace/environment 5. Validate access and test diff parsing Your site and docs drafts already emphasize cluster + IaC context, PR suggestions, and support for Helm/Terraform/Argo CD integrations. ([Kubegrade](https://kubegrade.com/)) ## **Verify access + health checks** After cluster and IaC connection, confirm the following: ### **Cluster connectivity** - Agent is online - Cluster API reachable - Inventory sync successful ### **Permissions** - Agent can read required resource types - Write/execute permissions align with intended mode (read-only vs execution-enabled) - Namespace restrictions are correctly enforced ### **Integrations** - Git repository access verified - GitOps system connection valid (if configured) - Observability integrations reachable (if configured) ### **Health checks (recommended)** - Run a metadata-only scan - Open dependency graph view - Test a dry-run recommendation (no execution) - Review audit log entries for setup actions ## **First workflow run (example: safe change via PR)** Use a low-risk, non-production example to validate end-to-end flow. ### **Example workflow: Fix configuration drift in a dev namespace** 1. Select cluster and namespace (dev) 2. Run drift detection scan 3. Review identified drift 4. Ask Kubegrade to propose remediation 5. Review generated PR (files, diffs, rationale) 6. Approve PR generation 7. Merge PR in Git provider 8. Let Argo CD / Flux apply the change 9. Confirm drift resolved in Kubegrade ### **What success looks like** - Drift warning cleared - PR linked in audit trail - GitOps rollout status visible - No unapproved direct mutation in cluster # **Platform Concepts** ## **Organizations, workspaces, projects** ### **Organization** Top-level account boundary for: - Users - Billing - Security settings - Shared policies and integrations ### **Workspace** Operational grouping for teams or business units. Your draft docs describe workspaces as grouping clusters, policies, and automation workflows. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) Use workspaces to separate: - Teams - Business domains - Regions - Compliance zones ### **Project** A scoped unit within a workspace used to group: - Related clusters - Applications/services - Policies - Workflow defaults Recommended pattern: - Org = company - Workspace = platform domain / BU / region - Project = product/service/application group ## **Clusters and environments (dev/stage/prod)** Kubegrade supports managing multiple clusters and mapping them to lifecycle environments. ### **Cluster** A Kubernetes environment (EKS/GKE/AKS/OpenShift/self-managed) connected to Kubegrade. ### **Environment** A logical delivery stage: - **dev**: experimentation, low-risk validation - **staging**: production-like testing and upgrade rehearsals - **prod**: business-critical execution with stricter controls ### **Best practices** - Tag clusters with environment metadata - Apply stricter guardrails to production - Rehearse upgrades/remediations in staging before prod - Use separate approval policies per environment Your draft docs already define clusters as logical units representing environments such as dev/staging/prod. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ## **Namespaces and scope controls** Namespaces are the primary unit for multi-tenant control within a cluster. Kubegrade supports scope control at multiple layers: - Cluster-level scope - Namespace-level scope - Workflow/action-level scope - Role-based access scope ### **Common use cases** - Developers can view and run approved workflows only in assigned namespaces - Platform admins can manage upgrades cluster-wide - Auditors get read-only access to dashboards and logs ### **Recommended rollout** - Start with read-only access across the cluster - Enable execution for non-prod namespaces first - Expand gradually with explicit policies ## **Policies and guardrails** Policies and guardrails define what Kubegrade can recommend or execute, and under what conditions. ### **Policy types (typical)** - Approval requirements (who must approve) - Time windows (maintenance windows only) - Scope restrictions (namespace/cluster/workspace) - Change restrictions (e.g., no prod write actions without multi-approval) - Execution mode (read-only, suggest-only, execute-via-GitOps) - Severity-based routing (critical issues escalate) ### **Guardrail goals** - Prevent unsafe actions - Maintain consistency - Enforce compliance requirements - Preserve human oversight ## **Human-in-the-loop approvals** Kubegrade is designed to support automation without removing operator control. Your site and troubleshooting page both emphasize human-in-the-loop operation. ([Kubegrade](https://kubegrade.com/)) ### **What human-in-the-loop means in Kubegrade** - Kubegrade can analyze and propose actions - Users review generated plans/PRs - Execution requires approval based on policy - GitOps remains the system of execution when enabled ### **Approval stages (common)** 1. Recommendation review 2. PR generation approval 3. Merge approval (in Git provider) 4. Post-change verification review (optional) ### **Why this matters** - Reduced risk - Better auditability - Easier adoption by platform teams - Compliance alignment ## **Audit trail model** Kubegrade maintains a traceable history of operational activity. ### **Typical audit events** - User sign-in / role changes - Cluster connection updates - Policy changes - Workflow runs - Recommendations generated - PRs created/linked - Approvals granted/denied - Execution outcomes - Rollback actions ### **Audit trail goals** - Explain what changed - Show who approved it - Link recommendations to actual diffs/PRs - Support incident reviews and compliance checks Your draft docs mention audit logs and historical audit timelines/execution trails. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ## **Roles & permissions (owner / admin / developer / viewer)** Your draft docs already describe a role model including organization owner, cluster administrator/admin, developer, and viewer/auditor. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ### **Owner** Full access to the organization. Can typically: - Manage billing and org-wide settings - Manage all users and roles - Configure identity integration - View all projects/clusters/workspaces - Override policy defaults (if permitted) ### **Admin** Administrative access within assigned workspace/project/cluster scopes. Can typically: - Connect clusters and integrations - Manage policies and team assignments - Run and approve operational workflows (within scope) - Configure execution settings ### **Developer** Operational access for application teams within assigned scopes. Can typically: - View cluster/workload state - Run approved workflows in assigned namespaces - Review recommendations and open PRs - Access dashboards and troubleshooting views ### **Viewer** Read-only access. Can typically: - View dashboards, graphs, and audit logs - Review compliance/audit posture - Export reports (if enabled) Cannot execute or approve changes. # **Security & Infrastructure** ## **Security overview** Kubegrade is designed for enterprise Kubernetes operations with strong emphasis on: - Least-privilege access - Human-in-the-loop approvals - GitOps-based change execution - Auditability - Support for private/on-prem/hybrid deployments Your public messaging already emphasizes secure, auditable workflows and human oversight. ([Kubegrade](https://kubegrade.com/)) ## **Data handling & what leaves the cluster** Kubegrade should be documented as using a **minimum necessary data** model, with deployment-mode differences. ### **Typical data used by Kubegrade** - Kubernetes object metadata and configuration state - Events / status conditions - Health and dependency context - Workflow execution metadata - Optional integration metadata (Git/IaC references, alert identifiers) ### **What may leave the cluster (SaaS / hybrid)** Depending on configuration: - Telemetry metadata - Resource state summaries - Policy/drift findings - Workflow request/response metadata - Optional logs/diagnostic snippets (if enabled) ### **What should be explicitly documented by deployment mode** - Whether payloads include object specs vs metadata only - Whether secrets are collected (recommended: no secret values) - Retention periods - Encryption in transit / at rest - Regional hosting options (if any) Your draft docs note agents collecting telemetry securely and not exfiltrating beyond telemetry metadata; formalize exact behavior here. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ## **Network requirements (egress, proxies)** ### **Outbound connectivity (typical)** Agents usually require outbound access to: - Kubegrade control plane endpoints - Identity endpoints (if SSO-enabled) - Git provider APIs (if PR generation enabled) - Optional observability APIs/integrations ### **Inbound connectivity** Prefer no inbound internet exposure to clusters for SaaS mode (outbound-only agent pattern), where possible. ### **Proxy support** Document support for: - HTTP/HTTPS proxy - Authenticated proxies - Custom CA bundles - NO\_PROXY exclusions for internal services ### **Recommended documentation items** - Exact FQDNs / ports - TLS requirements - Timeout/retry behavior - Firewall allowlist guidance ## **Private connectivity (VPC/VNet peering / PrivateLink equivalent if applicable)** If supported, document private connectivity options by cloud: - AWS: PrivateLink / VPC peering / Transit Gateway patterns - Azure: Private Endpoint / VNet peering - GCP: Private Service Connect / VPC peering - Enterprise private routing for on-prem hybrid deployments If not yet GA: - Mark as “available by deployment model / enterprise request” - Document supported reference architectures - Clarify what remains public (if anything) ## **IP allowlisting** For environments enforcing outbound restrictions: - Publish Kubegrade egress destination IPs/CIDRs (if static) - Or publish endpoint domains with certificate pinning guidance - Document IP rotation/change notification policy - Provide separate lists for SaaS regions if applicable For inbound (if any webhooks/callbacks used): - Publish Kubegrade source IPs for webhook delivery ## **TLS / certificates** ### **In transit** - TLS for agent ↔ control plane communication - TLS for web UI/API access - TLS for integration calls (Git, observability, identity providers) ### **Enterprise requirements** Document support for: - Custom CA trust bundles - Internal PKI certificates (on-prem) - TLS termination models (if self-hosted) - Certificate rotation practices ### **Best practice** - Never disable certificate verification in production - Rotate credentials/certs on schedule - Separate certs per environment where possible ## **Secrets management** Kubegrade should avoid collecting or storing secret values unless strictly required. ### **Recommended approach** - Use Kubernetes Secrets only for agent/integration credentials where necessary - Prefer external secret managers and short-lived credentials - Scope credentials per workspace/project/integration - Encrypt secrets at rest - Redact secrets from logs/UI/audit exports ### **Documentation should specify** - Where secrets live (SaaS/on-prem/hybrid differences) - Rotation procedures - Revocation procedures - How to re-authenticate integrations without downtime ## **RBAC model + least privilege** Your draft docs already emphasize access controls and agent respect for RBAC. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ### **RBAC principles** - Least privilege by default - Read-only initial onboarding mode - Explicit elevation for execution-enabled workflows - Namespace scoping where practical - Separate permissions for analysis vs action ### **Kubernetes-side RBAC** Document: - Required API groups/resources for read operations - Additional permissions required for execution workflows - Optional privileges by module (upgrades, drift remediation, troubleshooting actions) ### **Kubegrade app RBAC** Document: - Owner/Admin/Developer/Viewer permissions - Scope inheritance (org → workspace → project → cluster) - Approval authority vs execution authority separation ## **Compliance notes (SOC2/ISO)** Use this page as a factual capability page (not a marketing claim page). ### **Include only confirmed claims** Document: - Compliance frameworks supported by platform controls (e.g., audit logs, RBAC, approval workflows, least privilege) - Security practices relevant to customer audits - Deployment models for regulated environments (on-prem/hybrid) - Data handling controls ### **If certifications are in progress** State clearly: - “SOC 2 / ISO 27001 status” (in progress / planned / certified) - Scope of certification (company vs product vs hosting environment) Do not overstate certification or attestation status unless already formalized. # **Product Modules (mirrors your website product pages)** # **Upgrade Management** ## **Overview** Kubegrade Upgrade Management helps teams reduce manual effort and upgrade risk by turning Kubernetes upgrades into a guided, auditable workflow. Core outcomes: - Faster upgrade planning - Preflight validation before changes - Safer execution with approvals - PR-based, GitOps-compatible remediation where config changes are required - Better auditability and rollback readiness Your public messaging strongly emphasizes upgrade automation, safety checks, and reduced manual toil. ([Kubegrade](https://kubegrade.com/)) ## **Supported Kubernetes versions/providers** Document support as a compatibility matrix. ### **Providers** - EKS - GKE - AKS - OpenShift - Rancher/RKE (if supported) - Self-managed upstream Kubernetes ### **Version support format (recommended)** Provide: - Minimum supported version per provider - Tested upgrade paths (N → N+1, etc.) - Unsupported/skipped paths - Provider-specific notes (managed add-ons, control plane sequencing, nodegroup specifics) ## **Preflight checks (API deprecations, add-ons, constraints)** Preflight checks are run before upgrade execution to identify blockers and risk. ### **Typical checks** - Kubernetes API deprecations in manifests - Add-on compatibility (CNI, CSI, ingress controllers, metrics stack) - Version skew constraints - Pod disruption budget / capacity constraints - Node image/runtime prerequisites - Admission controller and policy conflicts - Deprecated Helm chart APIs / CRD compatibility - Cluster health baseline (unhealthy nodes, failing workloads) ### **Output** Preflight results should be categorized as: - Blockers (must fix) - Warnings (recommended fixes) - Informational notes ## **Upgrade plans (single / multi-cluster)** ### **Single-cluster upgrade plan** Includes: - Cluster target version - Preflight status - Required remediations - Maintenance window - Rollback strategy - Approval requirements ### **Multi-cluster upgrade plan** Adds: - Sequencing (canary → staging → production) - Cluster grouping by environment/provider/team - Parallelism constraints - Success gates between waves - Standardized runbook application ### **Recommended pattern** 1. Canary cluster 2. Staging cluster(s) 3. Limited production wave 4. Broad production rollout ## **Scheduling + maintenance windows** Kubegrade should support scheduling upgrades and workflows within controlled windows. ### **Scheduling capabilities (recommended)** - One-time scheduled execution - Recurring maintenance windows - Timezone-aware windows - Environment-specific restrictions - Freeze windows / blackout periods ### **Operational controls** - Preflight refresh before execution - Re-approval if conditions changed - Auto-cancel on critical health regressions (if configured) ## **What gets written where (PR contents, provenance)** When Kubegrade proposes upgrade-related changes, document exactly what gets generated. ### **Typical PR contents** - Manifest or values updates - Version pin changes - Compatibility fixes - Config changes required by target version - Explanatory summary of why changes are required - References to preflight findings ### **Provenance metadata (recommended)** - Workflow run ID - Cluster / environment - Policy checks passed/failed - Agent/version used - Timestamp - Approver(s) (linked in audit trail) This aligns with your PR suggestion + GitOps execution model and auditability positioning. ([Kubegrade](https://kubegrade.com/)) ## **Runbooks: common upgrade failure modes** ### **1) Node upgrade stalls** Possible causes: - Pod disruption budgets too strict - Capacity shortage - DaemonSet rollout blocking drain - Misconfigured eviction settings Response: - Review draining events - Temporarily scale capacity / relax constraints (approved) - Retry wave ### **2) Workloads fail after upgrade** Possible causes: - API deprecations missed - Admission/policy incompatibility - Ingress/controller changes - CRD/version mismatch Response: - Review failing workloads and dependency graph impact - Generate remediation PR - Roll back workload config or cluster wave if needed ### **3) Add-on incompatibility** Possible causes: - Unsupported add-on version for target K8s version - CSI/CNI version skew - Metrics/monitoring stack incompatibility Response: - Pin compatible add-on version - Upgrade add-ons in required order - Re-run preflight ### **4) Control plane/node version skew issues** Response: - Validate provider sequencing rules - Reconcile managed nodegroups - Apply supported path only # **Troubleshooting** ## **Overview** Kubegrade Troubleshooting turns Kubernetes troubleshooting from a reactive, manual process into a guided, context-rich workflow with automation and human control. Your current product page explicitly positions troubleshooting as intelligent/automated, with multi-layer diagnosis, dependency graphs, and human-in-the-loop controls. ([Kubegrade](https://kubegrade.com/troubleshooting/)) ## **Guided investigations (what Kubegrade inspects)** Kubegrade investigates issues using correlated signals across Kubernetes layers. ### **Common inspection domains** - Pod and container status - Events and recent changes - Resource pressure (CPU/memory) - Scheduling constraints - Service/endpoints/ingress routing - DNS/network indicators - ConfigMaps/Secrets references and misconfigurations - Dependency relationships across workloads/services Your troubleshooting page specifically mentions issue propagation via dependency graphs and common problem categories including DNS, scheduling, ConfigMaps/Secrets, and performance/resource bottlenecks. ([Kubegrade](https://kubegrade.com/troubleshooting/)) ## **Common incident playbooks (CrashLoopBackOff, OOM, 5xx, DNS, etc.)** ### **CrashLoopBackOff** Kubegrade checks: - Container logs - Exit codes - Liveness/readiness probe failures - Missing env/config/secret references - Recent config changes Typical outputs: - Root-cause candidates - Impacted dependent services - Proposed config or probe adjustments via PR ### **OOMKilled / resource exhaustion** Kubegrade checks: - Memory limits/requests - Actual usage patterns - Recent release changes - Node pressure and eviction signals Typical outputs: - Right-size recommendations - HPA/resource tuning suggestions - PR-ready config updates (if enabled) ### **5xx / ingress/service routing issues** Kubegrade checks: - Ingress/controller status - Service selectors/endpoints - Pod readiness - Upstream dependency health - DNS/network path issues ### **DNS / network connectivity issues** Kubegrade checks: - CoreDNS status - Service DNS resolution - NetworkPolicy constraints - Endpoint availability - Namespace/service name mismatches ### **Scheduling failures** Kubegrade checks: - Resource requests vs node capacity - Taints/tolerations - Node selectors / affinity - PDB interactions ## **Evidence collection + timelines** Kubegrade should present investigations as a structured evidence chain. ### **Evidence types** - Events - Status transitions - Metrics snapshots - Dependency graph changes - Config diffs / Git changes - Alert triggers (if integrated) ### **Timelines** A useful incident timeline should show: - First observed symptom - Related alerts and state transitions - Changes preceding the incident - Actions taken (manual or Kubegrade-assisted) - Recovery confirmation Your draft docs mention historical audit timelines and execution trails, which can be reused here. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ## **Suggested remediations → PR workflow** For supported cases, Kubegrade can convert troubleshooting findings into reviewable changes. Flow: 1. Investigation identifies root-cause candidates 2. Kubegrade proposes remediation options 3. User selects/edits preferred option 4. Kubegrade generates PR against Git/IaC source 5. User reviews and merges 6. GitOps applies 7. Kubegrade verifies recovery signals This directly matches your PR + GitOps model. ([Kubegrade](https://kubegrade.com/)) ## **Post-incident review / export** After resolution, Kubegrade can support review and reporting. ### **Post-incident review contents** - Incident summary - Affected clusters/namespaces/services - Timeline of events and changes - Root cause (confirmed/probable) - Actions taken - PR links / approvals - Prevention follow-ups ### **Export options (recommended)** - JSON/CSV event export - PDF/markdown incident report - Webhook push to ticketing systems - Linkable audit record # **Drift Detection** ## **Overview** Kubegrade Drift Detection identifies mismatches between your live cluster state and intended state (Git/IaC/policy baselines) so teams can correct drift before it becomes an outage or audit problem. Your homepage and draft docs explicitly reference drift detection and cluster+IaC context. ([Kubegrade](https://kubegrade.com/)) ## **Drift sources (cluster vs IaC vs Git)** ### **Cluster vs IaC drift** Live cluster resources differ from Terraform/Helm/Kustomize-defined state. ### **Cluster vs Git drift** Git-tracked manifests/configs no longer match what is running in the cluster. ### **Policy baseline drift** Cluster/workload configuration diverges from internal standards or approved baselines. ### **Cross-environment drift** Staging/prod environments diverge unintentionally over time. ## **Drift policies (what to flag / severity)** ### **What to flag (examples)** - Manual changes outside GitOps - Resource limits/requests changed in cluster only - NetworkPolicy differences - Ingress/service selector mismatches - Deprecated/unsupported API usage - Missing labels/annotations required by policy - Image tag/version deviations - RBAC drift ### **Severity model (recommended)** - **Critical**: security/compliance risk, production impact likely - **High**: likely operational risk or audit issue - **Medium**: inconsistency or future risk - **Low**: informational / cosmetic drift ## **Drift remediation via PR** Kubegrade should support turning drift findings into PRs that restore desired state. ### **Remediation modes** - Reconcile cluster to Git/IaC (preferred in GitOps environments) - Update Git/IaC to reflect approved runtime changes (controlled exceptions) - Open review-only recommendation without PR ### **PR contents** - Exact diff to restore alignment - Drift classification and severity - Evidence of where mismatch was detected - Impact notes (if applicable) ## **Exclusions and suppression rules** Not all drift should trigger alerts. ### **Common exclusions** - Auto-generated labels/annotations - Runtime status fields - Ephemeral resources - Known provider-managed mutations - Approved temporary overrides (time-bound) ### **Suppression best practices** - Scope suppressions narrowly (resource/namespace/path) - Add expiration date - Require reason/comment - Track suppressions in audit logs # **Intelligent Dependency Graphs** ## **Overview** Kubegrade provides deep cluster visualization and dependency mapping to help teams understand relationships, blast radius, and warning states quickly. Your site and docs drafts both reference interactive dependency graphs and dynamic visual dashboards. ([Kubegrade](https://kubegrade.com/)) ## **Object-level graph model (workloads/services/ingress/network)** Kubegrade’s graph should model object-level relationships such as: - Namespace → workload (Deployment/StatefulSet/DaemonSet) - Workload → pods - Service → endpoints/pods - Ingress → services - ConfigMap/Secret references - NetworkPolicy relationships - Storage dependencies (PVC/PV) - Optional external dependencies (via integrations/annotations) Purpose: - Faster root cause analysis - Change impact assessment - Upgrade dependency planning ## **Filters, scopes, saved views** ### **Filters** - Cluster / environment - Namespace - Application/service - Resource kind - Health state / warning severity - Owner/team labels ### **Scopes** Users should be able to graph: - Full cluster - Namespace only - Service-centric blast radius - Incident-specific subgraph ### **Saved views** Save and reuse views for: - Critical services - Upgrade canary scope - Audit/compliance reviews - Team handoffs ## **Warning states + “what changed” views** ### **Warning states** Overlay graph nodes/edges with states such as: - Degraded - Pending / unschedulable - Failed probes - Policy violation - Drift detected - Upgrade blocker ### **“What changed” views** Compare graph snapshots over time to identify: - New/removed dependencies - Routing changes - Config-linked impact - Service endpoint shifts This is especially useful for post-incident reviews and upgrade regressions. ## **Sharing and exporting views** ### **Sharing** - Link-based internal sharing (RBAC-respected) - Saved workspace/project views - Embed in incident tickets/docs (if supported) ### **Exporting** - Image export (PNG/SVG) - Data export (JSON graph metadata) - Snapshot attachment to incident reports # **AI Agents** ## **Overview (agents are “goal-oriented workflows”)** Kubegrade AI Agents are goal-oriented operational workflows designed to help teams complete specific Kubernetes tasks (e.g., troubleshooting, upgrades, PR generation) with strong context and human control. Your draft docs and public pages already describe agentic workflows, AI-driven assistance, and human-in-the-loop operation. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ## **Built-in agents (upgrade, troubleshoot, PR generation)** ### **Upgrade Agent** Purpose: - Analyze target version readiness - Run preflight checks - Propose upgrade sequencing and remediation steps - Generate PRs for required config changes ### **Troubleshooting Agent** Purpose: - Investigate symptoms across multiple signal layers - Identify probable causes - Suggest and optionally prepare remediations ### **PR Generation Agent** Purpose: - Convert approved remediation plans into repo-specific pull requests - Preserve format/structure conventions - Attach rationale and provenance metadata ## **Custom agents (your MCP / tool connectors model)** Custom agents let teams define workflows that combine Kubegrade context with external tool context. ### **Typical custom agent inputs** - Cluster metadata and object state - Dependency graph context - IaC source context (Terraform/Helm/Kustomize) - Git/GitOps state - External tools via connectors/MCPs (e.g., Terraform/ArgoCD/etc.) ### **Example custom use cases** - Policy enforcement workflows - Scheduled drift checks + PR creation - Environment readiness checks before releases - Cost/risk posture reviews across fleet ## **Prompting patterns + guardrails** Even for AI-assisted workflows, outputs should be constrained. ### **Recommended prompting patterns** - Goal + scope + constraints (cluster/env/namespace) - Desired action mode (analyze-only / propose / PR-ready) - Risk tolerance and change restrictions - Output format requirements (summary, checklist, PR body) ### **Guardrails** - Read-only by default for new workflows - Policy checks before proposing or executing - Explicit approval for PR generation/execution - Scope limits (namespace/workspace/project) - Logging/audit of agent actions and outputs ## **Approval + execution controls** Kubegrade keeps AI-assisted workflows under operator control. ### **Controls to document** - Approval requirements by severity/environment - Who can run agents vs who can approve results - Suggest-only mode vs PR generation mode - Execution via GitOps only (recommended default for prod) - Time-window restrictions ## **Feedback loop (how to improve suggestions)** Kubegrade can improve agent usefulness over time through operator feedback. ### **Feedback signals (recommended)** - Accepted vs rejected suggestions - Edits made before PR generation - Rollback frequency - Time to resolution after applying recommendation - “Incorrect root cause” or “missing context” tags ### **Why it matters** - Better remediation quality - Less noisy suggestions - More consistent outputs across teams Your draft docs already mention learning from feedback to refine future operations. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) # **GitOps Remediation** ## **Overview** GitOps Remediation is Kubegrade’s workflow for converting operational findings (drift, incidents, upgrade blockers, optimization recommendations) into reviewable pull requests and applying changes through your existing GitOps system. Your site explicitly describes PR suggestions and GitOps execution. ([Kubegrade](https://kubegrade.com/)) ## **Supported repo layouts** Document support patterns clearly. ### **Common layouts** - Monorepo (all environments/services) - Environment-per-folder - Service-per-folder - Repo-per-service - Platform repo + app repos split - Helm values repos - Terraform module + environment composition repos ### **Best practice** Require users to define: - Repo path mappings per cluster/environment - File ownership / CODEOWNERS expectations - Branch strategy ## **PR generation rules (naming, commit strategy, reviewers)** ### **Branch naming (recommended)** kubegrade/\/\/\ Examples: - kubegrade/drift/prod-eu1/restore-nginx-limits - kubegrade/upgrade/staging/1-29-preflight-fixes ### **Commit strategy** - One commit per logical remediation (preferred) - Squash option for noisy generated changes - Signed commits if required by org policy ### **Reviewers** - Auto-assign based on CODEOWNERS - Add platform approvers for prod scopes - Tag relevant service owners for workload-level changes ## **Policy checks before PR** Before opening a PR, Kubegrade should validate: - Scope and permissions - Policy compliance - Environment restrictions - Change-type restrictions - Required approvals - Optional dry-run validation/tests (if integrated) If checks fail: - Open recommendation without PR - Show blocking reasons and remediation steps ## **Merge-to-apply mechanics** Kubegrade should document how changes get applied after PR merge. ### **Typical flow** 1. PR merged in Git provider 2. GitOps tool (Argo CD / Flux) detects commit 3. GitOps sync applies changes to cluster 4. Kubegrade watches rollout status / post-change signals 5. Audit trail links PR → sync → outcome ### **Recommended controls** - Sync windows - Manual sync for production - Health checks before marking complete ## **Rollback PRs** Rollback support should be explicit and safe. ### **Rollback approaches** - Revert generated PR commit(s) - Generate rollback PR from known-good state - Partial rollback for specific resources (advanced / policy-controlled) ### **Rollback triggers (examples)** - Failed post-change health checks - Elevated error rates - Dependency graph degradation - SLO breach detected via integrations ## **Auditability: change provenance** Every GitOps remediation should be traceable. ### **Provenance fields (recommended)** - Workflow source (drift/troubleshoot/upgrade/etc.) - Cluster/environment/namespace - Agent/workflow version - Trigger type (manual/scheduled/alert-driven) - Approvers and timestamps - PR link and commit SHA - GitOps sync result - Verification result # **Alert Sorting (coming soon)** ## **Overview** Alert Sorting will help teams reduce alert noise by grouping related events, surfacing high-priority issues, and connecting alerts to Kubegrade workflows for faster action. This module is **coming soon** and details may evolve. ## **Ingest sources (Prometheus/Alertmanager etc.)** Planned/common sources: - Prometheus Alertmanager - Grafana alerts - Datadog monitors - PagerDuty incidents/events - Webhook-based alert feeds - Additional observability tools (as integrations expand) ## **Dedup, grouping, suppression** Planned capabilities: - Deduplicate repeated alerts - Group by service/namespace/cluster/root-cause candidate - Suppress known-noisy or low-signal alerts - Correlate alerts with active incidents and graph state ## **Routing rules (team/service/cluster)** Planned routing based on: - Team ownership - Cluster/environment - Namespace/service labels - Severity - Time windows/on-call schedules (via integrations) ## **Suggested actions + links into workflows** Planned actions: - Open troubleshooting investigation with alert context - Jump to impacted dependency graph - Trigger drift scan on affected namespace - Draft remediation PR (where applicable) - Create/update Jira/PagerDuty ticket ## **Noise tuning playbook** Recommended future playbook structure: 1. Identify top noisy alert classes 2. Define ownership labels and routing metadata 3. Add dedup/grouping rules 4. Add suppressions with expiry 5. Track MTTA/MTTR and false-positive reduction 6. Review monthly # **Fleet Management (coming soon)** ## **Overview** Fleet Management will help platform teams standardize and operate many clusters consistently across providers, teams, and environments. This module is **coming soon** and behavior/capabilities may evolve. ## **Standardization (policies/baselines across clusters)** Planned capabilities: - Shared policy baselines - Cluster posture templates - Environment-specific controls - Compliance baseline comparisons ## **Bulk actions (safe modes only)** Planned safe actions: - Bulk scans - Bulk policy checks - Bulk upgrade planning - Bulk drift posture refresh - Bulk PR generation (with per-cluster review gates) Recommended constraint: - No blind bulk execution in production without approval and wave controls ## **Drift/upgrade posture dashboards** Planned dashboards: - Upgrade readiness across fleet - Drift severity distribution - Policy compliance by cluster/team - Risk hotspots by provider/environment ## **Segmentation (by env/team/provider)** Fleet views should segment by: - Environment (dev/stage/prod) - Team/workspace/project - Cloud provider - Region - Compliance zone - Cluster criticality tier # **Integrations** # **Kubernetes Providers** ## **EKS** Connect Amazon EKS clusters to Kubegrade for: - Inventory and health visibility - Upgrade planning and preflight checks - Drift and troubleshooting workflows - GitOps remediation suggestions Document: - Auth method(s) - Required RBAC - EKS-specific upgrade sequence notes - Managed add-on compatibility considerations ## **GKE** Connect Google Kubernetes Engine clusters for: - Cluster/workload visibility - Troubleshooting and drift analysis - Upgrade readiness and planning workflows Document: - GKE auth options - Autopilot vs Standard support differences (if any) - GKE version/support caveats ## **AKS** Connect Azure Kubernetes Service clusters for: - Upgrade readiness and execution planning - Drift detection - Troubleshooting and dependency mapping Document: - Auth and identity setup - Nodepool considerations - AKS-specific maintenance windows/provider constraints ## **OpenShift** Connect Red Hat OpenShift clusters for: - Cluster operations visibility - Troubleshooting and compliance workflows - Drift and GitOps remediation support Document: - OpenShift API/auth specifics - SCC/RBAC considerations - OpenShift route/operator compatibility notes ## **Rancher / RKE (if applicable)** If supported, document whether Kubegrade connects: - Directly to downstream clusters - Via Rancher-managed contexts - To RKE/RKE2 clusters with standard Kubernetes APIs Include any version limitations. ## **Self-managed clusters** Kubegrade can support upstream/self-managed Kubernetes clusters where: - Agent installation is allowed - Required RBAC is granted - Network connectivity to Kubegrade endpoints exists (SaaS/hybrid) Document supported distributions and known constraints. # **IaC & Config** ## **Terraform** Use Terraform integration to: - Compare declared infrastructure/Kubernetes config to runtime state - Generate PRs for remediation or upgrade prep changes - Preserve auditable workflows via Git Your docs draft explicitly references Terraform support and imports/integrations. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ## **Helm** Use Helm integration to: - Inspect chart values and manifests - Detect value drift or incompatible settings - Generate PRs against values files / chart config ## **Kustomize** Use Kustomize integration to: - Parse overlays and environment-specific manifests - Compare desired state vs live cluster - Generate patch updates via PRs # **GitOps** ## **Argo CD** Kubegrade integrates with Argo CD to: - Map Git changes to cluster sync behavior - Track application sync/health after PR merge - Route remediation through existing GitOps controls Your docs draft references ArgoCD support. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ## **Flux** Kubegrade integrates with Flux for: - Git-driven remediation execution - Rollout tracking after merge - Drift reconciliation workflows in GitOps-first environments # **CI/CD** ## **GitHub Actions** Use GitHub Actions with Kubegrade to: - Trigger scans/checks on PRs - Validate generated remediations - Enforce policy checks before merge ## **GitLab CI** Use GitLab CI to: - Run Kubegrade checks in pipeline stages - Validate environment readiness - Gate merges with policy/drift/upgrade checks ## **Jenkins** Use Jenkins to: - Trigger Kubegrade workflows from existing enterprise pipelines - Ingest outputs into release processes - Link remediation and upgrade checks into change management flows Your docs draft mentions Jenkins, GitHub Actions, and GitLab CI integration ecosystem. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) # **Observability & Logging** ## **Prometheus** Prometheus can provide metrics context for: - Troubleshooting investigations - Upgrade risk/health validation - Alert correlation - Post-change verification Your docs draft references Prometheus exports/integration. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ## **Grafana** Grafana integration can support: - Linking dashboards to Kubegrade investigations - Visual context during incident analysis - Shared operational dashboards ## **Datadog** Datadog integration can support: - Alert and monitor context - Incident signal enrichment - Post-remediation verification checks Your docs draft mentions Datadog support/export. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ## **(others you support)** Create a generic pattern page for additional integrations documenting: - Authentication - Scope/permissions - Data ingested - Supported workflows - Limitations - Troubleshooting # **ChatOps & Ticketing** ## **Slack** Slack integration can be used to: - Send workflow notifications - Route approval requests - Share incident summaries - Link directly to Kubegrade investigations/PRs ## **Microsoft Teams** Teams integration can provide: - Notifications for alerts/workflows - Approval routing (if supported) - Incident timeline sharing ## **Jira** Jira integration can support: - Ticket creation from incidents/drift findings - Status updates with PR/execution links - Traceability between platform ops and change records ## **PagerDuty** PagerDuty integration can support: - Incident/event ingestion - Ownership and escalation routing - Linking PagerDuty incidents to Kubegrade investigations ## **Webhooks** Webhook integration supports custom automation and system interoperability. Document: - Event types - Payload schema - Signing/authentication - Retry behavior - IP allowlisting requirements # **Identity & Access** ## **SSO overview** Kubegrade supports centralized identity integration for enterprise access control (where enabled). Use SSO to: - Centralize authentication - Enforce MFA and org policies - Manage user lifecycle and access consistency Your draft docs reference external identity providers and access controls. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ## **SAML / OIDC** Document supported modes and setup steps. ### **SAML** Include: - Metadata exchange - ACS URL / Entity ID - Group/role mapping - Certificate rotation ### **OIDC** Include: - Issuer URL - Client ID/secret - Redirect URIs - Claims mapping - Group/role mapping ### **Role mapping best practices** - Map IdP groups to Kubegrade roles (Owner/Admin/Developer/Viewer) - Scope groups by workspace/project where possible - Keep production approval roles restricted # **Administration** ## **User management** Admins/Owners can: - Invite users - Remove users - Assign roles - Manage access scope by workspace/project/cluster Recommended documentation: - Invite flow - Pending invites - Deactivation behavior - Access review process ## **Team/role management** Document how to: - Create teams - Assign default roles - Map teams to clusters/projects - Enforce separation of duties (view vs approve vs execute) ## **Workspace/project settings** Document settings such as: - Naming and metadata - Cluster associations - Default policies - Notification settings - Integration mappings - Environment labels and conventions ## **Policy management** Admins should be able to: - Create and update policies - Assign policies by scope - Test policy impact (dry-run) - Review policy violations and exceptions - Track policy changes in audit logs ## **Audit logs** Provide admin guidance for: - Searching/filtering logs - Exporting logs - Retention and storage settings - Linking logs to incidents/workflows/PRs Your draft docs explicitly mention compliance audit logs. ([Archbee](https://app.archbee.com/public/PREVIEW-Ns_vUFBg6zZbf6KxOnspK/PREVIEW-S1V5qYZJFBXMUe_ifEJd6)) ## **Billing (if relevant in docs)** If exposed: - Seats/users - Cluster-based billing units - Usage dimensions (if any) - Invoice access - Billing contacts - Plan changes and renewals If not public yet, keep minimal and note “contact support/sales.” # **Reference** ## **Configuration reference** This page should be a canonical reference for: - Global settings - Cluster connection settings - Agent configuration - Network/proxy settings - Policy defaults - Integration configuration keys Format recommendation: - Option name - Type - Default - Required? - Example - Notes ## **Agent CRDs / manifests reference** If Kubegrade installs CRDs or custom resources, document: - CRD names and versions - Spec fields - Status fields - Examples - RBAC implications If no CRDs are exposed yet, document install manifests/Helm values instead. ## **API reference** Document: - Authentication - Base URLs (SaaS/on-prem) - Rate limits - Endpoints by resource (clusters, workflows, policies, audit logs, integrations) - Webhook subscription endpoints - Error codes and pagination ## **Webhooks reference** Document webhook events and payloads. Recommended fields: - event\_type - event\_id - timestamp - org/workspace/project - cluster/environment - severity (if applicable) - resource references - links (UI, PR, incident) Also document: - Signature verification - Retries - Idempotency guidance ## **Limits & quotas** Publish practical limits to reduce support friction. Examples: - Max clusters per workspace (plan-dependent) - Max concurrent workflow runs - Max integrations per workspace - Retention periods - Payload size limits - API rate limits ## **Release notes / changelog** Use a consistent format: - Date - Version - New - Improved - Fixed - Breaking changes - Migration notes (if needed) Include module tags (Upgrade, Troubleshooting, Drift, AI Agents, GitOps, Integrations). # **FAQ** ## **Security FAQ** ### **Does Kubegrade execute changes directly in my cluster?** Kubegrade supports controlled execution models and is designed to keep humans in the loop. In GitOps workflows, changes are proposed as PRs and applied through your GitOps system after review/merge. ([Kubegrade](https://kubegrade.com/)) ### **Can Kubegrade run in private or regulated environments?** Yes, Kubegrade publicly positions support for cloud, on-prem, and hybrid environments. Exact deployment architecture depends on your setup and requirements. ([Kubegrade](https://kubegrade.com/)) ### **What permissions does the agent need?** Permissions depend on enabled modules and whether you use read-only or execution-enabled workflows. Start with least privilege and expand only as needed. ### **Does Kubegrade collect secrets?** Kubegrade should not collect secret values unless explicitly required for a configured integration. Secret handling behavior must be documented by deployment mode and integration. ## **Pricing/billing FAQ (optional)** ### **How is Kubegrade priced?** Document your actual pricing model only (e.g., by cluster, node, seat, or enterprise plan). If still evolving, say so and direct to sales. ### **Is on-prem pricing different from SaaS?** Typically yes, because support, deployment complexity, and infrastructure requirements differ. Publish only confirmed commercial policy. ### **Are there limits on clusters or users?** List plan-based limits here and link to the Limits & Quotas page. ## **Troubleshooting FAQ** ### **Why is my cluster showing as disconnected?** Common causes: - Agent not running - Network egress blocked - TLS/certificate trust issue - Invalid credentials/token - Proxy misconfiguration ### **Why can’t Kubegrade generate a PR?** Common causes: - Git repo not connected - Insufficient repo permissions - Path mapping not configured - Policy blocked PR generation - Unsupported repo layout edge case ### **Why is drift reported for fields we don’t manage?** You may need exclusions/suppression rules for provider-managed or runtime-generated fields. ### **Why is execution blocked after approval?** A second gate may still apply: - Maintenance window closed - Policy check failed on re-validation - GitOps sync restrictions - Environment freeze window [title] Core workflow [path] Overview/ # **Scan** ### Kubegrade analyzes - Cluster resources and workload state - Events and signals - Configuration and policy posture - IaC / Git definitions (when connected) ### Purpose - Identify drift, risk, failures, or upgrade blockers - Build context for troubleshooting and change recommendations # **Visualize** ### Kubegrade presents interactive views of - Cluster objects and dependencies - Service relationships - Warning states and impacted components - Relevant context for investigations # **Propose PR** ### **Kubegrade’s GitOps agent ** - drafts Git-native pull requests (for example, Kubernetes manifests, Helm values, or Terraform changes) that encode the proposed upgrade or remediation steps, so teams can review, comment, and approve them in their existing Git workflow. ### **What’s inside a PR** - Includes concrete config changes plus human‑readable rationale and risk notes. - Links back to the originating incident, cluster, or recommendation in Kubegrade. ### **How this fits existing processes** - Integrates with your current Git provider and CI, so approvals, checks, and rollouts follow your normal process. - Supports progressive rollout patterns (e.g., staging first, then production) encoded directly in the PR content or labels. # ******Approve** Kubegrade is designed to keep humans in the loop. ### **Teams review** - Proposed changes (never auto‑applied) - Safety checks and policy outcomes - Scope of impact - Execution plan / timing # **Execute (GitOps)** ### **Changes are applied through your GitOps process (e.g., Argo CD or Flux), preserving** - Auditability - Review controls - Provenance of change - Rollback paths [title] Quickstart (10min) [path] Getting Started/ This quickstart guides you step by step through the sign up process to the point where you will see information about your first connected cluster. # **Sign up and log in** 1. **Sign Up:** [https://app.kubegrade.com/sign-up](https://app.kubegrade.com/sign-up) (Desktop recommended) 2. **Create an Account:** Enter your business email address, choose a secure password, and and click on 'Sign Up'. 3. **Verification**: Verify your email address by clicking on the verification link sent to you via email. 4. **Log In**: Use your email and password to log in on [https://app.kubegrade.com/log-in](https://app.kubegrade.com/log-in) to access the Kubegrade app. 5. **Forgot your password? **Click on "Forgot Password?" on [https://app.kubegrade.com/log-in](https://app.kubegrade.com/log-in) to reset your password by following the instructions in the email. # Connect your first cluster Kubegrade is a SaaS platform accessed via web browser, no local installation is required. For the initial setup, you need to deploy the in-cluster Kubegrade Agent. ### Here are the recommended steps 1. **Connect Clusters**: Click on the button "Connect Cluster" and follow the instructions on the modal to connect your Kubernetes clusters. 2. **Generate API Key:** If none has been generated, please generate a new one and select it. 3. **Name Cluster**: Please name your cluster the way you want it to be displayed in the application. 4. **Install scanner:** Kubegrade will generate a command for you. Run this on your cluster (requires kubectl and permission to create resources). The scanner agent safely establishes secure communication for upgrade, monitoring, and workflow execution. Wait for Kubegrade to detect and verify your cluster is online (give it a few minutes) and then review your cluster groups, cluster details and more on your dashboard. :::CtaButton{label="Having problems connecting your cluster?" docId="ICKNUhLfMYdXkMWnGMG2u" openInNewTab="true" noFollow="false"} ::: [title] Deployment options [path] Getting Started/ # **Web application** Kubegrade control plane is hosted by Kubegrade and accessible via any browser. You install the scanner agent in your clusters and connect supported integrations. # **On-prem** Kubegrade control plane and data plane components run in your environment. **Best for** - Regulated workloads - Strict data residency or network constraints - Enterprises requiring private deployment models # **Hybrid** A mix of hosted and private components (e.g., hosted control plane with private agents/connectivity and controlled data paths). **Best for** - Enterprises that want managed UX plus controlled execution/data boundaries [title] Connect IaC [path] Getting Started/ Kubegrade becomes more powerful when it can compare live state against declared configuration and generate changes in the same systems your team already uses. # **Supported IaC/config types** - Terraform - Helm - Kustomize # **What connecting IaC enables** - Runtime vs IaC comparison - Drift detection with source context - PR generation against your actual config repo - Safer remediation with reviewable changes # **Typical setup flow** 1. Connect Git provider/repository 2. Select repo path(s) 3. Identify config format (Terraform/Helm/Kustomize) 4. Map repo to cluster/workspace/environment 5. Validate access and test diff parsing [title] Fixing Kubernetes CrashLoopBackOff using Kubegrade (Automated Root Cause & PR Fix) (cloned) [path] Use Cases/ ::embed[]{url="https://www.youtube.com/embed/tZnMvVk3lw0"} # Overview This short demo shows how Kubegrade troubleshoots and fixes a real Kubernetes CrashLoopBackOff issue end-to-end. Kubegrade automates Kubernetes troubleshooting by detecting issues, explaining root causes, and fixing them through GitOps pull requests. # Scenario A pod is stuck in CrashLoopBackOff due to a failing container command that causes repeated restarts immediately after startup. # What happens in the demo - Identify a pod in CrashLoopBackOff via Argo CD - Confirm repeated container restarts in the terminal - Open Kubegrade cluster and namespace visualization - Provide deployment context to the Kubegrade agent - AI diagnoses the root cause automatically - Problematic container command is identified - AI generates a Git pull request with the fix - Pull request is reviewed and merged - Argo CD applies the change - Pods recover and return to a running state - No manual kubectl log hunting - No guesswork - No YAML trial-and-error # This is one example. The same workflow applies to: - CrashLoopBackOff errors - Container startup failures - Invalid commands or entrypoints - Misconfigured deployments - Runtime and configuration drift [title] Fixing Kubernetes OOMKilled Errors using Kubegrade (Out of Memory CrashLoop Fix) [path] Use Cases/ ::embed[]{url="https://www.youtube.com/embed/8W48b8awQaA"} # Overview This short demo shows how Kubegrade troubleshoots and fixes a real Kubernetes OOMKilled (Out of Memory) error end-to-end. Kubegrade automates Kubernetes troubleshooting by detecting issues, explaining root causes, and fixing them through GitOps pull requests. # Scenario A pod is terminated with exit code 137 after exceeding its memory limit, causing repeated restarts. # What happens in the demo - Identify an OOMKilled pod in Argo CD - Confirm exit code 137 in pod status - Open Kubegrade cluster and namespace visualization - Select the affected namespace as context for the AI agent - AI performs root cause analysis - Memory limits misconfiguration is identified - AI proposes remediation options - Memory limits are increased automatically - Kubegrade generates a Git pull request with the fix - Changes are reviewed and merged - Argo CD applies the update - Pod recovers and returns to a healthy running state - No manual resource tuning - No trial-and-error limit changes - No kubectl memory forensics # This is one example. The same workflow applies to: - OOMKilled pods - Exit code 137 errors - Memory and CPU misconfigurations - Resource limits and requests tuning - Runtime instability caused by under-provisioning [title] Fix Kubernetes Pending Pods Fast (CPU Limits Error) using Kubegrade [path] Use Cases/ ::embed[]{url="https://www.youtube.com/embed/R8DWeW2T2Gk"} # Overview This short demo shows how Kubegrade troubleshoots and fixes a real Kubernetes CPU limits misconfiguration end-to-end. Kubegrade automates Kubernetes troubleshooting by detecting issues, explaining root causes, and fixing them through GitOps pull requests. # Scenario A deployment is stuck in Pending state because the container is requesting an impossible CPU value, preventing scheduling. # What happens in the demo - Identify a pod stuck Pending - Confirm pod is not running from logs and terminal - Open Kubegrade visualization and select the affected namespace - Ask the agent why the deployment is not scheduling - AI performs root cause analysis - Misconfigured CPU request detected (100 CPUs typo) - Correct value suggested (100 millicores) - Kubegrade generates a Git pull request with the fix - PR is reviewed and merged - Argo CD applies the change - Pod schedules successfully and becomes healthy - No manual YAML debugging - No scheduler guesswork - No trial-and-error resource tuning # This is one example. The same workflow applies to: - Pods stuck Pending - CPU and memory resource errors - Scheduler failures - Invalid resource requests - Deployment configuration mistakes [title] Fix Kubernetes CPU Throttling using Kubegrade (Performance Issue Troubleshooting) [path] Use Cases/ ::embed[]{url="https://www.youtube.com/embed/oN0mp6UH4yU"} # Overview A pod is running but experiencing degraded performance due to CPU throttling caused by restrictive resource limits. Kubegrade automates Kubernetes troubleshooting by detecting issues, explaining root causes, and fixing them through GitOps pull requests. # Scenario This short demo shows how Kubegrade diagnoses and fixes a real Kubernetes CPU throttling performance issue end-to-end. # What happens in the demo - Identify a deployment with performance issues - Confirm pod is running but throttled - Inspect logs and terminal signals - Open Kubegrade visualization and select affected namespace - Ask the agent to diagnose throttling cause - AI performs root cause analysis - CPU demand exceeding defined limits detected - Kubegrade proposes increasing CPU limits - AI generates a Git pull request with the fix - PR is reviewed and merged - Changes are applied - CPU throttling disappears and performance stabilizes - No manual metrics digging - No resource tuning guesswork - No trial-and-error limit adjustments # This is one example. The same workflow applies to: - CPU throttling issues - Performance degradation in Kubernetes - Resource constraint bottlenecks - Improper CPU limits - Production performance instability [title] Fix Kubernetes Pod Restarting Issues using Kubegrade (Auto Root Cause + PR Fix) [path] Use Cases/ ::embed[]{url="https://www.youtube.com/embed/guHtGclGGR0"} # Overview This short demo shows how Kubegrade troubleshoots and fixes a real Kubernetes pod that keeps restarting. Kubegrade automates Kubernetes troubleshooting by detecting issues, explaining root causes, and fixing them through GitOps pull requests. # Scenario An NGINX pod repeatedly restarts due to a configuration issue, causing instability and failed container runs. # What happens in the demo - What happens in the demo: - Identify a pod stuck in a restart loop - Inspect logs and confirm failures in terminal - Open Kubegrade visualization and select the affected namespace - Ask the agent to diagnose why the pod keeps restarting - AI performs root cause analysis - Configuration error is detected - Multiple remediation options are suggested - Kubegrade generates a Git pull request with the fix - Faulty pod is deleted (immutable pod behavior) - PR is reviewed and merged - Pod is recreated automatically - Deployment stabilizes and runs successfully - No manual log digging - No trial-and-error YAML edits - No guesswork debugging # This is one example. The same workflow applies to: - Restarting pods - Failing containers - Misconfigured deployments - Runtime configuration errors - Production instability issues [title] What Kubegrade is (and isn’t) [path] Overview/ # **What Kubegrade is** ### Kubegrade is a Kubernetes operations platform focused on ***day-2 operations***, including: - Upgrade management - Troubleshooting - Drift detection - Visualization and dependency mapping - GitOps-based remediation - Guided/agentic workflows with human approval - Alert sorting - Fleet management It combines cluster state, configuration context, and workflow automation to help teams move from detection to action faster. # **What Kubegrade isn’t** ### Kubegrade is **not**: - A replacement for your Kubernetes provider (EKS/GKE/AKS/OpenShift) - A cluster provisioner by default - A generic CI/CD platform - A pure observability tool - A “black box” auto-remediator that bypasses human review - A simple “AI wrapper” or generic chatbot UI Kubegrade is designed to work **with** your existing tooling and processes, not replace them. The platform emphasizes human-in-the-loop approvals and GitOps execution. [title] Architecture at a glance [path] Overview/ # Kubegrade includes the following components ![]() **** # **Kubegrade Control Plane** ### The central service/UI where users - View cluster posture - Run investigations - Review plans and recommendations - Approve remediations - Manage policies, roles, and workflows # **In-Cluster Agent(s)** ### Lightweight agents/controllers deployed in target clusters that - Collect telemetry/metadata - Execute approved workflows - Respect cluster RBAC and network boundaries - Support upgrade planning and operational actions # **IaC / Git / GitOps Integrations** ### Kubegrade connects to - IaC/config sources (Terraform, Helm, Kustomize) - Git repositories - GitOps systems (Argo CD / Flux) to generate auditable PRs and align runtime state with declared state. # **Observability / Notification Integrations** Kubegrade can connect to monitoring, alerting, and collaboration systems to enrich context and route actions. [title] Network requirements [path] Security & Infrastructure/ # **Outbound connectivity (typical)** ### Agents usually require outbound access to - Kubegrade control plane endpoints - Identity endpoints (if SSO-enabled) - Git provider APIs (if PR generation enabled) - Optional observability APIs/integrations # **Inbound connectivity** Prefer no inbound internet exposure to clusters for SaaS mode (outbound-only agent pattern), where possible. ## **Proxy support** Document support for: - HTTP/HTTPS proxy - Authenticated proxies - Custom CA bundles - NO\_PROXY exclusions for internal services ## **Recommended documentation items** - Exact FQDNs / ports - TLS requirements - Timeout/retry behavior - Firewall allowlist guidance [title] Built-in and Custom Agents [path] Administration/ # Overview Agents let you define AI agents tailored to your workflows and connect them to external tools via MCP servers such as GitHub, Azure DevOps and more. The custom agents appear alongside built‑in agents on the **Agents** page so your teams can use them across clusters. ## Built-in agents - Navigate to Agents in the left sidebar -> [https://app.kubegrade.com/agents](https://app.kubegrade.com/agents) - Select an agent in the Built-in Agents section to customize them. - Fill out the Custom Instructions field if you want to further customize the agent - Example Assistant agent ## Accessing custom agents - Navigate to Agents in the left sidebar -> [https://app.kubegrade.com/agents](https://app.kubegrade.com/agents) - Use the **Create Agent** field to start a new custom agent or select an existing one to edit. -> [https://app.kubegrade.com/agents/create](https://app.kubegrade.com/agents/create) - Built‑in agents can be customised (prompt and MCP configuration) without affecting future Kubegrade updates to the base product. ## Agent configuration Each custom agent has three main configuration areas: - **Overview** - Agent name: Human‑readable name shown to users in the Agents list. - Description: Short explanation of what this agent does and when to use it. - **Instructions (Agent Prompt)** - The agent prompt defines the default behaviour and tone of the agent. - Click **Build Prompt** to open the prompt builder. - Enter a high‑level description of what the agent should do. - Click on the button with the arrows to generate a suggested prompt. - You can edit the prompt text directly at any time. - Click Insert to insert the suggested prompt in the Agent Prompt field as instructions for the agent. - **MCP Servers** - MCP servers connect the agent to external systems (for example GitHub, Azure DevOps and more). - An agent can have zero, one, or multiple MCP servers enabled given they are complementary (e.g. only one GitOps connection at a time). - MCP settings apply only to the selected agent and do not affect other agents. # Working with MCP Servers Custom Agent MCP Servers: go to [https://app.kubegrade.com/agents](https://app.kubegrade.com/agents) to customise built‑in agents or create, edit, and delete custom agents. When creating or editing a custom agent, you can connect MCP servers. ## Adding an MCP server 1. In the agent editor, scroll to the **MCP Servers** section. 2. Click **+ Connect MCP Servers**. 3. In the **Connect MCP Server** dialog, choose an MCP server type from the dropdown (for example **Github** or **Azure DevOps**). 4. Configure authentication for the selected server by entering the requested information and clicking **Connect**. 5. Select which tools from that MCP server should be available to this agent. 6. Click **Save** to persist the configuration. ## Example: GitHub MCP server ![](https://archbee-image-uploads.s3.amazonaws.com/7Lzjl4NBI_sc1o8F1zL-G/QoR1FzisGtR2U9EeccJLC_image.png) ### When you choose **Github** as the MCP server - Enter a **Personal Access Token** (PAT) with the scopes needed for the tools you plan to use (for example repository read or read/write for PR workflows). - Click **Connect** so Kubegrade can validate the token and load the available tools. - Under **Tools**, choose one or more tools (for example Git repository access or PR helpers) that the agent will be allowed to call. - Save the agent to make the GitHub integration active. # Publishing and managing agents - Draft vs published - New agents start as drafts and are only visible in the editor. - Click **Publish** to make the agent available to users in the Agents view and dropdown selection in the agent chats. - Editing and versioning - You can update the name, description, prompt, or MCP servers of an existing agent at any time. - Changes to a published agent take effect immediately for all future conversations that use that agent. - Deleting agents - From the **Agents** list, open the menu for an agent and choose **Delete** to remove it. - Deletion does not affect any GitHub or Azure DevOps resources; it only removes the configuration inside Kubegrade. [title] User management [path] Administration/ # Roles & Permissions Kubegrade uses organization‑level role-based access control (RBAC) to ensure that users have the appropriate access to features and settings. ## Organization Roles | Role | Description | Key capabilities | | ------------- | ------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | | **Org Owner** | Primary administrative role for the organization. Automatically assigned after signing up. | Full access to all settings, billing, user management (invite/remove), and organization-wide integrations. | | **Org Admin** | Administrative role for day-to-day management. | Can manage users (except Org Owners) and organization settings, but typically does not handle subscription-level ownership. | | **Org User** | Standard member role for core product usage. | Can access dashboards, clusters, and AI agents, but cannot access Organization Settings or manage other users. | ## Permissions Summary ### **Core product features** - **All roles** can: - View clusters and dashboards. - Use visualization tools. - Interact with AI agents. - Manage their own personal notification preferences. ### **Administrative & organization settings** - **Org Owners and Org Admins** can: - Access the **Organization Settings** menu. - **Invite new users** to the organization via email. - Manage **user roles** (promote/demote members). - Configure **organization-wide integrations**, such as GitHub/Azure DevOps PATs and MCP server connections. - **Org Users**: - Cannot access Organization Settings. - Cannot invite or manage other users. - Cannot configure organization-wide integrations. ### **Integration ownership** - **Connection creators** Regardless of role, the user who creates a specific MCP connection or custom agent is considered its owner for notifications. They receive system alerts for that resource (for example, disconnection events or token expiration). [title] Fixing ImagePullBackOff in Kubernetes using Kubegrade (Automated Root Cause & PR Fix) [path] Use Cases/ ::embed[]{url="https://www.youtube.com/embed/LVAuPgUqx3w"} # Overview This short demo shows how Kubegrade troubleshoots and fixes a real Kubernetes ImagePullBackOff issue end-to-end. Kubegrade automates Kubernetes troubleshooting by detecting issues, explaining root causes, and fixing them through GitOps pull requests. # Scenario A pod fails to start because Kubernetes cannot pull the container image due to an invalid or unavailable image version. # What happens in the demo - Identify an unhealthy pod stuck in ImagePullBackOff - Confirm the image pull failure at the cluster level - Open Kubegrade cluster and namespace visualization - Ask the Kubegrade AI agent to diagnose the issue - AI identifies the root cause of the image pull failure - AI proposes remediation options - AI generates a Git pull request with the correct image version - Pull request is reviewed and merged - Argo CD applies the change - Pod recovers and returns to a healthy running state - No manual image guessing - No registry trial-and-error - No digging through YAML files # This is one example. The same workflow applies to: - ImagePullBackOff errors - ErrImagePull failures - Invalid or missing image tags - Registry access and configuration issues - Git and cluster configuration drift [title] Welcome to Kubegrade [path] Overview/ Kubegrade is an operational layer for Kubernetes teams that need to keep clusters reliable, auditable, and maintainable as environments grow. ### It helps platform teams and developers - Understand what is running across clusters - Detect risks (drift, misconfigurations, upgrade issues) - Investigate and troubleshoot incidents faster - Propose safe fixes as pull requests - Execute changes through existing GitOps workflows Kubegrade is built for real production environments and supports cloud, on-prem, and hybrid deployments. ::embed[]{url="www.youtube.com/watch?v=uzFFdyqGiL8"} :::CtaButton{label="Start using Kubegrade ->" docId="Rl2X2urit-J3brg7CG5UI" openInNewTab="false" noFollow="false"} ::: [title] First workflow run [path] Getting Started/ Use a low-risk, non-production example to validate end-to-end flow. # **Example workflow: Fix configuration drift in a dev namespace** 1. Select cluster and namespace (dev) 2. Run drift detection scan 3. Review identified drift 4. Ask Kubegrade to propose remediation 5. Review generated PR (files, diffs, rationale) 6. Approve PR generation 7. Merge PR in Git provider 8. Let Argo CD / Flux apply the change 9. Confirm drift resolved in Kubegrade # **What success looks like** - Drift warning cleared - PR linked in audit trail - GitOps rollout status visible - No unapproved direct mutation in cluster [title] Verify access + health checks [path] Getting Started/ After cluster and IaC connection, confirm the following: # **Cluster connectivity** - Agent is online - Cluster API reachable - Inventory sync successful # **Permissions** - Agent can read required resource types - Write/execute permissions align with intended mode (read-only vs execution-enabled) - Namespace restrictions are correctly enforced # **Integrations** - Git repository access verified - GitOps system connection valid (if configured) - Observability integrations reachable (if configured) # **Health checks (recommended)** - Run a metadata-only scan - Open dependency graph view - Test a dry-run recommendation (no execution) - Review audit log entries for setup actions [title] Requirements [path] Getting Started/ # **Minimum platform requirements** - A Kubernetes cluster you can administer (or delegated access to install an agent) - Permission to create Kubernetes resources (agent install) - Outbound network access from the cluster to Kubegrade endpoints (for SaaS/hybrid modes) - Access to your Git/IaC repositories for PR workflows (optional but strongly recommended) # **Recommended operational prerequisites** - One GitOps tool (Argo CD or Flux) - Namespace/environment conventions (dev/stage/prod) - Basic RBAC model for platform/dev/auditor roles - Staging environment for testing before production rollout # **Supported targets (high level)** - Managed Kubernetes providers (EKS, GKE, AKS) - Enterprise/self-managed clusters (OpenShift, Rancher/RKE, upstream Kubernetes) - Hybrid/multi-cluster environments [title] Fleet Management (coming soon) [path] Product Modules/ # **Overview** Fleet Management will help platform teams standardize and operate many clusters consistently across providers, teams, and environments. This module is **coming soon** and behavior/capabilities may evolve. ![]() ![]() ![]() # **Standardization ** Planned capabilities: - Shared policy baselines - Cluster posture templates - Environment-specific controls - Compliance baseline comparisons # **Bulk actions (safe modes only)** Planned safe actions: - Bulk scans - Bulk policy checks - Bulk upgrade planning - Bulk drift posture refresh - Bulk PR generation (with per-cluster review gates) Recommended constraint: - No blind bulk execution in production without approval and wave controls # **Drift/upgrade posture dashboards** Planned dashboards: - Upgrade readiness across fleet - Drift severity distribution - Policy compliance by cluster/team - Risk hotspots by provider/environment # **Segmentation ** Fleet views should segment by: - Environment (dev/stage/prod) - Team/workspace/project - Cloud provider - Region - Compliance zone - Cluster criticality tier [title] Drift Detection [path] Product Modules/ # **Overview** Kubegrade Drift Detection identifies mismatches between your live cluster state and intended state (Git/IaC/policy baselines) so teams can correct drift before it becomes an outage or audit problem. # **Drift sources (cluster vs IaC vs Git)** ## **Cluster vs IaC drift** Live cluster resources differ from Terraform/Helm/Kustomize-defined state. ## **Cluster vs Git drift** Git-tracked manifests/configs no longer match what is running in the cluster. ## **Policy baseline drift** Cluster/workload configuration diverges from internal standards or approved baselines. ## **Cross-environment drift** Staging/prod environments diverge unintentionally over time. # **Drift policies (what to flag / severity)** ## **What to flag (examples)** - Manual changes outside GitOps - Resource limits/requests changed in cluster only - NetworkPolicy differences - Ingress/service selector mismatches - Deprecated/unsupported API usage - Missing labels/annotations required by policy - Image tag/version deviations - RBAC drift ## **Severity model (recommended)** - **Critical**: security/compliance risk, production impact likely - **High**: likely operational risk or audit issue - **Medium**: inconsistency or future risk - **Low**: informational / cosmetic drift # **Drift remediation via PR** Kubegrade should support turning drift findings into PRs that restore desired state. ## **Remediation modes** - Reconcile cluster to Git/IaC (preferred in GitOps environments) - Update Git/IaC to reflect approved runtime changes (controlled exceptions) - Open review-only recommendation without PR ## **PR contents** - Exact diff to restore alignment - Drift classification and severity - Evidence of where mismatch was detected - Impact notes (if applicable) # **Exclusions and suppression rules** Not all drift should trigger alerts. ## **Common exclusions** - Auto-generated labels/annotations - Runtime status fields - Ephemeral resources - Known provider-managed mutations - Approved temporary overrides (time-bound) ## **Suppression best practices** - Scope suppressions narrowly (resource/namespace/path) - Add expiration date - Require reason/comment - Track suppressions in audit logs [title] Secrets management [path] Security & Infrastructure/ Kubegrade should avoid collecting or storing secret values unless strictly required. ### **Recommended approach** - Use Kubernetes Secrets only for agent/integration credentials where necessary - Prefer external secret managers and short-lived credentials - Scope credentials per workspace/project/integration - Encrypt secrets at rest - Redact secrets from logs/UI/audit exports ### **Documentation should specify** - Where secrets live (SaaS/on-prem/hybrid differences) - Rotation procedures - Revocation procedures - How to re-authenticate integrations without downtime [title] IP allowlisting [path] Security & Infrastructure/ ### For environments enforcing outbound restrictions - Publish Kubegrade egress destination IPs/CIDRs (if static) - Or publish endpoint domains with certificate pinning guidance - Document IP rotation/change notification policy - Provide separate lists for SaaS regions if applicable ### For inbound (if any webhooks/callbacks used) - Publish Kubegrade source IPs for webhook delivery [title] TLS / certificates [path] Security & Infrastructure/ ### **In transit** - TLS for agent ↔ control plane communication - TLS for web UI/API access - TLS for integration calls (Git, observability, identity providers) ### **Enterprise requirements** Document support for: - Custom CA trust bundles - Internal PKI certificates (on-prem) - TLS termination models (if self-hosted) - Certificate rotation practices ### **Best practice** - Never disable certificate verification in production - Rotate credentials/certs on schedule - Separate certs per environment where possible [title] Private connectivity [path] Security & Infrastructure/ ### If supported, document private connectivity options by cloud - AWS: PrivateLink / VPC peering / Transit Gateway patterns - Azure: Private Endpoint / VNet peering - GCP: Private Service Connect / VPC peering - Enterprise private routing for on-prem hybrid deployments ### If not yet GA - Mark as “available by deployment model / enterprise request” - Document supported reference architectures - Clarify what remains public (if anything) [title] Compliance notes [path] Security & Infrastructure/ Use this page as a factual capability page (not a marketing claim page). ### **Include only confirmed claims** Document: - Compliance frameworks supported by platform controls (e.g., audit logs, RBAC, approval workflows, least privilege) - Security practices relevant to customer audits - Deployment models for regulated environments (on-prem/hybrid) - Data handling controls ### **If certifications are in progress** State clearly: - “SOC 2 / ISO 27001 status” (in progress / planned / certified) - Scope of certification (company vs product vs hosting environment) Do not overstate certification or attestation status unless already formalized. [title] RBAC model [path] Security & Infrastructure/ ### **RBAC principles** - Least privilege by default - Read-only initial onboarding mode - Explicit elevation for execution-enabled workflows - Namespace scoping where practical - Separate permissions for analysis vs action ### **Kubernetes-side RBAC** Document: - Required API groups/resources for read operations - Additional permissions required for execution workflows - Optional privileges by module (upgrades, drift remediation, troubleshooting actions) ### **Kubegrade app RBAC** Document: - Owner/Admin/Developer/Viewer permissions - Scope inheritance (org → workspace → project → cluster) - Approval authority vs execution authority separation [title] GitOps [path] Administration/ # Overview Kubegrade delivers fixes via GitOps. Instead of mutating your cluster directly, Kubegrade generates pull requests (PRs) to your Git repository. Your GitOps engine (for example, ArgoCD) then syncs approved changes into the cluster. This guide shows how to enable GitOps with GitHub or Azure DevOps and run the full workflow end to end. Kubegrade assumes that your GitOps repository follows a clear, environment‑oriented structure so it can discover and link workloads correctly. ## **Recommended conventions (adapt as needed)** - One repo for infra + apps, with a top‑level clusters/ directory per environment. - Each environment has its own kustomization.yaml or Helm release definitions that reference shared app and infra components. - Application‑specific configuration (values, overlays) lives under apps/\/ and is referenced from the environment’s config. Kubegrade uses your configured paths to map clusters and workloads to the corresponding Git directories, so consistent naming is important. # GitHub integration This section describes how to connect a GitOps repository to Kubegrade. ## Prerequisites - A connected Kubernetes cluster in Kubegrade (agent installed and reporting) - A Git provider account and repository: - GitHub: ability to create a Personal Access Token (PAT) with repo scope - Azure DevOps: ability to create a PAT and know your Organization name - A GitOps engine set up to sync from your repository (for example, ArgoCD) ## Configuration steps ### Option A: GitHub What you will do - Create a GitHub Personal Access Token (PAT) with repo permissions - Provide the PAT and repository details in Kubegrade to enable PR generation Create a GitHub PAT: 1. In GitHub, create a Personal Access Token with scope: repo (full control of private repositories). Limit the token lifetime and repository access as appropriate for your security policies. 2. Copy the token value securely. Connect in Kubegrade: 1. Open Kubegrade and navigate to Integrations → GitOps → GitHub. 2. Enter your repository details (owner, repo name, default branch) and paste the PAT. 3. Save to validate connectivity and permissions (repo read/write, pull request creation). ### Option B: Azure DevOps (preview/workaround) Azure DevOps integration is currently in a preview/workaround phase. You will use a PAT and Organization name to connect. Capabilities are evolving and may change. Create an Azure DevOps PAT: 1. From Azure DevOps, create a Personal Access Token with permissions to read and write code and create pull requests for the target project/repository. 2. Note your Organization name and the project/repository you plan to use. Connect in Kubegrade: 1. Open Kubegrade and navigate to Integrations → GitOps → Azure DevOps. 2. Enter your Organization name and paste the PAT. 3. Select the project and repository, then save. Expect minor UI or capability changes while in preview. ## Operator and repo setup (outside Kubegrade) - Flux: - Deploy Flux into the cluster and configure it with the Azure DevOps repo URL (HTTPS or SSH). - Provide credentials (PAT or SSH key) to let Flux pull from Azure DevOps. - Argo CD: - Register the Azure DevOps repo as a Git source and create Application resources pointing at environment‑specific paths. [title] Administration [path] / [title] Security [path] FAQs/ # **Does Kubegrade execute changes directly in my cluster?** Kubegrade supports controlled execution models and is designed to keep humans in the loop. In GitOps workflows, changes are proposed as PRs and applied through your GitOps system after review/merge. # **Can Kubegrade run in private or regulated environments?** Yes, Kubegrade publicly positions support for cloud, on-prem, and hybrid environments. Exact deployment architecture depends on your setup and requirements. # **What permissions does the agent need?** Permissions depend on enabled modules and whether you use read-only or execution-enabled workflows. Start with least privilege and expand only as needed. # **Does Kubegrade collect secrets?** Kubegrade should not collect secret values unless explicitly required for a configured integration. Secret handling behavior must be documented by deployment mode and integration. [title] Pricing & Billing [path] FAQs/ # **How is Kubegrade priced?** Kubegrade offers several tiers, from free 14-day trials to enterprise-level subscriptions, providing flexibility based on your team size and functionality demands. See more on [https://kubegrade.com/pricing/](https://kubegrade.com/pricing/) # **Is on-prem pricing different from SaaS?** Typically yes, because support, deployment complexity, and infrastructure requirements differ. Please get in contact with our sales team for details -> [https://ro.am/kubegrade/sales](https://ro.am/kubegrade/sales)