Introduction
Most Kubernetes security controls are preventive — RBAC stops an unauthorized call, Pod Security Admission blocks a privileged pod, a NetworkPolicy drops unwanted traffic. But security engineering has a second half that the KCSA exam expects you to understand: detective controls. When something does happen — a service account reads a Secret it shouldn’t, someone execs into a production pod, an attacker probes the API as system:anonymous — how would you ever know? The answer is Kubernetes audit logging.
Audit logging is the API server’s tamper-evident record of who did what, when, and to which resource. It’s the raw material for incident response, threat hunting, and compliance evidence, and it maps directly onto two KCSA domains: the observability side of Platform Security and the API server focus of Cluster Component Security. Yet it’s one of the more misunderstood topics, partly because it isn’t on by default in a vanilla cluster and partly because getting it right means balancing signal against a firehose of noise — and never leaking secrets into your logs in the process.
This guide walks audit logging the way the KCSA frames it: what the audit pipeline is, how policies decide what gets recorded through levels and stages, how the log and webhook backends ship events, what you should and shouldn’t capture, and how the resulting data turns into real threat detection. If you want the surrounding context first, review Kubernetes Cluster Component Security (which covers securing the API server) and Kubernetes Security Fundamentals, then come back here to go deep on the audit trail.
Where Audit Logging Fits: The Detective Layer
The KCSA threat model teaches that no preventive control is perfect — misconfigurations happen, credentials leak, and insiders exist. Detective controls close that gap. In Kubernetes there are two complementary detective sources, and the exam wants you to tell them apart:
| Source | What it observes | Where it runs |
|---|---|---|
| API server audit logs | Every request to the Kubernetes control plane (API calls, verbs, users, resources) | kube-apiserver |
| Runtime security tooling (e.g. Falco) | Syscalls and behavior inside containers/nodes (a shell spawned, a sensitive file read) | Nodes, via eBPF/kernel modules |
Audit logs answer “who called the API and what did they ask for?” Runtime tools answer “what actually happened on the node/inside the container?” An attacker who runs kubectl exec leaves an audit trail at the API server; the shell they open and the commands they run inside the pod are what a runtime tool like Falco catches. You need both for full coverage — a favorite KCSA nuance. This guide focuses on the audit-log half; the runtime half is touched on in Platform Security.
The Audit Pipeline: How a Request Becomes an Event
Every request that reaches the API server passes through the audit subsystem. The flow is:
Request → Authentication → Authorization → Admission → Audit policy evaluated
│
(matching rule assigns an audit LEVEL)
│
Event emitted at one or more STAGES → Backend(s)Two knobs control what actually gets recorded: the level (how much detail) and the stage (at what point in the request’s life). A single audit policy file defines both. Understanding these two dimensions is the core of the KCSA audit material.
Audit Levels: How Much Detail to Capture
The level decides how much of each request is written. There are four, from least to most verbose:
| Level | What it records | Typical use |
|---|---|---|
| None | Nothing — suppress matching events | Silence noisy, low-value requests |
| Metadata | Request metadata: user, timestamp, verb, resource, namespace — no request or response body | Safe default for most traffic; mandatory for Secrets |
| Request | Metadata + the request body | When you need to see what was submitted |
| RequestResponse | Metadata + request body + response body | Full fidelity for high-value, sensitive operations |
The single most important safety rule the exam rewards: never log Secret (or other sensitive) contents at Request or RequestResponse. Doing so writes the plaintext secret values straight into your audit logs, turning your detective control into a data-leak. Secrets, ConfigMaps with sensitive data, and TokenReview/token requests should be captured at Metadata only.
Audit Stages: When to Capture
A request isn’t instantaneous, so audit events can be emitted at up to four stages of its lifecycle:
| Stage | When it fires |
|---|---|
| RequestReceived | Immediately when the audit handler receives the request, before it’s processed |
| ResponseStarted | After response headers are sent but before the body — relevant to long-running requests like watch |
| ResponseComplete | When the response body has finished and no more bytes will be sent |
| Panic | When the request triggers a panic |
Most events you care about land at ResponseComplete, which shows the request and its outcome (allowed, denied, error). Because RequestReceived roughly doubles event volume without adding much value, it’s commonly suppressed with omitStages. Long-running watch connections are why ResponseStarted exists — the response never really “completes” while the watch is open.
Writing an Audit Policy
An audit policy is a YAML document with an ordered list of rules. The API server evaluates rules top to bottom and stops at the first match, applying that rule’s level. Ordering therefore matters enormously — put your most specific, most sensitive rules first and your catch-all last.
apiVersion: audit.k8s.io/v1
kind: Policy
# Skip the noisy RequestReceived stage for every rule
omitStages:
- "RequestReceived"
rules:
# 1. Never log the contents of Secrets — metadata only, to avoid leaking values
- level: Metadata
resources:
- group: ""
resources: ["secrets", "configmaps"]
# 2. Capture full detail on RBAC changes — a key privilege-escalation signal
- level: RequestResponse
verbs: ["create", "update", "patch", "delete"]
resources:
- group: "rbac.authorization.k8s.io"
resources: ["roles", "rolebindings", "clusterroles", "clusterrolebindings"]
# 3. Record pod exec/attach at Metadata (who ran a command in which pod)
- level: Metadata
resources:
- group: ""
resources: ["pods/exec", "pods/attach"]
# 4. Drop high-volume, low-value health/version probes
- level: None
nonResourceURLs:
- "/healthz*"
- "/version"
- "/readyz*"
# 5. Catch-all: everything else at Metadata
- level: MetadataNotice the pattern: sensitive-but-secret data → Metadata, security-relevant changes → RequestResponse, noise → None, and a Metadata catch-all so nothing slips through unrecorded. Rules can also match on verbs, users, userGroups, namespaces, and resourceNames for finer control.
The Backends: Where Events Go
Once the policy decides an event should be recorded, a backend ships it. The API server supports two.
Log backend (file)
Writes events as JSON to a file on the control-plane node (or to stdout). It’s configured with kube-apiserver flags:
| Flag | Purpose |
|---|---|
--audit-policy-file | Path to the policy YAML |
--audit-log-path | Output file (- = stdout) |
--audit-log-maxage | Days to retain old log files |
--audit-log-maxbackup | Maximum number of rotated files to keep |
--audit-log-maxsize | Size in MB before a file is rotated |
--audit-log-format | json (default) or legacy |
kube-apiserver \
--audit-policy-file=/etc/kubernetes/audit/policy.yaml \
--audit-log-path=/var/log/kubernetes/audit/audit.log \
--audit-log-maxage=30 \
--audit-log-maxbackup=10 \
--audit-log-maxsize=100The catch: logs sit on the node. A local file is fine for a lab but weak for security — an attacker who compromises the control plane can tamper with it. Production clusters forward these logs to a central, write-once store (a SIEM, object storage) so the record is durable and tamper-resistant.
Webhook backend
Sends audit events over HTTP to an external service — a SIEM, a log aggregator, or a cloud logging endpoint — configured with --audit-webhook-config-file (a kubeconfig-format file pointing at the receiver). Webhook delivery can run in batch mode (buffer and send in groups, higher throughput) or blocking mode (send each event before continuing, stronger guarantees but higher latency). This decouples the audit trail from the node, which is exactly what you want for integrity.
A sample audit event (JSON) looks like this — note how much investigative value even a Metadata-level record carries:
{
"kind": "Event",
"level": "Metadata",
"verb": "create",
"user": { "username": "system:serviceaccount:default:deployer" },
"sourceIPs": ["10.0.3.14"],
"objectRef": { "resource": "pods", "namespace": "prod", "subresource": "exec" },
"responseStatus": { "code": 201 },
"requestReceivedTimestamp": "2026-07-30T09:14:52Z",
"stage": "ResponseComplete"
}Turning Audit Logs Into Threat Detection
Collecting events is only half the job; the KCSA cares that you know what to look for. High-signal patterns worth alerting on:
pods/execandpods/attach— interactive access into a running pod. Rare in a healthy automated environment; a strong compromise or insider signal.- Reads and lists of
secrets— especially by unexpected service accounts or from unusual source IPs. Credential theft leaves this trail. - RBAC changes — new
ClusterRoleBindings(particularly anything binding tocluster-admin) is a classic privilege-escalation move. - Anonymous or unexpected users — requests from
system:anonymousor unknown identities indicate misconfigured authentication or probing. 403 Forbiddenbursts — repeated authorization failures from one identity suggest reconnaissance or an attacker mapping permissions.- Deletion of audit/logging resources — attempts to cover tracks.
The workflow is: ship audit events to a central store, parse them (each event’s verb, user, objectRef, sourceIPs, and responseStatus are the key fields), and alert on the patterns above. This is precisely why audit logging is classified as a detective control — it doesn’t stop the action, but it makes the action visible so you can respond. Correlating these signals against the Kubernetes threat model is how attack techniques become detections.
Managed Clusters and Compliance
On managed offerings (EKS, GKE, AKS) you don’t run kube-apiserver yourself, so control-plane audit logging is exposed as a provider feature you enable — control-plane logs are streamed to the cloud’s native logging service (e.g. CloudWatch, Cloud Logging, Azure Monitor). The concepts are identical; only the configuration surface differs. Either way, a durable audit trail is a common requirement in compliance frameworks — CIS Kubernetes Benchmark controls, SOC 2, PCI-DSS — which expect evidence of who accessed what. That ties audit logging back to the Compliance and Security Frameworks domain.
Audit Logging Best Practices (Exam-Ready Checklist)
- Enable it deliberately — it’s off by default in a plain cluster; a policy and backend must be configured.
- Metadata-only for Secrets — never write secret values to logs.
- Order rules specific-first — first match wins, so sensitive rules go before the catch-all.
- Suppress noise — drop health/readiness probes and often
RequestReceivedto keep signal high and volume manageable. - Ship off-node — forward to a tamper-resistant central store (webhook backend or log forwarding), never rely solely on a local file.
- Full detail for high-value changes — RBAC and admission-related mutations warrant
RequestResponse. - Pair with runtime detection — audit logs cover the API; a runtime tool covers in-container behavior.
Study Strategy for This Topic
Audit logging questions on the KCSA are conceptual, not hands-on, so focus your energy on the distinctions the exam actually tests:
- Levels vs. stages — know all four of each and, crucially, that
RequestResponseon Secrets is a mistake. - First-match rule ordering — understand why order changes behavior.
- Log vs. webhook backend — local file vs. external service, and why off-node shipping matters for integrity.
- Audit logs vs. runtime tooling — API-server activity vs. in-container behavior.
Reading fixes the model, but the KCSA is a timed, scenario-based exam, and the reliable way to confirm recall is practice. Warm up with the free KCSA practice questions, and when you want full-length, explained mock exams spanning every domain, the Sailor.sh KCSA Mock Exam Bundle lets you rehearse under real exam conditions and close gaps before test day. To map audit logging into your wider prep, follow the KCSA study plan and the KCSA exam guide for 2026.
Conclusion
Kubernetes audit logging is the cluster’s memory — the API server’s authoritative account of every request that touched the control plane. For the KCSA it sits squarely in the detective-controls story: preventive measures reduce risk, but audit logs are how you see what actually happened when prevention fails. Master the two dimensions of an audit policy (levels for detail, stages for timing), the two backends (local log file vs. external webhook), and the golden rule that Secrets are recorded at Metadata only, and you’ll handle the exam’s audit questions with confidence.
More importantly, the same knowledge makes you a better operator: a cluster with a well-tuned audit policy, forwarded to a tamper-resistant store and paired with runtime detection, is a cluster where an intruder can’t move without leaving a trace.
FAQ
Is Kubernetes audit logging enabled by default?
No. A vanilla cluster does not record audit events until you provide an audit policy file and configure a backend (log or webhook) on the kube-apiserver. On managed platforms (EKS, GKE, AKS) it’s an opt-in provider feature you enable in the console or API.
What’s the difference between audit levels and audit stages?
Levels control how much detail is captured — None, Metadata, Request, or RequestResponse. Stages control when an event is emitted during a request’s lifecycle — RequestReceived, ResponseStarted, ResponseComplete, or Panic. A policy rule sets a level; omitStages can suppress specific stages.
Why shouldn’t I log Secrets at the RequestResponse level?
Because Request and RequestResponse write the request/response bodies into the audit log — for a Secret, that means the plaintext secret values end up in your logs. Always capture Secrets (and similarly sensitive resources) at Metadata so you record that they were accessed without exposing what they contained.
How do audit logs differ from Falco or other runtime tools?
Audit logs record activity at the API server — who called the API, with which verb, on which resource. Runtime tools like Falco observe behavior on the node and inside containers — a spawned shell, an unexpected file read, an outbound connection. They’re complementary: an exec shows up in audit logs, but the commands run inside the shell only show up in runtime monitoring.
How does the API server decide which audit rule applies?
Rules in the policy are evaluated in order, and the first one that matches wins. That’s why you place specific, sensitive rules (Secrets, RBAC) at the top and a broad catch-all at the bottom — otherwise a general rule could match first and override the specific handling you intended.
Where should audit logs be sent in production?
Off the node. Relying only on a local file (--audit-log-path) is risky because an attacker who compromises the control plane could alter it. Use the webhook backend or forward the log file to a central, tamper-resistant store such as a SIEM or cloud logging service, which also supports compliance evidence requirements.