Linux Runtime Security with auditd & Falco: Detecting Anomalies in AI & Container Workloads (2026 Guide)
As organizations accelerate their adoption of microservices, autonomous agent swarms, and self-hosted inference clusters, traditional security perimeters are being severely tested. Static vulnerability scanning and image signing are essential hygiene, but they cannot defend against runtime exploits: zero-day vulnerabilities in container runtimes, malicious prompt injections hijacking tool execution, or unauthorized privilege escalations inside GPU worker nodes.
When an autonomous artificial intelligence system runs custom Model Context Protocol (MCP) servers or hosts high-throughput models by running local LLMs in production, the container environment possesses dynamic execution capabilities. If an attacker manages to trigger arbitrary command execution or escape a container sandbox, detection must happen at the operating system kernel level in milliseconds.
In this comprehensive, production-grade guide, we explore how to build a robust defense-in-depth Linux runtime security posture using the two premier open-source standards in 2026: the Linux Audit Daemon (auditd) and Falco (powered by modern eBPF probes). We analyze system call interception, construct tailored anomaly detection rules for container and AI workloads, and demonstrate real-time alerting workflows.
1. The Shift to Linux Runtime Security in 2026
Modern security engineering categorizes infrastructure protection into two core disciplines: Build-time (Shift-Left) security and Runtime (Shield-Right) security. While static analyzers scan images for known CVEs, modern threat vectors exploit active runtime state:
- Autonomous Tool Execution Exploitation: AI agents running shell tools or code execution interpreters can be manipulated through indirect prompt injections to spawn unauthorized binaries (e.g., netcat, curl to external command-and-control servers, or crypto-miners).
- Container Breakouts & Namespace Escapes: Misconfigured capabilities or shared host namespaces allow compromised container processes to compromise the underlying Linux kernel. Review our manual on Docker container security best practices for baseline hardening.
- Model Weight & Data Tampering: Malicious actors targeting AI inference hosts frequently attempt to exfiltrate private training data or poison cached weights stored in shared volumes (such as model directories in cloud storage).
To detect and neutralize these threats immediately, systems engineers require visibility directly into the Linux kernel where every process creation, network socket binding, and file modification takes place.
2. Deep Dive: Linux Audit Daemon (auditd)
The Linux Audit Subsystem is a battle-tested, kernel-native framework designed to track security-relevant information on your system. Unlike user-space monitoring daemons, auditd receives audit events directly from the kernel over Netlink sockets, making it virtually impossible for unprivileged user processes to bypass.
Core Subsystem Architecture
The audit architecture operates via three primary components:
- Kernel Audit Core: Hooks into system calls (e.g., execve, openat, connect) and evaluates events against active rule filters in kernel memory.
- auditd Daemon: A user-space daemon that collects records from the kernel and writes them to the system audit log.
- Analysis Utilities: Tools such as ausearch and aureport that parse, query, and aggregate audit event records.
Production auditd Rules for Container & AI Hosts
Modern production hosts require focused, optimized rules to prevent audit log flooding while capturing critical operational anomalies. Place the following rules in
1 | /etc/audit/rules.d/99-production-security.rules |
:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28 # Self-Protection: Make audit rules immutable until reboot
-e 2
# Set buffer size for high-throughput container hosts
-b 8192
-f 1
# Monitor critical authentication and privilege configuration files
-w /etc/security/ -p wa -k identity_modification
-w /etc/pam.d/ -p wa -k identity_modification
-w /etc/sudoers -p wa -k privilege_escalation
-w /etc/sudoers.d/ -p wa -k privilege_escalation
# Monitor <a class="wpil_keyword_link" href="https://www.howto-do.it/what-is-docker/" title="Docker" data-wpil-keyword-link="linked" data-wpil-monitor-id="2237">Docker</a> and Containerd sockets & configurations
-w /var/run/docker.sock -p rw -k docker_socket_access
-w /run/containerd/containerd.sock -p rw -k containerd_socket_access
-w /etc/docker/daemon.json -p wa -k docker_daemon_config
# Monitor GPU drivers and AI Model weight directories
-w /dev/nvidia0 -p wa -k gpu_driver_tampering
-w /dev/nvidiactl -p wa -k gpu_driver_tampering
-w /opt/models -p wa -k ai_model_tampering
# Audit process execution by unprivileged system users
-a always,exit -F arch=b64 -S execve -F auid>=1000 -F auid!=4294967295 -k user_execution
# Audit dynamic kernel module loading and unloading
-a always,exit -F arch=b64 -S init_module -S finit_module -S delete_module -k kernel_modules
To compile and load the new rules into the active kernel, reload the audit configuration:
1
2 <a class="wpil_keyword_link" href="https://www.howto-do.it/what-is-sudo-superuser-do/" title="sudo" data-wpil-keyword-link="linked" data-wpil-monitor-id="2228">sudo</a> augenrules --load
sudo auditctl -l
Querying and Investigating with ausearch
When an alert triggers, use ausearch to quickly extract the full forensic context:
1
2
3
4
5 # Search for all events flagged with the ai_model_tampering key
sudo ausearch -k ai_model_tampering --interpret
# Search for failed system calls initiated by a specific user
sudo ausearch -ua 1001 --success no -i
3. Deep Dive: Falco and Cloud-Native eBPF Runtime Security
While auditd is the gold standard for compliance and forensic audit logging, it was designed before the container era. It lacks native awareness of Kubernetes pods, container namespaces, and container image metadata. Furthermore, under heavy I/O, synchronous audit logging can introduce measurable latency.
This is where Falco, the CNCF graduated runtime security project, excels. Falco leverages extended Berkeley Packet Filters (eBPF) to run safe, sandboxed bytecode programs inside the Linux kernel, capturing system calls asynchronously with less than 1–2% CPU overhead.
Why eBPF Transforms Container Security
- Rich Context Enrichment: Falco automatically enriches raw kernel system calls with container runtime metadata (container ID, container name, image repository, Kubernetes pod name, namespace).
- Real-time Behavioral Filtering: Events are evaluated inside the kernel against expressive declarative rules before being dispatched to user space via high-speed ring buffers.
- Zero Kernel Modification: Modern eBPF probes require no proprietary out-of-tree kernel modules, ensuring rock-solid kernel stability across LTS distributions.
Tailored Falco Rules for AI Inference & Agent Workloads
AI inference containers (such as vLLM or Ollama) and autonomous agent containers should run dedicated, predictable workloads. A shell being spawned or a package manager executing inside an inference container is an immediate indicator of compromise (IoC).
Create a custom rule file at
1 | /etc/falco/rules.d/ai-workload-rules.yaml |
:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43 - list: ai_inference_containers
items: [vllm-production, ollama-service, tgi-production, agent-executor]
- rule: Unauthorized Shell Spawned in AI Container
desc: Detects interactive shell execution inside dedicated AI model serving containers
condition: >
spawned_process and
container and
container.name in (ai_inference_containers) and
proc.name in (<a class="wpil_keyword_link" href="https://www.howto-do.it/what-is-bash-bourne-again-shell/" title="bash" data-wpil-keyword-link="linked" data-wpil-monitor-id="2233">bash</a>, sh, zsh, ksh, dash)
output: >
CRITICAL: Unexpected shell spawned in AI container
(user=%user.name container=%container.name image=%container.image.repository
proc=%proc.name cmdline=%proc.cmdline pproc=%proc.pname)
priority: CRITICAL
tags: [security, ai, container, mitre_execution]
- rule: Model Weights Directory Modification
desc: Detects write or unlink attempts inside protected AI model storage paths
condition: >
(open_write or unlink or rmdir) and
container and
fd.name startswith "/opt/models"
output: >
ALERT: Unauthorized modification attempt on AI model storage
(user=%user.name container=%container.name file=%fd.name
proc=%proc.name cmdline=%proc.cmdline)
priority: WARNING
tags: [security, ai, integrity]
- rule: Outbound Network Connection from Isolated Inference Worker
desc: Detects unexpected outbound connection attempts from model inference nodes
condition: >
outbound and
container and
container.name in (ai_inference_containers) and
not fd.sip in (127.0.0.1, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16)
output: >
CRITICAL: External egress detected from AI worker
(container=%container.name ip=%fd.rip port=%fd.rport
proc=%proc.name cmdline=%proc.cmdline)
priority: CRITICAL
tags: [security, ai, network, egress]
4. Architectural Comparison: auditd vs. Falco
Choosing between auditd and Falco is not an either/or decision—they serve complementary tiers of enterprise defense:
| Feature & Capability | Linux Audit Daemon (auditd) | Falco (eBPF Engine) |
|---|---|---|
| Kernel Hooking Mechanism | Native Linux Kernel Audit Subsystem via netlink sockets. | Modern eBPF probes (bpf_probe) and ring buffers. |
| Performance Overhead | Moderate to high under heavy I/O; synchronous syscall tracking. | Ultra-low (<1-2% CPU); asynchronous in-kernel ring buffer dispatch. |
| Container & Pod Awareness | Minimal; logs process IDs (pid), user IDs, and cgroup paths. | Rich native metadata: container ID, image, name, namespace, pod. |
| Rule Language & Expressiveness | Low-level syscall flags (-a always,exit -S execve). | High-level YAML conditions (spawned_process and container). |
| Alerting & Integration | Log-based file output to system audit files; requires log forwarder. | Direct gRPC, webhook, stdout, Slack, Datadog via Falcosidekick. |
| Best Enterprise Role | Mandatory compliance, forensic audit trails, host file integrity. | Real-time threat detection, AI agent security, container defense. |
5. Step-by-Step Linux Deployment Guide
Step 1: Installing and Configuring auditd
Install the audit subsystem packages on Debian 12 or Ubuntu 24.04 LTS:
1
2
3 sudo apt-get update
sudo apt-get install -y auditd audispd-plugins
sudo systemctl enable --now auditd
Step 2: Installing Falco with the Modern eBPF Driver
Add the official Falco package repository and install the modern eBPF driver:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17 # Download Falco repository signing key
curl -fsSL -o /tmp/falco-key.asc https://falco.org/repo/falcosecurity-packages.asc
sudo gpg --dearmor -o /usr/share/keyrings/falco-archive-keyring.gpg /tmp/falco-key.asc
# Add repository source entry
echo "deb [signed-by=/usr/share/keyrings/falco-archive-keyring.gpg] https://download.falco.org/packages/deb stable main" | sudo tee /etc/apt/sources.list.d/falcosecurity.list
# Install Falco package
sudo apt-get update
sudo apt-get install -y falco
# Configure Falco to use the modern eBPF driver in /etc/falco/falco.yaml
sudo sed -i 's/kind: .*/kind: modern_ebpf/' /etc/falco/falco.yaml
# Enable and start the Falco background daemon
sudo systemctl enable --now falco
sudo systemctl status falco
Step 3: Real-Time Alerting with Falcosidekick
To avoid manual terminal monitoring, deploy Falcosidekick to route Falco alerts directly to Slack, PagerDuty, or your central SIEM webhook:
1
2
3
4
5
6
7 docker run -d \
--name falcosidekick \
--restart always \
-p 2801:2801 \
-e SLACK_WEBHOOKURL="https://hooks.slack.com/services/T00/B00/XXXXX" \
-e SLACK_MINIMUMPRIORITY="warning" \
falcosecurity/falcosidekick:latest
Update
1 | /etc/falco/falco.yaml |
to forward notifications to the sidekick daemon:
1
2
3
4 json_output: true
http_output:
enabled: true
url: "http://localhost:2801/"
1 sudo systemctl restart falco
6. Incident Scenario: Detecting an Exploit in Real Time
Imagine an autonomous AI agent container (
1 | agent-executor |
) is manipulated via indirect prompt injection into attempting unauthorized privilege discovery:
1
2 # Simulating attacker action inside the container
docker exec -it agent-executor whoami
Immediately, both layers capture and record the incident:
- Falco Output (Real-Time Notification within 50ms):
1
2
3
4
5
6{
"time": "2026-09-24T19:15:02.102341231Z",
"priority": "Critical",
"rule": "Unauthorized Shell Spawned in AI Container",
"output": "CRITICAL: Unexpected shell spawned in AI container (user=agent container=agent-executor image=corp/agent-runner proc=sh cmdline=whoami pproc=containerd-shim)"
} - auditd Log (Immutable Forensic Trace in Audit Subsystem):
1
2
3type=SYSCALL arch=c000003e syscall=execve success=yes
ppid=1204 pid=15432 auid=1000 uid=1000 gid=1000 euid=1000 tty=(none) ses=4
comm="whoami" exe="/usr/bin/whoami" key="user_execution"
Summary & Security Best Practices Checklist
By pairing auditd for compliance, immutable forensic logging, and host file system monitoring with Falco (eBPF) for sub-second, container-aware behavioral anomaly detection, you establish an ironclad runtime defense perimeter for your Linux infrastructure.
- Always make auditd rules immutable with
1-e 2
after bootstrapping host configuration.
- Use Falco’s modern eBPF probe to minimize CPU overhead on intensive AI GPU hosts.
- Create explicit whitelists of authorized binaries per container and alert immediately on any shell execution.
- Forward runtime alerts via Falcosidekick to centralized response systems for rapid containment.
Harden Your Linux & AI Infrastructure with Experts
Securing enterprise Linux environments against sophisticated agentic and container threats requires comprehensive architecture design. Contact our security and DevOps engineering team via our Contact Form to audit your server infrastructure, configure tailored eBPF runtime rules, and ensure total operational compliance.
- About the Author
- Latest Posts
Mark is a senior content editor at Text-Center.com and has more than 20 years of experience with linux and windows operating systems. He also writes for Biteno.com