Enforcing Security Policy at the Kernel’s Decision Points: eBPF LSM Hooks

Every time a process opens a file, spawns a child, or connects to a socket, the Linux kernel reaches an internal checkpoint. Before completing the operation, it consults a chain of security modules — SELinux, AppArmor, Smack — each of which may approve or deny the request. For decades, adding custom logic to that chain meant writing a kernel module: compiled C, loaded as root, living in kernel address space, one bad pointer away from a panic. eBPF LSM changes that equation entirely.

---

The Linux Security Module Framework

The Linux Security Module (LSM) framework was merged in 2001 as a clean abstraction over the kernel’s access control paths. Rather than scattering permission checks through every syscall handler, the kernel defines over 240 discrete hook points at which security decisions must be made. security_file_open fires before every file open. security_bprm_check fires before every execve. security_socket_connect fires before every outbound connection.

Each hook is a call into a linked list of registered handlers. SELinux registers its handlers at boot; AppArmor registers its own. The kernel walks the list — if any handler returns a non-zero error code, the operation is denied. Security logic runs at precisely the right moment: after the kernel has gathered enough context to make an informed decision, before the operation takes effect.

The problem is the registration mechanism. Until BPF LSM arrived, only modules compiled into the kernel image or loaded via insmod could register handlers. Deploying custom policy meant maintaining out-of-tree kernel code across upgrades — a maintenance burden that scales poorly and carries serious stability risk.

---

BPF LSM: Attaching Programs to the Security Chain

BPF LSM, merged in Linux 5.7 under the name KRSI (Kernel Runtime Security Instrumentation), introduces a new program type: BPF_PROG_TYPE_LSM. A BPF program of this type can be attached to any LSM hook using BPF_LSM_MAC attach type. The kernel adds the BPF handler to the security hook list alongside — and evaluated after — any compiled-in modules.

The attach mechanism uses bpf_program__attach_lsm() in libbpf. The BPF program is declared with the SEC("lsm/hook_name") section attribute, where hook_name matches the LSM hook exactly: SEC("lsm/file_open"), SEC("lsm/bprm_check_security"), SEC("lsm/socket_connect"). The verifier enforces that the program’s argument types match the hook’s expected parameters, which are derived from BTF.

The return value carries policy weight. A BPF LSM program attached to a MAC hook returns an integer: zero permits the operation, a negative errno denies it. Return -EPERM from lsm/file_open and the open() syscall fails in userspace. This is a binding enforcement decision, not a trace event.