hidden executable pages in linux kernel, part 2

In part 1 I've described how memory managed by hardware. Now lets dig into how kernel sees memory. Not surprisingly that we should check the same structures that malicious drivers update while hiding

Modules

List of module structures with head in modules and lock modules_mutex. It has projection on file /proc/modules but sizes in those file are sloppy - function module_total_size calculates total size of driver (including discarded sections!). So we should use only some selected fields:

• on kernel >= 6.4 mem[MOD_TEXT].base & mem[MOD_TEXT].size
• on kernel < 4.5 module_core & core_text_size
• otherwise core_layout.base & core_layout.text_size
  

vmap_area_list

It has projection on file /proc/vmallocinfo and requires root access. Sure sophisticated rootkits can intercept it but that's ok since we use it for cross-scan only

 

False positives

As you can guess not every executable page belongs to some driver - there are couple exceptions

 

kprobes

under x86 they usually allocated within function arch_ftrace_update_trampoline. Funny fact - they are "frozen in amber" - even after uninstalling some kprobe page for trampoline remains marked as executable so probably you even can recover (with disassembler) for which functions kprobes were installed - this can be very valuable for forensic

ebpf JIT

Oh yeah, this is my favorite! Lets check what happens on my old ubuntu:

bpftool prog show


951: cgroup_skb  tag 6deef7357e7b4530  gpl
    loaded_at 2024-09-05T07:28:09+0300  uid 0
    xlated 64B  jited 54B  memlock 4096B
952: cgroup_skb  tag 6deef7357e7b4530  gpl
    loaded_at 2024-09-05T07:28:09+0300  uid 0
    xlated 64B  jited 54B  memlock 4096B
953: cgroup_skb  tag 6deef7357e7b4530  gpl
    loaded_at 2024-09-05T07:28:09+0300  uid 0
    xlated 64B  jited 54B  memlock 4096B
954: cgroup_skb  tag 6deef7357e7b4530  gpl
    loaded_at 2024-09-05T07:28:09+0300  uid 0
    xlated 64B  jited 54B  memlock 4096B
955: cgroup_skb  tag 6deef7357e7b4530  gpl
    loaded_at 2024-09-05T07:28:09+0300  uid 0
    xlated 64B  jited 54B  memlock 4096B
956: cgroup_skb  tag 6deef7357e7b4530  gpl
    loaded_at 2024-09-05T07:28:09+0300  uid 0
    xlated 64B  jited 54B  memlock 4096B
957: cgroup_skb  tag 6deef7357e7b4530  gpl
    loaded_at 2024-09-05T07:28:09+0300  uid 0
    xlated 64B  jited 54B  memlock 4096B
958: cgroup_skb  tag 6deef7357e7b4530  gpl
    loaded_at 2024-09-05T07:28:09+0300  uid 0
    xlated 64B  jited 54B  memlock 4096B

As you can see it has 8 very small ebpf programs each JITed into 54 byte of code. Try to guess how much memory do they actually take up? lkcd shows

Memory summary:
1150976 0xffffffffaee3856e bpf_jit_alloc_exec+E
8192 0xffffffffaec89d04 arch_ftrace_update_trampoline+124

1.15Mb! We can even check content of those pages:

lkcd -M ... | grep bpf_jit_alloc_exec
     [158] PTE 0xffff936bc6b664f0 14CC1B161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC169E000 alloced by bpf_jit_alloc_exec+E
     [190] PTE 0xffff936bc6b665f0 160CC7161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC16BE000 alloced by bpf_jit_alloc_exec+E
     [192] PTE 0xffff936bc6b66600 1627BD161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC16C0000 alloced by bpf_jit_alloc_exec+E
     [194] PTE 0xffff936bc6b66610 1627C5161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC16C2000 alloced by bpf_jit_alloc_exec+E
     [196] PTE 0xffff936bc6b66620 1627B7161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC16C4000 alloced by bpf_jit_alloc_exec+E
     [198] PTE 0xffff936bc6b66630 149C7F161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC16C6000 alloced by bpf_jit_alloc_exec+E
     [200] PTE 0xffff936bc6b66640 1BB539161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC16C8000 alloced by bpf_jit_alloc_exec+E
     [201] PTE 0xffff936bc6b66648 14CDD4161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC16C9000 alloced by bpf_jit_alloc_exec+E
     [203] PTE 0xffff936bc6b66658 1F2C8D161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC16CB000 alloced by bpf_jit_alloc_exec+E
     [204] PTE 0xffff936bc6b66660 1CF8F0161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC16CC000 alloced by bpf_jit_alloc_exec+E
     [277] PTE 0xffff936bc6b668a8 16548B161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC1715000 alloced by bpf_jit_alloc_exec+E
     [279] PTE 0xffff936bc6b668b8 165774161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC1717000 alloced by bpf_jit_alloc_exec+E
     [281] PTE 0xffff936bc6b668c8 165778161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC1719000 alloced by bpf_jit_alloc_exec+E
     [282] PTE 0xffff936bc6b668d0 165779161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC171A000 alloced by bpf_jit_alloc_exec+E
     [283] PTE 0xffff936bc6b668d8 16577A161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC171B000 alloced by bpf_jit_alloc_exec+E
     [285] PTE 0xffff936bc6b668e8 16577C161 addr FFFFFFFFC1600000 final_addr FFFFFFFFC171D000 alloced by bpf_jit_alloc_exec+E
     [210] PTE 0xffff936bc9c20690 205CC3161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18D2000 alloced by bpf_jit_alloc_exec+E
     [211] PTE 0xffff936bc9c20698 18E437161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18D3000 alloced by bpf_jit_alloc_exec+E
     [218] PTE 0xffff936bc9c206d0 10F890161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18DA000 alloced by bpf_jit_alloc_exec+E
     [219] PTE 0xffff936bc9c206d8 1142FB161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18DB000 alloced by bpf_jit_alloc_exec+E
     [221] PTE 0xffff936bc9c206e8 1EB948161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18DD000 alloced by bpf_jit_alloc_exec+E
     [222] PTE 0xffff936bc9c206f0 17943B161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18DE000 alloced by bpf_jit_alloc_exec+E
     [224] PTE 0xffff936bc9c20700 113F59161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18E0000 alloced by bpf_jit_alloc_exec+E
     [225] PTE 0xffff936bc9c20708 1E08E0161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18E1000 alloced by bpf_jit_alloc_exec+E
     [239] PTE 0xffff936bc9c20778 2E63D2161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18EF000 alloced by bpf_jit_alloc_exec+E
     [240] PTE 0xffff936bc9c20780 1CE54A161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18F0000 alloced by bpf_jit_alloc_exec+E
     [242] PTE 0xffff936bc9c20790 1F01EE161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18F2000 alloced by bpf_jit_alloc_exec+E
     [243] PTE 0xffff936bc9c20798 1F01EF161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18F3000 alloced by bpf_jit_alloc_exec+E
     [245] PTE 0xffff936bc9c207a8 3115CA161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18F5000 alloced by bpf_jit_alloc_exec+E
     [246] PTE 0xffff936bc9c207b0 3115CB161 addr FFFFFFFFC1800000 final_addr FFFFFFFFC18F6000 alloced by bpf_jit_alloc_exec+E
kdps FFFFFFFFC169E000 6

FFFFFFFFC169E000: 01 00 00 00 CC CC CC CC  ........ 0xcccccccc00000001
FFFFFFFFC169E008: CC CC CC CC CC CC CC CC  ........ 0xcccccccccccccccc
FFFFFFFFC169E010: CC CC CC CC CC CC CC CC  ........ 0xcccccccccccccccc
FFFFFFFFC169E018: CC CC CC CC CC CC CC CC  ........ 0xcccccccccccccccc
FFFFFFFFC169E020: CC CC CC CC CC CC CC CC  ........ 0xcccccccccccccccc
FFFFFFFFC169E028: CC CC CC CC CC CC CC CC  ........ 0xcccccccccccccccc


I think ebpf should be illegal in every self-respecting country