пятница, 29 июля 2022 г.

dirty secrets of ld.so

As you can know you can set library path under linux with several ways:

  • envvar LD_LIBRARY_PATH, but it can be removed somewhere inside program so /proc/pid/environ is useless (as usually they expose via official API only useless trash but carefully hiding any really important things)
  • via option --library-path to ld.so - like /lib64/ld-linux-x86-64.so.2 --library-path path someprogram Again command line can be patched
  • via /etc/ld.so.conf - this file also can be patched after your program was launched
So good question is "is there some trusted source to see what library path was installed for some running program?"

Yes, this is ld,so itself - because it uses this data while dynamically loading some modules, So long story short: value from --library-path & LD_LIBRARY_PATH stored in variable library_path and whole directory set in rtld_search_dirs
Bad news - they are not exported and even worse - they are hard to find even using disassembler

суббота, 9 июля 2022 г.

PoC to blind pamspy

Lets disasm jit code from this spyware:

 [8] prog 0xffffb02dc0133000 id 160 len 46 jited_len 215 aux 0xffff8ccb58fab400 used_maps 1 used_btf 0 func_cnt 0
     tag: 0F 86 19 76 BC 37 68 B3
  stack_depth: 16
  num_exentries: 0
  type: 2 BPF_PROG_TYPE_KPROBE
  expected_attach_type: 0 BPF_CGROUP_INET_INGRESS
  used maps:
   [0] 0xffff8ccbc1b1c600 - rb
...
ffffffffc07bc801 e80a38e6f1  call 0xffffffffb2620010 ; bpf_ringbuf_submit
ffffffffc07bc806 31c0        xor eax, eax
ffffffffc07bc808 415e        pop r14
ffffffffc07bc80a 415d        pop r13
ffffffffc07bc80c 5b          pop rbx
ffffffffc07bc80d c9          leave
ffffffffc07bc80e c3          ret

and in ebpf opcodes:
43 85 00 00 00 C0 CF 02 00 call 0x2CFC0 ; bpf_ringbuf_submit
44 B7 00 00 00 00 00 00 00 mov r0, 0
45 95 00 00 00 00 00 00 00 ret

Here 0x2CFC0 is offset to bpf_ringbuf_submit from __bpf_call_base
The last call submit some data to bpf map rb with type BPF_MAP_TYPE_RINGBUF. If we could patch this function no data will be passed to usermode. How are these native function addresses filled in at all?

четверг, 30 июня 2022 г.

size of ebpf jit code on different processors

it doesn't make much sense but bcs I have now several jit compilers - why not compare how much size have jitted code for different processors?

I chose 3 ebpf programs

  1. simple BPF_PROG_TYPE_CGROUP_SKB with only comparison, 8 opcodes
  2. BPF_PROG_TYPE_RAW_TRACEPOINT with 3 maps, 68 opcodes
  3. enough complex BPF_PROG_TYPE_RAW_TRACEPOINT with 6 maps, 1824 opcodes
results

processor1st2nd3rd
x64543128195
arm649956712959
powerpc7854611462
risc-v1024709494
s3907853412622
sparc7948210446

среда, 29 июня 2022 г.

verification of jitted ebpf code

There are some projects for ebpf in usermode, but for verification purposes you need the same code which was used in kernel. So I ripped out some jit code to run it in usermode

  • x64
  • powerpc
  • risc-v
  • s390
  • sparc
  • sunway sw64
And now we can make verification of jitted code - we have actual generated code for some ebpf, next we run JIT for ebpf opcodes in usermode, and finally can compare them

суббота, 25 июня 2022 г.

pmu events

Some details

pmu stored in tree pmu_idr and synced with mutex pmus_lock. and as usually can be used to blind EBPF. How? Lets see:

General speaking there are usually four steps involved to attach an eBPF program to a perf event:

  1. Open the perf event
  2. Load the eBPF program
  3. Set the eBPF program on the perf event
  4. Enable the perf event
We interested in point 4 - enabling of the perf event involves calling of pmu->event_init & pmu->add methods. And worse - all pmu structures located in .data section and thus writable. So I add today some code to dump them:

понедельник, 20 июня 2022 г.

ebpf opcodes patching

I made today disasm for eBPF opcodes. Lets see how they looks like:
85 00 00 00 C0 10 02 00 call 0x210C0

in jitted code this is call 0xffffffffb4c14110. ffffffffb4c14110 - 210C0 = FFFFFFFFB4BF3050, address of __bpf_call_base. Suppose that we have some paranoidal code in kernel mode and don`t want to be traced with all this ebpf black magic, what we can do on machine without JIT?

First, we could just patch first opcode to
95 00 00 00 00 00 00 00 ret

Second - we could find some empty native function in kernel (or even reuse __bpf_call_base) and patch address let`s say htab_map_update_elem to it. Can some linux ebpf-based EDR detect this?

среда, 15 июня 2022 г.

epbf maps

As you can see from function bpf_map_alloc_id all bpf maps stored in balanced tree map_idr and synced on spinlock map_idr_lock. No surprise that you can`t view them in user-mode - there is bpf command BPF_MAP_GET_NEXT_ID but it can only enumerate ID of maps. So I add today some code to view bpf maps: lkmem -c -d -B gives output like

bpf_maps at 0xffffffff929c1880: 15
 [0] id 3 UDPrecvAge at 0xffff99e344f48000
  type: 1 BPF_MAP_TYPE_HASH
  key_size 8 value_size 8
 [1] id 4 UDPsendAge at 0xffff99e344cb4c00
  type: 1 BPF_MAP_TYPE_HASH
  key_size 38 value_size 8

also disasm of jitted ebpf code began to look better:
mov rdi, 0xffff99e344f48000 ; UDPrecvAge
call 0xffffffff90c191f0 ; __htab_map_lookup_elem

This letter explains that JIT replacing sequence of opcodes
bpf_mov r1, const_internal_map_id
bpf_call bpf_map_lookup

with direct loading of 64bit address of map (BPF_LD_IMM64 pseudo op). But this code is not optimal - every instruction occupy 10 bytes. Lets consider case where we employ constants pool and put all map addresses somewhere after function - sure this will require at least 8 bytes for each address + perhaps some space for alignment. But now we can produce code like:
mov rdi, qword [map1_addr wrt rip] ; 7 bytes
call __htab_map_lookup_elem
...
; somewhere after function
map1_addr: resq 1 ; jit should put real address of map here

if function has 3 or more reference to the same map we can have some decreasing of jitted code size