суббота, 15 апреля 2023 г.

custom attributes in gcc and dwarf

Lets check if we can add our own attributes (if Google can afford it, then why is it forbidden to mere mortals?). For example I want to have in gcc and dwarf flag about functions/methods parameters direction - is some param IN or OUT. I chose the value of this dwarf attribure 0x28ff

It`s pretty obviously that we can add our own custom attribute in gcc - they even have example how to do this. But what about dwarf producer? Long story short - seems that you cannot do it from plugin. The only dwarf related pass for plugins is pass_dwarf2_frame. So we need to patch gcc. But before this we need to 

build gcc from sources

At moment of writing latest stable version of gcc was 12.0 so run 

git clone --branch releases/gcc-12 https://github.com/gcc-mirror/gcc.git

and then follows instructions

patch gcc

Lets see how gcc produces dwarf output. All symbol table formatters implement gcc_debug_hooks and currently gcc has 3 (btw there are patches for mingw to produce PDB, so in theory you could have vmlinux.pdb):

so lets add function add_param_direction in dwarf2out.cc:
bool add_param_direction(tree decl, dw_die_ref parm_die)
{
  bool pa1 = lookup_attribute ("param_in", DECL_ATTRIBUTES (decl));
  bool pa2 = lookup_attribute ("param_out", DECL_ATTRIBUTES (decl));
  if ( !(pa1 ^ pa2) )
    return false;
  unsigned char pa_value = 0;
  // seems that you can`t have flag with value 1 - see gcc_assert at line 9599
  if ( pa1 )
    pa_value = 2;
  if ( pa2 )
    pa_value = 3;
  add_AT_flag(parm_die, (dwarf_attribute)0x28ff, pa_value);
  return true;
}
It first checks if parameter has attribute param_in or param_out (but not both at the same time bcs this is senseless) and adds custom dwarf flag attribute via add_AT_flag call. Then we just need to call this function from gen_formal_parameter_die
 
Now we can add our custom attributes - this can be done via plugin but I preferred to patch c-family/c-attribs.cc:
 
tree handle_param_in_attribute (tree *node, tree name, tree ARG_UNUSED (args),
                         int ARG_UNUSED(flags), bool *no_add_attrs)
{
  if ( !DECL_P (*node) )
  {
    warning (OPT_Wattributes, "%qE attribute can apply to params declarations only", name);
    *no_add_attrs = true;
    return NULL_TREE;
  }
  tree decl = *node;
  if (TREE_CODE (decl) != PARM_DECL)
  {
    warning (OPT_Wattributes, "%qE attribute can apply to params only", name);
    *no_add_attrs = true;
  } else {
    // check presense of param_out
    if ( lookup_attribute ("param_out", DECL_ATTRIBUTES (decl)) )
    {
      warning (OPT_Wattributes, "%qE attribute useless when param_out was used", name);
      *no_add_attrs = true;
      DECL_ATTRIBUTES (decl) = remove_attribute("param_out", DECL_ATTRIBUTES (decl));
    }
  }
  return NULL_TREE;
}

Function handle_param_in_attribute checks that this attribute linked with function/method parameter. Then it checks that the same parameter don`t have attribute param_out - in this case it just removes both
 
All patches located here 

results

понедельник, 10 апреля 2023 г.

custom dwarf attributes in golang

Finally I found them

0x2900

DW_AT_go_kind, form DW_FORM_data1. Internal golang types kind. For example DW_TAG_structure_type can have kind Struct, Slice or String. I made script to extract statistic which kind can be attached to dwarf tags

0x2901

DW_AT_go_key, form DW_FORM_ref_addr - tag ID. Can be attached to kindMap

0x2902

DW_AT_go_elem, form DW_FORM_ref_addr - tag ID. Can be attached to
  • kindChan
  • kindMap
  • kindSlice

0x2903

DW_AT_go_embedded_field, form DW_FORM_flag. If non-zero member is embedded structure

0x2904

DW_AT_go_runtime_type, form DW_FORM_addr. I don`t know what is it - sure this is not real VA bcs it can point to random sections and even be out of elf module

0x2905

DW_AT_go_package_name, form DW_FORM_string. Just package name for compilation unit

0x2906

DW_AT_go_dict_index, form DW_FORM_udata
index of the dictionary entry describing the real type of this type shape
 
 

пятница, 31 марта 2023 г.

DWARF size overhead

I made today simple script to estimate size overhead due types duplication. This is hard task for C++ - bcs some types can have specialized (or partially specialized) template parameters and sure this types should be considered as different. But for plain C we can safely get all high-level types and assume that types with the same name and declared at the same line and column are equal

Next I ran this script on objdump -g dump for linux kernel. Script gave me digit 252741370

Lets find size of .debug_info section

objdump -h vmlinux | grep debug_info
 35 .debug_info   118205ec  0000000000000000  0000000000000000  03037230  2**0

Size is 0x118205ec = 293733868

And finally lets calculate share of unnecessary info: 252741370 /  293733868 = 0,8604

I am shocked - 86%!!! Looks like hd manufacturers conspiracy 

Update: for C++ I made another version of this script to support namespaces and got following results:

  • cc1 from gcc 111858917 / 130272407 = 0,8587
  • gdb 105034993 / 139899598 = 0,7508
  • llvm-dwarfdump 194031570 / 263053224 = 0,7376

четверг, 30 марта 2023 г.

dwarfdump

I made pale analog of world famous pdbdump to dump types and functions from DWARF. Before introducing my tool I have several words about DWARF - it is excess, compiler-specific, inconsistent and dangerous

Redudancy

gcc and llvm put every used types set in each compilation unit. This is really terrible if you use lots of templates like STL/boost - you will have duplicated declarations of std::map, std::string etc. Yep, this is main reason why stripped binaries becomes much smaller:

ls -l llvm-dwarfdump llvm-dwarfdump.stripped

-rwxrwxr-x 1 redp redp 471241104 mar 29 00:52 llvm-dwarfdump
-rwxrwxr-x 1 redp redp 22170696  mar 29 17:49
llvm-dwarfdump.stripped

Another example - lets check how many times function console_printk declared in debug info from linux kernel:
grep console_printk vm.g | wc -l
2883

It is the same function declared in file include/linux/printk.h line 65 column 0xc - why linker can`t merge it`s type producing debug output?
 
Golang tries to fix this problem using types declarations once and then referring to them from another units (and at the same time compressing debug sections with zlib) - this is very ironically bcs anyway binaries on go typically have size in several Mb (btw llvm-dwarfdump cannot process compressed sections)

 

compiler-specific 

This is pretty obvious - each programming language has some unique features and DWARF must deal with all of them
But just look at this:
 <0><b>: Abbrev Number: 1 (DW_TAG_compile_unit)
    <c>   DW_AT_name        : internal/cpu
    <19>   DW_AT_language    : 22       (Go)
    <1a>   DW_AT_stmt_list   : 0x0
    <1e>   DW_AT_low_pc      : 0x401000
    <26>   DW_AT_ranges      : 0x0
    <2a>   DW_AT_comp_dir    : .
    <2c>   DW_AT_producer    : Go cmd/compile go1.13.8
    <44>   Unknown AT value: 2905: cpu

I was unable to find in golang sources meaning of this custom attributes

 

Inconsistency

DWARF specification don`t define lots of important things. Just to name few:
  • order of tags, so you can have mix of formal parameters with types at the same nesting level
  • which attributes are mandatory for tags - I saw lots of missed DW_AT_sibling for example
  • when locations info should be placed in separate section .debug_loc - seems that this happens for inlined subroutines only
  • encoding of addresses. You have DW_AT_low_pc for functions address. But also there is DW_AT_abstract_origin (and DW_AT_specification). The same function can have different addresses even in plain C via this attributes: 
     <1><191cde>: Abbrev Number: 194 (DW_TAG_subprogram)
        <191ce0>   DW_AT_external    : 1
        <191ce0>   DW_AT_name        : (indirect string, offset: 0x24d2f): perf_events_lapic_init
        <191ce4>   DW_AT_decl_file   : 1
        <191ce5>   DW_AT_decl_line   : 1719
        <191ce7>   DW_AT_decl_column : 6
        <191ce8>   DW_AT_prototyped  : 1
        <191ce8>   DW_AT_inline      : 1    (inlined)
     <1><19a945>: Abbrev Number: 96 (DW_TAG_subprogram)
        <19a946>   DW_AT_abstract_origin: <0x191cde>
        <19a94a>   DW_AT_low_pc      : 0xffffffff81004dc0
     <1><19b3c7>: Abbrev Number: 96 (DW_TAG_subprogram)
        <19b3c8>   DW_AT_abstract_origin: <0x191cde>
        <19b3cc>   DW_AT_low_pc      : 0xffffffff81007930


 All of this lead us to conclusion that DWARF is just

Dangerous

True ant-debugging trick - what if attribute DW_AT_type for DW_TAG_pointer_type points to the same tag? How about negative offset in DW_AT_sibling? I believe that this is very reach area for fuzzing

 

Features of dwarfdump

среда, 7 декабря 2022 г.

timers in linux kernel

timers are very important artifact for forensics, for example Volatility even has plugin to dump timers from windows kernel. Unfortunately Volatility cannot dump timers from linux kernel so I made such dump in my lkcd (with -T option)

kernel timers are just structures timer_list and the most important field is

void (*function)(unsigned long); 

bcs if your machine rootkited - probably one of timers will contains address from some unknown module. timers are chained in linked list via entry field and lots of this lists stored in array vectors into per-cpu variable timer_base. As you can see there can be 2 instances of this structure - this depends from undocumented config option CONFIG_NO_HZ_COMMON

Some timers are part of so called workqueue - structure delayed_work. In such case timer_list.function contains address of exported function delayed_work_timer_fn

вторник, 29 ноября 2022 г.

linux drivers cross-compilation

Just reminder for myself how to build driver for arm64 having x64 based machine with ubuntu

Install right gcc

for arm64 we need gcc-aarch64-linux-gnu:

sudo apt-get install gcc-aarch64-linux-gnu

Build Kernel

You cannot use installed kernel and must build one for appropriate architecture - in my case for arm64 (note - gcc has prefix aarch64, C - consistency). Clone or unpack kernel source tree to some directory KROOT and then

make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- menuconfig
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- modules

Patch Makefile

Usual trick is to use something like

MACHINE ?= $(shell uname -m)
ifeq ($(MACHINE),x86_64)

but this gives you arch of host machine, so you must rewrite all such cases to use ARCH variable (and to setup make -C $(KROOT))

Building

and finally you can cross-compile your driver with something like

make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -f Makefile.arm64

понедельник, 31 октября 2022 г.

BTI incompatible exported functions in kernel 5.15.0-53

if BTI is enabled, the first instruction encountered after an indirect jump must be a special BTI instruction

from here

I downloaded Ubuntu for arm64 (jammy-desktop-arm64.iso) and decided to check if there are some functions with don`t contain BTI c at start

17804 such functions. System.map-5.15.0-53-generic contains 62819 functions in total. Next I just intersected them with exported - 1269

This is obvious bug - maybe in gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0

at least some of this functions are really important - like register_ftrace_function