ced: sed-like cubin editor

Unfortunately, the only sass assembler I know of has several drawbacks:

• it's inactive last couple of years. I dropped email to his author and he didn't replied. Hope he is well
• it don't support modern sm architectures sm1xx
• it's matmul solver sometimes produces wrong instructions
• and it don't support many EIATTRS 

The last problem is not related with CuAssembler itself - it is more general: seems that nvdisasm produces output which cannot be used to assembly cubin files

Also we still don't know format of some sections like SHT_CUDA_RELOCINFO. All this makes task of rebuilding cubin files very hard

However do we really need to rebuild cubin files? In my experience 99.9% of desired patches are just set/remove some instructions attributes like register reusing/caching policy/wait groups for USCHED_INFO etc - just boring tuning to squeeze out the last couple of percent of productivity

So the flow of thought was something like

• it would be good to make plugin for hex-editor to disasm sass instruction at some known offset and show GUI where I could patch some fields
• I am talentless at creating GUI - so perhaps it would be better to dump instructions fields in text form and then just edit it
• hey - if you can parse this text representation and patch it back to sass - you don't need hex-editor at all - you could just use sed-like tool to patch instructions via script

and so being lazy and impatient I wrote such tool - it's called ced. Name similarity to sed is not coincidence - it allows you run text script to patch or replace some sass instructions inside cubin files

 

Syntax of ced scripts

comments starts with # at start of line

# this is sample of clueless comment

 

selecting section/function to patch

First you need select section or function for patching. This can be done with one of 3 commands:

1. s section_index - can be obtained from readelf -S or from my nvd
2. sn section_name
3. fn function_name. Note that single section can contain several functions
   

 

replacing instruction

Frequently you want to nop some instruction. Format is

offset nop

Or you can use full-featured sass asm to replace whole instruction:

offset r MOV R1, c[0][0x128] 

You also can patch only predicate of instruction - for example if you change predicate to !@PT this effectively converts current instruction to NOP:

offset !@7

 

Field offset is just hexadecimal offset within some code section without 0x prefix - you can just copy-paste them from output of nvdisasm/nvd

 

patching selected fields of instruction

finally you can patch only some specific instruction fields, like in example above with MOV R1 you could patch destination register:

offset p Rd R2

There can be several fields for patching in single instruction so you can chain them - just put leading space on next line:

10 p Rd R2

 p  Sa_offset 0x12c

This pair of patch commands will produce instruction MOV R2, c[0][0x12c]

 

The good question is where you can get fields names? Well, you can use my ina for selecting specific instruction and it's form. Or check MDs in data directory. Or you can run ced with -v option and use dirty hack:

s section_index

offset <space>

@Pg.D(7) MOV E:Rd c:srcConst[constBank:UImm][Sa_offset:SImm*] ,quadMask:UImm(15) req_sb_bitset:BITSET E:usched_info
 

There can be problem when several fields forms single value in instruction via table. Lets assume we have table with 2 columns:

    abSize
          CInteger@U8 CInteger@U8 -> 0
          CInteger@U8 CInteger@S8 -> 1
          CInteger@U8 CInteger@U16 -> 2
          CInteger@U8 CInteger@S16 -> 3
          CInteger@U8 CInteger@"32" -> 4

...


it may happens that particular combination of fields is invalid - like you patched first field with new value and not patched second yet. ced will warn you about this cases, also invalid table values won't be written

 

Command line options

you can run ced like

ced [options] path2cubin path2script

if patch2script omitted stdin will be used. Options are:

• -k - dump fields and values of instructions
• -v - verbose mode, like dump fields names etc
• -t - dump symbols
• -d - dump many useless stuff for debugging
  

Don't forget to make first backup of your cubin files

 

Limitations

The main one is that by design ced don't modify any ELF sections/symbols/relocs, so you can't enlarge/shrink some section, patch reloc/symbol offset etc. ced can only patch sass instruction inline

As consequence you cannot patch/remove instructions having offsets in EIATTRS like EXIT (has record in EIATTR_EXIT_INSTR_OFFSETS), redux/vote (must have record in EIATTR_INT_WARP_WIDE_INSTR_OFFSETS) etc. Unfortunately exhaustive list of such instructions

1. arch specific
2. unknown
   

so ced won't check this

Also it's bad idea to patch instructions having relocs - ced will warn you about this

 

You can't have label names in 'r' sass instruction like BRA `(label) - just because ced has no ideas about value of your "label"

 

Highly likely there are also many unknown so far

 

Building

ced was written on c++20 so you need appropriate c++ compiler - gcc-12+ or clang-10+. I don't know if it will work on non-linux systems

I tried to keep dependencies as small as possible - as result you need to clone from git only couple of libraries:

• ELFIO for ELF files parsing
• FP16 for working with fp16 imm values

For historical reason all source code contained in subdir test. First edit test/Makefile for ELFIO/FP16 headers directory and then just run
cd test
make 

Next you need to build smxx.so for your CUDA card - this involves translation from MD to c++ code with giant perl script so you also must install perl (I used 5.30 but probably any standard perl5 in your linux distro will be ok). I used Confess module for debugging - you can either install it from CPAN or just remove it from PERL_OPTS in test/Makefile

Final step - set and export env var SM_DIR to full path of directory with smxx.so shared libraries:

export SM_DIR=`pwd`

Happy hacking!