Patching of cubin files is good, but loading and running them requires lots of code and using of Driver API. It would be much more convenient to patch SASS directly in binaries produced by nvcc
Unfortunately evil nvidia as usually shows it's paranoia:
- cuobjdump can list & extract content but not replace. Also it is extremely buggy on old libraries like libcublas.so v7
- official fatbinary is too complex and rebuilds whole file from scratch
- format of fatbinary is undocumented
- list files with -v option
- extract file at some index: -i idx -o output.filename
- replace file at some index: -i idx -r replace.filename
Perl binding
Being lazy I prefer to use perl scripts to automate as much as possible, so I also made perl XS module ELF::FatBinary. Having also module ELF::Reader this allows more fine filtering of ELF files - like if file contains section/symbol with some specific name etc. See simple example how it might look like
Limitation
The tool can replace files inside fatbinary only in-place, so
- compressed fatbinaries not supported
- size of files must be the same