I asked 7 month ago in r/llvm question how I can insert PTX asm right in LLVM IR and got exactly zero answers. So, as usual, I had to figure it out on my own (depressing little song "No Help is Coming" is playing in the background)
How inline PTX looks like in text form:%7 = call i32 asm sideeffect "madc.hi.cc.u32 $0,$1,$2,$3;", "=r,r,r,r"(i32 %.sroa.018.0.extract.trunc, i32 %.sroa.282.0.extract.trunc, i32 0) #5, !srcloc !9
So basically it is just call result-type asm with some arguments in parentheses (note that type of result $0 is i32 and it described after keyword call). If you need result of PTX instruction - just assign it to some variable. Official documentation says that #5 is attributes list - somewhere below it defined asattributes #5 = { nounwind }
and !9 is metadata - is this case for debug info srcloc:!9 = !{i32 46731}
Well, that was easy part of story - and now Something Completely Different (tm). LLVM IR is strictly typed (I would say - redundantly strictly), so types first time defined for each argument - like i32 for $1, $2 and $3. Second time - in string called operand constraint codes - in my case it is "=r,r,r,r". And official documentation blatantly lies about them. Let's check some source code - method getRegForInlineAsmConstraint in NVPTXISelLowering.cpp. As you can see it accepts following codes:
- b - 1bit, predicates
- c & h - 16bit, like (.b16 / .u16 / .s16)
- r & f - 32bit, like (.b32 / .u32 / .s32) and .f32 for f
- l, N, d - 64bit, (.b64 / .u64 / .s64) & .f64 for d
- q - 128bit since sm70+
- 0 - meaning is still unknown
Symbol '=' is so called Constraint Modifier:
- = Write-only output operand (overwrites previous contents)
- + Read-write operand (input and output tied to the same register)
- & Early-clobber operand (modified before inputs are consumed)
- ~ Clobber list marker (tells LLVM a register or memory/flags are modified implicitly
call i32 asm sideeffect "madc.hi.cc.u32 $0,$2,$1,$3;", "=r,r,r,r"(i32 %.sroa.282.0.extract.trunc, i32 %.sroa.018.0.extract.trunc, i32 0)This makes the task of parsing & comparison of PTX instructions non-trivial - especially in complex cases like%1 = call { i32, i32, i32, i32 } asm sideeffect "tex.grad.1d.v4.u32.f32 {$0, $1, $2, $3}, [$4, {$5}], {$6}, {$7};", "=r,=r,=r,=r,l,f,f,f"(i64 %tmp5, float %tmp6, float %tmp7, float %tmp8)
PTX from cicc
Once you understand how inline PTX is represented in LLVM IR, the next step is examining how nvidia’s own internal toolchain leverages it.
While doing some RE of nvidia's llvm-based back-end I dumped inline PTX instructions. Now when I have PTX parser the next logic step is try to parse PTX from cicc and for example try find some undocumented instruction/attributes (which nvidia uses for unfair competitive advantage). So I added to my parser option -r to dump instructions with unrecognized attributes, and also wrote little perl script to collect them. Then run whole pipe like
../ptx.parse/tp -r < ptx.txt | perl ../ptx.parse/ra.pl
And try to guess what happened? Yes - nvidia uses ~5-7% of instructions with undocumented attributes
