gcc plugin to collect cross-references, part 9

Lets extract some useful results from my gcc plugin for collecting cross-references: 1, 2, 3, 4, 5, 6, 7 & 8. 

I've noticed that plugin worked unbearably slowly on big source files (like compiling itself) - something above 30 minutes, so I add profiling to it (see details here). Profiler showed that consumed user-time was only 18 seconds - this is glare sign that plugin spending most of time on some kind of lock. After some meditation (and strace/ltrace) I finally found that root of problem was in sqlite - it sleeping in fdatasync on each data writing to its DB! The cure is to execute
PRAGMA synchronous=OFF
this gives a non-illusory x100 speed up. Can I now consider myself as one of mythical "x100 programmer", he-he?

The next problem is how to store data from multiple gcc processes into single sqlite DB (like parallel make -j X). I've decided to use Apache Thrift - old and unfashionable but it really works and you can make multithreaded RPC server in 500 lines of c++ code. Server just listen on some port and stores all data from RPC clients into sqlite DB. Command line arguments path2db port_number .To shutdown RPC server use rpcq port_number. Of course you can make your own implementation of RPC server, for example with PHP & DB/2 (unfortunately Thrift does not supports Cobol)

As a consequence now plugin has 3 implementations for data storing:

1. in plain text files, self-test takes 14s
   
2. into sqlite, self-test takes 18s
3. in sqlite via RPC server listened on some arbitrary TCP port. Self-test takes 23s. Connection string looks like -fplugin-arg-gptest-db=localhost:port_number 
   

Btw sqlite version has size 2.67Mb, Thrift rpc client - 4.65Mb. Something is fundamentally wrong with "modern frameworks"

DB schema 

is very simple - in essence it consists of two tables:

symtab for storing symbols

• id - primary key
• mangled name of symbol/function or value of literal
• fname - for functions this is name of file where they were declared
• bcount - amount of basic blocks of function, can be used as some complexity metric

xrefs to link symbols and functions from which they are referred

• id of function
• bb - index of basic block
• arg - if non-zero - index of function's argument
• what - id of referred symbol
• kind - one letter type of xref:
  • 'c' - function call
  • 'v' - call of virtual method
  • 'l' - literal
  • 'r' - just reference to some global symbol
  • 'f' - field of some struct/class/union
  • 'F' - constant (with -fplugin-arg-gptest-ic option)
    

Fetching results

I wrote perl script to search and dump xrefs. If I want to find functions calling virtual methods of class FPersistence:

perl sgp.pl 1.db v FPersistence

4 is _ZN9my_PLUGIN10start_fileEPKc
 [0] FPersistence.cu_start
4 is _ZN9my_PLUGIN9stop_fileEv
 [0] FPersistence.cu_stop
5 is _ZL20callback_finish_unitPvS_
 [0] FPersistence.cu_stop
6 is _ZN9my_PLUGIN10pass_errorEPKcz
 [0] FPersistence.report_error
8 is _ZL19callback_start_unitPvS_
 [0] FPersistence.cu_start
16 is _ZN9my_PLUGIN7connectEv
 [0] FPersistence.connect
20 is _ZN9my_PLUGIND2Ev
 [0] FPersistence.__dt_del
102 is _ZN9my_PLUGIN8dump_rtxEPK7rtx_defi
 [0] FPersistence.add_xref
110 is _ZN9my_PLUGIN16dump_rtx_operandEPK7rtx_defcii
 [0] FPersistence.add_literal
 [0] FPersistence.add_xref
 [0] FPersistence.add_comment
 [0] FPersistence.add_ic
112 is _ZN9my_PLUGIN11dump_methodEPK9tree_node
 [0] FPersistence.add_xref
113 is _ZN9my_PLUGIN19add_fref_from_equalEi
 [0] FPersistence.add_xref
 [0] FPersistence.add_xref
180 is _ZN9my_PLUGIN13dump_comp_refEPK9tree_nodeR15aux_type_clutch
 [0] FPersistence.add_xref
223 is _ZN9my_PLUGIN7executeEP8function
 [0] FPersistence.bb_start
 [0] FPersistence.bb_stop
 [0] FPersistence.func_start
 [0] FPersistence.func_stop
 [0] FPersistence.func_proto
277 is _ZN9my_PLUGIN12dump_mem_refEPK9tree_nodeR15aux_type_clutch
 [0] FPersistence.add_xref
 [0] FPersistence.add_xref
 [0] FPersistence.add_xref
 [0] FPersistence.add_xref
 

Script's arguments:

1. path to sqlite DB
2. kind of xref
3. pattern of symbol to search for