Use Python to time c++ binaries from the command line

Often I want to benchmark c++ code to try to understand if I’ve made things faster or slower. Though I don’t doubt there are possibly better and possibly more accurate measures out there, I have yet to find a straightforward and clear answer to this question. But this may be trivially easy in Python, using timeit.

This assumes you have Python installed (of some compatible version, this is only tested on 2.7). Assuming your c++ binary name is called “a.out” and is located in the directory you’re currently in, you can run:

$ python -m timeit -n 1 'import subprocess; subprocess.call("./a.out", shell=True)'

What I know:
0. Respect the quotes.
1. The option “-n 1” tells you the number of times to run the function. n=1 appears to run the function 3 times and report the best time of those 3.
2. The python function we’re using here is in the module subprocess and is called “call”.

A few real notes:
0. Note that this function will not purely benchmark your code, as there is technically some overhead with import subprocess and running subprocess.call. If you care about this, then you’ll need another solution. You might be able to get a sense of the overhead on your machine by running:

$ python -m timeit -n 1 'import subprocess; subprocess.call("", shell=True)'

This runs in 2.5 ms on my laptop. You could assume linearity here and just subtract off this time, if you don’t mind the back-of-the-envelopeness of this.

1. There might be a hundred caveats on this, with respect to subprocess and shell=True, which are details I have yet to fully figure out. But again this is a qnd benchmark, good enough for my uses, and possibly useful to others.

Leave a comment