- This is a simple python script that can measure python program running time in fine levels.
- It’s simpler than a full profiler, easier to use than other currently available similar scripts.
I have been revisiting Python recently and walked through Google’s tutorial. Interestingly my first solution to one of the exercise word count took a lot of time to analysis a 600K text.
I wanted some profile tool to locate the bottle neck. There are lots of profiler tools but they seemed to be too “heavy weighted” for my simple purpose. This great Python profiler guide introduced a simple timing context manager script to measure script execution. You will need to
wrap blocks of code that you want to time with Python’s with keyword and this Timer context manager. It will take care of starting the timer when your code block begins execution and stopping the timer when your code block ends.
from timer import Timer
I used this method and found out the problem in my script is that I used
if key in dict.keys()
which seems natural since I saw this before:
for key in sorted(dict.keys()):
However, this is a linear look up in a list which didn’t utilize the constant time access of HashMap. The right way is to use dict directly without keys().
if key in dict
So the problem was solved but I still found the timing context manager not simple and intuitive enough. Adding timer means lots of edits and indents of original code, and moving checkpoints involves more editing and indenting.
Therefore, I wrote a simple script that have similar function of timing code blocks by checkpoints, also kept the usage effort minimal.
- import the script
times.start(digit)in the place that you want the timer start.
digitcontrol the digits after the decimal point, default at 7.
times.seg_stop("msg")enclose a code block and print the time of start and stop.
msgcan be used to identify the code block in the output.
- You can also add single checkpoint
times.last_seganywhere, it will print the time elapsed since last checkpoint which could be of any type.
Here is an example of using timing script in Wordcount. You can search and highlight
times to see all the edits needed in script.
I put some efforts in the print indenting to make the output aligned:
Here is the script. You can also download it from the link in right top corner of code block.
Of course sometimes you want more features like execution frequency and memory analysis, then you can always use more powerful profiler like line_profiler and memory profiler. The profiler guide I mentioned earlier have detailed introductions on them.
That being said, I still found my simple timing script is often useful enough and easy to use with minimal overhead.
- 2015-01-15 : The first version of this article on wordpress
- 2015-10-20 : Some edits when I moved my blog to github pages.
- 2016-05-10 : Added Summary.
- 2016-08-19 : Syntax highlighting for all code chunks