Profiling

Computational methods are often limited by the time a calculation requires, and different methods are often employed to reduce this time. Thus measuring the time your code takes to execute is often an important aspect of computationally intensive programs. The technical name for this is called code profiling, which extends beyond the total run time of your code to the run time and memory use of each function used in your code. Profiling is used to increase the speed of code by first understanding where the code is spending most of its time. Let’s look at three python packages that can help profile your code: time, timeit and cProfile.

time

The time package has a number of methods that deal with time. In can be used for example to print the local time if you wanted to add a time stamp to your plots. It is also an easy way to get the approximate run time of your program. The time.time() method returns the time since Jan. 1, 1970 in seconds. So if you call this at the start of your program and at the end then difference will be how long it took to run your code:

import time
start=time.time()
...my program...
end=time.time()
print(f"Program took :{end-start} seconds to run")

timeit

The timeit package basically does what we just did with time, but runs your code snippet many many times so that you get more reliable results. The default number of times it runs is 1,000,000, so timeit it usually used to test the time of one short section of code or a function, not a long program that does a lot of things. For example:
import timeit
print(timeit.timeit("5+5/3"))
#timeit only knows the code in the string, if you want to
#add something before that you can follow with another string
print(timeit.timeit("5+5/math.sqrt(3)","import math")
print(timeit.timeit('"Hello"+" "+"World"',number=10000000)

cProfile

The cProfiler package records information about all the routines in your code. It is really useful for large programs that may call functions many times. It does not return information in an easy to understand format, but other programs can be used to help interpret what it returns.
It can be run from the command line, or called in your program:

>python -m cProfile my_code.py

or

import cProfile
pr = cProfile.Profile()
pr.enable()
...my_code...
pr.disable()
pr.print_stats(sort='time')

There are a lot of other packages to help make sense of what cProfile records since it can be rather complex.

tqdm

Finally you may want to show the a status bar instead of measuring the time. The tqdm package can do this for you.

Print this page