16.3. Micro-benchmarking

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil -- Donald Knuth

../../_images/performance-optimization-knuth.jpg

16.3.1. Evaluation

  • Fresh start of Python process

  • Clean memory before start

  • Same data

  • Same start conditions, CPU load, RAM usage, iostat

  • Do not measure how long Python wakes up

  • Check what you measure

16.3.2. Timeit - Programmatic use

  • timeit

Code 16.35. Timeit simple statement
from timeit import timeit


setup = """name = 'José Jiménez'"""
stmt = """result = f'My name... {name}'"""

duration = timeit(stmt, setup, number=10000)

print(duration)
# 0.0005737080000000061
Code 16.36. Timeit multiple statements with setup code
from timeit import timeit


setup = """
firstname = 'José'
lastname = 'Jiménez'
"""

TEST = dict()
TEST[0] = 'name = f"{firstname} {lastname}"'
TEST[1] = 'name = "{0} {1}".format(firstname, lastname)'
TEST[2] = 'name = firstname + " " + lastname'
TEST[3] = 'name = " ".join([firstname, lastname])'


for stmt in TEST.values():
    duration = timeit(stmt, setup, number=10000)
    print(f'{duration:.5}\t{stmt}')

# 0.00071559    name = f"{firstname} {lastname}"
# 0.0026514     name = "{0} {1}".format(firstname, lastname)
# 0.001015      name = firstname + " " + lastname
# 0.0013494     name = " ".join([firstname, lastname])
Code 16.37. Timeit with globals()
from timeit import timeit


def factorial(n: int) -> int:
    if n == 0:
        return 1
    else:
        return n * factorial(n-1)


duration = timeit(
    stmt='factorial(500); factorial(400); factorial(450)',
    globals=globals(),
    number=10000,
)

duration = round(duration, 6)

print(f'factorial time: {duration} seconds')
# factorial time: 2.845382 seconds

16.3.3. Timeit - Console use

Code 16.38. Timeit
python3 -m timeit -n100000 -r100 --setup='name="Jose Jimenez"' 'output = f"My name... {name}"'
# 100000 loops, best of 100: 55.9 nsec per loop

python3 -m timeit -n100000 -r100 --setup='name="Jose Jimenez"' 'output = "My name... {name}".format(name=name)'
# 100000 loops, best of 100: 327 nsec per loop

python3 -m timeit -n100000 -r100 --setup='name="Jose Jimenez"' 'output = "My name... %s" % name'
# 100000 loops, best of 100: 124 nsec per loop

-n N, --number=N
how many times to execute 'statement'

-r N, --repeat=N
how many times to repeat the timer (default 5)

-s S, --setup=S
statement to be executed once initially (default pass)

-p, --process
measure process time, not wallclock time, using time.process_time() instead of time.perf_counter(), which is the default

-u, --unit=U
specify a time unit for timer output; can select nsec, usec, msec, or sec

-v, --verbose
print raw timing results; repeat for more digits precision

-h, --help
print a short usage message and exit

16.3.4. PyPerformance

  • pip install pyperformance

  • pyperformance run -b django_template - run django template benchmark

$ python3.10 -m venv venv-py310
$ venv-py310/bin/pip install pyperformance
$ venv-py310/bin/pyperformance run -b django_template

$ python3.11 -m venv venv-py311
$ venv-py311/bin/pip install pyperformance
$ venv-py311/bin/pyperformance run -b django_template

16.3.5. References