4.1. Profiling CPU

A profile is a set of statistics that describes how often and for how long various parts of the program executed
The profiler modules are designed to provide an execution profile for a given program, not for benchmarking purposes (for that, there is timeit for reasonably accurate results). This particularly applies to benchmarking Python code against C code: the profilers introduce overhead for Python code, but not for C-level functions, and so the C code would seem faster than any Python one.
cProfile (CPython built-in)
yappi https://github.com/sumerc/yappi
PyCharm IDE
vmprof https://vmprof.readthedocs.io/en/latest/

4.1.1. Tracing vs Sampling Profilers

Tracing profilers record every function call and return, which can be very slow and produce large amounts of data.
Sampling profilers take periodic samples of the program's state, which is less intrusive and produces smaller amounts of data.
https://docs.python.org/dev/whatsnew/3.14.html#pep-768-safe-external-debugger-interface-for-cpython
https://docs.python.org/dev/howto/perf_profiling.html#perf-profiling
https://docs.python.org/dev/library/sys.monitoring.html#module-sys.monitoring
https://peps.python.org/pep-0799/
https://creators.spotify.com/pod/profile/corepy/
https://creators.spotify.com/pod/profile/corepy/episodes/The-Megahertz-e35ffoi

4.1.2. PyCharm Profiler

PyCharm Profiler is a built-in profiler in PyCharm IDE.
python -m pip install yappi

../../_images/performance-profiler-factorial.png — Figure 4.1. PyCharm Profiler Factorial [2]

../../_images/performance-profiler-fibonacci.png — Figure 4.2. PyCharm Profiler Fibonacci [1]

../../_images/performance-profiler-pycharm.png — Figure 4.3. PyCharm Profiler [3]

4.1.3. profile

Python's built-in profiler
Written in Python
Only for educational purposes

4.1.4. cProfile

Python's built-in profiler
Written in C

Table 4.2. cProfile
Name	Description
ncalls	for the number of calls
tottime	for the total time spent in the given function (and excluding time made in calls to sub-functions)
percall	is the quotient of tottime divided by ncalls
cumtime	is the cumulative time spent in this and all subfunctions (from invocation till exit)
percall	is the quotient of cumtime divided by primitive calls
filename:lineno(function)	provides the respective data of each function

Table 4.3. cProfile
Name	Description
calls	call count
cumulative	cumulative time
cumtime	cumulative time
file	file name
filename	file name
module	file name
ncalls	call count
pcalls	primitive call count
line	line number
name	function name
nfl	name/file/line
stdname	standard name
time	internal time
tottime	internal time

$ python -m cProfile [-o output_file] [-s sort_order] FILE.py

$ python -m cProfile myfile.py

 $ python -m cProfile -s cumtime passwd.py
function calls (948 primitive calls) in 0.002 seconds

Ordered by: cumulative time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   2/1    0.000    0.000    0.002    0.002 {built-in method builtins.exec}
  0.000    0.000    0.002    0.002 passwd.py:1(<module>)
   2/1    0.000    0.000    0.002    0.002 <frozen importlib._bootstrap>:1349(_find_and_load)
   2/1    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap>:1304(_find_and_load_unlocked)
   2/1    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap>:911(_load_unlocked)
  0.000    0.000    0.001    0.001 <frozen importlib._bootstrap_external>:1020(exec_module)
   5/2    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap>:480(_call_with_frames_removed)
  0.000    0.000    0.001    0.001 datetime.py:1(<module>)
  0.000    0.000    0.001    0.001 <frozen importlib._bootstrap>:806(module_from_spec)
  0.000    0.000    0.001    0.001 <frozen importlib._bootstrap_external>:1318(create_module)
  0.001    0.001    0.001    0.001 {built-in method _imp.create_dynamic}
  0.000    0.000    0.000    0.000 passwd.py:857(main)
  0.000    0.000    0.000    0.000 passwd.py:528(read)
  0.000    0.000    0.000    0.000 {method 'read' of '_io.TextIOWrapper' objects}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:1240(_find_spec)
  0.000    0.000    0.000    0.000 {built-in method _io.open}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1551(find_spec)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1522(_get_spec)
  0.000    0.000    0.000    0.000 {method '__exit__' of '_io._IOBase' objects}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1624(find_spec)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1093(get_code)
  0.000    0.000    0.000    0.000 passwd.py:738(asshadow)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1326(exec_module)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1214(get_data)
  0.000    0.000    0.000    0.000 {built-in method _imp.exec_dynamic}
  0.000    0.000    0.000    0.000 passwd.py:544(asrecords)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1500(_path_importer_cache)
  0.000    0.000    0.000    0.000 {built-in method posix.getcwd}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:145(_path_stat)
  0.000    0.000    0.000    0.000 {built-in method _io.open_code}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:416(__enter__)
  0.000    0.000    0.000    0.000 {built-in method posix.stat}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:131(_path_join)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:304(acquire)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:733(_init_module_attrs)
  0.000    0.000    0.000    0.000 passwd.py:636(aspasswd)
  0.000    0.000    0.000    0.000 passwd.py:479(asint)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:164(_path_isfile)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1619(_get_spec)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:155(_path_is_mode_type)
  0.000    0.000    0.000    0.000 passwd.py:491(asdate)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:632(cached)
  0.000    0.000    0.000    0.000 passwd.py:699(asgroups)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:513(cache_from_source)
  0.000    0.000    0.000    0.000 {method 'split' of 'str' objects}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:642(_get_cached)
  0.000    0.000    0.000    0.000 passwd.py:600(aspassword)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:162(__enter__)
  0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
  0.000    0.000    0.000    0.000 passwd.py:561(asmembers)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:426(_get_module_lock)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:124(setdefault)
  0.000    0.000    0.000    0.000 __init__.py:101(find_spec)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:833(spec_from_file_location)
  0.000    0.000    0.000    0.000 {method 'rstrip' of 'str' objects}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:782(_compile_bytecode)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:420(__exit__)
  0.000    0.000    0.000    0.000 <frozen codecs>:322(decode)
  0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}
  0.000    0.000    0.000    0.000 passwd.py:463(asstr)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:372(release)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:137(_path_split)
  0.000    0.000    0.000    0.000 passwd.py:586(asaccounts)
  0.000    0.000    0.000    0.000 {method 'get' of 'dict' objects}
  0.000    0.000    0.000    0.000 {built-in method marshal.loads}
  0.000    0.000    0.000    0.000 {method 'read' of '_io.BufferedReader' objects}
  0.000    0.000    0.000    0.000 {method 'startswith' of 'str' objects}
  0.000    0.000    0.000    0.000 {method 'splitlines' of 'str' objects}
  0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
  0.000    0.000    0.000    0.000 {method 'isspace' of 'str' objects}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:74(__new__)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:697(_classify_pyc)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:491(_verbose_message)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:1226(__exit__)
  0.000    0.000    0.000    0.000 {built-in method builtins.max}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:89(_unpack_uint32)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:1222(__enter__)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1233(path_stats)
  0.000    0.000    0.000    0.000 passwd.py:514(aslist)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:1128(find_spec)
  0.000    0.000    0.000    0.000 {built-in method builtins.getattr}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:82(remove)
  0.000    0.000    0.000    0.000 <frozen codecs>:312(__init__)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:232(__init__)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:445(cb)
  0.000    0.000    0.000    0.000 {method 'rpartition' of 'str' objects}
  0.000    0.000    0.000    0.000 {built-in method _codecs.utf_8_decode}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:730(_validate_timestamp_pyc)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:190(_path_abspath)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:982(find_spec)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:139(<genexpr>)
  0.000    0.000    0.000    0.000 {built-in method _imp.acquire_lock}
  0.000    0.000    0.000    0.000 {built-in method builtins.locals}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:79(__init__)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:173(__exit__)
  0.000    0.000    0.000    0.000 {built-in method _imp.release_lock}
  0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:185(_path_isabs)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:645(parent)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:67(_relax_case)
  0.000    0.000    0.000    0.000 {built-in method builtins.len}
  0.000    0.000    0.000    0.000 {method 'format' of 'str' objects}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:674(_check_name_wrapper)
  0.000    0.000    0.000    0.000 {built-in method _imp.is_builtin}
  0.000    0.000    0.000    0.000 {method 'write' of '_io.TextIOWrapper' objects}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:48(_new_module)
  0.000    0.000    0.000    0.000 {built-in method _imp.find_frozen}
  0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
  0.000    0.000    0.000    0.000 {method '__exit__' of '_thread.RLock' objects}
  0.000    0.000    0.000    0.000 <frozen codecs>:189(__init__)
  0.000    0.000    0.000    0.000 {method 'remove' of 'list' objects}
  0.000    0.000    0.000    0.000 {method 'endswith' of 'str' objects}
  0.000    0.000    0.000    0.000 {method 'pop' of 'dict' objects}
  0.000    0.000    0.000    0.000 {built-in method _thread.get_ident}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:599(__init__)
  0.000    0.000    0.000    0.000 {built-in method from_bytes}
  0.000    0.000    0.000    0.000 {built-in method __new__ of type object at 0x103c87cf0}
  0.000    0.000    0.000    0.000 {built-in method posix.fspath}
  0.000    0.000    0.000    0.000 {method 'rfind' of 'str' objects}
  0.000    0.000    0.000    0.000 _virtualenv.py:52(find_spec)
  0.000    0.000    0.000    0.000 <frozen codecs>:263(__init__)
  0.000    0.000    0.000    0.000 {method 'pop' of 'list' objects}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:653(has_location)
  0.000    0.000    0.000    0.000 {built-in method _thread.allocate_lock}
  0.000    0.000    0.000    0.000 {built-in method _weakref._remove_dead_weakref}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:158(__init__)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:412(__init__)
  0.000    0.000    0.000    0.000 __init__.py:108(<lambda>)
  0.000    0.000    0.000    0.000 {built-in method _imp._fix_co_filename}
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1184(__init__)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1307(__init__)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1209(get_filename)
  0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1017(create_module)

>>>
... import cProfile
...
... cProfile.run('import re; re.compile("foo|bar")')
       216 function calls (209 primitive calls) in 0.000 seconds
Ordered by: standard name
ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  0.000    0.000    0.000    0.000 <string>:1(<module>)
  0.000    0.000    0.000    0.000 enum.py:284(__call__)
  0.000    0.000    0.000    0.000 enum.py:526(__new__)
  0.000    0.000    0.000    0.000 enum.py:836(__and__)
  0.000    0.000    0.000    0.000 pydev_import_hook.py:16(do_import)
  0.000    0.000    0.000    0.000 re.py:232(compile)
  0.000    0.000    0.000    0.000 re.py:271(_compile)
  0.000    0.000    0.000    0.000 sre_compile.py:249(_compile_charset)
  0.000    0.000    0.000    0.000 sre_compile.py:276(_optimize_charset)
  0.000    0.000    0.000    0.000 sre_compile.py:453(_get_iscased)
  0.000    0.000    0.000    0.000 sre_compile.py:461(_get_literal_prefix)
  0.000    0.000    0.000    0.000 sre_compile.py:492(_get_charset_prefix)
  0.000    0.000    0.000    0.000 sre_compile.py:536(_compile_info)
  0.000    0.000    0.000    0.000 sre_compile.py:595(isstring)
  0.000    0.000    0.000    0.000 sre_compile.py:598(_code)
   3/1    0.000    0.000    0.000    0.000 sre_compile.py:71(_compile)
  0.000    0.000    0.000    0.000 sre_compile.py:759(compile)
  0.000    0.000    0.000    0.000 sre_parse.py:111(__init__)
  0.000    0.000    0.000    0.000 sre_parse.py:160(__len__)
  0.000    0.000    0.000    0.000 sre_parse.py:164(__getitem__)
  0.000    0.000    0.000    0.000 sre_parse.py:172(append)
   3/1    0.000    0.000    0.000    0.000 sre_parse.py:174(getwidth)
  0.000    0.000    0.000    0.000 sre_parse.py:224(__init__)
  0.000    0.000    0.000    0.000 sre_parse.py:233(__next)
  0.000    0.000    0.000    0.000 sre_parse.py:249(match)
  0.000    0.000    0.000    0.000 sre_parse.py:254(get)
  0.000    0.000    0.000    0.000 sre_parse.py:286(tell)
  0.000    0.000    0.000    0.000 sre_parse.py:417(_parse_sub)
  0.000    0.000    0.000    0.000 sre_parse.py:475(_parse)
  0.000    0.000    0.000    0.000 sre_parse.py:76(__init__)
  0.000    0.000    0.000    0.000 sre_parse.py:81(groups)
  0.000    0.000    0.000    0.000 sre_parse.py:903(fix_flags)
  0.000    0.000    0.000    0.000 sre_parse.py:919(parse)
  0.000    0.000    0.000    0.000 {built-in method _sre.compile}
  0.000    0.000    0.000    0.000 {built-in method builtins.__import__}
  0.000    0.000    0.000    0.000 {built-in method builtins.exec}
  0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
     29/26    0.000    0.000    0.000    0.000 {built-in method builtins.len}
  0.000    0.000    0.000    0.000 {built-in method builtins.max}
  0.000    0.000    0.000    0.000 {built-in method builtins.min}
  0.000    0.000    0.000    0.000 {built-in method builtins.ord}
  0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
  0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
  0.000    0.000    0.000    0.000 {method 'find' of 'bytearray' objects}
  0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}

4.1.5. perf

A powerful Python profiler that can be used to profile Python code and C extensions.
Linux and macOS only.

$ PYTHONPERFSUPPORT=1 perf record -F 9999 -g -o perf.data python my_script.py
$ perf report -g -i perf.data

$ perf record -F 9999 -g -o perf.data python -X perf my_script.py
$ perf report -g -i perf.data

or:

import sys

sys.activate_stack_trampoline("perf")
do_profiled_stuff()
sys.deactivate_stack_trampoline()

non_profiled_stuff()

then:

$ perf record -F 9999 -g -o perf.data python ./example.py
$ perf report -g -i perf.data

or with JIT:

$ perf record -F 9999 -g -k 1 --call-graph dwarf -o perf.data python -Xperf_jit my_script.py
$ perf inject -i perf.data --jit --output perf.jit.data
$ perf report -g -i perf.jit.data

or with JIT:

$ PYTHON_PERF_JIT_SUPPORT=1 perf record -F 9999 -g --call-graph dwarf -o perf.data python my_script.py
$ perf inject -i perf.data --jit --output perf.jit.data
$ perf report -g -i perf.jit.data

4.1.6. Samply

Command-line sampling profiler for macOS, Linux, and Windows
Samply is a sampling profiler for Python that can be used to profile Python code and C extensions.
It is designed to be fast and efficient, making it suitable for profiling large applications.
https://github.com/mstange/samply

$ python -m pip install samply
$ python -m samply record PYTHONPERFSUPPORT=1 python my_script.py

4.1.7. Yappi

Yappi is a high-performance statistical profiler for Python. It is well-suited for profiling multi-threaded applications and handling partial loads of a program.
https://github.com/sumerc/yappi
https://pypi.org/project/yappi/
pip install yappi
python -m yappi myfile.py
yappi.set_clock_type("cpu") # Use set_clock_type("wall") for wall time
yappi.start()
yappi.get_func_stats().print_all()
yappi.get_thread_stats().print_all()

$ python -m pip install yappi

$ python -m yappi myfile.py

>>>
... import yappi
...
... def a():
...     for _ in range(1_000):  # do something CPU heavy
...         pass
...
... yappi.set_clock_type("cpu") # Use set_clock_type("wall") for wall time
... yappi.start()
... a()
...
... yappi.get_func_stats().print_all()
... yappi.get_thread_stats().print_all()
Clock type: CPU
Ordered by: totaltime, desc

name ncall tsub ttot tavg doc.py:5 a 1 0.117907 0.117907 0.117907

name id tid ttot scnt _MainThread 0 139867147315008 0.118297 1

>>>
... import yappi
... import time
... import threading
...
... _NTHREAD = 3
...
...
... def _work(n):
...     time.sleep(n * 0.1)
...
...
... yappi.start()
...
... threads = []
... # generate _NTHREAD threads
... for i in range(_NTHREAD):
...     t = threading.Thread(target=_work, args=(i + 1, ))
...     t.start()
...     threads.append(t)
... # wait all threads to finish
... for t in threads:
...     t.join()
...
... yappi.stop()
...
... # retrieve thread stats by their thread id (given by yappi)
... threads = yappi.get_thread_stats()
... for thread in threads:
...     print("Function stats for (%s) (%d)" % (thread.name, thread.id)
...     )  # it is the Thread.__class__.__name__
...     yappi.get_func_stats(ctx_id=thread.id).print_all()
Function stats for (Thread) (3)

name ncall tsub ttot tavg ..hon3.7/threading.py:859 Thread.run 1 0.000017 0.000062 0.000062 doc3.py:8 _work 1 0.000012 0.000045 0.000045

Function stats for (Thread) (2)

name ncall tsub ttot tavg ..hon3.7/threading.py:859 Thread.run 1 0.000017 0.000065 0.000065 doc3.py:8 _work 1 0.000010 0.000048 0.000048

Function stats for (Thread) (1)

name ncall tsub ttot tavg ..hon3.7/threading.py:859 Thread.run 1 0.000010 0.000043 0.000043 doc3.py:8 _work 1 0.000006 0.000033 0.000033

Async application:

>>>
... import asyncio
... import yappi
...
... async def foo():
...     await asyncio.sleep(1.0)
...     await baz()
...     await asyncio.sleep(0.5)
...
... async def bar():
...     await asyncio.sleep(2.0)
...
... async def baz():
...     await asyncio.sleep(1.0)
...
...
... yappi.set_clock_type("WALL")
...
... with yappi.run():
...     asyncio.run(foo())
...     asyncio.run(bar())
...
... yappi.get_func_stats().print_all()
Clock type: WALL
Ordered by: totaltime, desc

name ncall tsub ttot tavg doc4.py:5 foo 1 0.000030 2.503808 2.503808 doc4.py:11 bar 1 0.000012 2.002492 2.002492 doc4.py:15 baz 1 0.000013 1.001397 1.001397

4.1.8. Line Profiler

$ python -m pip install line_profiler
$ python -m kernprof -l myfile.py
$ python -m line_profiler -rmt myfile.py.lprof

4.1.9. References

4.1.10. Assignments

# %% About
# - Name: File Read Passwd
# - Difficulty: hard
# - Lines: 77
# - Minutes: 89

# %% License
# - Copyright 2025, Matt Harasymczuk <matt@python3.info>
# - This code can be used only for learning by humans
# - This code cannot be used for teaching others
# - This code cannot be used for teaching LLMs and AI algorithms
# - This code cannot be used in commercial or proprietary products
# - This code cannot be distributed in any form
# - This code cannot be changed in any form outside of training course
# - This code cannot have its license changed
# - If you use this code in your product, you must open-source it under GPLv2
# - Exception can be granted only by the author

# %% English
# 0. Commit and push your current state of repository
# 1. In the current directory, the following files were saved:
#    - `FILE_PASSWD` with content of `etc-passwd.txt` file
#    - `FILE_SHADOW` with content of `etc-shadow.txt` file
#    - `FILE_GROUP` with content of `etc-group`.txt file
# 2. Define variable `result: list[dict]` with user data,
#    whose `UID` is greater or equal to 1000, dict keys are:
#    - groups_names: list[str]
#    - passwd_comment: str
#    - passwd_gid: int
#    - passwd_home: str
#    - passwd_login: str
#    - passwd_password: str
#    - passwd_shell: str
#    - passwd_uid: int
#    - shadow_algorithm: str | None
#    - shadow_expiration: date | None
#    - shadow_hash: str | None
#    - shadow_inactive: int | None
#    - shadow_last_change: date | None
#    - shadow_locked: bool
#    - shadow_max_age: int | None
#    - shadow_min_age: int | None
#    - shadow_password: str
#    - shadow_reserved: str
#    - shadow_salt: str | None
#    - shadow_warning: int | None
# 3. Run doctests - all must succeed

# %% Polish
# 0. Zakomituj i pusznij (commit i push) obecny stan repozytorium
# 1. W obecnym katalogu zapisano następujące pliki:
#    - `FILE_PASSWD` z zawartością pliku `etc-passwd.txt`
#    - `FILE_SHADOW` z zawartością pliku `etc-shadow.txt`
#    - `FILE_GROUP` z zawartością pliku `etc-group.txt`
# 2. Zdefiniuj zmienną `result: list[dict]` z danymi użytkowników,
#    których `UID` jest większy lub równy niż 1000, klucze to:
#    - groups_names: list[str]
#    - passwd_comment: str
#    - passwd_gid: int
#    - passwd_home: str
#    - passwd_login: str
#    - passwd_password: str
#    - passwd_shell: str
#    - passwd_uid: int
#    - shadow_algorithm: str | None
#    - shadow_expiration: date | None
#    - shadow_hash: str | None
#    - shadow_inactive: int | None
#    - shadow_last_change: date | None
#    - shadow_locked: bool
#    - shadow_max_age: int | None
#    - shadow_min_age: int | None
#    - shadow_password: str
#    - shadow_reserved: str
#    - shadow_salt: str | None
#    - shadow_warning: int | None
# 3. Uruchom doctesty - wszystkie muszą się powieść

# %% References
# [1] passwd(5) - File Formats Manual
#     Year: 2017
#     Retrieved: 2025-02-14
#     URL: https://manpages.debian.org/unstable/passwd/passwd.5.en.html
#
# [2] shadow(5) - File Formats Manual
#     Year: 2017
#     Retrieved: 2025-02-14
#     URL: https://manpages.debian.org/unstable/passwd/shadow.5.en.html
#
# [3] crypt(5) - File Formats Manual
#     Year: 2017
#     Retrieved: 2025-02-14
#     URL: https://manpages.debian.org/unstable/libcrypt-dev/crypt.5.en.html

# %% Doctests
"""
>>> import sys; sys.tracebacklimit = 0
>>> assert sys.version_info >= (3, 9), \
'Python 3.9+ required'

>>> from os import remove

>>> remove(FILE_PASSWD)
>>> remove(FILE_SHADOW)
>>> remove(FILE_GROUP)
"""

# %% Run
# - PyCharm: right-click in the editor and `Run Doctest in ...`
# - PyCharm: keyboard shortcut `Control + Shift + F10`
# - Terminal: `python -m doctest -f -v myfile.py`

# %% Imports
from datetime import date, timedelta

# %% Types
result: list[dict[str, str|int|bool]]

# %% Data
FILE_PASSWD = 'etc_passwd.txt'
FILE_SHADOW = 'etc_shadow.txt'
FILE_GROUP = 'etc_group.txt'

ALGORITHMS = {
    '1': 'md5crypt',
    '2a': 'bcrypt (Blowfish)',
    '2b': 'bcrypt (Blowfish)',
    '2x': 'bcrypt (Blowfish)',
    '2y': 'bcrypt (Blowfish)',
    '3': 'NTHASH',
    '5': 'sha256crypt',
    '6': 'sha512crypt',
    '7': 'scrypt',
    'y': 'yescrypt',
    '8': 'PBKDF2',
    'gy': 'gost-yescrypt',
    'md5': 'Solaris MD5',
    'sha1': 'PBKDF1 with SHA-1',
    '': 'DES',
}

with open(FILE_PASSWD, mode='wt', encoding='utf-8') as file:
    file.write("""# File: /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
alice:x:1000:1000:Alice:/home/alice:/bin/bash
bob:x:1001:1001:Bob:/home/bob:/bin/bash
carol:x:1002:1002:Carol:/home/carol:/bin/bash
dave:x:1003:1003:Dave:/home/dave:/bin/bash
eve:x:1004:1004:Eve:/home/eve:/bin/bash
mallory:x:1005:1005:Mallory:/home/mallory:/bin/bash
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
""")

with open(FILE_SHADOW, mode='wt', encoding='utf-8') as file:
    file.write("""# File: /etc/shadow
root:*:10957:0:99999:7:::
daemon:*:10957:0:99999:7:::
bin:*:10957:0:99999:7:::
sys:*:10957:0:99999:7:::
alice:$6$5H0QpwprRiJQR19Y$bXGOh7dIfOWpUb/Tuqr7yQVCqL3UkrJns9.7msfvMg4ZO/PsFC5Tbt32PXAw9qRFEBs1254aLimFeNM8YsYOv.:10957:0:99999:7:::
bob:$6$P9zn0KwR$tgfvvFWJJ5FKmoXiP5rXWOjwoEBOEoAuBi3EphRbJqqjWYvhEM2wa67L9XgQ7W591FxUNklkDIQsk4kijuhE50:10957:0:99999:7:::
carol:!$1$.QKDPc5E$SWlkjRWexrXYgc98F.:10957:0:99999:7:::
dave:!$6$wXtY9ZoG$MxaKfj3Z8F9G8wKz7LU0:10957:0:99999:7:::
eve:!!:10957:0:90:5:30:10988:
mallory: :::::::
nobody:*:10957:0:99999:7:::
""")

with open(FILE_GROUP, mode='wt', encoding='utf-8') as file:
    file.write("""# File: /etc/group
root::0:root
daemon::1:root,daemon
bin::2:root,bin,daemon
sys::3:root,bin,sys,adm
users::1001:alice,bob,carol,dave,eve
staff::1002:alice,bob,eve
admins::1003:eve
nogroup::65534:
""")

# %% Result
result = ...