13.3. Generator Yield

• yield keyword turns function into generator function

Generators can return (or yield) something:

>>> def run():
...     yield 1

>>> def run():
...     yield 'something'

Generators can be defined with required and optional parameters just like a regular function:

>>> def run(a, b, c=0):
...     yield a + b + c

13.3.1. Problem

• Python do not execute code after function returns

• In order to return two values from the function you must use tuple

>>> def run():
...     return 1
>>>
>>>
>>> result = run()
>>> print(result)
1

>>> def run():
...     return 1
...     return 2  # this will not execute
>>>
>>>
>>> result = run()
>>> print(result)
1

>>> def run():
...     return 1, 2
>>>
>>>
>>> result = run()
>>> print(result)
(1, 2)

13.3.2. Solution

>>> def run():
...     yield 1
...     yield 2
>>>
>>>
>>> result = run()
>>> tuple(result)
(1, 2)

13.3.3. Call Generator

• Generators are called just like a regular function

• The rule with positional and keyword arguments are identical to regular functions

Generators are called just like a regular function:

>>> def run():
...     yield 1
>>>
>>>
>>> result = run()

The rule with positional and keyword arguments are identical to regular functions:

>>> def run(a, b, c=0):
...     yield a + b + c
>>>
>>>
>>> result = run(1, b=2)

13.3.4. Result Object

• Calling a generator will return a generator object

• This object is an iterator

One does not simple get data from generator:

>>> def run():
...     yield 1
>>>
>>>
>>> result = run()
>>>
>>> print(result)  
<generator object run at 0x...>

13.3.5. Evaluate Eagerly

>>> def run():
...     yield 1
...     yield 2
...     yield 3
>>>
>>> result = run()
>>>
>>> tuple(result)
(1, 2, 3)

13.3.6. Evaluate Lazily

• All generators implements Iterator protocol

• Iterator has obj.__iter__() method which enable use of iter(obj)

• Iterator has obj.__next__() method which enable use of next(obj)

In Order to do so, you need to generate next item using next()

>>> def run():
...     yield 1
...     yield 2
...     yield 3
>>>
>>>
>>> result = run()
>>>
>>> next(result)
1
>>>
>>> next(result)
2
>>>
>>> next(result)
3
>>>
>>> next(result)
Traceback (most recent call last):
StopIteration

13.3.7. Evaluate Loop

>>> def run():
...     yield 1
...     yield 2
...     yield 3
>>>
>>> result = run()
>>>
>>> for x in result:
...     print(x)
...
1
2
3

13.3.8. Evaluate Iterator

>>> def run():
...     yield 1
...     yield 2
...     yield 3
>>>
>>> result = run()
>>>
>>> it = iter(result)  # result.__iter__()
>>>
>>> try:
...     x = next(it)   # result.__next__()
...     print(x)
...
...     x = next(it)   # result.__next__()
...     print(x)
...
...     x = next(it)   # result.__next__()
...     print(x)
...
...     x = next(it)   # result.__next__()
...     print(x)
... except StopIteration:
...     pass
...
1
2
3

13.3.9. Execution

• There can be one or many yield keywords in a generator function

• Each yield keyword pauses the function and returns the value

• After last yield raises StopIteration

Execute code before and after yield:

>>> def run():
...     print('one')
...     print('two')
...     yield 1
...     print('three')
...     print('four')
...     yield 2

>>> result = run()

>>> next(result)
one
two
1

>>> next(result)
three
four
2

>>> next(result)
Traceback (most recent call last):
StopIteration

13.3.10. Yield from a Loop

>>> def run():
...     for x in range(0,3):
...         yield x
>>>
>>>
>>> result = run()
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
Traceback (most recent call last):
StopIteration

13.3.11. Yields from Loops

>>> def run():
...     for x in range(0, 3):
...         yield x
...     for y in range(10, 13):
...         yield y
>>>
>>>
>>> result = run()
>>>
>>> type(result)
<class 'generator'>
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
10
>>> next(result)
11
>>> next(result)
12
>>> next(result)
Traceback (most recent call last):
StopIteration

13.3.12. Yield in a Zip Loop

• firstnames() - generator (a stream) returning firstnames

• lastnames() - generator (a stream) returning lastnames

• for fname, lname in zip(firstnames(), lastnames())

>>> def firstnames():
...     yield 'Mark'
...     yield 'Melissa'
...     yield 'Rick'
>>>
>>>
>>> def lastnames():
...     yield 'Watney'
...     yield 'Lewis'
...     yield 'Martinez'
>>>
>>>
>>> for fname, lname in zip(firstnames(), lastnames()):
...     print(f'{fname=}, {lname=}')
fname='Mark', lname='Watney'
fname='Melissa', lname='Lewis'
fname='Rick', lname='Martinez'

13.3.13. Case Study

• range() implementation using function and generator

>>> def myrange(start, stop, step):
...     result = []
...     current = start
...     while current < stop:
...         result.append(current)
...         current += step
...     return result
>>>
>>> result = myrange(0, 10, 2)
>>> tuple(result)
(0, 2, 4, 6, 8)

>>> def myrange(start, stop, step):
...     current = start
...     while current < stop:
...         yield current
...         current += step
>>>
>>> result = myrange(0, 10, 2)
>>> tuple(result)
(0, 2, 4, 6, 8)

13.3.14. Use Case - 1

Function:

>>> def even(data):
...     result = []
...     for x in data:
...         if x % 2 == 0:
...             result.append(x)
...     return result
>>>
>>>
>>> result = even(range(0,10))
>>>
>>> print(result)
[0, 2, 4, 6, 8]

Generator:

>>> def even(data):
...     for x in data:
...         if x % 2 == 0:
...             yield x
>>>
>>>
>>> result = even(range(0,10))
>>>
>>> print(result)  
<generator object even at 0x...>
>>>
>>> list(result)
[0, 2, 4, 6, 8]

13.3.15. Use Case - 2

>>> class User:
...     def __init__(self, firstname, lastname):
...         self.firstname = firstname
...         self.lastname = lastname
...
...     def __str__(self):
...         return f'{self.firstname} {self.lastname}'
>>>
>>>
>>> class Order:
...     def __init__(self, product, price):
...         self.product = product
...         self.price = price
...
...     def __str__(self):
...         return f'product {self.product} for {self.price} USD'

>>> def users():
...     yield User('Mark', 'Watney')
...     yield User('Melissa', 'Lewis')
...     yield User('Rick', 'Martinez')
>>>
>>>
>>> def orders():
...     yield Order(product='A', price=123.45)
...     yield Order(product='B', price=123.45)
...     yield Order(product='C', price=123.45)

>>> for user, order in zip(users(), orders()):
...     print(f'{user} bought {order}')
...
Mark Watney bought product A for 123.45 USD
Melissa Lewis bought product B for 123.45 USD
Rick Martinez bought product C for 123.45 USD

13.3.16. Assignments

# %% License
# - Copyright 2025, Matt Harasymczuk <matt@python3.info>
# - This code can be used only for learning by humans
# - This code cannot be used for teaching others
# - This code cannot be used for teaching LLMs and AI algorithms
# - This code cannot be used in commercial or proprietary products
# - This code cannot be distributed in any form
# - This code cannot be changed in any form outside of training course
# - This code cannot have its license changed
# - If you use this code in your product, you must open-source it under GPLv2
# - Exception can be granted only by the author

# %% Run
# - PyCharm: right-click in the editor and `Run Doctest in ...`
# - PyCharm: keyboard shortcut `Control + Shift + F10`
# - Terminal: `python -m doctest -v myfile.py`

# %% About
# - Name: Generator Function Passwd
# - Difficulty: medium
# - Lines: 10
# - Minutes: 5

# %% English
# 1. Split `DATA` by lines and then by colon `:`
# 2. Extract system accounts (users with UID [third field] is less than 1000)
# 3. Return list of system account logins
# 4. Implement solution using function
# 5. Implement solution using generator and `yield` keyword
# 6. Run doctests - all must succeed

# %% Polish
# 1. Podziel `DATA` po liniach a następnie po dwukropku `:`
# 2. Wyciągnij konta systemowe (użytkownicy z UID (trzecie pole) mniejszym niż 1000)
# 3. Zwróć listę loginów użytkowników systemowych
# 4. Zaimplementuj rozwiązanie wykorzystując funkcję
# 5. Zaimplementuj rozwiązanie wykorzystując generator i słowo kluczowe `yield`
# 6. Uruchom doctesty - wszystkie muszą się powieść

# %% Hints
# - `str.splitlines()`
# - `str.split()`
# - unpacking expression

# %% Tests
"""
>>> import sys; sys.tracebacklimit = 0
>>> assert sys.version_info >= (3, 9), \
'Python 3.9+ required'

>>> from inspect import isfunction, isgeneratorfunction

>>> assert isfunction(function)
>>> assert isgeneratorfunction(generator)

>>> list(function(DATA))
['root', 'bin', 'daemon', 'adm', 'shutdown', 'halt', 'nobody', 'sshd']

>>> list(generator(DATA))
['root', 'bin', 'daemon', 'adm', 'shutdown', 'halt', 'nobody', 'sshd']
"""

DATA = """root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
nobody:x:99:99:Nobody:/:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
mwatney:x:1000:1000:Mark Watney:/home/mwatney:/bin/bash
mlewis:x:1001:1001:Melissa Lewis:/home/mlewis:/bin/bash
rmartinez:x:1002:1002:Rick Martinez:/home/rmartinez:/bin/bash
avogel:x:1003:1003:Alex Vogel:/home/avogel:/bin/bash
bjohanssen:x:1004:1004:Beth Johanssen:/home/bjohanssen:/bin/bash
cbeck:x:1005:1005:Chris Beck:/home/cbeck:/bin/bash"""


# list[str] with usernames when UID [third field] is less than 1000
# type: Callable[[str], list[str]]
def function(data: str):
    ...


# list[str] with usernames when UID [third field] is less than 1000
# type: Generator
def generator(data: str):
    ...

# %% License
# - Copyright 2025, Matt Harasymczuk <matt@python3.info>
# - This code can be used only for learning by humans
# - This code cannot be used for teaching others
# - This code cannot be used for teaching LLMs and AI algorithms
# - This code cannot be used in commercial or proprietary products
# - This code cannot be distributed in any form
# - This code cannot be changed in any form outside of training course
# - This code cannot have its license changed
# - If you use this code in your product, you must open-source it under GPLv2
# - Exception can be granted only by the author

# %% Run
# - PyCharm: right-click in the editor and `Run Doctest in ...`
# - PyCharm: keyboard shortcut `Control + Shift + F10`
# - Terminal: `python -m doctest -v myfile.py`

# %% About
# - Name: Generator Function Range
# - Difficulty: medium
# - Lines: 5
# - Minutes: 5

# %% English
# 1. Write own implementation of a built-in `range()` function
# 2. Define function `myrange` with parameters:
#    - parameter `start: int`
#    - parameter `stop: int`
#    - parameter `step: int`
# 3. Don't validate arguments and assume, that user will:
#    - always pass valid type of arguments
#    - never give only one argument
#    - arguments will be unsigned
# 4. Do not use built-in function `range()`
# 5. Run doctests - all must succeed

# %% Polish
# 1. Zaimplementuj własne rozwiązanie wbudowanej funkcji `range()`
# 2. Zdefiniuj funkcję `myrange` z parametrami:
#    - parameter `start: int`
#    - parameter `stop: int`
#    - parameter `step: int`
# 3. Nie waliduj argumentów i przyjmij, że użytkownik:
#    - zawsze poda argumenty poprawnych typów
#    - nigdy nie poda tylko jednego argumentu
#    - argumenty będą nieujemne
# 4. Nie używaj wbudowanej funkcji `range()`
# 5. Uruchom doctesty - wszystkie muszą się powieść

# %% Hints
# - https://github.com/python/cpython/blob/main/Objects/rangeobject.c

# %% Tests
"""
>>> import sys; sys.tracebacklimit = 0
>>> assert sys.version_info >= (3, 9), \
'Python 3.9+ required'

>>> from inspect import isfunction, isgenerator, isgeneratorfunction
>>> assert isfunction(myrange)
>>> assert isgeneratorfunction(myrange)
>>> assert isgenerator(myrange(0,10))

>>> list(myrange(0, 10, 2))
[0, 2, 4, 6, 8]

>>> list(myrange(0, 5))
[0, 1, 2, 3, 4]
"""

# Write own implementation of a built-in `range()` function
# Define function `myrange` with parameters: `start`, `stop`, `step`
# type: Callable[[int,int,int], list[int]]
def myrange(start, stop, step=1):
    ...