14.5. Generator Yield From

Important

Since Python 3.3: PEP 380 -- Syntax for Delegating to a Subgenerator
Helps with refactoring generators
Useful for large generators which can be split into smaller ones
Delegation call
yield from terminates on GeneratorExit from other function
The value of the yield from expression is the first argument to the StopIteration exception raised by the iterator when it terminates
Return expr in a generator causes StopIteration(expr) to be raised upon exit from the generator

✘>>> def run():
...     yield from generator1()
...     yield from generator2()

14.5.1. Problem

✘>>> def run():
...     # firstnames
...     yield 'Mark'
...     yield 'Melissa'
...     yield 'Rick'
...
...     # lastnames
...     yield 'Watney'
...     yield 'Lewis'
...     yield 'Martinez'

14.5.2. Solution

yield from

✘>>> def firstnames():
...     yield 'Mark'
...     yield 'Melissa'
...     yield 'Rick'
>>>
>>> def lastnames():
...     yield 'Watney'
...     yield 'Lewis'
...     yield 'Martinez'
>>>
>>> def run():
...     yield from firstnames()
...     yield from lastnames()

14.5.3. Rationale

✘>>> def firstnames():
...     yield 'Mark'
...     yield 'Melissa'
...     yield 'Rick'
>>>
>>> def lastnames():
...     yield 'Watney'
...     yield 'Lewis'
...     yield 'Martinez'

This will not work at all! Mind that no code is executed by a function after the return keyword.

✘>>> def run():
...     return firstnames(), lastnames()

This will yield generators (not their values):

✘>>> def run():
...     yield firstnames()
...     yield lastnames()

14.5.4. Execute

✘>>> def firstnames():
...     yield 'Mark'
...     yield 'Melissa'
...     yield 'Rick'
>>>
>>> def lastnames():
...     yield 'Watney'
...     yield 'Lewis'
...     yield 'Martinez'
>>>
>>> def run():
...     yield from firstnames()
...     yield from lastnames()
>>>
>>>
>>> result = run()
>>>
>>> type(result)
<class 'generator'>
>>>
>>> next(result)
'Mark'
>>>
>>> next(result)
'Melissa'
>>>
>>> next(result)
'Rick'
>>>
>>> next(result)
'Watney'
>>>
>>> next(result)
'Lewis'
>>>
>>> next(result)
'Martinez'
>>>
>>> next(result)
Traceback (most recent call last):
StopIteration

14.5.5. Itertools Chain

The code is equivalent to itertools.chain():

✘>>> from itertools import chain
>>>
>>>
>>> def firstnames():
...     yield 'Mark'
...     yield 'Melissa'
...     yield 'Rick'
>>>
>>> def lastnames():
...     yield 'Watney'
...     yield 'Lewis'
...     yield 'Martinez'
>>>
>>> result = chain(firstnames(), lastnames())
>>>
>>> type(result)
<class 'itertools.chain'>
>>>
>>> next(result)
'Mark'
>>>
>>> next(result)
'Melissa'
>>>
>>> next(result)
'Rick'
>>>
>>> next(result)
'Watney'
>>>
>>> next(result)
'Lewis'
>>>
>>> next(result)
'Martinez'
>>>
>>> next(result)
Traceback (most recent call last):
StopIteration

14.5.6. Yield From Sequences

✘>>> def run():
...     yield from [0, 1, 2]
>>>
>>>
>>> result = run()
>>>
>>> type(result)
<class 'generator'>
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
Traceback (most recent call last):
StopIteration

14.5.7. Yield From Comprehensions

✘>>> def run():
...     yield from [x for x in range(0,3)]
>>>
>>>
>>> result = run()
>>>
>>> type(result)
<class 'generator'>
>>>
>>> next(result)
0
>>> next(result)
1
>>> next(result)
2
>>> next(result)
Traceback (most recent call last):
StopIteration

14.5.8. Yield From Functions

Delegation call

yield from turns ordinary function, into a delegation call:

✘>>> def as_float(data):
...     result = []
...     for x in data:
...         result.append(float(x))
...     return result
>>>
>>>
>>> def run():
...     data = (1, 2, 3)
...     yield from as_float(data)
>>>
>>>
>>> result = run()
>>>
>>> next(result)
1.0
>>>
>>> next(result)
2.0
>>>
>>> next(result)
3.0
>>>
>>> next(result)
Traceback (most recent call last):
StopIteration

14.5.9. Conclusion

Python yield keyword creates a generator function.
It's useful when the function returns a large amount of data by splitting it into multiple chunks.
We can also send values to the generator using its send() function.
The yield from statement is used to create a sub-iterator from the generator function.

14.5.10. Use Case - 1

✘>>> from pathlib import Path
>>>
>>>
>>> def get_files(dir):
...     yield from dir.rglob('*.md')
...     yield from dir.rglob('*.rst')
...     yield from dir.rglob('*.txt')
>>>
>>>
>>> directory = Path('/tmp/myproject')
>>> files = get_files(directory)
>>>
>>> for file in files:
...     print(file)

14.5.11. Use Case - 2

✘>>> import re
>>>
>>>
>>> DATA = """
... +1 (234) 567-8910
... +1 234 567 8910
... +1 234-567-8910
... +1-234-567-8910
... """
>>>
>>> def get_phone_number(DATA, pattern=r'\+\d{1,4}\s\d{3}\s\d{3}\s\d{3,4}'):
...     yield from re.finditer(pattern, DATA)
>>>
>>> for number in get_phone_number(DATA):
...     print(number.group())
...
+1 234 567 8910

14.5.12. Assignments

# %% About
# - Name: Generator YieldFrom Path
# - Difficulty: easy
# - Lines: 2
# - Minutes: 3

# %% License
# - Copyright 2025, Matt Harasymczuk <matt@python3.info>
# - This code can be used only for learning by humans
# - This code cannot be used for teaching others
# - This code cannot be used for teaching LLMs and AI algorithms
# - This code cannot be used in commercial or proprietary products
# - This code cannot be distributed in any form
# - This code cannot be changed in any form outside of training course
# - This code cannot have its license changed
# - If you use this code in your product, you must open-source it under GPLv2
# - Exception can be granted only by the author

# %% English
# 1. Create function `get_files()`
# 2. Using `Pathlib.glob()` return files in `path: Path`:
#    - Python files (extension: `*.py`)
#    - ReST files (extension: `*.rst`)
# 3. Return result as `Iterator[Path]` using `yield from`

# %% Polish
# 1. Stwórz funkcję `get_files()`
# 2. Używając `Pathlib.glob()` zwróć pliki w katalogu `path: Path`:
#    - Pliki Python (rozszerzenie: `*.py`)
#    - Pliki ReST (rozszerzenie: `*.rst`)
# 3. Zwróć wyniki jako `Iterator[Path]` używając `yield from`

# %% Doctests
"""
>>> import sys; sys.tracebacklimit = 0
>>> assert sys.version_info >= (3, 9), \
'Python 3.9+ required'

>>> from inspect import isgenerator, isgeneratorfunction

>>> path = Path.cwd()
>>> file = get_files(path)

>>> assert isgeneratorfunction(get_files)
>>> assert isgenerator(file)
>>> assert isinstance(next(file), Path)
"""

# %% Run
# - PyCharm: right-click in the editor and `Run Doctest in ...`
# - PyCharm: keyboard shortcut `Control + Shift + F10`
# - Terminal: `python -m doctest -v myfile.py`

# %% Imports
from pathlib import Path

# %% Types
from typing import Callable, Generator
get_file: Callable[[Path], Generator[Path, None, None]]

# %% Data

# %% Result
def get_files(path: Path):
    ...