The essentials
Quick reference
One focused task per row. Jump to the related section for complete, working examples.
| Use | Syntax | Examples |
|---|---|---|
| Concatenate iterables lazily | items = chain(first, second) | View examples |
| Flatten one iterable level | items = chain.from_iterable(groups) | View examples |
| Slice an iterator | page = islice(source, start, stop, step) | View examples |
| Create fixed-size batches | chunks = batched(source, 3) | View examples |
| Require complete batches | chunks = batched(source, 3, strict=True) | View examples |
| Iterate adjacent pairs | edges = pairwise(values) | View examples |
| Compute running totals | totals = accumulate(values) | View examples |
| Accumulate with a function | products = accumulate(values, operator.mul, initial=1) | View examples |
| Fork an iterator | left, right = tee(source, 2) | View examples |
| Group adjacent equal keys | groups = groupby(records, key=operator.itemgetter('team')) | View examples |
| Select with a Boolean mask | selected = compress(data, selectors) | View examples |
| Build a Cartesian product | coordinates = product(rows, columns) | View examples |
| Choose without ordering | pairs = combinations(values, 2) | View examples |
| Create an item lookup | by_score = operator.itemgetter('score') | View examples |
| Create an attribute lookup | by_name = operator.attrgetter('profile.name') | View examples |
| Create a method call | normalize = operator.methodcaller('strip') | View examples |
| Reduce to one value | total = reduce(operator.add, values, 0) | View examples |
| Pre-bind call arguments | parse_hex = partial(int, base=16) | View examples |
| Adapt a comparison function | key = cmp_to_key(compare) | View examples |
The itertools, functools, and operator modules turn Python's iteration and callable protocols into a compact toolkit for data processing. itertools builds lazy streams, functools adapts or combines callables, and operator supplies reusable functions for ordinary operators and lookups. The concise forms are most valuable when they preserve the business rule: keep pipelines bounded, understand when an input is consumed or cached, and prefer a named function whenever a dense expression would hide intent.
Step by step
Detailed examples
Compose lazy pipelines and bound their consumption
chain and chain.from_iterable traverse inputs sequentially without copying their contents. islice applies nonnegative start, stop, and step positions to any iterable, including a stream that cannot be indexed. These tools return one-pass iterators: creating them does no useful work until consumption, and an unbounded source needs an explicit stop. Fully consuming islice can advance the shared source farther than the number of values yielded, so do not treat it as a non-destructive view.
from itertools import chain, islice
pages = ([1, 2], [], [3, 4, 5])
stream = chain.from_iterable(pages)
print(list(islice(stream, 1, 4)))
print(list(stream))[2, 3, 4]
[5]Choose disjoint batches or overlapping pairs
batched consumes up to n items per tuple and is lazy; its default permits a short final tuple. Python 3.12 introduced batched, and Python 3.13 added strict=True for protocols that require exact block sizes. pairwise, available since Python 3.10, instead emits overlapping adjacent pairs and is ideal for edges, deltas, and transitions. Neither operation pads missing data, so padding must be an explicit domain decision.
from itertools import batched, pairwise
values = [10, 13, 20, 21, 30]
print(list(batched(values, 2)))
print([right - left for left, right in pairwise(values)])
try:
list(batched(values, 2, strict=True))
except ValueError as error:
print(type(error).__name__)[(10, 13), (20, 21), (30,)]
[3, 7, 1, 9]
ValueErrorMake cumulative state and iterator sharing explicit
accumulate exposes every intermediate reduction, while functools.reduce returns only the final value. Supplying initial to accumulate emits that seed before consuming input. tee creates multiple logical iterators from one source, but it is not thread-safe and may retain a large buffer when consumers advance at different speeds; materializing the source can be cheaper when one consumer will lag far behind. Do not continue using the original iterator after tee has forked it.
from itertools import accumulate, tee
import operator
print(list(accumulate([2, 3, 4], operator.mul, initial=1)))
source = iter('ABC')
first, second = tee(source)
print(next(first), next(first))
print(''.join(second))
print(''.join(first))[1, 2, 6, 24]
A B
ABC
CSort before global grouping and preserve ephemeral groups
groupby groups consecutive runs, unlike SQL GROUP BY; equal keys separated in the input produce separate groups. Sort by the same key first when the requirement is one group per key. Each group iterator shares groupby's source and becomes unavailable when the outer iterator advances, so materialize a group when it must survive. compress stops as soon as either data or selectors is exhausted, which makes selector-length validation the caller's responsibility.
from itertools import compress, groupby
from operator import itemgetter
rows = [('blue', 3), ('red', 2), ('blue', 5), ('red', 7)]
rows.sort(key=itemgetter(0))
summary = [(key, sum(value for _, value in group)) for key, group in groupby(rows, key=itemgetter(0))]
print(summary)
print(list(compress(summary, [True, False])))[('blue', 8), ('red', 9)]
[('blue', 8)]Estimate combinatorial size before generating candidates
product models nested loops, permutations models ordered arrangements, and combinations models unordered selections. These iterators avoid storing every result, but product first pools each input and the number of outputs can still grow explosively. Elements are distinguished by input position, not value, so duplicate input values can produce duplicate-looking tuples. Use math.prod, math.perm, or math.comb to estimate work before launching an expensive search.
from itertools import combinations, product
from math import comb
coordinates = list(product('AB', range(2)))
print(coordinates)
print(list(combinations('ABCD', 2)))
print(comb(4, 2))[('A', 0), ('A', 1), ('B', 0), ('B', 1)]
[('A', 'B'), ('A', 'C'), ('A', 'D'), ('B', 'C'), ('B', 'D'), ('C', 'D')]
6Replace trivial lambdas with inspectable operator callables
itemgetter, attrgetter, and methodcaller create callables suitable for sorted, min, max, map, and groupby. They communicate a simple lookup or call more directly than a lambda and can retrieve multiple fields. attrgetter supports dotted traversal, but failures still surface at call time when an attribute is missing. These helpers do not sanitize user-selected attribute or method names; expose an allowlist when names cross a trust boundary.
from operator import itemgetter, methodcaller
records = [
{'name': 'Ada', 'score': 91},
{'name': 'Linus', 'score': 88},
{'name': 'Grace', 'score': 95},
]
print([row['name'] for row in sorted(records, key=itemgetter('score'), reverse=True)])
print(list(map(methodcaller('strip'), [' ready ', ' done '])))
print(itemgetter('name', 'score')(records[0]))['Grace', 'Ada', 'Linus']
['ready', 'done']
('Ada', 91)Adapt callable signatures without obscuring intent
reduce folds a binary function from left to right and should usually receive an initializer so empty input has defined behavior. partial freezes selected leading positional arguments and keywords. Python 3.14 added functools.Placeholder for reserving arbitrary positional slots; code supporting earlier versions should use a small named wrapper instead. cmp_to_key is mainly a migration adapter for comparison APIs—native key functions are typically simpler, faster, and easier to test.
from functools import partial, reduce
import operator
parse_hex = partial(int, base=16)
values = list(map(parse_hex, ['0a', '10', 'ff']))
print(values)
print(reduce(operator.add, values, 0))
print(reduce(operator.add, [], 0))[10, 16, 255]
281
0Local code tester
Compose a lazy order pipeline
Group sorted records, calculate totals, and take a bounded view while changing the source data.
Press Run to load Python locally.
Sources and further reading
References
Authoritative documentation used to verify and expand this cheat sheet.
- Python Software Foundationitertools — Functions creating iterators for efficient loopingdocs.python.org
- Python Software Foundationfunctools — Higher-order functions and operations on callable objectsdocs.python.org
- Python Software Foundationoperator — Standard operators as functionsdocs.python.org
- Python Software FoundationIterator typesdocs.python.org
Help us improve
Found a typo or missing example?
Tell us what would make this cheat sheet clearer, more complete, or more useful.