Python Generators in Python: A Complete Guide with Examples

Syntax

Basic structure of a generator function

def generator_function():
    yield value

Example: Python Generator

Let’s create a generator to yield square numbers one by one:

def square_numbers(limit):
    for i in range(1, limit + 1):
        yield i * i

# Using the generator
squares = square_numbers(5)
for square in squares:
    print(square)

You can also try this code with Online Python Compiler

Run Code

Output:

1
4
9
16
25

Explanation:

1. The square_numbers function uses yield to return one square number at a time.
    
2. Each call to the generator produces the next square number, resuming execution from where it last paused.

Python Generator Expression

A generator expression in Python is a concise way to create generators without defining a full generator function. It is similar to list comprehensions but uses parentheses () instead of square brackets []. Generator expressions are memory-efficient because they generate values lazily, producing one item at a time instead of storing the entire sequence in memory.

Generator Expression Syntax:

gen = (expression for item in iterable if condition)

Example 2: Python Generator Expression

Create a generator to produce even numbers up to 10:

even_numbers = (x for x in range(11) if x % 2 == 0)


for num in even_numbers:
    print(num)

You can also try this code with Online Python Compiler

Run Code

Output:

0
2
4
6
8
10

Explanation:

1. The generator expression (x for x in range(11) if x % 2 == 0) filters even numbers from 0 to 10.
    
2. It produces values one at a time as we iterate over it.

Advantages of Generator Expressions

1. Memory Efficiency: Generates items on demand without storing them in memory.
    
2. Short and Readable: A single line of code can replace a full generator function.
    
3. Compatibility: Can be used directly with functions like sum(), max(), and min().

Use of Python Generators

Python generators are widely used in scenarios where you need to process large datasets or streams of data efficiently. Let’s look at some key advantages and use cases:

1. Memory Efficiency: Generators produce values on the fly without storing the entire sequence in memory.
    
2. Infinite Sequences: Generators can represent infinite sequences without running out of memory.
    
3. Pipeline Processing: Generators are ideal for processing data pipelines, where each step transforms or filters the data.

Example: Infinite Sequence

Here’s a generator to produce an infinite sequence of Fibonacci numbers:

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b
# Using the generator
fib = fibonacci()
for _ in range(5):
    print(next(fib))

You can also try this code with Online Python Compiler

Run Code

Output:

0
1
1
2
3

Create a Generator in Python

Let’s build a generator to process a dataset efficiently. Imagine a scenario where we need to filter numbers greater than a threshold.

Example

def filter_numbers(data, threshold):
    for num in data:
        if num > threshold:
            yield num

# Using the generator
data = [10, 20, 5, 30, 15]
thresh_gen = filter_numbers(data, 15)

for value in thresh_gen:
    print(value)

You can also try this code with Online Python Compiler

Run Code

Output:

20
30

Explanation:

1. The generator filter_numbers processes the dataset data.
    
2. Only numbers greater than the threshold are yielded.

Yield vs Return in Python: Key Differences

The yield and return keywords are both used to send values from a function, but they work differently:

Example: Using yield

def countdown(n):
    while n > 0:
        yield n
        n -= 1


# Using the generator
for number in countdown(3):
    print(number)

You can also try this code with Online Python Compiler

Run Code

Output:

3
2
1

Explanation:

The function pauses at yield and resumes from that point, generating one value at a time.

Example: Using return

def sum_numbers(n):
    return sum(range(1, n + 1))


# Using the function
print(sum_numbers(5))

You can also try this code with Online Python Compiler

Run Code

Output:

15

Explanation:

The function ends with return, providing a single output immediately.

Frequently Asked Questions

What is the difference between a generator and an iterator?

 Generators are a simpler way to create iterators. While all generators are iterators, not all iterators are generators.

Can a generator be reused after it is exhausted? 

No, once a generator is exhausted, you need to create a new instance to iterate again.

What are the advantages of using generators? 

Generators are memory-efficient, allow infinite sequences, and are ideal for lazy evaluation and data pipelines.

Conclusion

In this article, you learned about Python generators, how to create them using the yield keyword, and their applications for efficient data handling. We also discussed generator expressions and the differences between yield and return. By understanding and using generators effectively, you can write memory-efficient and scalable Python code.