Iterators and Generators in Python

Introduction

In Python programming or any other programming language, looping over the sequence (or traversing) is the most common aspect. While loops and for loops are two loops in python that can handle most of the repeated tasks executed by programs. Iterating over sequences is so widespread that Python offers extra capabilities to make it easier and more efficient. One of the tools for traversing is Iterators and Generators in Python. This chapter kicks off our investigation of these tools. Now, let’s get started with Iterators and Generators in python. 

Also See, Intersection in Python and Convert String to List Python.

Iterators in Python

In Python, an iterator is an object used to iterate over iterable objects such as lists, tuples, dictionaries, and sets. An object is called iterable if we can get an iterator from it or loop over it. 

Let us understand this with one example:

Suppose you have a list of even numbers:

Numbers = [2, 6, 8, 10, 12]

You can efficiently process the traversing using a For or While loop to get each element one by one.

They are neatly implemented within for loops, comprehensions, generators, etc., but concealed from view. In simple words, an iterator is just an object that can be iterated on.

A Python iterator object must implement two specific methods, __iter__() or iter() and __next__() or next() , which are referred to collectively as the iterator protocol.

Python iter()

The iter() function in Python returns an iterator for the supplied object. The iter() generates a thing that can be iterated one element at a time. These items are handy when combined with loops such as for loops and while loops. 

Syntax:

iter( object , sentinel )

iter() function takes two parameters:

• Object: An object whose iterator needs to be created (lists, sets, tuples, etc.).
• Sentinel (optional): Special value that represents the end of the sequence.

 

Python next()

The next() function returns the next item from the iterator. The next() function holds the value one at a time. 

Syntax:

next( iterator , default )

The next() method accepts two parameters:

• Iterator : next( ) function retrieves the next item from the iterator.
• default(optional): this value is returned if the iterator is exhausted (not tired, but no next item to retrieve).
   

Let’s consider an example for better understanding:

Assume we have a list of different types as given below. 

list1 = [ 25 , 78, ‘coding’, ‘is’, ‘<3’ ]  # list of different types

Let’s print it with the help of Iterators ( or iter() and next() ):-

# Program to print the list using Iterator protocols
X = [25, 78, 'Coding', 'is', '<3']
# Get an iterator using iter() 
a = iter(X)

# Printing the a iterator
print(a)

# next() for fetching the 1st element in the list that is 25
print(next(a))

# Fetch the 2nd element in the list that is 78
print(next(a))

# Fetching the consecutive elements
print(next(a))
print(next(a))
print(next(a))

 

Output

<list_iterator object at 0x000001B91F9CFFD0>
25
78
Coding
is
<3

As you can see, when we are trying to print the iterator a, it shows its type and the memory address it is located on. One by one, next() fetches the element from the list.

Let's use the same example to figure out why there's an exception at the end of the sequence in the above diagram. 

Let's look at the next() method again in the above program to see what happens next.

# Trying to fetch the elements at the end of the sequence
print(next(a))

 

Output

<list_iterator object at 0x0000028A1D0EFFD0>
25
78
Coding
is
<3
Traceback (most recent call last):
  File "C:\Users\HP\PycharmProjects\pythonProject\main.py", line 15, in <module>
    print(next(a))
StopIteration

When we attempted to fetch the next value, we received an exception. Usually, a StopIteration Exception is raised when the next() method attempts to proceed to the next value, but there are no new values in the container.

What exactly is going on in the code's background? Let’s have a look at the flowchart given below to understand it. 

Source: Techbeamers

Now, how to avoid the StopIteration Exception?

StopIteration is an iterator's means of signalling that it has reached the end. When you use a for loop to iterate, the exception is handled internally and exploited to terminate the loop. This is one of the distinctions between loops and iterators. When you explicitly call next(), you should be prepared to catch the exception yourself. A more elegant way to print the elements by using the loop is given below. 

We can wrap the code inside the try block, as shown below.

# Program to print the tuple using Iterator protocols
tup = (87, 90, 100, 500)

# get an iterator using iter()
tup_iter = iter(tup)

# Infinite loop
while True:
   try:
       # To fetch the next element
       print(next(tup_iter))
       # if exception is raised, break from the loop
   except StopIteration:
       break

 

Output

87
90
100
500

Create your Iterator

Example 1: 

class MyNumbers:
 # __iter__() is same as iter()
  def __iter__(self):
   self.a = 1
   return self

  # __next__() is same as next()
  def __next__(self):
    # 20th is the highest value
    if self.a <= 5:
       x = self.a
      # Manually increment
       self.a += 1
      # returning the iterator to the function call
       return x

# Create the object of the class
myclass = MyNumbers()
# get an iterator using iter()
myiter = iter(myclass)

# printing the values using a for-in loop
for x in myiter:
 print(x)

Output

1
2
3
4
5
None
None
None
None
None
None
None
None
#------ goes on until control exit

After the maximum is hit, we will get the None an infinite number of times. The StopIteration statement can be used to halt the iteration from continuing indefinitely.

We can add a termination condition to the next() method to produce an error if the iteration is repeated a certain amount of times.

Example 2: 

class MyNumbers:
 # __iter__() is same as iter()
  def __iter__(self):
   self.a = 1
   return self

  # __next__() is same as next()
  def __next__(self):
    # 20th is the highest value
    if self.a <= 5:
       x = self.a
      # Manually increment
       self.a += 1
      # returning the iterator to the function call
       return x
      # added the terminating statement to prevent the iteration to go on forever
    else:
        raise StopIteration

# Create the object of the class
myclass = MyNumbers()
# get an iterator using iter()
myiter = iter(myclass)

# printing the values using a for-in loop
for x in myiter:
 print(x)

Output

1
2
3
4
5

Great, We made our iterator by defining its maximum length and including the terminating condition.

For a better understanding, consider the following example.

class PowerTwo:
   # Class to implement an iterator of powers of two
   # Constructor accepting the max value
   def __init__(self, max=0):
       self.max = max

   # defined __iter__() to point the first element
   def __iter__(self):
       self.n = 1
       return self

   # __next__() to fetch the next value from the iterator
   def __next__(self):
       if self.n <= self.max:
           result = 2 ** self.n
           self.n += 1
           return result

   # Terminating condition
       else:
           raise StopIteration

# create an object
numbers = PowerTwo(4)

# create an iterable from the object
i = iter(numbers)

# Using for-in loop to print the elements up to max
for it in i:
   print(it)

Output

2
4
8
16

In the above code, we are attempting to implement an iterator of powers of two manually. In the __iter__() method, we direct the n variable to the initial value. In the __next__() function, we manually increase the n variable to its maximum value. Simultaneously, the result is returned to an iterable object.

You can compile it with online python compiler.