Python Basics : Yield keyword, Iterables and Generators

Introduction

Often times you must've seen folks using yield keyword inside of python loops. In this blog post, we will learn about yield keyword in detail with examples.

To understand yield keyword in detail, we first need to understand what iterables and generators are.

What are Python Iterables ?

In Python, anything that can be iterated on, is called an iterable. In other words, if we can loop on a Python object and access it's items one by one it's called an Iterable.

We have different types of iterables available in python, like lists, tuples, set, strings etc. Let's understand by an example :

For example, we create a list actors

actors = ["Jared Dunn", "Jian Yang", "Erlich Bachman"]

# Iterate on actors using a for loop
for actor in actors:
    print(actor)

This list is an iterable, because we can loop over it in order and access items one by one. Iterables are widely used in python, and they are handy to store items that needs access over and over again. However, in iterables all the items of an object are saved in memory - which is not always optimal.

For example, in the list of actors above, just 3 items won't hog our memory but if the list were to contain names of over a million actors - it might not be the best way to store and access the actors.

What are Python Generators ?

In Python, generators are iterators - a kind of iterable that can only be accessed once. Generators use lazy-iterators behind the scenes, meaning that they don't store all the items of iterable at once, instead they are stored in memory as we access them.

This makes the whole storing of huge lists very efficient, while working with big data-sets Generators are a life-saver in Python. Let's understand generators with an example :

squares = (x*x for x in range(10))
for i in squares:
    print(i)

Here we created a generator object named squares. Let's try running this code snippet :

>>> mygenerator = (x*x for x in range(3))
>>>
>>> mygenerator
<generator object <genexpr> at 0x7f90744bb660>
>>> for i in mygenerator:
...     print(i)
...
0
1
4
>>>
>>> for i in mygenerator:
...     print(i)
...
>>>

We saw that we were able to iterate and print all the values once, but when we re-tried iterating it did not return anything. That's how generators work.

What is yield keyword ?

yield keyword is used to make a function return a generator. It's used like return keyword, the only difference is that instead of returning a value, it will return a generator object. Let's understand this with an example :

>>> def my_generator():
...    num_list = range(3)
...    for i in num_list:
...        yield i*i
...
>>> obj_generator = my_generator() # creates a generator
>>> print(obj_generator) # obj_generator is a generator object!
<generator object my_generator at 0x7f90744bb661>
>>> for i in obj_generator:
...     print(i)
0
1
4

You might ask why go through all this hassle when the list itself has only 3 items in it. Remember, this comes in handy when even using list puts a strain on your memory, i.e. working with large iterables or data-sets.

Conclusion

In this blog post, we learnt learn iterables, generators and yield keyword by example. I hope this helped, feel free to reach out for any questions. Haapy coding!