Python Topics : Generators & Generator Expressions
Introduction to Generators
to pause a function midway and resume from where the function was paused use the yield statement
when a function contains at least one yield statement, it's a generator function.
when calling a generator function, it returns a new generator object
the call doesn't start the function
generator objects (or generators) implement the iterator protocol
generators are lazy iterators
to execute a generator function call the next() built-in function on it

A Simple Generator Example
a generator function 
def greeting():
    print('Hi!')
    yield 1
    print('How are you?')
    yield 2
    print('Are you there?')
    yield 3
the yield statement is like a return statement in a function
when Python encounters the yield statement, it returns the value specified in the yield
it also pauses the execution of the function
if calling the same function again, Python will resume from where the previous yield statement was encountered 
def greeting():
    print('Hi!')
    yield 1
    print('How are you?')
    yield 2
    print('Are you there?')
    yield 3
    
messenger = greeting()

result = next(messenger)
print(result)

result = next(messenger)
print(result)

result = next(messenger)
print(result)

# this call will raise a StopIteration exception
result = next(messenger)
print(result)
Using Generators to Create Iterators
example defines an iterator which returns a square number of an integer 
class Squares:
    def __init__(self, length):
        self.length = length
        self.current = 0

    def __iter__(self):
        return self

    def __next__(self):
        result = self.current ** 2
        self.current += 1
        if self.current > self.length:
            raise StopIteration
        return result

length = 5
square = Squares(length)
for s in square:
    print(s)
the class contains much boilerplate code
a simpler version of the class can be created by using a generator 
def squares(length):
    for n in range(length):
        yield n ** 2

length = 5
square = squares(length)
for s in square:
    print(s)
Introduction to Generator Expressions
a generator expression is an expression that returns a generator object
the squares generator function returns a generator object that produces square numbers of integers from 0 to length - 1
a generator object is an iterator
can use a for loop to iterate over its elements 
for square in squares(5):
    print(square)
a generator expression provides a simpler way to return a generator object 
squares = (n** 2 for n in range(5))

for square in squares:
    print(square)
a generator expression is similar to a list comprehension in terms of syntax
a generator expression also supports complex syntaxes including
  • if statements
  • multiple nested loops
  • nested comprehensions
Generator Expressions vs List Comprehensions
Syntax
in terms of syntax, a generator expression uses parentheses () while a list comprehension uses the square brackets [] 
square_list = [n** 2 for n in range(5)]

square_generator = (n** 2 for n in range(5))
Memory Utilization
a list comprehension returns a list while a generator expression returns a generator object
a list comprehension returns a complete list of elements upfront
a generator expression returns a list of elements, one at a time, based on request
A list comprehension is eager while a generator expression is lazy

Iterable vs Iterator
a list comprehension returns an iterable
can iterate over the result of a list comprehension again and again

generator expression returns a lazy iterator
a generator object can be iterated just once

index