Python Iterables Are Not Just About Sequences

Lots of times when I read code, I see people thinking that python iterables are just about python sequences like python lists, tuples, or strings. The most culprit are python lists. When they want to create a custom class that is iterable, they would rather make the underlying data structure a list in order to make use of the methods that are supported by sequences. I want to use this post to make you understand that python iterables are not just sequences. Iterables include a whole lot of objects than just sequences.

 

First, what is an iterable?

The definition of a python iterable.

Basically, a python iterable is any object that you can loop over. The object can be a python sequence like lists, strings, tuples, bytes, or they can be python collections like dictionaries, sets, frozensets, or they can even be file objects. These are all objects that are capable of returning their members or elements one item at a time. If you so desire, you can define your own user defined objects and can make them an iterable. I will show you how in later examples of loops in python.

Also, on a practical level, you can define a python iterable as anything that can appear in a for loop. I really don’t need to give an example here but think of anything you have put on the right side of a for loop in your code and that object is an iterable. The list goes on and on. Also, anything that you can put as an argument to the zip and map functions are python iterables. Therefore, knowing how the for loop operates, we can give a technical definition of an iterable.

Technically, an iterable is any object whose class implements the __iter__() special method or if you want to specify sequence semantics, which implements the __getitem__() special method. You really need to implement __iter__() method for your iterable when you need a generalized python iterator. But if you want to play with a sequence type in your object, then all you need to do is implement the __getitem__() special method.

As I do to in all my posts, let’s illustrate the definitions above with examples. Let’s first give examples of python iterables that are not python sequences. We’ll be using the practical definition: ability to participate in python for loops.

First, we’ll show that python dictionaries are iterables. Using for loop directly on the dictionary, python iterates over the dictionary based on the keys, but it has a powerful method, items, that can help one to iterate over the keys and values at the same time.

File objects are also iterables. You can replace the ‘eba.txt’ file in the code below with any text file of your choice. All I wanted to show was that the file handling object, fh, is a python iterable since it can participate in a for loop.

with open('eba.txt') as fh:
    for line in fh:
        print(line)

Then finally what you must be familiar with, python sequences. All sequence types are iterables. But not all iterables are sequence types as we have noted above.

Python strings are iterable sequences.

Lists and tuples are sequence types and also iterable. In fact, all sequence types are iterable. They give examples of loops in python.

All the types above that are iterable are custom data types. What about user defined types? I said above that user defined types can be made iterable. How? By making them implement the __iter__() method or if you desire sequence semantics, the __getitem__() method. Let’s use the __iter__() method because later I will show you how to implement the __getitem__() method.

The Fruits class below uses a python list as the underlying data structure. We implemented the __iter__() method which returns an iterator object, itself. All implementers of this method will return themselves as iterator objects. To enable the for loop to access each of the items in the iterator object, we need to implement another method, the __next__() method. The __next__() method defines an object as a python iterator and iterators are also iterables. What the __next__() method below does is just to go through each of the items using their index, which is also a data attribute, and returning each of the items with that index. When it gets to the end of the list, it returns the Stopiteration exception to the python for loop which then stops asking for more items.

One thing I want you to note from above code is that all the built-in iterables implement the __iter__() method that is why when you explicitly call iter(object) on them, they will give you an iterator object. You can read on python iterators here.

Now, let me discuss on one special type of iterable and those are sequences.

What are sequences?

Sequences are iterables but they support looping through the items in the sequence using indices. So, everything you do with a python iterable, you can also do with a python sequence. That is why when you read code, you wonder if everyone thinks only sequences are iterables. For an object to be a sequence, it must implement the __getitem__() and __len__() special methods. I discussed using the __len__() special method on user defined objects in another post. So, all python sequences like lists, tuples, strings implement these two methods.

Let’s give an example of a user defined object that acts as an iterable by mimicking the sequence semantics. Here, the Fruits class implements the __getitem__() and __len__() methods.

This code is not different from the earlier user defined code except that this time it is implementing the __getitem__() method rather than the __iter__() method.

If you ask me, which should I use, the __iter__() method or the __getitem__() method for user defined objects? The answer is – it depends on what you want to do with your user defined objects. It is rare to see implementations of the __iter__() method because of the limitations associated with iterators and python has made it possible to produce iterators easily using generators. But it is common to implement the __getitem__() method if you want your object to behave like a sequence, or even a collection.

And if you want further functionality, you could make collections.abc.Sequence the base class for your class. When you do this, then you could be able to carry out further functionalities that sequences have like find out the count of an item, get the index of an item, find out if an item is in the object etc.

Let me give some examples. First, I imported the Sequence abstract base class from collections.abc. Then, I made it the parent class to my class, Fruits. I also implemented the __contains__() special method to be able to use the “in” operator on instances of the class. Here is some code that shows the added functionality of my new Fruits class that is separate from what the native Fruits class above could do.

We cannot end the discussion without noting how python iterables support lazy evaluation of values.

Lazy evaluation in python.

According to Wikipedia.org article on lazy evaluation strategy, this is a technique which delays the evaluation of an expression until the value is needed and which also avoids repeated evaluations. Python supports the lazy evaluation technique in iterables. For example, with the built-in python range function, we don’t need to explicitly produce all the values that are needed for the range but instead use them as when needed due to the lazy evaluation technique. This helps us to save memory. For example, take the following call to range to evaluate a million items. I decided I didn’t need to print the values beyond the 100th, so I decided to break the loop at the 101th item.

for i in range(1000000):
    if i == 101:
        break
    print(i)

If python did not use the lazy evaluation technique while producing the items, it could have produced a million items while I only required just the first 100.

Lazy evaluation is also seen when you are using the items or values functions of a dictionary. Python would only get the key or value based on when and whether you need them or not. It doesn’t just populate memory with all the keys and values. Here is a python iteration over dictionary keys and values.

fruits_dict = {'mango': 1, 'orange': 3, 'pineapple': 7, 'melon': 4}
for key, value in fruits_dict.items():
    print(key, value)

Lazy evaluation saves time and memory space. This is one feature that is very powerful in python programming.

But if on the other hand, you do need all the values from the iterator that is created during lazy evaluation, you can just cast it to a list or tuple. For example, using the range earlier, If I really needed all the million items, instead of retrieving them one at a time, I can cast it to a list and get everything at once.

range_list = list(range(1000000))

You can read up the documentation glossary on iterables and use them to your pleasure. Happy pythoning.

Python Decision Control Structures: Python while And for Loops Part 2

 

The last post discussed on the use of the python if else, and elif statements as part of a selection control structure. Today, the second part, we will discuss on repetition control structures using python while and for loops. These control flows are used when we want to reuse parts of some code a number of times based on a condition.

The looping constructs provided by python, the while and for loops, are distinct, yet sometimes they can be interchanged except in some cases. First, we will discuss the python while loop.

The python while loop.

A while loop helps one to carry out repeated execution of a block of code based on repeated testing of a Boolean condition.

The syntax of a while loop is as follows:

while condition:
    body

Just similar to the python if statement, the condition is a Boolean expression that evaluates to True or False, and the body is a block of code. The block of code can even be nested with other control structures. The while loops starts its execution by testing the Boolean condition. If the condition evaluates to True, the body of the loop is executed. After the execution of the body, the condition is retested again. If the condition evaluates to True again, another iteration of the body is done. This iteration is repeated as long as the condition evaluates to True. The moment it evaluates to False, the loop is exited and the flow of control transfers to the statement outside the python while loop.

To make sure that the while loop doesn’t run forever, it should come to a point where the condition will evaluate to False. To do this effectively, use a counter that you initialize before the loop and that is incremented or decremented inside the loop. If on the other hand you find that you didn’t do so and your loop runs forever, just press Ctrl+C to interrupt the process.

Here is a code that loops through a list of fruits and tells us whether our favorite fruit, which we have to input, is in the list of fruits. Please, pardon me that the list of recommended fruits is short.

Notice in line 5 that the python while loop is based on two Boolean expressions:

while j < len(fruits) and fruits[j] != fav_fruit:

The first Boolean expression checks to make sure we have not gone to the end of the list and the second checks by the index, j, to see if we have the fruit in our list. After the Boolean expressions, the body of the loop is just a one-liner that increments the index to the list, j. j is being used here as an index to the list to iterate through the list whenever the Boolean expressions evaluates to True; that means we have not found a match for favorite fruit in the list. There is some python if else statements after the while loop that is used as confirmation whether a match was found or not. You can see the post on python if else statements.

This code does not run forever; it terminates no matter what the user enters. Because either we will not find a favorite fruit on the list and get to the end of the list where the loop terminates or we will find a favorite fruit on the list and the loop then terminates.

Now, let’s go to the second looping structure: python’s for loop with some examples.

The python for loop.

When you want to iterate through a sequence of elements in an iterable, the for loop is more preferred to the while loop. It can be used on any structure that is iterable, whether a sequence or collection. Sequences are python strings, tuples, range, and bytes while collections are python dictionaries, sets, and frozensets. The syntax of a python for loop is:

for element in iterable:
    body

You often use the element variable in the body code; the reason why we are iterating through the iterable in the first place is to access the elements. Readers who are familiar with java programming language would realize that the python for loop is in some sense similar to the java “for each” loop.

To illustrate the way a for loop works, let us take a list of numbers, iterate through each of the numbers in the list and add them together to get a total sum.

In the code above, the variable num iterates through each of the numbers in the list of numbers in the for loop and then at the body code, it adds the numbers to total variable to give the total sum.

Let’s take another python for loop example of getting the biggest number in the list of numbers.

We first assigned biggest variable to an arbitrary number, this time 0, and then in the for loop num iterates through each of the numbers. Each time num gets the value of one of the numbers in the list. In the body of the for loop we compare num to the biggest each time and if num is bigger than the biggest, we assign that num to the biggest variable. This comparison happens each iteration through the list in the for loop.

We could achieve the above code using a while loop but we would need to use an indexing method. Indexing with for loops will be described below. But note that some collections like sets cannot be done using while loops because they cannot be indexed.

Now let’s implement a for loop using an index into the elements of the iterable.

Python for loops using an index

There are occasions where we might want to know where an element resides within an iterable. The traditional application of the for loop does not give us that benefit of location. But we can get that effect by indexing into the iterable using a python range function. The range function will generate a sequence of numbers that serve as the indices into the iterable or sequence. The syntax for the for loop is:

for element in range(len(iterable)):
    body

Note how the python length function provides the number that range will use to generate the indices for looping through the iterable. I have a post on the python length function. You can check it out to further understand the syntax above. Now, let’s take some illustrative example. Suppose we want to get the biggest number like we did above but using the indices in the loop, we could implement the code this way:

We eventually implemented the same code like before but instead of iterating directly through the numbers in the list, we used the indexing method to iterate through the numbers.

Note that the index variable above is an integer which is derived from the range of values generated by the range function.

I think we have basically covered the essential points for python while and for loops. But we will not close the chapter without talking about two important statements that have an influence on the iteration of a while and for loop – the python break and continue statements.

The python break and continue statements

Both of these statements interrupt the operation of a while or for loop but in different ways.

The break statement terminates the execution of the loop and transfers control to the next statement in the code. It is usually used to check for the trigger of a condition in the loop and when that condition is satisfied, the loop is terminated.

For example, let us say we want to loop through the list of numbers above to stop when we get a 9. This is how the code could be written. Watch how the python break statement was inserted into a control flow block.

If you run the code above, you will notice that the loop is iterating through each of the numbers until it gets triggered when num is 9. When this condition is reached, the loop terminates and the rest of the numbers are not referenced.

Use the break statement sparingly. It has great power.

The companion loop interruption statement is the python continue statement. The continue statement is usually employed when we don’t want a set of statements in the body of the loop to be executed when a condition is triggered. When triggered, control passes to the next item in the loop; the loop is not terminated.

An example will suffice.

I used a python while loop this time around. The Boolean expression in the while loop checks for when we have looped through all the elements in the numbers list. For each num we are looping, we first check if the number at that index is 0, if it is not zero, we use it to divide the numerator and print out the result, but if it is zero, we tell the program to ignore that number, don’t use it to divide the numerator, and move on to the next number in the loop using the continue statement. This is a very convenient way to program. That is why I love python.

Notice on line 6 and 10 that each time I increase the index by 1 so that the loop can proceed. This makes sure we do not find ourselves in an infinite loop that doesn’t end.

Take your knowledge to new heights. Experiment with the looping constructs introduced here. It’s a joyful thing programming in python.

Python Decision Control Structures: The Python if else Statement - Part 1

 

We all need to make decisions. We decide on what to eat, what to wear, what to learn, or where to work. Computers also make decisions; at least, the decisions we code. You designate what decision you want a program to make using control structures in your code.

A control structure is a block of programming that analyses one or more variables and then decides on what to do next based on the parameters given by the variables. Another word for control structures is flow of control. Given conditions and parameters, this is the decision making construct in any program.

There are three types of control structures or control flow. They are the sequential, the selection, and repetition control structures.

1. Sequential control Flow

This is when you execute statements one after the other in the order they appear in the program. Provided there are no syntax or semantic errors, the statement will run from the first to the last statement.

Here is the flow chart for the sequential control flow. 


 
2. Selection Control Flow

This involves choosing between two or more alternative paths, based on the evaluation of the conditional statement.

Here is a typical flowchart. 


 

Selection control flows are usually implemented as python conditional statements and could be a python if statement, python if else statement, or python if elif else statement.

3. Repetition control flow

This involves repeating a series of statements several times based on the evaluation of a conditional statement.

Here is a typical flowchart. 


 

Repetition control, or sometimes called iteration control, flows are usually implemented in python using the while and for loops.

Apart from the sequential control flow, the selection control flow which makes use of conditional and the repetition control flow which makes use of loops all consist of a condition to be evaluated and a body of statements. You use the python syntax for defining blocks of code to help python interpret your statements appropriately. This involves using the colon character to delimit the beginning of a block of code that acts as the body for a control structure. If the body is just a one-line statement, you can place it on the same line with the condition being evaluated and just after the colon. Or you could decide to place it on a separate line. But if the body is more than one line, you use python’s principles of indentation to place it on a separate line after the colon. Using indented blocks to designate the body helps python to interpret the code as a group of statements belonging to the same control flow. You should already be familiar with python’s principle of indentation. But let me just give some examples with a python while loop.

# using a while loop to show block indentation
while n < 5:  #use colon to show block of code comes next
    # indent the block of code in body by 4 spaces
    print(n)
    n += 1

In any program, you could end up using one, two or all of the control flows. In this post, we will discuss about the selection control flow, while in the next part we will discuss the repetition control flow.

Python implements the selection control flow using conditionals.

Conditonal statements.

Conditional statements in python, also known as the python if statement, if else statement, or if elif else statement, are a way for one to execute a chosen block of code based on the run-time evaluation of one or more Boolean expressions. You usually write the conditional or python if statement in the following way:

if first_condition:
    first_body
elif second_condition:
    second_body
elif third_condition:
    third_body
else:
    fourth_body

Each condition is a Boolean expression that evaluates to True or False, and each body consists of one or more statements that are executed conditionally. On the success of the first condition, the body is executed and no other condition is evaluated after that. But if it fails, the next condition is evaluated for whether it will succeed or fail. When any condition succeeds, all other conditions are discarded and only the body of the condition that succeeds is executed. If none of the conditions succeed, the else body is executed. Note that precisely only the body following a successful conditional will be executed, and sometimes none at all if there is no else statement.

In the example code above, I used two elif statements. You can use any number of elif statements, including zero.

Note that when you want to evaluate a conditional in the if statement, you are evaluating it based on whether it resolves to True or False. In some cases, if you are not evaluating the Boolean expression based on a specific value but on whether the variable has value, you don’t need to evaluate for True or False because every variable is already True when it has a value. You only write out the variable name.

For example, don’t do this:

if fruits == 'True':
    print('Fruits variable has a value')                

Rather you produce more optimized code if you write it like this, omitting the test for True:

#I removed the test for True
if fruits: 
    print('Fruits variable has a value')                

The elif and else statements in conditional constructs are optional. Let’s illustrate all with examples.

1. Where only the python if statement is used.

if hungry:
    eat_food()

2. Where the python if else statements are used.

Some call this the python if then else statement which is a correspondence to other programming languages like java.

if hungry:
    eat_food()
else:
    sleep()

3. Where only the python if elif statements are used.

if hungry:
    eat_food()
elif tired:
    rest()

4. Where all three statements, if elif else statements, are used.

This looks like a case switch statement in python; a throwback from java.

 
if hungry:
    eat_food()
elif tired:
    rest()
elif bored:
    watch_tv()
else:
    sleep()

So I have outlined four different ways the conditional construct can be used. Note that only the if statement is required; the others are optional.

One other thing I need you to know about the block indentation so you don’t run into problems. When you have code you want to specify that is not included in the body of the conditional construct, you need to take back your indentation by 4 steps. For example, if after the else block below I want my program to shift control to another activity which does not lie in the conditional construct, i.e do_next_activity(), my indentation goes back 4 steps.

if hungry:
    eat_food()
else:
    sleep()
# this is not part of the conditional construct
# it goes back 4 steps in indentation    
do_next_activity() 

From above, do_next_acitivity() goes back 4 steps and is not part of the indentation for the conditional constructs. It does not participate in the indentation.

Lastly, we have python nested if statements.

Python nested if statements.

We may nest one control structure within another, and to do this successfully, we rely on the indentation to make our intent clear to python. Let’s take our if statement a bit further and nest it.

if hungry:
    if food_exists:
        cook_food()
    else:
        buy_food()
    eat_food()
do_next_activity()                                   

You can now see that we have a nested if statement within another if statement. All you need to do is to be careful about making sure your indentation is correct.

We can illustrate the nested conditional construct above with a traditional flowchart. That makes for easy understanding.

In the next post, I will discuss on the repetition control flow which consists of the while and for loops.