Are you writing nested for loops again and again and making your code more complex ? Beware! as you might me dishonoring “Zen of Python” Principle. In this article we will learn how to use list comprehension in our daily code integrations. We will discuss about what is Python list comprehension, how to use it, when to use it and when not to use it. And by the end of this article, you’ll understand the full potential of Python list comprehensions and its features. Let’s first understand what actually is list comprehension.

What is List comprehension ?

Python list comprehension is a way of making your code look elegant, clean and more pythonic.
With python’s list comprehension you can write multiple nested for loops in just one line of code.

>>> cubes = []
>>> for cube in range(10):
        cubes.append(cube**3)
>>> cubes
#Output
[0, 1, 8, 27, 64, 125, 216, 343, 512, 729]         

>>> cubes = [cube**3 for cube in range(10)]
#Output
[0, 1, 8, 27, 64, 125, 216, 343, 512, 729]

You can see the potential of list comprehension above. Now that’s said, lets explore more of what list comprehension can do by first learning the syntax.

Syntax for List Comprehension

Python list comprehension consists of starting ” [ “ and closing ” ] “ brackets same as list with an expression followed by a for clause, then zero or more for or if clauses. The result will be a new list resulting from evaluating the expression.

list_of_item = [ expression for item in iterable ]
# or
list_of_item = [ expression for item in iterable if conditional]

In order to start writing list comprehension you need to understand, some important elements in the above syntax.

1. expression: this is the expression which you want to apply on each individual item of the list. This expression could be any mathematical expressions, any function call on the item, or any operation on the item etc. For ex. In the above code snippet , we applied power (**) operator of python as a expression on the item.
   
   
2. item: item is the object, iterable or something in the list. In the above code snippet, the item was cube( 1, 2, 3, …..9) that was changing each time as we were looping over a range of 10.
   
   
3. iterable: iterable is any Python object capable of returning its items or members one at a time. An iterable also allow iteration on its items/members over a for loop. For example in Python iterables are list, string, dictionary, tuples, set, iterator, generator etc.
   
   
4. conditional: conditionals are basically any conditions like if-else.

Combining the above elements , you can easily create python comprehensions in list.

List comprehension in practice

Let’s write some list comprehension with if conditionals.

no_list = [3, 4, 2, 33, 65, 5, 72, 32, 43, 78, 23]
even_list = [num for num in no_list if num%2==0]
"""
Output: even_list
[4, 2, 4, 72, 32, 78]
"""

In the above snippet on line 3 we are using for loop to iterate/loop over no_list and then on each iteration we are checking using the modulo operator ( % ) whether the remainder after dividing with 2 is 0 or not. If it is 0, then add the num to the result list to be returned.

item_price = [ 110, 21, 78, 23 ]
def billing_price(item):
   return item + (item * 0.18)
#getting the final price list using list comprehension
item_final_price = [billing_price(itm) for itm in item_price]

Nested list comprehension

Most of the time we are using mathematical operations in our code for computing and if you are form a mathematical background It is common for you to use nested “for loops” in your code, but you can optimize that using nested list comprehension. Let’s see an example from python website..
Consider the following example of a 3×4 matrix implemented as a list of 3 lists of length 4 each

>>> matrix = [
               [1, 2, 3, 4],
       ...     [5, 6, 7, 8],
       ...     [9, 10, 11, 12],
         ... ]

The following list comprehension will transpose rows and columns:

>>> [[row[i] for row in matrix] for i in range(4)]
#Output
[[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]

>>> transposed = []
>>> for i in range(4):
...     # the following 3 lines implement the nested listcomp
...     transposed_row = []
...     for row in matrix:
...         transposed_row.append(row[i])
...     transposed.append(transposed_row)
>>> transposed [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]

TIP : You can also use zip in real world scenarios for doing these calculations.

-Python website

When not to use List comprehension

While list comprehension can be a life saver, sometimes it is ok not to use it. You should always keep in mind the “Zen of Python” while writing code. And you will often find one principle contradicting the other. In that cases, use your common sense and make a wise decision. Even I during my coding hours get stuck in this kind of situation. I give preference to code runtime over code complexity.

1) List is long in length

Avoid using list comprehension when there is multiple(more than 3 or 4) for loops or when the logic is too long with multiple if-else conditions. In this case give preference to readability and to code that is easy to understand.

accounts = [ acc.details() for acc in total_acc if acc.credit_card_due() != 0 for due in acc.due_balance() if sum(due) > 100]

In the above list comprehension, we cannot easily understand what the code is doing because it is lacking comment. Also there are multiple for-if conditionals making the code look more complex. Now let’s look it in more Pythonic way below…

bank_acc = []
#Iterate over list of acc object in total acc list
for acc in total_acc:
    # Check credit card due
    if acc.credit_card_due() == 0:
        continue 
    # Check total due on all balances
    for due in acc.due_balances():     
        if sum(due) < 100:         
            continue     
    bank_acc.append(acc.details())

Due to good commenting style and better readablity one can easily understand what’s going on. In the above snippet we are iterating over account object in total_acc list which contains a list of account object. After that we are first checking credit card due on each account in question. If credit card due is found then we check total due on the account including credit card and other linked checking accounts to that account in question. If the sum of the total due is greater than 100 we add that account to the bank_acc list for further analysis.

2) List is large in size

The major reason why you should avoid using list is when they grow large in size. When list are larger is size, (and when I say large that means taking too much memory) the machine will become slow making it hard for the computations and other processes.

List and other data structures like dictionary and sets work by loading the data in memory. And this is ok for medium length computations but when this grow exponentially it may cause a problem. For example..In the first example if you calculate square of a number till 1000, python list comprehension will work with almost no issue. As you move to the second example when you try to do so for the first billion integers, then you might notice that the computer become non-responsive for some moments. That is because the computer is creating all the squares of the number till billion and storing it in memory. Because of this the available memory is slowly decreasing causing the programs to hang.

>>> sum([i * i for i in range(1000)]) 
332833500
>>> sum(i * i for i in range(1000000000)) 
333333332833333333500000000

Source : Real Python

In such scenarios , you can use generators which returns an iterable. The benefit of returning an iterable is that the whole list is not created instantly in the memory but every time you call the next() function on the iterable it returns one item by storing one item in memory at a time.

Summary

• List comprehensions are widely used in coding practices to make the code compact.
• List comprehension can save you a lot of space and provide good readability.
• Syntax for list comprehension list_of_item = [ expression for item in iterable if conditional]
• Nested list comprehensions are also used in practice.
• Don’t over use list comprehensions in your code just to look cool.
• Make wise decision while using list comprehension based on “Zen of Python” principles.
• General rule of thumb, don’t use list comprehension if the "for" loops and "if" conditional count is more than 2.
• Use generators if list grow large in size.