Using List Comprehension in Python to speed up your code

,

Are you trying to reduce the runtime of your Python code? Faster code can reduce cloud costs, especially with serverless solutions like Azure Functions or AWS Lambda.

One way to have your script run faster is by using so called “list comprehension” in Python. List comprehensions are a concise and often faster way to create lists compared to traditional for-loops. List comprehension can be used to iterate over a certain range of items in an iterable object (like a range or a list) and create a new list based on certain filters or selections.

Imagine you have a dictionary like the one shown below:

dictionary = {
    'items': [
        {'item1': 'value1'}, 
        {'item2': 'value2'}
        ],
    'count': 2
    }

If you want to gather all info within the ‘items’ array (item1 and item2 in this case), you could use a traditional for-loop like below:

item_lijst = []

for item in dictionary['items']:
    item_lijst.append(item)

This would work perfectly well. Using list comprehension, you could get the same result using the snippet of code below:

item_lijst = [item for item in dictionary['items']]

List comprehension will however, get the job done faster than the for-loop in this case. Don’t believe me? Check this out:

# Having run the code below 10 times, runtime for list comprehension was faster in almost all cases. Your results may vary based on hardware among other things. 
import time

# Example data
dictionary = {'items': range(100000)}

# List comprehension
start = time.time()
item_lijst = [item for item in dictionary['items']]
end = time.time()
print(f"List comprehension time: {end - start:.6f} seconds")    # 0.003144 seconds

# For loop with append
start = time.time()
item_lijst = []
for item in dictionary['items']:
    item_lijst.append(item)
end = time.time()
print(f"For loop time: {end - start:.6f} seconds")              # 0.005277 seconds

List comprehensions are faster because they are implemented in C within the Python interpreter, avoiding the overhead of repeatedly calling methods like .append()

Next time you are using a for-loop to iterate over an object and you’d like to speed things up, try using list comprehension! The AI overlord that I used to fact check this blog post suggested to add limitations of list comprehension so here you go: limitations of list comprehension include reduced readability and loading the entire dataset into memory (as is usual with lists). When dealing with a large dataset, a generator expression may be more appropriate. See the next blog posts for more about that.

Leave Comment