Абстракция множеств и словарей Python

Введение

List Comprehension - Абстракция списков или списковое включение или генератор списков.

Абстракция списков или списковое включение (англ. list comprehension) в синтаксисе некоторых языков программирования — это способ компактного описания операций обработки списков.

Список можно создать и без применения абстракции списка. Например, с помощью цикла for

>>> squares = [] >>> for i in range(10): squares.append(i * i) >>> squares

Изучив синтаксис абстракции списка можно переписать это решение всего в одну строку

>>> squares = [i * i for i in range(10)] >>> squares

Результат в обоих случаях одинаковый

[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Базовый синтаксис

Вместо того, чтобы создавать пустой список и добавлять каждый элемент в конец, вы просто определяете список и его содержимое одновременно, следуя этому формату:

new_list = [expression for member in iterable]

[ expr(item) for item in iterable ]

Каждая абстракция списка в Python состоит из трёх элементов (expression, member, iterable):

• expression is the member itself a call to a method, or any other valid expression that returns a value. В примере
  
  squares = [i * i for i in range(10)]
  
  , the expression i * i is the square of the member value.
  
  
  
  
• member это объект или значение в списке или другом итерируемом объекте. В данном примере значение member это i.
  
  
  
  
• iterable может быть списком, множеством, последовательностью, генератором или любым другим объектом, который способен возвращать свои элементы по одному за раз.
  
  В данном примере iterable это range(10).

Ещё один простейший пример:

sites = ["urn.su", "heihei.ru", "devhops.ru", "topbicycle.ru"] print([len(site) for site in sites])

[6, 9, 10, 13]

Because the expression requirement is so flexible, a list comprehension in Python works well in many places where you would use map(). You can rewrite the pricing example with its own list comprehension:

>>> txns = [1.09, 23.56, 57.84, 4.56, 6.78] >>> TAX_RATE = .08 >>> def get_price_with_tax(txn): ... return txn * (1 + TAX_RATE) >>> final_prices = [get_price_with_tax(i) for i in txns] >>> final_prices

[1.1772000000000002, 25.4448, 62.467200000000005, 4.9248, 7.322400000000001]

Единственная разница между этим решение и использованием map() состоит в том, что абстракция списка в Python возвращает список а map() возвращает map object.

Выгода от использования абстракции списка

List comprehensions are often described as being more Pythonic than loops or map().

But rather than blindly accepting that assessment, it’s worth it to understand the benefits of using a list comprehension in Python when compared to the alternatives.

Later on, you’ll learn about a few scenarios where the alternatives are a better choice.

One main benefit of using a list comprehension in Python is that it’s a single tool that you can use in many different situations.

In addition to standard list creation, list comprehensions can also be used for mapping and filtering.

You don’t have to use a different approach for each scenario.

This is the main reason why list comprehensions are considered Pythonic, as Python embraces simple, powerful tools that you can use in a wide variety of situations.

As an added side benefit, whenever you use a list comprehension in Python, you won’t need to remember the proper order of arguments like you would when you call map().

List comprehensions are also more declarative than loops, which means they’re easier to read and understand.

Loops require you to focus on how the list is created.

You have to manually create an empty list, loop over the elements, and add each of them to the end of the list.

With a list comprehension in Python, you can instead focus on what you want to go in the list and trust that Python will take care of how the list construction takes place.

How to Supercharge Your Comprehensions

In order to understand the full value that list comprehensions can provide, it’s helpful to understand their range of possible functionality. You’ll also want to understand the changes that are coming to the list comprehension in Python 3.8.

Использование Conditional Logic

Ранее вы уже видели формулу создания list comprehensions:

new_list = [expression for member in iterable]

С этой формулой всё в порядке но обычно используется более сложная версия с поддержкой логических условий.

Самый распростанённый способ добавить условие (conditional logic) к абстракции списка - это добавить его в конец выражения

new_list = [expression for member in iterable (if conditional)]

В этом примере условное выражение находится прямо перед закрывающей скобкой.

Условные выражения полезны тем, что они позволяют абстракциям списков отфильтровывать нежелательные значения без вызова функции filter():

Пример с условием:

sites = ["urn.su", "heihei.ru", "devhops.ru", "topbicycle.ru"] it = ["urn.su", "devhops.ru"] print([len(site) for site in sites if site in it])

[6, 10]

Ещё один пример

>>> sentence = 'the rocket came back from mars' >>> vowels = [i for i in sentence if i in 'aeiou'] >>> vowels

['e', 'o', 'e', 'a', 'e', 'a', 'o', 'a']

В этом примере условное выражение отфильтровывает все символы, которые не являются гласными.

Фильтр в отдельной функции

С помощью условий можно проверить любое валидное выражение. Если нужна более сложная фильтрация её можно вынести в отдельную функцию:

sentence = 'The rocket, who was named Ted, came back \ from Mars because he missed his friends.' def is_consonant(letter): vowels = 'aeiou' return letter.isalpha() and letter.lower() not in vowels consonants = [i for i in sentence if is_consonant(i)] print(consonants)

['T', 'h', 'r', 'c', 'k', 't', 'w', 'h', 'w', 's', 'n', 'm', 'd', \ 'T', 'd', 'c', 'm', 'b', 'c', 'k', 'f', 'r', 'm', 'M', 'r', 's', 'b', \ 'c', 's', 'h', 'm', 's', 's', 'd', 'h', 's', 'f', 'r', 'n', 'd', 's']

В этом примере создан сложный фильтр is_consonant() который передаётся в качестве условного выражения в абстракцию списка.

Обратите внимание, что member значение i передаётся как аргумент в функцию.

Предварительное изменение

Условное выражение можно поставить в конец для простой фильтрации, но если нужно изменить значение member вместо фильтрации условное выражение нужно поместить в начало:

new_list = [expression (if conditional) for member in iterable]

С помощью этой формулы можно использовать услоные выражения для обработки различных входных значений.

Например, если у вас есть список цен, вам может быть потребуется заменить отрицательные значения на нули а положительные оставить без изменений:

>>> original_prices = [1.25, -9.45, 10.22, 3.78, -5.92, 1.16] >>> prices = [i if i > 0 else 0 for i in original_prices] >>> prices

[1.25, 0, 10.22, 3.78, 0, 1.16]

В этом примере выражение i содержит условное выражение if i > 0 else 0.

Это говорит Python вывести значение i если число положительное, но если число отрицательное вывести 0.

Возможно, более понятным будет вариант с отдельной функцией:

>>> def get_price(price): ... return price if price > 0 else 0 >>> prices = [get_price(i) for i in original_prices] >>> prices

[1.25, 0, 10.22, 3.78, 0, 1.16]

Теперь ваше условное выражение содержится внутри функции get_price(), и вы можете использовать его как часть выражения для абстракции списка.

Using Set and Dictionary Comprehensions While the list comprehension in Python is a common tool, you can also create set and dictionary comprehensions. A set comprehension is almost exactly the same as a list comprehension in Python. The difference is that set comprehensions make sure the output contains no duplicates. You can create a set comprehension by using curly braces instead of brackets:

>>> quote = "life, uh, finds a way" >>> unique_vowels = {i fori in quote if i in 'aeiou'} >>> unique_vowels

{'a', 'e', 'u', 'i'}

Your set comprehension outputs all the unique vowels it found in quote. Unlike lists, sets don’t guarantee that items will be saved in any particular order. This is why the first member of the set is a, even though the first vowel in quote is i. Dictionary comprehensions are similar, with the additional requirement of defining a key:

>>> squares = {i: i * i fori in range(10)} >>> squares

{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}

To create the squares dictionary, you use curly braces ({}) as well as a key-value pair (i: i * i) in your expression. Using the Walrus Operator Python 3.8 will introduce the assignment expression, also known as the walrus operator. To understand how you can use it, consider the following example. Say you need to make ten requests to an API that will return temperature data. You only want to return results that are greater than 100 degrees Fahrenheit. Assume that each request will return different data. In this case, there’s no way to use a list comprehension in Python to solve the problem. The formula expression for member in iterable (if conditional) provides no way for the conditional to assign data to a variable that the expression can access. The walrus operator solves this problem. It allows you to run an expression while simultaneously assigning the output value to a variable. The following example shows how this is possible, using get_weather_data() to generate fake weather data:

>>> import random >>> def get_weather_data(): ... return random.randrange(90, 110) >>> hot_temps = [temp for_ in range(20) if (temp := get_weather_data()) >= 100] >>> hot_temps

[107, 102, 109, 104, 107, 109, 108, 101, 104]

You won’t often need to use the assignment expression inside of a list comprehension in Python, but it’s a useful tool to have at your disposal when necessary. When Not to Use a List Comprehension in Python List comprehensions are useful and can help you write elegant code that’s easy to read and debug, but they’re not the right choice for all circumstances. They might make your code run more slowly or use more memory. If your code is less performant or harder to understand, then it’s probably better to choose an alternative. Watch Out for Nested Comprehensions Comprehensions can be nested to create combinations of lists, dictionaries, and sets within a collection. For example, say a climate laboratory is tracking the high temperature in five different cities for the first week of June. The perfect data structure for storing this data could be a Python list comprehension nested within a dictionary comprehension:

>>> cities = ['Austin', 'Tacoma', 'Topeka', 'Sacramento', 'Charlotte'] >>> temps = {city: [0 for_ in range(7)] forcity in cities} >>> temps

{ 'Austin': [0, 0, 0, 0, 0, 0, 0], 'Tacoma': [0, 0, 0, 0, 0, 0, 0], 'Topeka': [0, 0, 0, 0, 0, 0, 0], 'Sacramento': [0, 0, 0, 0, 0, 0, 0], 'Charlotte': [0, 0, 0, 0, 0, 0, 0] }

You create the outer collection temps with a dictionary comprehension. The expression is a key-value pair, which contains yet another comprehension. This code will quickly generate a list of data for each city in cities. Nested lists are a common way to create matrices, which are often used for mathematical purposes. Take a look at the code block below:

>>> matrix = [[i fori in range(5)] for_ in range(6)] >>> matrix

[ [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4] ]

The outer list comprehension [... for_ in range(6)] creates six rows, while the inner list comprehension [i fori in range(5)] fills each of these rows with values. So far, the purpose of each nested comprehension is pretty intuitive. However, there are other situations, such as flattening nested lists, where the logic arguably makes your code more confusing. Take this example, which uses a nested list comprehension to flatten a matrix:

matrix = [ ... [0, 0, 0], ... [1, 1, 1], ... [2, 2, 2], ... ] >>> flat = [num forrow in matrix fornum in row] >>> flat

[0, 0, 0, 1, 1, 1, 2, 2, 2]

The code to flatten the matrix is concise, but it may not be so intuitive to understand how it works. On the other hand, if you were to use for loops to flatten the same matrix, then your code will be much more straightforward:

>>> matrix = [ ... [0, 0, 0], ... [1, 1, 1], ... [2, 2, 2], ... ] >>> flat = [] >>> forrow in matrix: ... fornum in row: ... flat.append(num) ... >>> flat

[0, 0, 0, 1, 1, 1, 2, 2, 2]

Now you can see that the code traverses one row of the matrix at a time, pulling out all the elements in that row before moving on to the next one. While the single-line nested list comprehension might seem more Pythonic, what’s most important is to write code that your team can easily understand and modify. When you choose your approach, you’ll have to make a judgment call based on whether you think the comprehension helps or hurts readability. Choose Generators for Large Datasets A list comprehension in Python works by loading the entire output list into memory. For small or even medium-sized lists, this is generally fine. If you want to sum the squares of the first one-thousand integers, then a list comprehension will solve this problem admirably:

>>> sum([i * i fori in range(1000)])

332833500

But what if you wanted to sum the squares of the first billion integers? If you tried then on your machine, then you may notice that your computer becomes non-responsive. That’s because Python is trying to create a list with one billion integers, which consumes more memory than your computer would like. Your computer may not have the resources it needs to generate an enormous list and store it in memory. If you try to do it anyway, then your machine could slow down or even crash. When the size of a list becomes problematic, it’s often helpful to use a generator instead of a list comprehension in Python. A generator doesn’t create a single, large data structure in memory, but instead returns an iterable. Your code can ask for the next value from the iterable as many times as necessary or until you’ve reached the end of your sequence, while only storing a single value at a time. If you were to sum the first billion squares with a generator, then your program will likely run for a while, but it shouldn’t cause your computer to freeze. The example below uses a generator:

>>> sum(i * i fori in range(1000000000))

333333332833333333500000000

You can tell this is a generator because the expression isn’t surrounded by brackets or curly braces. Optionally, generators can be surrounded by parentheses. The example above still requires a lot of work, but it performs the operations lazily. Because of lazy evaluation, values are only calculated when they’re explicitly requested. After the generator yields a value (for example, 567 * 567), it can add that value to the running sum, then discard that value and generate the next value (568 * 568). When the sum function requests the next value, the cycle starts over. This process keeps the memory footprint small. map() also operates lazily, meaning memory won’t be an issue if you choose to use it in this case:

>>> sum(map(lambda i: i*i, range(1000000000)))

333333332833333333500000000

It’s up to you whether you prefer the generator expression or map(). Profile to Optimize Performance So, which approach is faster? Should you use list comprehensions or one of their alternatives? Rather than adhere to a single rule that’s true in all cases, it’s more useful to ask yourself whether or not performance matters in your specific circumstance. If not, then it’s usually best to choose whatever approach leads to the cleanest code! If you’re in a scenario where performance is important, then it’s typically best to profile different approaches and listen to the data. timeit is a useful library for timing how long it takes chunks of code to run. You can use timeit to compare the runtime of map(), for loops, and list comprehensions:

>>> import random >>> import timeit >>> TAX_RATE = .08 >>> txns = [random.randrange(100) for_ in range(100000)] >>> def get_price(txn): ... return txn * (1 + TAX_RATE) ... >>> def get_prices_with_map(): ... return list(map(get_price, txns)) ... >>> def get_prices_with_comprehension(): ... return [get_price(txn) fortxn in txns] ... >>> def get_prices_with_loop(): ... prices = [] ... fortxn in txns: ... prices.append(get_price(txn)) ... return prices ... >>> timeit.timeit(get_prices_with_map, number=100)

2.0554370979998566

>>> timeit.timeit(get_prices_with_comprehension, number=100)

2.3982384680002724

>>> timeit.timeit(get_prices_with_loop, number=100)

3.0531821520007725

Here, you define three methods that each use a different approach for creating a list. Then, you tell timeit to run each of those functions 100 times each. timeit returns the total time it took to run those 100 executions. As the code demonstrates, the biggest difference is between the loop-based approach and map(), with the loop taking 50% longer to execute. Whether or not this matters depends on the needs of your application. Conclusion In this tutorial, you learned how to use a list comprehension in Python to accomplish complex tasks without making your code overly complicated. Now you can: Simplify loops and map() calls with declarative list comprehensions Supercharge your comprehensions with conditional logic Create set and dictionary comprehensions Determine when code clarity or performance dictates an alternative approach Whenever you have to choose a list creation method, try multiple implementations and consider what’s easiest to read and understand in your specific scenario. If performance is important, then you can use profiling tools to give you actionable data instead of relying on hunches or guesses about what works the best. Remember that while Python list comprehensions get a lot of attention, your intuition and ability to use data when it counts will help you write clean code that serves the task at hand. This, ultimately, is the key to making your code Pythonic!