Логотип Хекслета

Введение в Python

Теория: Python list comprehensions and generators

List comprehensions and generators in Python

What is a list comprehension

  • An expression that creates a collection based on another collection
  • Produces a list in general
  • Short clear syntax
  • Supports predicate clause to filter values
  • Might be lazy
  • One of the most powerful Python features

Just to compare

numbs = [1, 2, 3, 4, 5]
result = []
for x in numbs:
if x > 3:
y = x * x
result.append(y)
numbs = [1, 2, 3, 4, 5]
result = [x * x for x in numbs if x > 3]

List comprehension in depth

[stmt for var in iterable if predicate]

where:

  • stmt — any statement,
  • var — variable(s) on each iteration step,
  • iterable — any iterable value,
  • predicate — True/False statement with var in scope
[x for x in [1, 2, 3]]                           [1, 2, 3]
[x * x for x in [1, 2, 3]]                       [1, 4, 9]
[x for x in [1, 2, 3, 4, 5] if x % 2 == 0]       [2, 4]
[x + y for x, y in ((1, 2), (2, 3), (3, 4))]     [3, 5, 7]
strs = ['foo', 'bar', 'test']
[x.upper() for x in strs if len(x) < 4]          ['FOO', 'BAR']

users = [{'name': 'ivan', 'age': 29},
         {'name': 'juan', 'age': 31}]
[user['name'] for user in users if user['age'] > 30]       ['juan']

mydict = {'foo': 'test', 'bar': None}
[k for (k, v) in mydict.iteritems() if v is not None]      ['foo']

List comprehension — conclusion

  • Square brackets syntax
  • Creates list
  • Computes all values when created
  • Aware of long lists (save memory)
  • Use it rather than cycles

Set comprehensions

  • Same as a list comprehension, but...
  • Curly brackets syntax
  • Returns a set as a result, so...
  • Guaranties unique elements
{x for x in [1, 2, 3]}                      {1, 2, 3}
{x for x in [1, 2, 2, 2, 3, 1]}             {1, 2, 3}
list({x for x in [5, 2, 1, 1, 2, 5]})       [1, 2, 5]

users = [{'age': 20}, {'age': 90}]
{user for user in users if user['age'] > 30}
>>> TypeError: unhashable type: 'dict'

Dict comprehension

  • Curly brackets syntax
  • key: value statement
{k: v for (k, v) in [('foo', 1), ('bar', 2)]}
# {'bar': 2, 'foo': 1}

# fix string keys
mydict = {'1': 'value1', '2': 'value2'}
{int(k): v for (k, v) in mydict.iteritems()}
# {1: 'value1', 2: 'value2'}

# reverse a dictionary
{v: k for k, v in mydict.iteritems()}
# {'value1': '1', 'value2': '2'}

# drop keys with None values
mydict = {'foo': 42, 'bar': None}
{k: v for k, v in mydict.iteritems() if v is not None}
# {'foo': 42}

Generator object

  • Round brackets syntax
  • It's not a tuple!
  • Lazy sequence
  • Each element computes only when it requires
  • Saves memory and CPU

Lazy!

data = [1.0 / x for x in [3, 2, 1, 0]]
>>> ZeroDivisionError: float division by zero

data = (1.0 / x for x in [3, 2, 1, 0])
for x in data:
    print x

>>> 0.333333333333   0.5   1.0
>>> ZeroDivisionError

Iteration only

  • you cannot access value by index
  • you don't know the length
  • you can pass through it only one time
  • you cannot iterate backwards
  • pass it to list/tuple/set to get a common collection
gen = (x for x in [1, 2, 3])
print gen
<generator object <genexpr> at 0x105a12e60>

print gen[1]             TypeError: 'generator' object has no attribute '__getitem__'
len(gen)                 TypeError: object of type 'generator' has no len()
data = list(gen)         [1, 2, 3]

for x in gen:
    print x
# will print nothing! Why?

range vs xrange

range(1, 10000) >>> [1, 2, 3, ..., 9998, 9999, 10000] xrange(1, 10000) >>> xrange object

Generators are everywhere

  • Mathematical sequence (Fibonacci, progressions)
  • Network calls results = (get_data_from_server(user) for user in (...))
  • ORM: Django, SQLAlchemy (querysets)
  • Parsers (json, xml)
  • File readers
  • Almost everything in Py3 became lazy

Look for itertools module

chain('ABC', 'DEF')                        A B C D E F
combinations('ABCD', 2)                    AB AC AD BC BD CD
compress('ABCDEF', [1,0,1,0,1,1])          A C E F
count(2.5, 0.5)                            2.5  3.0  3.5 ...
cycle('ABCD')                              A B C D A B C D A B C D
izip_longest('ABCD', 'xy', fillvalue='-')  Ax By C- D-
product('ABCD', 'xy')                      Ax Ay Bx By Cx Cy Dx Dy
imap, islice, ifilter, etc...
tonnes of recipes

Next lesson

  • How does iteration work under cover
  • Create generators using functions
  • Yield operator
Навигация
Теория
Тесты