In the previous episodes

basic stuff, syntax
functions (wrote our cool min function) + functional stuff
scopes
PEP-8
strings
bytes
collections
classes basics
decorators – try more here
functools

‼️ Exceptions

Exceptions are for exceptional situations.

>>> [0] * int(1e16)
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
MemoryError
>>> import foobar
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
ImportError: No module named 'foobar'
>>> [1, 2, 3] + 4
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
TypeError: can only concatenate list to list

They are errors, which we can process.

👇 How to process

>>> try:
... 	something_dangerous()
... except (ValueError, ArithmeticError): 
...		pass
... except TypeError as e:
... 	pass

Variable stays inside

>>> try:
...     1 + "42"
... except TypeError as e:
... 	pass
>>> e
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
NameError: name 'e' is not defined

BaseException

>>> BaseException.__subclasses__()
[<class 'Exception'>, <class 'GeneratorExit'>,
   <class 'KeyboardInterrupt'>, <class 'SystemExit'>]
>>> try:
... 	something_dangerous()
... except Exception: 
... 	pass

Finally, assert

>>> assert 2 + 2 == 5, ("Math", "still", "works") 
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
AssertionError: ('Math', 'still', 'works')

do not except AssertionError

More exceptions…

>>> import foobar
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
ImportError: No module named 'foobar'
>>> foobar
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
NameError: name 'foobar' is not defined

And more…

>>> object().foobar
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
AttributeError: 'object' object has no attribute 'foobar'
>>> {}["foobar"]
Traceback (most recent call last):
File "<stdin>", line 1, in <module> KeyError: 'foobar'
>>> [][0]
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
IndexError: list index out of range

And even more…

>>> "foobar".split("")
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
ValueError: empty separator
>>> b"foo" + "bar"
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
TypeError: can't concat bytes to str

see https://docs.python.org/3/library/exceptions.html

Good practice: make your own base class

>>> class OurException(Exception):
... 	pass
...
>>> class TestFailure(OurException): 
... 	def __str__(self):
... 		return "lecture test failed"

Why?

This will allow you to

>>> try:
... 	do_something() 
... except OurException: 
...     # ...

The exception interface is easy

>>> try:
... 	1 + "42"
... except Exception as e:
... 	caught = e
...
>>> caught.args
("unsupported operand type(s) for +: 'int' and 'str'",) 
>>> caught.__traceback__
<traceback object at 0x10208d148>
>>> import traceback
>>> traceback.print_tb(caught.__traceback__)
File "<stdin>", line 2, in <module>

Rise and shine, new exception

>>> raise TypeError("type mismatch")
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
TypeError: type mismatch
>>> raise 42
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
TypeError: exceptions must derive from BaseException
>>> raise
Traceback (most recent call last):
	File "<stdin>", line 1, in <module>
RuntimeError: No active exception to reraise

raise from…

>>> try:
... 	{}["foobar"]
... except KeyError as e:
... 	raise RuntimeError("Ooops!") from e 
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
KeyError: 'foobar'
The [...] exception was the [...] cause of the following [...]:
Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
RuntimeError: Ooops!

else!

>>> try:
... 	handle = open("example.txt", "wt")
... else:
... 	report_success(handle)
... except IOError as e:
... 	print(e, file=sys.stderr)

finally, finally:

>>> try:
... 	{}["foobar"]
... finally:
... 	"foobar".split("")
...
Traceback (most recent call last):
	File "<stdin>", line 2, in <module>
KeyError: 'foobar'
During handling of [...] exception, [...] exception occurred:
Traceback (most recent call last):
	File "<stdin>", line 4, in <module>
ValueError: empty separator

Not having any exception handler is slightly faster (but blows up in your face when the exception happens), and try/except is faster than an explicit if as long as the condition is not met.

Exceptions wrap-up

similar to C++/Java, but Python has else
raise is like throw in C++/Java
inherite from Exception
try to minimize size of try
try not to use too broad exceptions

🤔 Context managers

>>> r = acquire_resource()
... try:
... 	do_something(r)
... finally:
... 	release_resource(r)

to this

>>> with acquire_resource() as r:
... 	do_something(r)

Context manager implementation

implement __enter__
implement __exit__

with syntax

>>> with acquire_resource() as r:
...

>>> r = manager.__enter__()
>>> try:
... 	do_something(r)
... finally:
... 	exc_type, exc_value, tb = sys.exc_info()
... 	suppress = manager.__exit__(exc_type, exc_value, tb)
... if exc_value is not None and not suppress:
... 	raise exc_value

Toy example

>>> from functools import partial
>>> class opened:
... 	def __init__(self, path, *args, **kwargs):
... 		self.opener = partial(open, path, *args, **kwargs)
...
... 	def __enter__(self):
... 		self.handle = self.opener()
... 		return self.handle
...
... 	def __exit__(self, *exc_info):
... 		self.handle.close()
... 		del self.handle # where is return???
...
>>> with opened("./example.txt", mode="rt") as handle:
... 	pass

with tempfile

>>> import tempfile
>>> with tempfile.TemporaryFile() as handle:
... 	path = handle.name
... 	print(path)
...
/var/folders/nj/T/tmptpy6nn5y
>>> open(path)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory

contextlib: closing

>>> from contextlib import closing
>>> from urllib.request import urlopen
>>> url = "http://compscicenter.ru"
>>> with closing(urlopen(url)) as page:
... 	do_something(page)

contextlib: redirect_stdout

>>> from contextlib import redirect_stdout
>>> import io
>>> handle = io.StringIO()
>>> with redirect_stdout(handle):
... 	print("Hello, World!")
...
>>> handle.getvalue()
'Hello, World!
'

contextlib: supress

>>> from contextlib import suppress
>>> with suppress(FileNotFoundError):
... 	os.remove("example.txt")

supress realization

>>> class supress:
... 	def __init__(self, *suppressed):
... 		self.suppressed = suppressed
...
... 	def __enter__(self):
... 		pass
...
... 	def __exit__(self, exc_type, exc_value, tb):
... 		return (exc_type is not None and
... 			issubclass(exc_type, suppressed))

contextlib: ContextDecorator

def f():
    with context():
	    # ...

to…

@context()
def f():
    # ...

Saves 4 spaces in every string!

>>> from contextlib import (suppress as _suppress,
... 	ContextDecorator)
>>> class suppressed(_suppress, ContextDecorator):
... 	pass
...
>>> @suppressed(IOError)
... def do_something():
... 	pass

many resources?

>>> def merge_logs(output_path, *logs):
... 	handles = open_files(logs)
... 	with open(output_path, "wt") as output:
... 		merge(output, handles)
... 	close_files(logs)

contextlib: ExitStack

>>> from contextlib import ExitStack
>>> def merge_logs(output_path, *logs):
... 	with ExitStack() as stack:
... 		handles = [stack.enter_context(open(log))
... 				   for log in logs]
... 	output = open(output_path, "wt")
... 	stack.enter_context(output)
... 	merge(output, handles)

ExitStack() is a context manager too

>>> with ExitStack() as stack:
... 	stack.enter_context(some_resource)
... 	stack.enter_context(other_resource)
... 	do_something(some_resource, other_resource)

Calls __exit__ in every context manager (reversed order)

Context managers wrap-up

context manager is a useful way to control resources lifecycle in Python
used with with, to implement in your object just create __enter_ and __exit_
some default types (e.g. files) already support with – they have context managers, use it
contextlib sometimes can make your life easier

🏃‍♀️ Iterators

for…

>>> import dis
>>> dis.dis("for x in xs: do_something(name)")
	0 SETUP_LOOP 		24 (to 27)
	3 LOAD_NAME 		0 (xs)
	6 GET_ITER
>> 	7 FOR_ITER 			16 (to 26)
	10 STORE_NAME 		1 (x)
	[...]
>> 	26 POP_BLOCK
>> 	27 LOAD_CONST 		0 (None)
	30 RETURN_VALUE

Iterator protocol

GET_ITER calls __iter__ which returns an iterator
FOR_ITER calls __next__ which returns an element from iterable, until StopIteration exception.

iter() behaviour

gets an iterator and calls __iter__
gets a function and calls it until the needed value

from functools import partial
with open(path, "rb") as handle:
	read_block = partial(handle.read, 64)
	for block in iter(read_block, ""):
		do_something(block)

next() behaviour

gets an iterator and calls __next__

>>> next(iter([1, 2, 3]))
1
>>> next(iter([]), 42)
42

for x in xs:
	do_something(x)

is like

it = iter(xs)
while True:
    try:
        x = next(it)
    except StopIteration:
        break
 do_something(x)

in, not in

Default implementation:

class object:
    # ...
    def __contains__(self, target):
        for item in self:
            if item == target:
                return True
        return False

>>> id = Identity()
>>> 5 in id # ≡ id.__contains__(5)
True
>>> 42 not in id # ≡ not id.__contains__(42)
True

Simpler iterator protocol implementation

__getitem__ gets 1 argument – an index

returns element from given index
raises IndexError otherwise

>>> class Identity:
...     def __getitem__(self, idx):
...         if idx > 5:
...             raise IndexError(idx)
...         return idx
...
>>> list(Identity())
[0, 1, 2, 3, 4, 5]

Iterators wrap-up

iterator is an object of a class which has __init__ and __next__ methods.
alternatively, it’s an object of a class which has __getitem__ (to get default implementation of next and iter)
for/in/not in

4 · Exceptions · context managers · iterators