{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Understanding Python Errors" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Reading errors" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By now, you have likely encountered Python errors very often. Here is an example of an error:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "s = 'hello'\n", "s[0] = 'a'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is called an **exception traceback**. The **exception** is the error itself, and the **traceback** is the information that shows where it occured. The above traceback is quite simple (because the code producing the error is quite simple). The most important thing you should look at in an error is the last line, in this case:\n", "\n", " 'str' object does not support item assignment\n", " \n", "Now let's look at a slightly more complex error:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "\n", "def subtract_smooth(x, y):\n", " y_new = y - median_filter(x, y, 2.5)\n", " return y_new\n", "\n", "def median_filter(x, y, width):\n", " y_new = np.zeros(y.shape)\n", " for i in range(len(x)):\n", " y_new[i] = np.median(y[np.abs(x - x[i]) < width * 0.5])\n", " return y_new" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "subtract_smooth(np.array([1,2,3,4,5]),np.array([4,5,6,8]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The error is now more complex. The first line shows what top-level code was executed when the error occured - in this case the call to ``subtract_smooth``:\n", "\n", " IndexError Traceback (most recent call last)\n", " in \n", " ----> 1 subtract_smooth(np.array([1,2,3,4,5]),np.array([4,5,6,8]))\n", " \n", "The next chunk shows where the error occured inside ``subtract_smooth``:\n", "\n", " in subtract_smooth(x, y)\n", " 2 \n", " 3 def subtract_smooth(x, y):\n", " ----> 4 y_new = y - median_filter(x, y, 2.5)\n", " 5 return y_new\n", "\n", " \n", "you can see it happened when calling ``median_filter``. Finally, we can see where the error occured inside ``median_filter``:\n", "\n", " in median_filter(x, y, width)\n", " 8 y_new = np.zeros(y.shape)\n", " 9 for i in range(len(x)):\n", " ---> 10 y_new[i] = np.median(y[np.abs(x - x[i]) < width * 0.5])\n", " 11 return y_new\n", " \n", "So tracebacks show you the full history of the error!\n", "\n", "Now in the above case, the final error is:\n", "\n", " IndexError: boolean index did not match indexed array along dimension 0; dimension is 4 but corresponding \n", " boolean dimension is 5\n", " \n", "Why is this occuring? The only place that boolean indices are used here is when doing:\n", "\n", " np.abs(x - x[i]) < width * 0.5\n", " \n", "The issue is that if we look back at the original function call, there are more values for ``x`` than for ``y``!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using the debugger" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the above example, the code was still simple enough that we could guess the solution, but sometimes things are not so simple. One way to diagnose the issue would have been to print out the content of the variables in ``median_filter`` and run it again to see what was going on.\n", "\n", "However, Python includes a *debugger*, which allows you to jump right in to where the error happened, and look at the variables. In the IPython notebook or in IPython, once an error has happened, you can run ``%debug``, and you will see a ``ipdb>`` prompt (IPython debugger). You can then print out variables to see what they are set to. Let's try the above example again:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "subtract_smooth(np.array([1,2,3,4,5]),np.array([4,5,6,8]))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%debug" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see that the boolean array that is being used as indices to ``y`` is too big. Much simpler! Type ``exit`` to exit the debugger." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Catching exceptions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In some cases, we know that errors might happen, and we don't want them to crash the code. For example, if we have to read in 1000 files, a few might be corrupt, and we just want to skip over them. Or we want to try something, and if it doesn't work, do something else. To do this, we can **catch** exceptions:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "s = 'hello'" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "s[1] = 'a'" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "try:\n", " s[1] = 'a'\n", "except:\n", " print(\"Can't set s[1]\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The ``try...except`` contruct above catches *all* exceptions, but in some cases we want to be a bit more specific. The error that occurs above is a ``TypeError``, which is just one kind of error (others include ``ValueError``, ``SystemError``, etc.). To catch just ``TypeError``, you can do:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "try:\n", " s[1] = 'a'\n", "except TypeError:\n", " print(\"Can't set s[1]; TypeError\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you catch other errors, TypeError will pass" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "try:\n", " s[1] = 'a'\n", "except ValueError: \n", " print(\"Can't set s[1]\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Catching exceptions is tricky business, however, because it can hide important hints for debugging." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Raising Errors" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If something goes wrong in your code, rather than just printing some warning and going on, it's usually much better to raise an exception yourself. You'll make sure execution halts where an error has occurred, rather than having confusing junk results much later, when it is hard to figure out what went wrong where." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Raising exceptions is not hard:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "raise ValueError(\"But writing good, descriptive error messages is.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "``raise`` is another reserved word in Python. What's behind it is an exception class; you can define these yourself once you know how to deal with objects and inheritance, but there's nothing wrong with re-using exceptions you know from the standard library: ``ValueError`` if some value doesn't meet your expectation, ``IOError`` if something is wrong with a file, ``KeyError`` if a lookup of something failed... You can always use ``Exception`` (which is a “base class” of all these exceptions) if unsure." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note, however, that it's almost always a bad idea to catch an exception and re-raise one of your design. You'll most likely be hiding the root cause of the problem, making analysis of the problem much harder." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Also, don't claim more than you know in your error message (and don't execute the following code if you're running your notebook as root (which of course one should not do anyway!)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# THIS IS HOW *NOT* TO DO IT!\n", "try:\n", " with open(\"/etc/passwd\", \"a\") as f:\n", " pass\n", "except IOError: \n", " # don't do this anyway: Catch and raise something else\n", " # And in particular don't claim more than you know.\n", " raise Exception(\"/etc/passwd does not exist\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "At least on unix systems, the existence claim is patently untrue, as you can verify with ls. What's failed is that you're not allowed to write to that file. By claiming more than you actually know in your error message, you will confuse users." ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 1 }