5 Hidden Python Features You Probably Never Heard Of

In this article, I’m going to show you the top 5 hidden features you can find in Python. Experienced Python developers might recognize some of them. However, others will still be unknown. Regardless, I find all of them very cool.

...
An Elegant Unpacking
Can You Flat This List?
What else?
Comparing Things Like a Boss
Conclusion

`...`

Yes, you’re reading it right, ... is a valid construct in Python. ... is a singleton object called Ellipsis. If you type it into the Python interpreter, you can actually see it:

>>> ...
Ellipsis

According to the official docs , Ellipsis is a "special value used mostly in conjunction with extended slicing syntax for user-defined container data types.". There’s two major use cases for it. One is to serve as a placeholder body in an empty function. The other is on Numpy, as a slice item, just as described in the docs.

Function Placeholder

def my_awesome_function():
    ...

This is equivalent to:

def my_awesome_function():
    Ellipsis

And this:

def my_awesome_function():
    pass

Beware, I'm not saying that pass == ..., I'm just saying that as a function body, the outcome is the same. In fact, you can use anything as placeholder.

def my_awesome_function():
    "An empty, but also awesome function"

Numpy

The code below basically means create an array of matrices. Each matrix is 3x3. Then get the second column (numpy arrays are 0-based) of all innermost matrix.

>>> import numpy as np
>>> array = np.arange(27).reshape(3, 3, 3)
>>> array
array([[[ 0,  1,  2],
        [ 3,  4,  5],
        [ 6,  7,  8]],

       [[ 9, 10, 11],
        [12, 13, 14],
        [15, 16, 17]],

       [[18, 19, 20],
        [21, 22, 23],
        [24, 25, 26]]])
>>> array[..., 1] 
array([[ 1,  4,  7],
       [10, 13, 16],
       [19, 22, 25]])
>>> # This is equivalent to
>>> array[:, :, 1] 
array([[ 1,  4,  7],
       [10, 13, 16],
       [19, 22, 25]])

Beware: Python's list doesn't work with ....

An Elegant Unpacking

Iterable unpacking is a remarkably convenient feature and has been there for a while. Most people use it to unpack iterables with multiple items. As example, consider the following use cases.

>>> a, *b, c = range(1, 11)
>>> a
1
>>> c
10
>>> b
[2, 3, 4, 5, 6, 7, 8, 9]

Or just:

>>> a, b, c = range(3)
>>> a
0
>>> b
1
>>> c
2

But one nice use case that many people do not take advantage of is unpacking a single iterable. Why this is useful? It makes the code a little bit more elegant, IMHO.

Instead of doing this:

>>> lst = [1]
>>> a = lst[0]
>>> a
1
>>> (a, ) = lst
>>> a
1

You can do this:

>>> lst = [1]
>>> [a] = lst
>>> a
1

I know that it may seem silly, but at least to me, it looks more elegant.

Can You Flat This List?

Flattening a list can be done in several ways. The simplest one is using list comprehension.

>>> l = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> flattened = [elem for sublist in l for elem in sublist]
>>> flattened
[1, 2, 3, 4, 5, 6, 7, 8, 9]

If you're more inclined to functional programming, you can use a reducer.

>>> from functools import reduce
>>> reduce(lambda x,y: x+y,l)
[1, 2, 3, 4, 5, 6, 7, 8, 9]

However, there's yet another way. You can use the sum function!

>>> sum(l, [])
[1, 2, 3, 4, 5, 6, 7, 8, 9]

This works because the sum function iterates through each element in the list and concatenates them with the default value you pass as the second argument. Since lists in Python can be concatenated with + operator, then you get something like this:

>>> sum(l, []) ==> [] + [1, 2, 3] + [4, 5, 6] + [7, 8, 9]
[1, 2, 3, 4, 5, 6, 7, 8, 9]

Even though this trick is brilliant, it’s by no means readable. Also, it has a terrible performance.

Can we do this with string?

No, Python forbids doing the same with strings, even though you can concatenated strings with + operator.

>>> s = ["abc", "def", "ghf"]

>>> sum(s, "")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-f60d89b81305> in <module>
----> 1 sum(s, "")

TypeError: sum() can't sum strings [use ''.join(seq) instead]

If we dig intro CPython source code, we can find where this happens:

static PyObject *
builtin_sum_impl(PyObject *module, PyObject *iterable, PyObject *start)
/*[clinic end generated code: output=df758cec7d1d302f input=162b50765250d222]*/
{
    PyObject *result = start;
    PyObject *temp, *item, *iter;

    iter = PyObject_GetIter(iterable);
    if (iter == NULL)
        return NULL;

    if (result == NULL) {
        result = PyLong_FromLong(0);
        if (result == NULL) {
            Py_DECREF(iter);
            return NULL;
        }
    } else {
        /* reject string values for 'start' parameter */
        if (PyUnicode_Check(result)) {
            PyErr_SetString(PyExc_TypeError,
                "sum() can't sum strings [use ''.join(seq) instead]");
            Py_DECREF(iter);
            return NULL;
        }
        if (PyBytes_Check(result)) {
            PyErr_SetString(PyExc_TypeError,
                "sum() can't sum bytes [use b''.join(seq) instead]");
            Py_DECREF(iter);
            return NULL;
        }
        if (PyByteArray_Check(result)) {
            PyErr_SetString(PyExc_TypeError,
                "sum() can't sum bytearray [use b''.join(seq) instead]");
            Py_DECREF(iter);
            return NULL;
        }
        Py_INCREF(result);
    }

github.com/python/cpython/blob/c96d00e88ead..

The `_`

This one is really interesting and very handy when working with the REPL. It not only works with the default Python interpreter, but with IPython as well.

Whenever you run an expression in the REPL, Python binds the output to the _ variable.

>>> nums = [1, 3, 7]
>>> sum(nums)
11
>>> _
11
>>>

Since _ is a variable like any other, you can re-bind it, or do anything else with it.

>>> 9 + _
20
>>> a = _
>>> a
20

What `else`?

The else statement in can serve several purposes. Few people know, but you can use it outside the classical ‘if else` block. Python allow it to be used on loops and also on exception blocks.

Loops

Python has two different loops, for and while. Both of them can be "broken". That is, if a certain condition is met, we can break out of the loop. For example:

In [7]: while a < 10:
   ...:     if a == 3:
   ...:         print("a == 3. exiting loop.")
   ...:         break
   ...:     a += 1
   ...: 
a == 3. exiting loop.

Now, let's say that we are looking for a particular condition. If that condition is satisfied, we save the result in a flag called found. Then, if we don't find it, we print a message.

found = False
a = 0

while a < 10:
    if a == 12:
        found = True
    a += 1
if not found:
    print("a was never found")

Since a never becomes 12, the program outputs a was never found.

Ok, but how can we use else in this context?

The else can be used to replace the flag. Basically, what we actually want is to run the loop and, if not found, then print a message. This is how it looks like with else:

a = 0

while a < 10:
    if a == 12:
        break
    a += 1
else:
    print("a was never found")

And since it works with any loop, you can use a for instead of while.

for a in range(10):
    if a == 12:
        break
    a += 1
else:
    print("a was never found")

Exceptions

The else in Python is so versatile that you can even use it in a try ... except block. The idea here is to capture a nonoccurrence of an exception.

In [13]: try:
    ...:     {}['lala']
    ...: except KeyError:
    ...:     print("Key is missing")
    ...: else:
    ...:     print("Else here")
    ...: 
Key is missing

In this example, we try looking up a key named “lala” in an empty dictionary. Since “lala” is not there, the code will raise an KeyError exception. When I run this snippet in IPython, I got an expected result.

What about a case where the program raises no exception?

In [14]: try:
    ...:     {'lala': 'bla'}['lala']
    ...: except KeyError:
    ...:     print("Key is missing")
    ...: else:
    ...:     print("Else here")
    ...: 
Else here

Now we can see it in action. The {’lala’: ‘bla’}[‘lala’] block won’t raise KeyError, so the else comes into play.

Remember, few people know this, so I personally avoid using this feature to not confuse other developers working in the same code base. It’s nice to impress friends, though!

Comparing Things Like a Boss

This is one of my favorites and not so hidden, to be honest. Unlike many programming languages, like Java, C or C++, Python allows you to chain comparison operators. Let's imagine that you have a variable x that holds the value of 10. Now, let's say that you want to assert that x is within a range, like 5..20, inclusive. You could do something like this:

In [16]: x = 10
In [17]: if x >= 5 and x <= 20:
    ...:     print("x is within range")
    ...: else:
    ...:     print("x is outside range")
    ...: 
is within range

It turns out, this can be simplified by chaining the operators. So, we can refactor the code to this:

In [18]: if 5 <= x <= 20:
    ...:     print("is within range")
    ...: else:
    ...:     print("x is outside range")
    ...: 
is within range

This code achieves the exact same result, but it's much more elegant. You can chain using any kind of comparison operator.

>>> x = 10
>>> 20 == x > 1
False
>>> 25 > x <= 15
True
>>> x < 20 < x*10 < 1000
True

Very cool!

Conclusion

That’s the end of this post. Python is a very friendly language and has some nice, but not well-known, features. In this post, I showed you my favorites and I hope you’ve learned something new.