Learn Python with Hockey - Part 3

This is the third in a series of posts that teach Python and data science using NHL data.

If you want to be notified when new guides like this come out you can enter your email here:

Let's get started!

All these posts are heavy on examples and meant to be followed along with. Like last time (assuming you've got everything installed) go to:

https://raw.githubusercontent.com/nathanbraun/learn-python-hockey/main/code/part3.py

And copy and paste everything that's there into Spyder (either temp.py or a new file).

Last time we talked about basic Python types like strings, numbers and bools. These are called primitives; they're the basic building block types.

There are other container types that can hold other values. Two important container types are lists and dicts. Sometimes containers are also called collections.

Lists are built with square brackets and are basically a simple way to hold other, ordered pieces of data.

Every spot in a list has a number associated with it. The first spot is 0. You can get sections (called slices) of your list by separating numbers with a colon. Both single numbers and slices are called inside square brackets, i.e. [].

A single integer inside a bracket returns one element of your list, while a slice returns a smaller list. Note a slice returns up to the last number, so [0:2] returns the 0 and 1 items, but not item 2.

Passing a negative number gives you the end of the list. To get the last two items you could do:

Also note how when you leave off the number after the colon the slice will automatically use the end of the list.

Lists can hold anything, including other lists. Lists that hold other lists are often called nested lists.

A dict is short for dictionary. You can think about it like an actual dictionary if you want. Real dictionaries have words and definitions, Python dicts have keys and values.

Dicts are basically a way to hold data and give each piece a name. They're written with curly brackets, like this:

You can access items in a dict like this:

And add new things to dicts like this:

Notice how keys are strings (they're surrounded in quotes). They can also be numbers or even bools. They cannot be a variable that has not already been created. You could do this:

Because when you run it Python is just replacing pos with 'lw'.

But you will get an error if pos is undefined. You also get an error if you try to use a key that's not present in the dict (note: assigning something to a key that isn't there yet — like we did with 'john carlson' above — is OK).

While dictionary keys are usually strings, dictionary values can be anything, including lists or other dicts.

Now that we've seen an example of container types, we can mention unpacking. Unpacking is a way to assign multiple variables at once, like this:

That does the exact same thing as assigning these separately on their own line.

One pitfall when unpacking values is that the number of whatever you're assigning to has to match the number of values available in your container. This would give you an error:

Unpacking isn't used that frequently. Shorter code isn't always necessarily better, and it's probably clearer to someone reading your code if you assign lw and c on separate lines.

However, some built-in parts of Python (including material below) use unpacking, so we needed to touch on it briefly.

Loops are a way to "do something" for every item in a collection.

For example, maybe I have a list of lowercase player names and I want to go through them and change them all to proper name formatting using the title string method, which capitalizes the first letter of every word in a string.

One way to do that is with a for loop:

What's happening here is the last two lines are run multiple times, once for every item in the list. The first time player has the value 'alex ovechkin', the second 'nicklas backstrom', etc. We're also using a variable i to keep track of our position in our list. The last line in the body of each loop is to increment i by one, so that we'll be working with the correct spot the next time we go through it.

The programming term for "going over each element in some collection" is iterating. Collections that allow you to iterate over them are called iterables.

Dicts are also iterables. The default behavior when iterating over dicts is you get access to the keys only. So:

But what if we want access to the values too? One thing we could do is write first_line_dict[x], like this:

But Python has a shortcut that makes things easier: we can add .items() to our dict to get access to the value.

Notice the for x, y... part of the loop. Adding .items() unpacks the key and value into our two loop variables (we choose x and y).

Loops are occasionally useful. And they're definitely better than copying and pasting a bunch of code over and over and making some minor change.

But in many instances, there's a better option: comprehensions.

Comprehensions are a way to modify lists or dicts with not a lot of code. They're like loops condensed onto one line.

Mark Pilgrim, author of Dive into Python, says that every programming language has some complicated, powerful concept that it makes intentionally simple and easy to do. Not every language can make everything easy, because all language decisions involve tradeoffs. Pilgrim says comprehensions are that feature for Python.

When you want to go from one list to another, different list you should be thinking comprehension. Our first for loop example, where we wanted to take our list of lowercase players and make a list where they're all properly formatted, is a great candidate.

The list comprehension way of doing that would be:

All list comprehensions take the form [a for b in c] where c is the list you're iterating over (starting with), and b is the variable you're using in a to specify exactly what you want to do to each item.

In the above example a is x.title(), b is x, and c is first_line.

Note, it's common to use x for your comprehension variable, but — like loops — you can use whatever you want. So this:

does exactly the same thing as the version using x did.

Comprehensions can be tricky at first, but they're not that bad once you get the hang of them. They're useful and we'll see them again though, so if the explanation above is fuzzy, read it again and look at the example until it makes sense.

A comprehension evaluates to a regular Python list. That's a fancy way of saying the result of a comprehension is a list.

And we can slice it and do everything else we could do to a normal list:

There is literally no difference.

Let's do another, more complicated, comprehension:

Remember, all list comprehensions take the form [a for b in c]. The last two are easy: c is just first_line and b is full_name.

That leaves a, which is full_name.split(' ')[1].

Sometimes its helpful to prototype this part in the REPL with an actual item from your list.

We can see split is a string method that returns a list of substrings. After calling it we can pick out each player's last name in spot 1 of our new list.

The programming term for how we've been using comprehensions so far — "doing something" to each item in a collection — is mapping. As in, I mapped title to each element of first_line.

We can also use comprehensions to filter a collection to include only certain items. To do this we add if some criteria that evaluates to a boolean at the end.

Updating our notation, a comprehension technically has the form [a for b in c if d], where if d is optional.

Above, d is x.startswith('a'). The startswith string method takes a string and returns a bool indicating whether the original string starts with it or not. Again, it's helpful to test it out with actual items from our list.

Interestingly, in this comprehension the a in our [a for b in c if d] notation is just x. That means we're doing nothing to the value itself (we're taking x and returning x); the whole purpose of this comprehension is to filter roster_list to only include items that start with 'a'.

You can easily extend this to map and filter in the same comprehension:

Dict comprehensions work similarly to list comprehensions. Except now, the whole thing is wrapped in {} instead of [].

And — like with our for loop over a dict, we can use .items() to get access to the key and value.

Comprehensions make it easy to go from a list to a dict or vice versa. For example, say we want to total up all the money in our dict salary_per_player.

Well, one way to add up numbers in Python is to pass a list of them to the sum() function.

If we want to get the total salary in our salary_per_player dict, we make a list of just the salaries using a list comprehension, then pass it to sum like:

This is still a list comprehension even though we're starting with a dict (salary_per_player). When in doubt, check the surrounding punctuation. It's brackets here, which means list.

Also note the for _, salary in ... part of the code. The only way to get access to a value of a dict (i.e., the salary here) is to use .items(), which also gives us access to the key (the player name in this case). But since we don't actually need the key for summing salary, the Python convention is to name that variable _. This lets people reading our code know we're not using it.

In the last section we saw sum(), which is a Python built-in that takes in a list of numbers and totals them up.

sum() is an example of a function. Functions are code that take inputs (the function's arguments) and return outputs. Python includes several built-in functions. Another common one is len, which finds the length of a list.

Using the function — i.e. giving it some inputs and having it return its output — is also known as calling or applying the function.

Once we've called a function, we can use it just like any other value. There's no difference between len(['ruben dias', 'gabriel jesus', 'riyad mahrez']) and 3. We could define variables with it:

Or use it in math.

Or whatever. Once it's called, it's the value the function returned, that's it.

It is very common in all programming languages to define your own functions.

After defining a function (making sure to highlight it and send it to the REPL) you can call it like this:

Note the arguments wins, ot_losses and losses. These work just like normal variables, except they're only available inside your function (the function's body).

So, even after defining and running this function it you try to type:

You'll get an error - wins only exists inside the team_pts function.

The programming term for where you have access to a variable (inside the function for arguments) is scope.

You could put the print statement inside the function:

And then when we call it:

Note the 62 in the REPL. Along with returning a bool, team_pts_noisy prints the value of wins. This is a side effect of calling the function. A side effect is anything your function does besides returning a value.

Printing variable values isn't a big deal (it can be helpful if your function isn't working like you expect), but apart from that you should avoid side effects in your functions.

Here's a question: what happens if we leave out any of the arguments when calling our function?

Let's try it:

We get an error. We gave it 62 and 16, which got assigned to wins and losses but ot_losses didn't get a value.

We can avoid this error by including default values. Let's make ot_losses default to 0.

Now ot_losses is optional because we gave it a default value. Note wins and losses are still required (no default values). Also note this mix of required and optional arguments — this is fine. Python's only rule is any optional arguments have to come after required arguments.

Now this function call works:

But if we run it without wins or losses we still get an error:

Up to this point we've just passed the arguments in order, or by position.

So when we call:

The function assigns 62 to wins, 16 to losses and 4 to ot_losses. It's in that order (wins, losses, ot_losses) because that's the order we wrote them when we defined team_pts above.

These are called positional arguments.

We wrote this function, so we know the order the arguments go, but often we'll use third party code with functions we didn't write.

In that case we'll want to know the function's Signature — the arguments it takes, the order they go, and what's required vs optional.

It's easy to check in the REPL, just type the name of the function and a question mark:

The alternative to passing all the arguments in the correct positions is to use keyword arguments, like this:

Keyword arguments are useful because you no longer have to remember the exact argument order. In practice, they're also required to take advantage of default values.

Think about it: presumably your function includes defaults so that you don't have to type in a value for every argument, every time. But if you're passing some values and not others, how's Python supposed to know which is which?

The answer is keyword arguments.

You're allowed to mix positional and keyword arguments:

But Python's rule is that positional arguments have to come first.

One thing this implies is it's a good idea to put your most "important" arguments first, leaving your optional arguments for the end of the function definition.

For example, later we'll learn about the read_csv function in Pandas, whose job is to load your csv data into Python. The first argument to read_csv is a string with the path to your file, and that's the only argument you'll use 95% of the time. But it also has more than 40 optional arguments, everything from skip_blank_lines (defaults to True) to parse_dates (defaults to False).

What this means is usually you can just use the function like this:

And on the rare occasions when you do need to tweak some option, change the specific settings you want using keyword arguments:

Python's argument rules are precise, but pretty intuitive when you get used to them. See the end of chapter exercises for more practice.

A cool feature of Python is that functions can take other functions as arguments.

Now let's also make a function to use this on.

And try it out:

The function do_to_list can work on built in functions too.

You can also create functions on the fly without names, usually for purposes of passing to other, flexible functions.

These are called anonymous or lambda functions.

There is much more to basic Python than this, but this is enough of a foundation to learn the other libraries we'll be using.

Libraries are just a collection of user defined functions and types ^7 that other people have written using Python ^8 and other libraries. That's why it's critical to understand the concepts in this section. Libraries are Python, with lists, dicts, bools, functions and all the rest.

we covered defining your own functions, we did not cover defining your own types — sometimes called classes — in Python. Working with classes is sometimes called object-oriented programming. While object-oriented programming and being able to write your own classes is sometimes helpful, it's definitely not required for everyday data analysis. I hardly ever use it myself.

sometimes they use other programming languages too. Parts of the data analysis library Pandas, for example, are written in the programming language C. But we don't have to worry about that.

Some libraries come built-in to Python. One example we'll use is the os (for operating system) library. To use it, we have to import it, like this:

That lets us use all the functions written in the os library. For example, we can call cpu_count to see the number of computer cores we currently have available.

Libraries like os can contain sub-libraries too. The sub-library we'll use from os is path, which is useful for working with filenames. One of the main function is join, which takes a directory (or multiple directories) and a filename and puts them together in a string. Like this:

With join, you don't have to worry about trailing slashes or operating system differences or anything like that. You can just replace DATA_DIR with the directory that holds the csv files that came with this book and you'll be set.