# Loops and files¶

This chapter presents loops, which are used to express repeated computation, and files, which are used to store data. As an example, we will download the famous book War and Peace from Project Gutenberg and write a loop that reads the book and counts the words.

In the next chapter, we’ll extend this example to count the number of unique words and the number of times each word appears. This example presents some computational tools you will need; it is also an introduction to working with textual data.

## Loops¶

One of the most important elements of computation is repetition, and the most common way to express repetition is a `for` loop. As a simple example, suppose we want to display the elements of a tuple. Here’s a tuple of three integers:

```t = 1, 2, 3
```

And here’s a `for` loop that prints the elements.

```for x in t:
print(x)
```
```1
2
3
```

The first line of the loop is a header that specifies the tuple, `t`, and a variable name `x`. The tuple already exists, but `x` does not; the loop will create it. The header has to end with a colon, `:`.

Inside the loop is a `print` statement, which displays the value of `x`.

So here’s what happens:

1. When the loop starts, it gets the first element of `t`, which is `1`, and assigns it to `x`. It executes the `print` statement, which displays the value `1`.

2. Then it gets the second element of `t`, which is `2`, and displays it.

3. Then it gets the third element of `t`, which is `3`, and displays it.

After printing the last element of the tuple, the loop ends.

We can also loop through the letters in a string:

```word = 'Data'

for letter in word:
print(letter)
```
```D
a
t
a
```

When the loop begins, `word` already exists, but `letter` does not. Again, the loop creates `letter` and assign values to it.

The variable created by the loop is called the “loop variable”. You can give it any name you like; in this example, I chose `letter` to remind me what kind of value it contains.

After the loop ends, the loop variable contains the last value.

```letter
```
```'a'
```

Exercise: Create a list, called `sequence` with four elements of any type. Write a `for` loop that prints the elements. Call the loop variable `element`.

You might wonder why I didn’t call the list `list`. I avoided it because Python has a function named `list` that makes new lists. For example, if you have a string, you can make a list of letters, like this:

```list('string')
```
```['s', 't', 'r', 'i', 'n', 'g']
```

If you create a variable named `list`, you can’t use the function any more.

## Looping and counting¶

War and Peace is a famously long book; let’s see how long it is. To count the words in a book, we need to

1. Loop through a book and

2. Count.

We’ll start with counting. We’ve already seen that you can create a variable and give it a value, like this:

```count = 0
count
```
```0
```

If you assign a different value to the same variable, the new value replaces the old one.

```count = 1
count
```
```1
```

You can increase the value of a variable by reading the old value, adding `1`, and assigning the result back to the original variable.

```count = count + 1
count
```
```2
```

Increasing the value of a variable is called an increment; decreasing the value is called a decrement. These operations are so common that there are special operators for them.

```count += 1
count
```
```3
```

In this example, the `+=` operator reads the value of `count`, adds `1`, and assigns the result back to `count`. Python also provides `-=` and other update operators like `*=` and `/=`.

Exercise: The following is a number trick from Learn With Math Games at https://www.learn-with-math-games.com/math-number-tricks.html:

Finding Someone’s Age

• Ask the person to multiply the first number of their age by 5.

• Tell them to add 3.

• Now tell them to double this figure.

• Finally, have the person add the second number of their age to the figure and have them tell you the answer.

• Deduct 6 and you will have their age.”

Test this algorithm using your age. Use a single variable and update it using `+=` and other update operators.

## Files¶

Now that we know how to count, let’s see how we can read words from a file. We can download War and Peace from Project Gutenberg, which is a repository of free books at https://www.gutenberg.org.

In order to read the contents of the file, you have to “open” it, which you can do with the `open` function.

```fp = open('2600-0.txt')
fp
```
```<_io.TextIOWrapper name='2600-0.txt' mode='r' encoding='UTF-8'>
```

The result is a `TextIOWrapper`, which is a type of file pointer. The file pointer contains the name of the file, the mode (which is `r` for “reading”) and the encoding (which is `UTF` for “Unicode Transformation Format”). A file pointer is like a bookmark; it keeps track of which parts of the file you have read.

If you use a file pointer in a `for` loop, it loops through the lines in the file. So we can count the number of lines like this:

```fp = open('2600-0.txt')
count = 0
for line in fp:
count += 1
```

And then display the result.

```count
```
```66054
```

There are about 66,000 lines in this file.

## if statements¶

We’ve already see comparison operators, like `>` and `<`, which compare values and produce a Boolean result, `True` or `False`. For example, we can compare the final value of `count` to a number:

```count > 60000
```
```True
```

We can use a comparison operator in an `if` statement to check for a condition and take action accordingly.

```if count > 60000:
print('Long book!')
```
```Long book!
```

The first line of the `if` statement specifies the condition we’re checking for. Like the header of a `for` statement, the first line of an `if` statement has to end with a colon.

If the condition is true, the indented statement runs; otherwise, it doesn’t. In the previous example, the condition is true, so the `print` statement runs. In the following example, the condition is false, so the `print` statement doesn’t run.

```if count < 1000:
print('Novella!')
```

We can put a `print` statement inside a `for` loop. In this example, we only print a line from the book when `count` is `1`. The other lines are read, but not displayed.

```fp = open('2600-0.txt')
count = 0
for line in fp:
if count == 1:
print(line)
count += 1
```
```The Project Gutenberg EBook of War and Peace, by Leo Tolstoy
```

Notice the indentation in this example:

• Statements inside the `for` loop are indented.

• The statement inside the `if` statement is indented.

• The statement `count += 1` is outdented from the previous line, so it ends the `if` statement. But it is still inside the `for` loop.

It is legal in Python to use spaces or tabs for indentation, but the most common convention is to use four spaces, never tabs. That’s what I’ll do in my code and I strongly suggest you follow the convention.

## break statement¶

If we display the final value of `count`, we see that the loop reads the entire file, but only prints one line:

```count
```
```66054
```

We can avoid reading the whole file by using a `break` statement, like this:

```fp = open('2600-0.txt')
count = 0
for line in fp:
if count == 1:
print(line)
break
count += 1
```
```The Project Gutenberg EBook of War and Peace, by Leo Tolstoy
```

The `break` statement ends the loop immediately, skipping the rest of the file. We can confirm that by checking the last value of `count`:

```count
```
```1
```

Exercise: Write a loop that prints the first 5 lines of the file and then breaks out of the loop.

## Whitespace¶

If we run the loop again and display the final value of `line`, we see the special sequence `\n` at the end.

```fp = open('2600-0.txt')
count = 0
for line in fp:
if count == 1:
break
count += 1

line
```
```'The Project Gutenberg EBook of War and Peace, by Leo Tolstoy\n'
```

This sequence represents a single character, called a newline, that puts vertical space between lines. If we use a `print` statement to display `line`, we don’t see the special sequence, but we do see extra space after the line.

```print(line)
```
```The Project Gutenberg EBook of War and Peace, by Leo Tolstoy
```

In other strings, you might see the sequence `\t`, which represents a “tab” character. When you print a tab character, it adds enough space to make the next character appear in a column that is a multiple of 8.

```print('01234567' * 6)
print('a\tbc\tdef\tghij\tklmno\tpqrstu')
```
```012345670123456701234567012345670123456701234567
a	bc	def	ghij	klmno	pqrstu
```

Newline characters, tabs, and spaces are called whitespace because when they are printed they leave white space on the page (assuming that the background color is white).

## Counting words¶

So far we’ve managed to count the lines in a file, but each line contains several words. To split a line into words, we can use a function called `split` that returns a list of words. To be more precise, `split` doesn’t actually know what a word is; it just splits the line wherever there’s a space or other whitespace character.

```line.split()
```
```['The',
'Project',
'Gutenberg',
'EBook',
'of',
'War',
'and',
'Peace,',
'by',
'Leo',
'Tolstoy']
```

Notice that the syntax for `split` is different from other functions we have seen. Normally when we call a function, we name the function and provide values in parentheses. So you might have expected to write `split(line)`. Sadly, that doesn’t work.

The problem is that the `split` function belongs to the string `line`; in a sense, the function is attached to the string, so we can only refer to it using the string and the dot operator (the period between `line` and `split`). For historical reasons, functions like this are called methods.

Now that we can split a line into a list of words, we can use `len` to get the number of words in each list, and increment `count` accordingly.

```fp = open('2600-0.txt')
count = 0
for line in fp:
count += len(line.split())
```
```count
```
```566317
```

By this count, there are more than half a million words in War and Peace.

Actually, there aren’t quite that many, because the file we got from Project Gutenberg has some introductory text and a table of contents before the text. And it has some license information at the end. To skip this “front matter”, we can use one loop to read lines until we get to `CHAPTER I`, and then a second loop to count the words in the remaining lines.

The file pointer, `fp`, keeps track of where it is in the file, so the second loop picks up where the first loop leaves off. In the second loop, we check for the end of the book and stop, so we ignore the “end matter” at the end of the file.

```first_line = "CHAPTER I\n"
last_line = "End of the Project Gutenberg EBook of War and Peace, by Leo Tolstoy\n"

fp = open('2600-0.txt')
for line in fp:
if line == first_line:
break

count = 0
for line in fp:
if line == last_line:
print(line)
break
count += len(line.split())
```
```End of the Project Gutenberg EBook of War and Peace, by Leo Tolstoy
```
```count
```
```562482
```

• When we compare two values to see if they are equal, we use the `==` operator, not to be confused with `=`, which is the assignment operator.

• The string we compare `line` to has a newline at the end. If we leave that out, it doesn’t work.

Exercise:

1. In the previous program, replace `==` with `=` and see what happens. This is a common error, so it is good to see what the error message looks like.

2. Correct the previous error, then remove the newline character after `CHAPTER I`, and see what happens.

The first error is a “syntax error”, which means that the program violates the rules of Python. If your program has a syntax error, the Python interpreter prints an error message, and the program never runs.

The second error is a “logic error”, which means that there is something wrong with the logic of the program. The syntax is legal, and the program runs, but it doesn’t do what we wanted. Logic errors can be hard to find because we don’t get any error messages.

If you have a logic error, here are two strategies for debugging:

1. Add print statements so the program displays additional information while it runs.

2. Simplify the program until it does what you expect, and then gradually add more code, testing as you go.