7. Reading and writing data to files#

This notebook can be downloaded for local use from here

7.1. Open and read the contents of a csv file line by line.#

Import the csv library. This is certainly not the only library that could be used, but it is simple and robust for text files in a csv format.

import csv

If necessary, find out where your data files are stored using shell commands from within Jupyter

!pwd

/home/grosedj/work/work-env/STOR-601-2023-24/python

Download the file pines.csv Have a look at the file. This assumes you are using unix.

!head pines.csv

1,0.1,9.9
2,0.1,7.2
3,0.2,6.2
4,0.2,8.4
5,0.7,4.5
6,0.9,7.5
7,1.1,8.5
8,1.3,6.3
9,1.4,1.6
10,1.6,5.4

So, three columns of integers - put each column into a list.

# open the file
with open('pines.csv',mode='r') as csv_file :
    # create a reader - this allows a delimiter to be specified and used
    csv_reader = csv.reader(csv_file, delimiter=',')
    # do not know how many rows - so initialise some empty lists
    C1 = []
    C2 = []
    C3 = []
    # read each row of the file ...
    for row in csv_reader :
        # .. and put the data into the lists
        C1.append(row[0])
        C2.append(row[1])
        C3.append(row[2])

# have a look
print(C1,C2,C3)  

['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '40', '41', '42', '43', '44', '45', '46', '47', '48', '49', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '60', '61', '62', '63', '64', '65', '66', '67', '68', '69', '70', '71'] ['0.1', '0.1', '0.2', '0.2', '0.7', '0.9', '1.1', '1.3', '1.4', '1.6', '1.7', '1.7', '2', '2.1', '2.3', '2.4', '2.5', '2.6', '2.7', '2.8', '2.8', '3.3', '3.3', '3.6', '3.7', '3.9', '3.9', '3.9', '4.1', '4.5', '4.8', '4.9', '5.2', '5.2', '5.3', '5.3', '5.6', '5.8', '5.9', '6', '6.1', '6.1', '6.1', '6.3', '6.5', '6.6', '6.6', '6.9', '7.2', '7.2', '7.3', '7.4', '7.4', '7.5', '7.6', '7.9', '8.1', '8.2', '8.3', '8.4', '8.5', '8.6', '8.7', '9.2', '9.2', '9.3', '9.4', '9.4', '9.5', '9.5', '9.5'] ['9.9', '7.2', '6.2', '8.4', '4.5', '7.5', '8.5', '6.3', '1.6', '5.4', '2.6', '4.1', '0.2', '8', '8.7', '1.1', '7', '5.7', '5.4', '4.2', '2.6', '5.2', '6.3', '5.9', '8', '3.4', '4.7', '9.1', '1.7', '2.3', '5.3', '6.3', '4.3', '7.5', '9.8', '1.3', '7.8', '1.8', '6.8', '5.5', '4.3', '3.2', '2.5', '4.1', '8.1', '8.9', '1.6', '0.4', '5.2', '6', '4.2', '3.6', '5.4', '9.3', '1.7', '8.5', '4.5', '6.7', '1', '3.2', '2', '0.9', '8.7', '6', '8.1', '0.8', '1', '2.6', '2.3', '4.6', '6.2']

Note - the data are strings !! Need to conver to appropriate type. For example …

C1 = [int(x) for x in C1]
C2 = [float(x) for x in C2]
C3 = [float(x) for x in C3]
print(C1,C2,C3)

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71] [0.1, 0.1, 0.2, 0.2, 0.7, 0.9, 1.1, 1.3, 1.4, 1.6, 1.7, 1.7, 2.0, 2.1, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.8, 3.3, 3.3, 3.6, 3.7, 3.9, 3.9, 3.9, 4.1, 4.5, 4.8, 4.9, 5.2, 5.2, 5.3, 5.3, 5.6, 5.8, 5.9, 6.0, 6.1, 6.1, 6.1, 6.3, 6.5, 6.6, 6.6, 6.9, 7.2, 7.2, 7.3, 7.4, 7.4, 7.5, 7.6, 7.9, 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 9.2, 9.2, 9.3, 9.4, 9.4, 9.5, 9.5, 9.5] [9.9, 7.2, 6.2, 8.4, 4.5, 7.5, 8.5, 6.3, 1.6, 5.4, 2.6, 4.1, 0.2, 8.0, 8.7, 1.1, 7.0, 5.7, 5.4, 4.2, 2.6, 5.2, 6.3, 5.9, 8.0, 3.4, 4.7, 9.1, 1.7, 2.3, 5.3, 6.3, 4.3, 7.5, 9.8, 1.3, 7.8, 1.8, 6.8, 5.5, 4.3, 3.2, 2.5, 4.1, 8.1, 8.9, 1.6, 0.4, 5.2, 6.0, 4.2, 3.6, 5.4, 9.3, 1.7, 8.5, 4.5, 6.7, 1.0, 3.2, 2.0, 0.9, 8.7, 6.0, 8.1, 0.8, 1.0, 2.6, 2.3, 4.6, 6.2]

7.2. Writing a csv file a few lines at a time#

with open('csv-out-file.csv', mode='w') as csv_out_file:
    # csv_file_writer = csv.writer(csv_out_file, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
    csv_file_writer = csv.writer(csv_out_file, delimiter=',')
    csv_file_writer.writerow([1,2,3,4])
    csv_file_writer.writerow([5,6,7,8])

7.3. Reading a csv file a few lines at a time#

csv_file = open('pines.csv','r')
csv_reader = csv.reader(csv_file, delimiter=',')
line = next(csv_reader)
print(line)
line = next(csv_reader)
print(line)
csv_file.close()

['1', '0.1', '9.9']
['2', '0.1', '7.2']

7.3.1. Exercise 1#

Create a csv file containing three columns of data with random data. Write at least a hundred rows.

7.3.2. Exercise 2#

Read the data from the file you created in exercise 1

7.3.3. Exercise 3#

Read the first fifty rows of your file from exercise 1, writing the values to a new file as you proceed, but only if all values in the row are less than 0.8

7.3.4. Exercise 4#

What is the purpose of the with statement ?

What is the purpose of the close method ?

What are the ‘r’ and ‘w’ values for ?

STOR-601 Introductory Python

Reading and writing data to files

Contents