Reading and writing data to files¶

Open and read the contents of a csv file line by line.¶

Import the csv library. This is certainly not the only library that could be used, but it is simple and robust for text files in a csv format.

import csv

If necessary, find out where your data files are stored using shell commands from within Jupyter

!pwd

/home/grosedj/Dropbox/lancaster-university/stor-i/courses/python/STOR-601/further-python/notebooks

Have a look at the file. This assumes you are using unix.

!head ./../data/csv-data-1.csv

100,2,3
200,4,1
300,6,7
400,2,3

So, three columns of integers - put each column into a list.

# open the file
with open('./../data/csv-data-1.csv',mode='r') as csv_file :
    # create a reader - this allows a delimiter to be specified and used
    csv_reader = csv.reader(csv_file, delimiter=',')
    # do not know how many rows - so initialise some empty lists
    C1 = []
    C2 = []
    C3 = []
    # read each row of the file ...
    for row in csv_reader :
        # .. and put the data into the lists
        C1.append(row[0])
        C2.append(row[1])
        C3.append(row[2])

# have a look
print(C1,C2,C3)  

['100', '200', '300', '400'] ['2', '4', '6', '2'] ['3', '1', '7', '3']

Note - the data are strings !! Need to conver to appropriate type. For example …

C1 = [int(x) for x in C1]
C2 = [int(x) for x in C2]
C3 = [int(x) for x in C3]
print(C1,C2,C3)

[100, 200, 300, 400] [2, 4, 6, 2] [3, 1, 7, 3]

Writing a csv file a few lines at a time¶

with open('./../data/csv-out-file.csv', mode='w') as csv_out_file:
    # csv_file_writer = csv.writer(csv_out_file, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
    csv_file_writer = csv.writer(csv_out_file, delimiter=',')
    csv_file_writer.writerow([1,2,3,4])
    csv_file_writer.writerow([5,6,7,8])

Reading a csv file a few lines at a time¶

csv_file = open('./../data/csv-data-1.csv','r')
csv_reader = csv.reader(csv_file, delimiter=',')
line = next(csv_reader)
print(line)
line = next(csv_reader)
print(line)
csv_file.close()

['100', '2', '3']
['200', '4', '1']

Exercise 1¶

Create a csv file containg three columns of data containing randoms. Write at least a hundred rows.

Exercise 2¶

Read the data from the file you created in exercise 1

Exercise 3¶

Read the first fifty rows of your file from exercise 1, writing the values to a new file as you proceed, but only if all values in the row are less than 0.8

Exercise 4¶

What is the purpose of the with statement ?

What is the purpose of the close method ?

What are the ‘r’ and ‘w’ values for ?

STOR-601 Scientific Programming with Python