Append CSV Files in Python

Comma Separated Value Files (CSV) are used to store tabular data. The data items in the CSV file are separated by commas and are saved with a .csv extension.

There are different methods of handling CSV files and appending data to them. This includes:

writer( ) function of CSV module
DictWriter( ) function of CSV module
Pandas module

Python offers a module named csv, which helps to work with CSV files. The csv package can be installed using the pip command.

After installation, we need to import the csv module:

Before appending data into a CSV file, we must know about the read and write functions in CSV files. The CSV file can be read using different methods using the CSV module. The CSV module offers a reader( ) function to read the data from the csv file.

For example, we have a random csv file containing data on employees of a company. Using the reader( ) function, we will read the data of the csv file.

Code:

import csv
with open("demo.csv") as file:
    reader = csv.reader(file)
    for data in reader:
        print(data)

Output:

['Employee ID', ' NAME', ' DESIGNATION']
['010', ' JASS', ' ENGINEER']
['012', ' AMA', ' WEB DEVELOPER']
['015', ' TOM', ' ANALYST']
['020', ' HARRY', ' ASSOCIATE']
['022', ' PIX', ' MANAGER']
['027', ' SAM', ' HR']

We first imported the csv module, then, using the open( ) function, we read the csv file (demo.csv). Then, made an object to read the csv file. Using the csv.reader( ) function, we read the data of the csv file and printed the data.

Like other files, we can open the csv file with different modes:

read
write
append

Now, we will append the csv files using different methods:

1. writer( ) function

The writer( ) function in handling CSV files converts the string into a delimited string. The writer( ) function uses writerow( ) and writerows( ) functions to write the delimited string to a csv file using the writer-object.

Syntax of writer( ) function

where:

obj = object of the writer function
writer = function to write objects
file_obj = object of the file

Now, we will append the data using the writer( ) function in Python.

Code:

import csv

with open("demo.csv", newline="") as file:
    reader = csv.reader(file)
    data = []
    for rows in reader:
        data.append(rows)

with open("demo1.csv", "a", newline="") as file1:
    writer = csv.writer(file1)
    writer.writerows(data[0:])  

with open("demo1.csv", newline="") as file:
    reader = csv.reader(file)
    for rows in reader:
        print(rows)

Output:

['AGE', '  SALARY']
['30', ' 53000']
['22', ' 32000']
['45', ' 90000']
['33', ' 40000']
['32', ' 34000']
['27', ' 500000']
[]
['Employee ID', '  NAME', '  DESIGNATION']
['010', ' JASS', ' ENGINEER']
['012', ' AMA', ' WEB DEVELOPER']
['015', ' TOM', ' ANALYST']
['020', ' HARRY', ' ASSOCIATE']
['022', ' PIX', ' MANAGER']
['027', ' SAM', ' HR']

We have used two csv files (demo and demo1) and tried to append data from one file to another. We first imported the csv library. Then, using the open( ) function, we first opened the demo file in read mode. We made a reader object and used the csv.reader( ) function; we read the file. Then, using the append( ) function, we append the data to the demo file. Then, we opened the demo1 file in the write mode. Using the writer( ) function, we appended the data from the demo class defined earlier. To print the data, we again opened the demo1 file and printed the result. As an output, it will first print the data of the demo1 file and then the data of the demo file at the end.

2. DictWriter( ) method

The DictWriter( ) method is used to write csv files using the csv module in Python. This function creates an object that maps the data of each row to a dictionary where the keys are defined by any variables representing the field names. If no field name is provided, the first row of the csv file is treated as the fields of the file. The rows must match the number of field names. If the rows are more than the field names, the rest of the data are stored in a list under a default restkey field name. The writerows( ) function is used to add rows to the csv file, just like adding data to a dictionary.

Syntax of DictWriter( ) function

where:

obj = object of the DictWriter function
file_object = object of the csv file
fieldnames = object to store the field names
field_names = the list of the field names

Let's append the data using the DictWriter( ) function in Python.

import csv 
field_names = ["EMPLOYEE ID", "NAME", "DESIGNATION"]

data = [{'EMPLOYEE ID': 101, 'NAME': 'TIM', 'DESIGNATION': 'ANALYST'}, 
        {'EMPLOYEE ID': 102, 'NAME': 'GEN', 'DESIGNATION': 'ASSOCIATE'},
        {'EMPLOYEE ID': 103, 'NAME': 'BOB', 'DESIGNATION': 'MANAGER'},
        {'EMPLOYEE ID': 106, 'NAME': 'SAMMY', 'DESIGNATION': 'DEVELOPER'},
        {'EMPLOYEE ID': 108, 'NAME': 'GRAY', 'DESIGNATION': 'TEAM LEAD'}]

with open("demo.csv", 'a', newline ="\n") as file:
    obj = csv.DictWriter(file, fieldnames = field_names)
    obj.writerows(data)

with open("demo.csv", newline = "") as f:
    reader = csv.reader(f)
    for rows in reader:
        print(rows)

Output:

['Employee ID', '  NAME', '  DESIGNATION']
['010', ' JASS', ' ENGINEER']
['012', ' AMA', ' WEB DEVELOPER']
['015', ' TOM', ' ANALYST']
['020', ' HARRY', ' ASSOCIATE']
['022', ' PIX', ' MANAGER']
['027', ' SAM', ' HR']
['101', 'TIM', 'ANALYST']
['102', 'GEN', 'ASSOCIATE']
['103', 'BOB', 'MANAGER']
['106', 'SAMMY', 'DEVELOPER']
['108', 'GRAY', 'TEAM LEAD']

In this, we have used a csv file containing data of employees (demo.csv). Firstly, we imported the csv library. Then, we made a list of field names and a dictionary containing data to be added to a CSV file. Then, using the open( ) function, we opened the file in append mode ('a'). Then, made a DictWriter object and used the DictWriter( ) function in which we passed the list of field names and the file object. Then, we passed the dictionary of data in the writerows( ) function to write data in the file. Again, we opened the file in read mode to print the data of the file. As an output, the dictionary of the data will be printed after the demo file content.

We can append the csv file by adding data from another file, just like the writer( ) function.

Code:

import csv

with open("file2.csv",newline="") as file:
    reader = csv.DictReader(file)
    data = []
    for row in reader:
        data.append(row)
    file.close()

with open("file1.csv","a",newline="") as file1:
    writer = csv.DictWriter(file1, fieldnames = data[0].keys())
    writer.writerows(data)

with open("file1.csv",newline="") as file:
    reader = csv.DictReader(file)
    for rows in reader:
        print(rows)

Output:

{'Employee ID': '010', '  NAME': ' JASS', '  DESIGNATION': ' ENGINEER'}
{'Employee ID': '012', '  NAME': ' AMA', '  DESIGNATION': ' WEB DEVELOPER'}
{'Employee ID': '015', '  NAME': ' TOM', '  DESIGNATION': ' ANALYST'}
{'Employee ID': '020', '  NAME': ' HARRY', '  DESIGNATION': ' ASSOCIATE'}
{'Employee ID': '022', '  NAME': ' PIX', '  DESIGNATION': ' MANAGER'}
{'Employee ID': '027', '  NAME': ' SAM', '  DESIGNATION': ' HR'}
{'Employee ID': '101', '  NAME': ' TIM', '  DESIGNATION': ' ANALYST'}
{'Employee ID': '102', '  NAME': ' GEN', '  DESIGNATION': ' ASSOCIATE'}
{'Employee ID': '103', '  NAME': ' BOB', '  DESIGNATION': ' MANAGER'}
{'Employee ID': '106', '  NAME': ' SAMMY', '  DESIGNATION': ' DEVELOPER'}
{'Employee ID': '108', '  NAME': ' GRAY', '  DESIGNATION': ' TEAM LEAD'}
{'Employee ID': '011', '  NAME': ' JASMIN', '  DESIGNATION': ' SENIOR ENGINEER'}
{'Employee ID': '021', '  NAME': ' MAX', '  DESIGNATION': ' DEVELOPER'}
{'Employee ID': '035', '  NAME': ' TONY', '  DESIGNATION': ' DATA ANALYST'}
{'Employee ID': '043', '  NAME': ' HERY', '  DESIGNATION': ' TL ASSOCIATE'}
{'Employee ID': '029', '  NAME': ' RONY', '  DESIGNATION': ' SENIOR MANAGER'}
{'Employee ID': '027', '  NAME': ' SIM', '  DESIGNATION': ' ASSN HR'}

Similar to the writer( ) function, we have appended a csv file to another file. We have opened the file2 and appended its data in file2 using the DictWriter( ) function.

3. Pandas library

The Pandas library in Python is used for working with different data sets. It can be used to read and explore csv files. It can make data frames from the csv files. For appending data in csv files, the pandas library provides a to_csv( ) function.

Syntax of to_csv( ) function

where:

obj = object of the file
pd = pandas library object
file_name = name of the csv file
index = gives the index for the appended file. The default value is false.
header = gives the headers/ headings of the data. It is set as False by default.

Now, we will append the csv file using the pandas library.

Code:

import pandas as pd

data = pd.read_csv("file2.csv")
data.to_csv("file1.csv", index = False,  header = False, mode = "a")

data1 = pd.read_csv("file1.csv")
data1

Output:

	Employee ID	NAME	DESIGNATION
0	10	JASS	ENGINEER
1	12	AMA	WEB DEVELOPER
2	15	TOM	ANALYST
3	20	HARRY	ASSOCIATE
4	22	PIX	MANAGER
5	27	SAM	HR
6	101	TIM	ANALYST
7	102	GEN	ASSOCIATE
8	103	BOB	MANAGER
9	106	SAMMY	DEVELOPER
10	108	GRAY	TEAM LEAD
11	11	JASMIN	SENIOR ENGINEER
12	21	MAX	DEVELOPER
13	35	TONY	DATA ANALYST
14	43	HERY	TL ASSOCIATE
15	29	RONY	SENIOR MANAGER
16	27	SIM	ASSN HR

We imported the pandas library. Using the read_csv( ) function, we read a csv file (file2.csv). Then, using the to_csv( ) function, we have appended the data of file2 in file1.

Next TopicAppend-files-in-python

← prev next →