I wonder how to add newDataFrame data onto the end of an existing csv file? Theto_csv doesn't mention such functionality.
- Possible duplicate ofHow to add pandas data to an existing csv file?9769953– 97699532018-09-26 14:53:36 +00:00CommentedSep 26, 2018 at 14:53
4 Answers4
You can append usingto_csv by passing a file which isopen in append mode:
with open(file_name, 'a') as f: df.to_csv(f, header=False)Useheader=None, so as not to append the column names.
In fact, pandas has a wrapper to do this into_csv using themode argument (see Joe's answer):
df.to_csv(f, mode='a', header=False)10 Comments
to_csv or the filename? I recall a related issue wherenot closing the file led to a 99% speedup of their code (IIRC they were appending to the same file tens of thousands of times).You can also pass the file mode as an argument to theto_csv method
df.to_csv(file_name, header=False, mode = 'a')Comments
A little helper function I use (based on Joe Hooper's answer) with some header checking safeguards to handle it all:
def appendDFToCSV_void(df, csvFilePath, sep=","): import os if not os.path.isfile(csvFilePath): df.to_csv(csvFilePath, mode='a', index=False, sep=sep) elif len(df.columns) != len(pd.read_csv(csvFilePath, nrows=1, sep=sep).columns): raise Exception("Columns do not match!! Dataframe has " + str(len(df.columns)) + " columns. CSV file has " + str(len(pd.read_csv(csvFilePath, nrows=1, sep=sep).columns)) + " columns.") elif not (df.columns == pd.read_csv(csvFilePath, nrows=1, sep=sep).columns).all(): raise Exception("Columns and column order of dataframe and csv file do not match!!") else: df.to_csv(csvFilePath, mode='a', index=False, sep=sep, header=False)1 Comment
Thank to Andy, the complete solution:
f = open(filename, 'a') # Open file as append modedf.to_csv(f, header = False)f.close()1 Comment
with it cleans up that for you. :)Explore related questions
See similar questions with these tags.



