Python df.to_csv only prints last line even if for loop is present

Question

I am writing a code where the final results(denoted by x) to be exported to csv file. I used a for loop to iterate over but stil it only exports the last line of the result. My full code below is:

import csv
import itertools
import requests
import json
import pandas as pd


domainfile=open('domainsinfo.csv',newline='',encoding='utf_8')
reader=csv.reader(domainfile)
w=[]
  for row in reader:
    w.extend(row)

domain = list(itertools.permutations(w,1))
print(domain)

 def url_report(domain):
    url = 'https://www.virustotal.com/vtapi/v2/url/report'
    params = {'apikey': '', 'resource':domain}
    response = requests.get(url, params=params)
    return response

def pp_json(json_thing, sort=True, indents=4):
        if type(json_thing) is str:
            print(json.dumps(json.loads(json_thing), sort_keys=sort, 
    indent=indents))
      else:
              print(json.dumps(json_thing, sort_keys=sort, 
 indent=indents))
      return None


for d in domain:
    response = url_report(d)
    json_response = response.json()
    pretty_json = pp_json(json_response)

    response_list=[]
    for key in json_response['scans']:   

        if json_response['scans'][key] ['detected'] is True:      
            response_list.append(True)
        else:
            response_list.append(False)
    x=any(response_list)
    print(x)


    for d in domain:        
      final_list=[]

    final_list.append(x)
    result=(final_list)
    result_table = {'Domain': [d], 'Result':result}

    df=pd.DataFrame(data=result_table) 
    print(df)

    export_csv = df.to_csv (r'C:\csv', index=None, header=True)

print(pretty_json)


input()

Can someone explain why this does not work even if the for loop is present

Answer 1

This happens because in each loop, your .csv is overwritten by your next one.
The df.to_csv exports the df in a csv file.
You could try taking this code out of the loop, or changing the file name dynamically (depending on what you want.

Answer 2

Like liakoyras mentioned, you are writing your dataframe in the CSV. There are many ways to overcome your problem. While he suggested a merge, I am giving you another alternative here using append . Read the documentation for more information.

Here is how you need to modify your code block, explanation in comments inside the code:

######## Declare a dataframe outside the loop
final = pd.DataFrame()

for d in domain:
    ........................
       your other codes
    ........................
    result_table = {'Domain': [d], 'Result':result}

    df=pd.DataFrame(data=result_table) 

    ###### keep appending df to final 
    ###### the final df now gets updated in every loop
    final = final.append(df)

#### now outside the loop, write final dataframe to your csv
final.to_csv('your/path/file.csv')