Folder creation with timestamp in python

Question

Hi I am a beginner in python and not well versed with file operations.I am writing a python script for logging. Below is my code snippet:

infile = open('/home/nitish/profiles/Site_info','r')
lines = infile.readlines()
folder_output =      '/home/nitish/profiles/output/%s'%datetime.now().strftime('%Y-%m-%d-%H:%M:%S')
folder = open(folder_output,"w")
for index in range(len(lines)):
  URL = lines[index]

  cmd = "curl -L " +URL

  curl = subprocess.Popen(cmd,shell=True,stdout=subprocess.PIPE)

  file_data = curl.stdout.read()
  print file_data

  filename = '/home/nitish/profiles/output/log-%s.html'%datetime.now().strftime('%Y-%m-%d-%H:%M:%S')
  output = open(filename,"w")
  output.write(file_data)
output.close()
folder.close()
infile.close()

I don't know if this is correct. I wish to create a new folder with timestamp everytime the script is run and place all the output from the for loop into the folder with timestamp.

Thanks for your help in advance

Answer 1

You have trailing newlines on all the urls so that would not work, you won't get past folder = open(folder_output,"w") as you are trying to create a file not a folder, there is also no need for a subprocess. You can do it all using standard lib functions:

from os import mkdir
import urllib.request
from datetime import datetime

now = datetime.now

new_folder = '/home/nitish/profiles/output/{}'.format(now().strftime('%Y-%m-%d-%H:%M:%S'))
# actually make the folder
mkdir(new_folder)

# now open the urls file and strip the newlines 
with open('/home/nitish/profiles/Site_info') as f:
    for url in map(str.strip, f):
        # open a new file for each request and write to new folder
        with open("{}/log-{}.html".format(new_folder, now().strftime('%Y-%m-%d-%H:%M:%S')), "w") as out:
            out.write(urllib.request.urlopen(url).read())

For python2, use import urllib and `urllib.urlopen or better yet use requests