How can I create new directory and save file using urllib2 in Python?

Question

I'm trying to create a python script to check if a host is alive, if so, download the website into a results/ directory. Once I learn how to do this I will branch out on figuring out how to spider and launch other subprocesses (such as launching nikto/skipfish after checking is complete and loading the saved file).

#! /usr/bin/python

import os
import sys
import urllib
import urllib2
import subprocess

# Where the magic happens

str1 = raw_input("Enter your target: ")
print "Target = ", str1
print "commencing testing on", str1

# Let's set the user-agent headers
http_headers = {"User-Agent":"Mozilla/5.0"}

request = urllib2.Request(str1)
response = urllib2.urlopen(request)
payload = response.read()

dir_path = os.path.join(self.results)
os.makedirs(dir_path)
**with open(os.join.path(dir_path, 'index.html', 'wb') as file:
        file.write(payload)
print str1, "index written to file"**

# Send an email to notify us when complete
var = "world"
pipe = subprocess.Popen(["./email.sh", var], stdout=subprocess.PIPE)
result = pipe.stdout.read()
print result

I receive the following error message:

File "./webtest.py", line 43
    with open(os.join.path(dir_path, 'index.html', 'wb') as file:
                                                          ^
SyntaxError: invalid syntax

Error after closing the parenthesis (from Phil's Answer):

    Traceback (most recent call last):
      File "./webtest.py", line 41, in <module>
        dir_path = os.path.join(self.results)
NameError: name 'self' is not defined

Answer 1

You missed a parentheses:

with open(os.join.path(dir_path, 'index.html', 'wb')) as file:

EDIT

That line has to do with the directory that you want. It's giving errors because you're not in a class (so "self" doesn't exist). The best course of action would be to replace it with just "results" and specify where results are. For example:

results = "/resultsdir/"
dir_path = os.path.join(results)