python POST with file - convert from RAW data

Question

I have been trying to get this thing to work for days, with no luck. I have worked through the majority of my issues, but I am now at a new error and I don't know why.

I have read multiple posts, and even asked a few questions along the way, but now I am completely stumped and it seems that nothing that I search for helps.

Here is the scenario:

• I need to log into a website (let's call it site_A) and pull a report
• Next, I need to log into a separate website (site_B) so I can upload that report
• With site_B, I am required to log in, get a session ID, then use that session ID to upload my file

This all done with API https POST calls... (yes, the API requires POST for everything).

site_A works perfectly - I log in and pull the data, which gives me an XML file. This file is then converted into a CSV file, due to site_B only accepting .csv.

site_B however, is giving me errors.

Using BURP, I am able to intercept the raw data of a test API published by the company. This raw data looks something like this (some data removed):

POST /api/import HTTP/1.1
Host: hostURL
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
X-Requested-With: XMLHttpRequest
Content-Type: multipart/form-data; boundary=---------------------------28791504620783
Cookie: asc_session_id=sessionToken; SkipBrowserCheck=True; JSESSIONID=sessionStuff; ASP.NET_SessionId=ASPSession
Connection: close

-----------------------------28791504620783
Content-Disposition: form-data; name="token"

sessionToken
-----------------------------28791504620783
Content-Disposition: form-data; name="Name"

WH
-----------------------------28791504620783
Content-Disposition: form-data; name="uploadedfile"; filename="data.csv"
Content-Type: application/vnd.ms-excel

data,data,data,data
data,data,data,data

-----------------------------28791504620783--

I have converted this into the following python program:

# Login - get session token
app = requests.Session()
loginResponse = app.post(loginURL, headers=headers, data=json.dumps(post_body), verify=False)

if loginResponse.status_code != 200:
    print ('Login attempt failed. Please try again later.')

else:
    userData = json.loads(loginResponse.text)
    sessionToken = userData["sessionId"]
    print ('Login successful! Attempting to upload file...')

    # Now try to upload file
    uploadURL = 'uploadURL/'

    headers = {
        'token': sessionToken,
        'Name': 'WH',
        'POST': '/api/import/285/2 HTTP/1.1',
        'Accept': 'application/json, text/javascript, */*; q=0.01',
        'Accept-Language': 'en-US,en;q=0.5',
        'Accept-Encoding': 'gzip, deflate, br',
        'Connection': 'close'   
    }   

    files = {'data.csv': open('data.csv', newline='')}

    uploadResponse = app.post(uploadURL, headers=headers, files=files, verify=False)

    if uploadResponse.status_code != 200:
        print ('Status report: %i' % uploadResponse.status_code)

    else: 
        print ('Upload Complete')

I have tried to move things around a bit, and place somethings in the body rather than the header. But this also gives me errors either with logging in, or incorrect file type. I have established that I NEED to use the token and the Name in the header. I am not sure if I have to use the "files" variable in python requests (files='whatever')

But from this point, I get a 500 error.

Can someone please point me in the right direction?

I have read the following posts, and while they have helped a little, they aren't helping here: Post 1 , Post 2 , Post 3

NOTE: I have NOT confirmed that the problem isn't on the server. It may not be with my code at all. But it seems that since the API test code on the website works, it should work in python as well.

Answer 1

I am not sure if it's the API, or if I was messing something up, but in this case I had to create a body that looked similar to the RAW data. I was under the impression that "files=" would do this for me, but apparently it did not.

I defined a boundary in the multipart/form and then used that to add my sections.

The end result looked something like:

headers = {
            'Content-Type': 'multipart/form-data; boundary=159753',
            'asc_xsrf_token': sessionToken,
        }   

        with open('data.csv', newline='') as csvFile:
            post_body = (
                "\n--159753\n"
                "Content-Disposition: form-data; name=\"asc_xsrf_token\"\n\n"
                + sessionToken + 
                "\n--159753\n"
                "Content-Disposition: form-data; name=\"Name\"\n\n"
                "WH\n"
                "--159753\n"
                "Content-Disposition: form-data; name=\"uploadedfile\"; filename=\"data.csv\"\n"
                "Content-Type: application/vnd.ms-excel\n\n"
                + csvFile.read() + 
                "\n--159753--"
                )
            csvFile.close()