Blobstore Issues with GAE Cloud Tools and Local Development Server

Question

I'm trying to debug some GAE code on my local development server but have hit a wall. The code uses Google's Blobstore service to facilitate file uploads. The code works just fine on production but not on my local development server.

I'm using the standard Google pattern of including as my form action in my jsp blobstoreService.createUploadUrl("/uploadSurvey") and then calling blobstoreService.getUploads(request) in my servlet.

The file is correctly uploaded (I can see it using the local admin console) but the call to getUploads() throws the exception: java.lang.IllegalStateException: Must be called from a blob upload callback request.

Looking at the request in the debugger, the required blobkey attribute isn't found, nor are any of the other input parameters in the form. Looking at the raw request (the one dispatched to /_ah/upload/...), the form parameters are present.

I use the google cloud tools app-engine-plugin, which uses the gcloud python dev server to run the generated war.

I realize blobstore is an older GAE feature, but as this code is "working" on prod, I'd prefer to not have to switch to the newer subsystem.

Anyone able to give me a clue where to look to get this all working on my dev server?

Thanks,

-- Dave

ps below is the request forwarded on to my servlet after the uploaded file has been stripped out:

POST /uploadSurvey HTTP/1.1
Accept-Encoding: identity
X-APPENGINE-BACKEND-ID: 8
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
X-APPENGINE-SERVER-NAME: localhost
Cookie: JSESSIONID=5773y31x3eut
X-Appengine-User-Email: 
X-APPENGINE-DEFAULT-VERSION-HOSTNAME: localhost:8888
X-APPENGINE-SERVER-PROTOCOL: HTTP/1.1
X-Appengine-User-Organization: 
X-APPENGINE-DEV-SCRIPT: unused
ORIGIN: http://localhost:8888
X-Appengine-User-Id: 
Accept-Language: en-us
X-APPENGINE-SERVER-SOFTWARE: Development/2.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/602.4.8 (KHTML, like Gecko) Version/10.0.3 Safari/602.4.8
X-Appengine-User-Nickname: 
Host: localhost:8888
X-Appengine-Dev-Request-Id: wCTAonUKrB
Content-Type: multipart/form-data; boundary="===============1477989950756010976=="
Content-Length: 1372
X-APPENGINE-REQUEST-LOG-ID: 5e8eaef5aff4add89b774badea1fd3a30da8be
X-Appengine-User-Is-Admin: 0
UPGRADE-INSECURE-REQUESTS: 1
X-APPENGINE-SERVER-PORT: 8888
Referer: http://localhost:8888/settings
X-AppEngine-Country: ZZ
X-APPENGINE-REQUEST-ID-HASH: BFD4FDDA
X-APPENGINE-REMOTE-ADDR: ::1

Update:

I added some debugging to http_proxy.py under the gcloud devserver2 directory and observed this content type being forwarded. I'm even more confused now, as it looks like the content is present...

    --===============5516630363169856841==
Content-Type: message/external-body; blob-key="XOQvaKc1cdczcwkIHfRFOw=="; access-type="X-AppEngine-BlobKey"
Content-Disposition: form-data; name="myFile"; filename="Naro Group - SNHU - Sales Readiness Assessment - Dec 2016.xls"

Content-Type: application/vnd.ms-excel
Content-Length: 164352
Content-MD5: NjBiNjI0N2M3MjZiMzc3NWMxZDQxYmM5YTU2YmM5YmM=
content-disposition: form-data; name="myFile"; filename="Naro Group - SNHU - Sales Readiness Assessment - Dec 2016.xls"
X-AppEngine-Upload-Creation: 2017-02-16 20:17:05.729401


--===============5516630363169856841==
Content-Type: text/plain
Content-Disposition: form-data; name="newSurveyId"

10001
--===============5516630363169856841==
Content-Type: text/plain
Content-Disposition: form-data; name="newSurveyName"

N
--===============5516630363169856841==
Content-Type: text/plain
Content-Disposition: form-data; name="newUserMessage"


--===============5516630363169856841==
Content-Type: text/plain
Content-Disposition: form-data; name="selectedClient"

6
--===============5516630363169856841==
Content-Type: text/plain
Content-Disposition: form-data; name="selectedPsf"

3
--===============5516630363169856841==
Content-Type: text/plain
Content-Disposition: form-data; name="selectedSection"

1
--===============5516630363169856841==--

Answer 1

I figured out a workaround, but I do think there is a bug here the Google folks need to look at. The Cloud Tools python dev server wasn't putting the all important X-AppEngine-BlobUpload header element into the rewritten request header. I modified blob_upload.py and http_proxy.py to do this (in tools/devserver2 under google-cloud-sdk). I then had to steel some code from Google's own production server code base, ParseBlobUploadFilter.java, to process the non-standard request payload and build the missing request attributes and make accessible the original request parameters. Of course, this code path should only be taken when running on a local dev server; this code is correctly called by the Google App Engine.