How to get data (items) from eBay Finding API through Google App Engine?

Question

I was trying to retrieve data from eBay Finding API through Google App Engine for a project. It seems to have to do with Google App Engine blocking regular requests.

I tried using urlfetch and urllib3 but to no avail. I am trying to retrieve item data in JSON format.

This was my first attempt:

def get(self):
        requests_toolbelt.adapters.appengine.monkeypatch()
        http = urllib3.PoolManager()
        key = 'WailSahb-Analysis-PRD-4c970d9ce-c9a80d1e'
        search_term = 'laptop'
        url = ('http://svcs.ebay.com/services/search/FindingService/v1\
    ?OPERATION-NAME=findItemsByKeywords\
    &sortOrder=PricePlusShippingLowest\
    &buyerPostalCode=92128&SERVICE-VERSION=1.13.0\
    &SECURITY-APPNAME=' + key +
    '&RESPONSE-DATA-FORMAT=JSON\
    &REST-PAYLOAD\
    &itemFilter(0).name=Condition\
    &itemFilter(0).value=New\
    &itemFilter(1).paramName=Currency\
    &itemFilter(1).paramValue=EUR\
    &itemFilter(2).paramName=FeedbackScoreMin\
    &itemFilter(2).paramValue=10\
    &paginationIntput.entriesPerPage=100\
    &outputSelector(0)=SellerInfo\
    &descriptionSearch=FALSE\
    &paginationIntput.pageNumber=1\
    &keywords=' + search_term)
        url = url.replace(" ", "%20")
        result = http.request('GET', url)
        self.response.write(result)

With this approach I get the following error:

MaxRetryError: HTTPSConnectionPool(host='pages.ebay.com', port=443): Max retries exceeded with url: /messages/page_not_found.html?eBayErrorEventName=p4buoajkbnmbehq%60%3C%3Dosu71%2872%3A4505-2018.08.16.15.28.47.151.MST (Caused by ProtocolError('Connection aborted.', error(13, 'Permission denied')))

I also tried this approach:

def get(self):
        try:
            api = Connection(appid='WailSahb-Analysis-PRD-4c970d9ce-c9a80d1e', config_file=None)
            response = api.execute('findItemsAdvanced', {'keywords': 'legos'})

            assert(response.reply.ack == 'Success')
            assert(type(response.reply.timestamp) == datetime.datetime)
            assert(type(response.reply.searchResult.item) == list)

            item = response.reply.searchResult.item[0]
            assert(type(item.listingInfo.endTime) == datetime.datetime)
            assert(type(response.dict()) == dict)
            self.response.headers['Content-Type'] = 'text/plain'
            self.response.write(result.data)
        except ConnectionError as e:
            self.response.write(e.response.dict())

In which I get this error:

TypeError: super(type, obj): obj must be an instance or subtype of type

Could anyone please help me get through this.

Thank you for your time.

Answer 1

I tried to reproduce your problem and had to change a few things, but was eventually able to successfully get the page you're trying to request.

I first copied your first attempt verbatim, and the error I got was slightly different:

MaxRetryError: HTTPSConnectionPool(host='pages.ebay.com', port=443): Max retries exceeded with url: /messages/page_not_found.html?eBayErrorEventName=p4buoajkbnmbehq%60%3C%3Dsm%7E71%287147606-2018.08.23.14.59.22.829.MST (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",))

This indicated to me that the that the issue was a missing or invalid SSL module. You didn't share your app.yaml , but I had to add the following to mine to get the HTTPS request to succeed:

libraries:
- name: ssl
  version: latest

However, the eventual app engine response was incorrect, because result was a urllib3.response.HTTPResponse object, and not an actual response.

To fix this, I changed the line:

self.response.write(result)

to

self.response.write(result.content)

and then this worked as intended.

---

Here's the final versions of the files that made this work for me:

In app.yaml :

runtime: python27
api_version: 1
threadsafe: true

handlers:
- url: /.*
  script: main.app

libraries:
- name: ssl
  version: latest

In main.py :

import webapp2
from requests_toolbelt.adapters import appengine
import urllib3


class MainPage(webapp2.RequestHandler):
    def get(self):
        appengine.monkeypatch()
        http = urllib3.PoolManager()
        key = 'WailSahb-Analysis-PRD-4c970d9ce-c9a80d1e'
        search_term = 'laptop'
        url = (
            'http://svcs.ebay.com/services/search/FindingService/v1\
            ?OPERATION-NAME=findItemsByKeywords\
            &sortOrder=PricePlusShippingLowest\
            &buyerPostalCode=92128&SERVICE-VERSION=1.13.0\
            &SECURITY-APPNAME=' + key +
            '&RESPONSE-DATA-FORMAT=JSON\
            &REST-PAYLOAD\
            &itemFilter(0).name=Condition\
            &itemFilter(0).value=New\
            &itemFilter(1).paramName=Currency\
            &itemFilter(1).paramValue=EUR\
            &itemFilter(2).paramName=FeedbackScoreMin\
            &itemFilter(2).paramValue=10\
            &paginationIntput.entriesPerPage=100\
            &outputSelector(0)=SellerInfo\
            &descriptionSearch=FALSE\
            &paginationIntput.pageNumber=1\
            &keywords=' + search_term)
        url = url.replace(" ", "%20")
        result = http.request('GET', url)
        self.response.write(result.data)


app = webapp2.WSGIApplication([
    ('/', MainPage),
], debug=True)