GraphQL query Post with Python Request

Question

I've been trying to Post Request with Python to scrape some json data from a site. I have no experience with GraphQL, but I think this site is built with it. So anyways, the problem is even if I request it with the POST method, it keeps returning me a "GRAPHQL_PARSE_FAILED" (I'll post the result below). I'm guessing it's because there's a 'downwards arrow with corner leftwards' which is contained in my query. Since the message says it can't parse '\↵'.

Here's the code.

self.API_URL = 'https://pcmap-api.place.naver.com/graphql'

def get_location_info(self, query, start):
    header = {
        'authority': 'pcmap-api.place.naver.com',
        'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36'
    }

    query = """
    {
        "query getRestaurants($input: RestaurantsInput, $isNmap: Boolean!, $isBounds: Boolean!) {↵  restaurants(input: $input) {↵    total↵    items {↵      ...RestaurantItemFields↵      easyOrder {↵        easyOrderId↵        easyOrderCid↵        businessHours {↵          weekday {↵            start↵            end↵            __typename↵          }↵          weekend {↵            start↵            end↵            __typename↵          }↵          __typename↵        }↵        __typename↵      }↵      baemin {↵        businessHours {↵          deliveryTime {↵            start↵            end↵            __typename↵          }↵          closeDate {↵            start↵            end↵            __typename↵          }↵          temporaryCloseDate {↵            start↵            end↵            __typename↵          }↵          __typename↵        }↵        __typename↵      }↵      yogiyo {↵        businessHours {↵          actualDeliveryTime {↵            start↵            end↵            __typename↵          }↵          bizHours {↵            start↵            end↵            __typename↵          }↵          __typename↵        }↵        __typename↵      }↵      __typename↵    }↵    nlu {↵      ...NluFields↵      __typename↵    }↵    brand {↵      name↵      isBrand↵      type↵      menus {↵        order↵        id↵        images {↵          url↵          desc↵          __typename↵        }↵        name↵        desc↵        price↵        isRepresentative↵        detailUrl↵        orderType↵        catalogId↵        source↵        menuId↵        nutrients↵        allergies↵        __typename↵      }↵      __typename↵    }↵    optionsForMap @include(if: $isBounds) {↵      maxZoom↵      minZoom↵      includeMyLocation↵      maxIncludePoiCount↵      center↵      spotId↵      __typename↵    }↵    __typename↵  }↵}↵↵fragment RestaurantItemFields on RestaurantSummary {↵  id↵  dbType↵  name↵  businessCategory↵  category↵  description↵  hasBooking↵  hasNPay↵  x↵  y↵  distance↵  imageUrl↵  imageUrls↵  imageCount↵  phone↵  virtualPhone↵  routeUrl↵  streetPanorama {↵    id↵    pan↵    tilt↵    lat↵    lon↵    __typename↵  }↵  roadAddress↵  address↵  commonAddress↵  blogCafeReviewCount↵  bookingReviewCount↵  totalReviewCount↵  bookingReviewScore↵  bookingUrl↵  bookingBusinessId↵  talktalkUrl↵  options↵  promotionTitle↵  agencyId↵  businessHours↵  microReview↵  tags↵  priceCategory↵  broadcastInfo {↵    program↵    date↵    menu↵    __typename↵  }↵  michelinGuide {↵    year↵    star↵    comment↵    url↵    hasGrade↵    isBib↵    alternateText↵    __typename↵  }↵  broadcasts {↵    program↵    menu↵    episode↵    broadcast_date↵    __typename↵  }↵  tvcastId↵  naverBookingCategory↵  saveCount↵  uniqueBroadcasts↵  isDelivery↵  markerLabel @include(if: $isNmap) {↵    text↵    style↵    __typename↵  }↵  imageMarker @include(if: $isNmap) {↵    marker↵    markerSelected↵    __typename↵  }↵  isTableOrder↵  isPreOrder↵  isTakeOut↵  bookingDisplayName↵  bookingVisitId↵  bookingPickupId↵  popularMenuImages {↵    name↵    price↵    bookingCount↵    menuUrl↵    menuListUrl↵    imageUrl↵    isPopular↵    usePanoramaImage↵    __typename↵  }↵  visitorReviewCount↵  visitorReviewScore↵  detailCid {↵    c0↵    c1↵    c2↵    c3↵    __typename↵  }↵  streetPanorama {↵    id↵    pan↵    tilt↵    lat↵    lon↵    __typename↵  }↵  __typename↵}↵↵fragment NluFields on Nlu {↵  queryType↵  user {↵    gender↵    __typename↵  }↵  queryResult {↵    ptn0↵    ptn1↵    region↵    spot↵    tradeName↵    service↵    selectedRegion {↵      name↵      index↵      x↵      y↵      __typename↵    }↵    selectedRegionIndex↵    otherRegions {↵      name↵      index↵      __typename↵    }↵    property↵    keyword↵    queryType↵    nluQuery↵    businessType↵    cid↵    branch↵    franchise↵    titleKeyword↵    location {↵      x↵      y↵      default↵      longitude↵      latitude↵      dong↵      si↵      __typename↵    }↵    noRegionQuery↵    priority↵    showLocationBarFlag↵    themeId↵    filterBooking↵    repRegion↵    repSpot↵    dbQuery {↵      isDefault↵      name↵      type↵      getType↵      useFilter↵      hasComponents↵      __typename↵    }↵    type↵    category↵    __typename↵  }↵  __typename↵}↵"
    }
    """ 

    data = {
        'operationName': "getRestaurants",
        'query': query,
        'variables': {'input': {
            'deviceType': "pcmap",
            'display': "50",
            'isNmap': "false",
            'query': query,
            'start': start
        },
        'isBound': 'true',
        'isNmap': 'false'
        }
    }
    r = requests.post(self.API_URL, headers=header, json=data).json()

As you can see in the query, it has the arrow that GraphQL can't parse. I'm guessing that I have to convert the "query" into some kind that GraphQL can understand. But it's kinda frustrating since I just copied and pasted what Request Payload has just shown. I placed query outside of the data parameter, because it seemed too messy.

Here's the result.

{
"errors": [
    {
        "extensions": {
            "code": "GRAPHQL_PARSE_FAILED"
        },
        "locations": [
            {
                "column": 89,
                "line": 1
            }
        ],
        "message": "Syntax Error: Cannot parse the unexpected character \"\\u21B5\".",
        "status": 500
    }
]

}

I really would appreciate it you can think of a way to solve this. Thanks!

Answer 1

I've been trying to Post Request with Python to scrape some json data from a site. I have no experience with GraphQL, but I think this site is built with it. So anyways, the problem is even if I request it with the POST method, it keeps returning me a "GRAPHQL_PARSE_FAILED" (I'll post the result below). I'm guessing it's because there's a 'downwards arrow with corner leftwards' which is contained in my query. Since the message says it can't parse '\↵'.

Here's the code.

self.API_URL = 'https://pcmap-api.place.naver.com/graphql'

def get_location_info(self, query, start):
    header = {
        'authority': 'pcmap-api.place.naver.com',
        'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36'
    }

    query = """
    {
        "query getRestaurants($input: RestaurantsInput, $isNmap: Boolean!, $isBounds: Boolean!) {↵  restaurants(input: $input) {↵    total↵    items {↵      ...RestaurantItemFields↵      easyOrder {↵        easyOrderId↵        easyOrderCid↵        businessHours {↵          weekday {↵            start↵            end↵            __typename↵          }↵          weekend {↵            start↵            end↵            __typename↵          }↵          __typename↵        }↵        __typename↵      }↵      baemin {↵        businessHours {↵          deliveryTime {↵            start↵            end↵            __typename↵          }↵          closeDate {↵            start↵            end↵            __typename↵          }↵          temporaryCloseDate {↵            start↵            end↵            __typename↵          }↵          __typename↵        }↵        __typename↵      }↵      yogiyo {↵        businessHours {↵          actualDeliveryTime {↵            start↵            end↵            __typename↵          }↵          bizHours {↵            start↵            end↵            __typename↵          }↵          __typename↵        }↵        __typename↵      }↵      __typename↵    }↵    nlu {↵      ...NluFields↵      __typename↵    }↵    brand {↵      name↵      isBrand↵      type↵      menus {↵        order↵        id↵        images {↵          url↵          desc↵          __typename↵        }↵        name↵        desc↵        price↵        isRepresentative↵        detailUrl↵        orderType↵        catalogId↵        source↵        menuId↵        nutrients↵        allergies↵        __typename↵      }↵      __typename↵    }↵    optionsForMap @include(if: $isBounds) {↵      maxZoom↵      minZoom↵      includeMyLocation↵      maxIncludePoiCount↵      center↵      spotId↵      __typename↵    }↵    __typename↵  }↵}↵↵fragment RestaurantItemFields on RestaurantSummary {↵  id↵  dbType↵  name↵  businessCategory↵  category↵  description↵  hasBooking↵  hasNPay↵  x↵  y↵  distance↵  imageUrl↵  imageUrls↵  imageCount↵  phone↵  virtualPhone↵  routeUrl↵  streetPanorama {↵    id↵    pan↵    tilt↵    lat↵    lon↵    __typename↵  }↵  roadAddress↵  address↵  commonAddress↵  blogCafeReviewCount↵  bookingReviewCount↵  totalReviewCount↵  bookingReviewScore↵  bookingUrl↵  bookingBusinessId↵  talktalkUrl↵  options↵  promotionTitle↵  agencyId↵  businessHours↵  microReview↵  tags↵  priceCategory↵  broadcastInfo {↵    program↵    date↵    menu↵    __typename↵  }↵  michelinGuide {↵    year↵    star↵    comment↵    url↵    hasGrade↵    isBib↵    alternateText↵    __typename↵  }↵  broadcasts {↵    program↵    menu↵    episode↵    broadcast_date↵    __typename↵  }↵  tvcastId↵  naverBookingCategory↵  saveCount↵  uniqueBroadcasts↵  isDelivery↵  markerLabel @include(if: $isNmap) {↵    text↵    style↵    __typename↵  }↵  imageMarker @include(if: $isNmap) {↵    marker↵    markerSelected↵    __typename↵  }↵  isTableOrder↵  isPreOrder↵  isTakeOut↵  bookingDisplayName↵  bookingVisitId↵  bookingPickupId↵  popularMenuImages {↵    name↵    price↵    bookingCount↵    menuUrl↵    menuListUrl↵    imageUrl↵    isPopular↵    usePanoramaImage↵    __typename↵  }↵  visitorReviewCount↵  visitorReviewScore↵  detailCid {↵    c0↵    c1↵    c2↵    c3↵    __typename↵  }↵  streetPanorama {↵    id↵    pan↵    tilt↵    lat↵    lon↵    __typename↵  }↵  __typename↵}↵↵fragment NluFields on Nlu {↵  queryType↵  user {↵    gender↵    __typename↵  }↵  queryResult {↵    ptn0↵    ptn1↵    region↵    spot↵    tradeName↵    service↵    selectedRegion {↵      name↵      index↵      x↵      y↵      __typename↵    }↵    selectedRegionIndex↵    otherRegions {↵      name↵      index↵      __typename↵    }↵    property↵    keyword↵    queryType↵    nluQuery↵    businessType↵    cid↵    branch↵    franchise↵    titleKeyword↵    location {↵      x↵      y↵      default↵      longitude↵      latitude↵      dong↵      si↵      __typename↵    }↵    noRegionQuery↵    priority↵    showLocationBarFlag↵    themeId↵    filterBooking↵    repRegion↵    repSpot↵    dbQuery {↵      isDefault↵      name↵      type↵      getType↵      useFilter↵      hasComponents↵      __typename↵    }↵    type↵    category↵    __typename↵  }↵  __typename↵}↵"
    }
    """ 

    data = {
        'operationName': "getRestaurants",
        'query': query,
        'variables': {'input': {
            'deviceType': "pcmap",
            'display': "50",
            'isNmap': "false",
            'query': query,
            'start': start
        },
        'isBound': 'true',
        'isNmap': 'false'
        }
    }
    r = requests.post(self.API_URL, headers=header, json=data).json()

As you can see in the query, it has the arrow that GraphQL can't parse. I'm guessing that I have to convert the "query" into some kind that GraphQL can understand. But it's kinda frustrating since I just copied and pasted what Request Payload has just shown. I placed query outside of the data parameter, because it seemed too messy.

Here's the result.

{
"errors": [
    {
        "extensions": {
            "code": "GRAPHQL_PARSE_FAILED"
        },
        "locations": [
            {
                "column": 89,
                "line": 1
            }
        ],
        "message": "Syntax Error: Cannot parse the unexpected character \"\\u21B5\".",
        "status": 500
    }
]

}

I really would appreciate it you can think of a way to solve this. Thanks!

Answer 2

I've been trying to Post Request with Python to scrape some json data from a site. I have no experience with GraphQL, but I think this site is built with it. So anyways, the problem is even if I request it with the POST method, it keeps returning me a "GRAPHQL_PARSE_FAILED" (I'll post the result below). I'm guessing it's because there's a 'downwards arrow with corner leftwards' which is contained in my query. Since the message says it can't parse '\↵'.

Here's the code.

self.API_URL = 'https://pcmap-api.place.naver.com/graphql'

def get_location_info(self, query, start):
    header = {
        'authority': 'pcmap-api.place.naver.com',
        'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36'
    }

    query = """
    {
        "query getRestaurants($input: RestaurantsInput, $isNmap: Boolean!, $isBounds: Boolean!) {↵  restaurants(input: $input) {↵    total↵    items {↵      ...RestaurantItemFields↵      easyOrder {↵        easyOrderId↵        easyOrderCid↵        businessHours {↵          weekday {↵            start↵            end↵            __typename↵          }↵          weekend {↵            start↵            end↵            __typename↵          }↵          __typename↵        }↵        __typename↵      }↵      baemin {↵        businessHours {↵          deliveryTime {↵            start↵            end↵            __typename↵          }↵          closeDate {↵            start↵            end↵            __typename↵          }↵          temporaryCloseDate {↵            start↵            end↵            __typename↵          }↵          __typename↵        }↵        __typename↵      }↵      yogiyo {↵        businessHours {↵          actualDeliveryTime {↵            start↵            end↵            __typename↵          }↵          bizHours {↵            start↵            end↵            __typename↵          }↵          __typename↵        }↵        __typename↵      }↵      __typename↵    }↵    nlu {↵      ...NluFields↵      __typename↵    }↵    brand {↵      name↵      isBrand↵      type↵      menus {↵        order↵        id↵        images {↵          url↵          desc↵          __typename↵        }↵        name↵        desc↵        price↵        isRepresentative↵        detailUrl↵        orderType↵        catalogId↵        source↵        menuId↵        nutrients↵        allergies↵        __typename↵      }↵      __typename↵    }↵    optionsForMap @include(if: $isBounds) {↵      maxZoom↵      minZoom↵      includeMyLocation↵      maxIncludePoiCount↵      center↵      spotId↵      __typename↵    }↵    __typename↵  }↵}↵↵fragment RestaurantItemFields on RestaurantSummary {↵  id↵  dbType↵  name↵  businessCategory↵  category↵  description↵  hasBooking↵  hasNPay↵  x↵  y↵  distance↵  imageUrl↵  imageUrls↵  imageCount↵  phone↵  virtualPhone↵  routeUrl↵  streetPanorama {↵    id↵    pan↵    tilt↵    lat↵    lon↵    __typename↵  }↵  roadAddress↵  address↵  commonAddress↵  blogCafeReviewCount↵  bookingReviewCount↵  totalReviewCount↵  bookingReviewScore↵  bookingUrl↵  bookingBusinessId↵  talktalkUrl↵  options↵  promotionTitle↵  agencyId↵  businessHours↵  microReview↵  tags↵  priceCategory↵  broadcastInfo {↵    program↵    date↵    menu↵    __typename↵  }↵  michelinGuide {↵    year↵    star↵    comment↵    url↵    hasGrade↵    isBib↵    alternateText↵    __typename↵  }↵  broadcasts {↵    program↵    menu↵    episode↵    broadcast_date↵    __typename↵  }↵  tvcastId↵  naverBookingCategory↵  saveCount↵  uniqueBroadcasts↵  isDelivery↵  markerLabel @include(if: $isNmap) {↵    text↵    style↵    __typename↵  }↵  imageMarker @include(if: $isNmap) {↵    marker↵    markerSelected↵    __typename↵  }↵  isTableOrder↵  isPreOrder↵  isTakeOut↵  bookingDisplayName↵  bookingVisitId↵  bookingPickupId↵  popularMenuImages {↵    name↵    price↵    bookingCount↵    menuUrl↵    menuListUrl↵    imageUrl↵    isPopular↵    usePanoramaImage↵    __typename↵  }↵  visitorReviewCount↵  visitorReviewScore↵  detailCid {↵    c0↵    c1↵    c2↵    c3↵    __typename↵  }↵  streetPanorama {↵    id↵    pan↵    tilt↵    lat↵    lon↵    __typename↵  }↵  __typename↵}↵↵fragment NluFields on Nlu {↵  queryType↵  user {↵    gender↵    __typename↵  }↵  queryResult {↵    ptn0↵    ptn1↵    region↵    spot↵    tradeName↵    service↵    selectedRegion {↵      name↵      index↵      x↵      y↵      __typename↵    }↵    selectedRegionIndex↵    otherRegions {↵      name↵      index↵      __typename↵    }↵    property↵    keyword↵    queryType↵    nluQuery↵    businessType↵    cid↵    branch↵    franchise↵    titleKeyword↵    location {↵      x↵      y↵      default↵      longitude↵      latitude↵      dong↵      si↵      __typename↵    }↵    noRegionQuery↵    priority↵    showLocationBarFlag↵    themeId↵    filterBooking↵    repRegion↵    repSpot↵    dbQuery {↵      isDefault↵      name↵      type↵      getType↵      useFilter↵      hasComponents↵      __typename↵    }↵    type↵    category↵    __typename↵  }↵  __typename↵}↵"
    }
    """ 

    data = {
        'operationName': "getRestaurants",
        'query': query,
        'variables': {'input': {
            'deviceType': "pcmap",
            'display': "50",
            'isNmap': "false",
            'query': query,
            'start': start
        },
        'isBound': 'true',
        'isNmap': 'false'
        }
    }
    r = requests.post(self.API_URL, headers=header, json=data).json()

As you can see in the query, it has the arrow that GraphQL can't parse. I'm guessing that I have to convert the "query" into some kind that GraphQL can understand. But it's kinda frustrating since I just copied and pasted what Request Payload has just shown. I placed query outside of the data parameter, because it seemed too messy.

Here's the result.

{
"errors": [
    {
        "extensions": {
            "code": "GRAPHQL_PARSE_FAILED"
        },
        "locations": [
            {
                "column": 89,
                "line": 1
            }
        ],
        "message": "Syntax Error: Cannot parse the unexpected character \"\\u21B5\".",
        "status": 500
    }
]

}

I really would appreciate it you can think of a way to solve this. Thanks!