Accessing Mt Gox API via http-conduit-0.1.9.3: query string causes timeout

Question

I'm trying to access the Mt Gox REST API using http-conduit . Queries that just have a path (eg https://data.mtgox.com/api/2/BTCUSD/money/ticker ) work fine, but when I add a queryString to the request it times out.

So this works:

mtGoxRequest :: String -> QueryText -> Request m
mtGoxRequest p qt = def {
   secure = True,
   host = "data.mtgox.com",
   port = 443,
   method = "GET",
   path = fromString $ "api/2/" ++ p,
   queryString = renderQuery False $ queryTextToQuery qt,
   responseTimeout = Just 10000000
}

currencyTicker :: Request m
currencyTicker = mtGoxRequest "BTCUSD/money/ticker" []

But this times out:

tradeStream :: Currency -> UTCTime -> Request m
tradeStream t = mtGoxRequest
   "BTCUSD/money/trades/fetch"
   [("since", Just $ T.pack $ utcToGoxTime t)]

The difference seems to be in the use of a queryString: when I added the bogus query "foo=bar" to the currencyTicker that timed out too.

However all this works fine in the web browser: going to https://data.mtgox.com/api/2/BTCUSD/money/ticker?foo=bar instantly returns the correct error message instead of timing out. The trade fetch URL works as well, although I won't include a link because the "since" argument says how far back to go. Conversely, if I remove the queryString from the trades list request it correctly returns the entire available trade history.

So something about the http-conduit query string is obviously different. Anyone know what it might be?

Here is the Haskell Request object being sent (as printed by "Show"):

Request {
  host                 = "data.mtgox.com"
  port                 = 443
  secure               = True
  clientCertificates   = []
  requestHeaders       = []
  path                 = "api/2/BTCUSD/money/trades/fetch"
  queryString          = "since=1367142721624293"
  requestBody          = RequestBodyLBS Empty
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = Just 10000000
}

According to its returned headers Mt Gox is using cloudflare-nginx and PHP 5.

Edit: Forgot to mention that when I use http-conduit to send a request with a queryString to http://scooterlabs.com/echo I get the correct response as well, so it seems to be some interaction between the Mt Gox webserver and http-conduit.

Answer 1

Got it figured out. You need to add a User-Agent string. So

requestHeaders = [(CI.mk "User-Agent", "Test/0.0.1")],

in the middle of the request function makes it work.

Answer 2

$ time curl https://data.mtgox.com/api/2/BTCUSD/money/trades/fetch?since=1367142721624293
...
real    0m20.993s

Looks to me like everything is working correctly: the API call takes a while to return, so http-conduit throws a timeout exception since 20s is longer than 10s.