Python Requests get returns response code 401 for nse india website

Question

I use this program to get the json data from https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY but since this morning it's not working as it returns <Response [401]> . The link loads fine on chrome though. Is there any way to fix this without using selenium?

import json
import requests

headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, '
                         'like Gecko) '
                         'Chrome/80.0.3987.149 Safari/537.36',
           'accept-language': 'en,gu;q=0.9,hi;q=0.8', 'accept-encoding': 'gzip, deflate, br'}

res = requests.get("https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY", headers=headers)
print(res)

Answer 1

Try this:

import requests

baseurl = "https://www.nseindia.com/"
url = f"https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY"
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, '
                         'like Gecko) '
                         'Chrome/80.0.3987.149 Safari/537.36',
           'accept-language': 'en,gu;q=0.9,hi;q=0.8', 'accept-encoding': 'gzip, deflate, br'}
session = requests.Session()
request = session.get(baseurl, headers=headers, timeout=5)
cookies = dict(request.cookies)
response = session.get(url, headers=headers, timeout=5, cookies=cookies)
print(response.json())

To access the NSE (api's) site multiple times then set cookies in each subsequent requests:

response = session.get(url, headers=headers, timeout=5, cookies=cookies)

Answer 2

对于执行和操作，您首先必须执行登录请求调用，该调用将返回会话 ID。此会话 ID 将用于下一个请求的标头部分，以进行身份​​验证并返回有效响应

Answer 3

Even I'm getting the same error from today morning. Earlier I used to face this issue only on servers hosted outside India. But now looks like they are trying to stop scraping altogether.

Really interested in finding a solution to this.

Also for the time being the old website still seems to be working. I know it's not a permanent solution but I'm using that as a stop gap until I find a more permanent solution.

Answer 4

Got the same error with Java. Solution is

• first make connection to https://nseindia.com .
• And then read the cookie returned by the server.
• Add the same cookie in your request.

Depending on which implementation you are using to make ssl calls, you will have to find appropriate methods to get and set cookies. Dont know why NSE site has to look for a cookie when the data is publicly enabled. Some extra technology applied by nse for sure..

Answer 5

I tried same thing with node js please use node js ide link here for online ide please review my code

const axios = require('axios');


let cookie;
let url_oc = "https://www.nseindia.com/option-chain"
let url = "https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY"
let headers = {
  'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)',
  'accept-language': 'en,gu;q=0.9,hi;q=0.8', 'accept-encoding': 'gzip, deflate, br'
}

const instance = axios.create({
  baseURL: url_oc,
  headers: headers,
  cookie: cookie ? cookie : ""
});



const getCookies = async () => {
  try {
    const response = await instance.get(url_oc);
    cookie = response.headers['set-cookie'].join();
  } catch (error) {
    if (error.response.status === 403) {
      console.log("getCookies =========> error.status === 403");
      await getCookies()
    } else {
      console.log("getCookies =========> error");
    }
  }
}


const getAPIData = async () => {
  try {
    if (cookie) {
      const response = await instance.get(url);
      console.log(response.data.records.timestamp);
    }

  } catch (error) {
    if (error.response && error.response.status === 401) {
      console.log("getAPIData =========> error.status === 401");
      if (!cookie) {
        console.log("getAPIData =========> cookie not found");
        await getCookies()
      }
      await getAPIData()
    } else {
      console.log("getAPIData =========> error");
    }
  }
}



(async () => {
  setInterval(async () => {
    await getCookies()
  }, 5000);

  setInterval(async () => {
    await getAPIData()
  }, 3000);
})()