如何使用符号列表获取股票的价格历史数据

Question

I am trying to be able to pass a list into my function to retrieve stock price data for every symbol in that list.

I am using the TD Ameritrade API. The API only allows you to get 200 symbols per call. I have broken the symbols up into chunks of 200 by using:

# TD Ameritrade api has a limit to the number of symbols you can get data for
# at a time. So to get everything I have to chunk the list into 200 symbols at a time
def chunks(l, n):
    """
    :param l: takes in a list
    :param n: Lets you know how long you want each chunk to be
    """
    n = max(1, n)
    return(l[i:i+n] for i in range(0, len(l), n))

# symbols_chunked = list(chunks(list(set(stocks)), 200)) 

Once the symbols are in chunks of 200 I then pass that chunked list to the functions that I created to make api calls. Some reason the price history call isn't working though.

• I have three functions built so far:
  • Quote Data
  • fundamental data
  • price history

The first two are working but I can't get the price history to work.

This is what the other two look like:

def request_quotes(stocks):
    """
    :param stocks: makes an api call for a list of stock symbols
    :returns: a pandas dataframe of quote data 
    """
    
    url = BASE + "marketdata/quotes"
    
    params = {
        'apikey': API_KEY,
        'symbol': stocks
    }
    
    request = requests.get(
        url=url,
        params=params
    ).json()
    
    time.sleep(1)
    
    return pd.DataFrame.from_dict(request, orient='index').reset_index(drop=True)

and:

def get_fundamentals(stocks):
    """
    :param stocks: List of stocks chuncked into 200 symbol chunks 
    
    :return: This will return tons of information that will then
    be changed into dataframes and inserted into the database. 
    """
    
    url = BASE + "instruments"
    
    # pass params 
    params = {
        'apikey': API_KEY,
        'symbol': stocks, 
        'projection': 'fundamental'
    }
    
    request = requests.get(
        url=url,
        params=params
        ).json()
    
    time.sleep(1)
    
    return pd.DataFrame.from_dict(request, orient='index').reset_index(drop='True')

Those two work, but this doesn't:

def get_five_minute_ph(stocks):
    """
    :param stocks: Chunked list of stocks to be passed to the preset
    price history params that this function will retrieve. 
    
    :returns: Price history data for each company in the chuncked list. 
    """
    
    url = BASE + "marketdata/pricehistory"

    params = {
        'apikey': API_KEY, # possible params:
        'symbol': stocks, # DON'T CHANGE, change stocks exchange param in cleaned_symbols func
        'period': '10', # 1,2,3,4,5,10 
        'periodType': 'day', # day, week, month, year, ytd
        'frequency': '5', # minute=1,5,10,15,30; daily=1, weekly=1, monthly=1
        'frequencyType': 'minute' # minute, daily, weekly, monthly
    }
    
    # 5 minute daily data request
    request = requests.get(
        url=url,
        params=params
    ).json()
    
    time.sleep(1)
    
    # 5 minute dataframe 
    five_min_df = pd.DataFrame.from_dict(request, orient='index').reset_index(drop='True')
    
    return five_min_df

df = get_five_minute_ph(["MSFT","FB"])
print(df)

• I get this error code:

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
<ipython-input-10-679eb6b7471f> in <module>
     31     return five_min_df
     32 
---> 33 df = get_five_minute_ph(["MSFT","FB"])
     34 print(df)

<ipython-input-10-679eb6b7471f> in get_five_minute_ph(stocks)
     19 
     20     # 5 minute daily data request
---> 21     request = requests.get(
     22         url=url,
     23         params=params

~\anaconda3\lib\site-packages\requests\models.py in json(self, **kwargs)
    898                     # used.
    899                     pass
--> 900         return complexjson.loads(self.text, **kwargs)
    901 
    902     @property

~\anaconda3\lib\json\__init__.py in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    355             parse_int is None and parse_float is None and
    356             parse_constant is None and object_pairs_hook is None and not kw):
--> 357         return _default_decoder.decode(s)
    358     if cls is None:
    359         cls = JSONDecoder

~\anaconda3\lib\json\decoder.py in decode(self, s, _w)
    335 
    336         """
--> 337         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338         end = _w(s, end).end()
    339         if end != len(s):

~\anaconda3\lib\json\decoder.py in raw_decode(self, s, idx)
    353             obj, end = self.scan_once(s, idx)
    354         except StopIteration as err:
--> 355             raise JSONDecodeError("Expecting value", s, err.value) from None
    356         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

• I have even tried to do this:

def get_one_minute_ph(stocks):
    """
    :param stocks: Chunked list of stocks to be passed to the preset
    price history params that this function will retrieve. 
    
    :returns: Price history data for each company in the chuncked list. 
    """
    url = BASE + "marketdata/pricehistory"

    params = {
        'apikey': API_KEY, # possible params:
        'symbol': stocks, # DON'T CHANGE, change stocks exchange param in cleaned_symbols func
        'period': '10', # 1,2,3,4,5,10 
        'periodType': 'day', # day, week, month, year, ytd
        'frequency': '1', # minute=1,5,10,15,30; daily=1, weekly=1, monthly=1
        'frequencyType': 'minute' # minute, daily, weekly, monthly
    }
    
    # 10 day 1 minute data request
    request = requests.get(
        url=url,
        params=params
    ).json()
    
    time.sleep(1)
    #SELECT THE DICT KEY THAT CONTAINS A LIST AND ASSIGN IT A VARIABLE
    #data_dict_list = result['candles']
    # CREATE FUNCTION TO SEPERATE THE DICTIONARIES FROM THE LIST

    def _seperate_key_value(arg1=dict(),arg2='dict_key',arg3='symbol'):
        data_dict_list = arg1[arg2]
        company = arg1[arg3]
        """
        ==========================================================================
        :param:--> arg1(variable = to a dictionary):--> ['arg2'] is the ('key') in
        arg1's dictionary that contains the values we need returned:-->['arg3']
        returns the symbol for the company whos data it belongs to. MADE FOR:
        dictionary created by using the td ameritrade price history API
        ==========================================================================
        :return: List of dictionaries. used for the TD Ameritrade price history
        API. Each dictionary located in the list will represent one
        (candle) on a "chart".
        ==========================================================================
        :purpose: get price data ready to be sorted in values_list
        ==========================================================================
        """

        candle = list((company, data_dict_list))
        return candle
    
#     one_min_df = pd.DataFrame.from_dict(request, orient='index').reset_index(drop='True')
    candles = _seperate_key_value(request)
    
    def _new_values_list(arg1=list()):
        keys = arg1[1]
        sym = arg1[0]
        """
        ==========================================================================
        :param: values returned by the seperate_key_value() in the form of a
        variable. Probably something like candle_data, candleD, etc.
        ==========================================================================
        :returns: a sorted pandas DataFrame with columns for the open, high, low
        close, volume and datetime as index. It also, includes a row with the
        epoch time for easier time series calculations.
        ==========================================================================
        """

        o = []
        for vals in keys: o.append(vals['open'])

        h = []
        for vals in keys: h.append(vals['high'])

        l = []
        for vals in keys: l.append(vals['low'])

        c = []
        for vals in keys: c.append(vals['close'])

        etime = []
        for vals in keys: etime.append(vals['datetime'])

        time = []
        for vals in keys:
            vals = pd.to_datetime(vals['datetime'], unit='ms')
            time.append(vals)

        vol = []
        for vals in keys: vol.append(vals['volume'])

        df = pd.DataFrame({'unix_time':etime, 'datetime': time, 'open': o, 'high': h, 'low': l, 'close': c,'volume': vol})

        return df
    
    one_min_df = _new_values_list(candles)
    # 1 minute dataframe 
    return one_min_df 

df = get_one_minute_ph(["FB","MSFT"])
print(df)

and I get the same error code. The reason, I created the function this way was to loop through the JSON data and just pull out the price history data before it is turned into a dataframe.

This is how the basic version looks:

def get_price_history(stock):
    """
    :param stock: company symbol/ticker
    :Example: MSFT 10 day minute 10

    :returns: raw json data (Open, High, Low, close, Volume, and Time (epoch time))
    """
    url = "https://api.tdameritrade.com/v1/marketdata/{}/pricehistory".format(stock)

    params = {
        'apikey': key, # possible params:
        'symbol': stock, # DON'T CHANGE, change stocks exchange param in cleaned_symbols func
        'period': '10', # 1,2,3,4,5,10 
        'periodType': 'day', # day, week, month, year, ytd
        'frequency': '1', # minute=1,5,10,15,30; daily=1, weekly=1, monthly=1
        'frequencyType': 'minute' # minute, daily, weekly, monthly
    }
    params.update({'apikey': key}) # Other users will need their own TD Ameritrade API Key

    return requests.get(url, params=params).json() # price data tied to candle element

candles = get_price_history("MSFT")

This returns:

{'candles': [{'open': 339.88, 'high': 339.88, 'low': 339.88, 'close': 339.88, 'volume': 266, 'datetime': 1637150400000}, {'open': 339.87, 'high': 339.96, 'low': 339.87, 'close': 339.96, 'volume': 1402, 'datetime': 1637150460000}, {'open': 340.1, 'high': 340.1, 'low': 340.1, 'close': 340.1, 'volume': 195, 'datetime': 1637150520000}, {'open': 340.02, 'high': 340.13, 'low': 340.02, 'close': 340.13, 'volume': 397, 'datetime': 1637150580000}, {'open': 340.02, 'high': 340.02, 'low': 340.02, 'close': 340.02, 'volume': 202, 'datetime': 1637150640000}, {'open': 340.06, 'high': 340.06, 'low': 340.06, 'close': 340.06, 'volume': 140, 'datetime': 1637150700000}, {'open': 340.06, 'high': 340.06, 'low': 340.06, 'close': 340.06, 'volume': 4191, 'datetime': 1637150760000}, {'open': 339.95, 'high': 339.95, 'low': 339.95, 'close': 339.95, 'volume': 900, 'datetime': 1637150880000}, {'open': 339.8, 'high': 339.8, 'low': 339.62, 'close': 339.62, 'volume': 771, 'datetime': 1637150940000}, {'open': 339.55, 'high': 339.55, 'low': 339.54, 'close': 339.54, 'volume': 370, 'datetime': 1637151000000}, {'open': 339.56, 'high': 339.56, 'low': 339.56, 'close': 339.56, 'volume': 102, 'datetime': 1637151060000}, {'open': 339.57, 'high': 339.57, 'low': 339.55, 'close': 339.55, 'volume': 297, 'datetime': 1637151180000}, {'open': 339.55, 'high': 339.6, 'low': 339.55, 'close': 339.6, 'volume': 868, 'datetime': 1637151240000}, {'open': 339.5, 'high': 339.5, 'low': 339.21, 'close': 339.21, 'volume': 869, 'datetime': 1637151300000}, {'open': 339.12, 'high': 339.13, 'low': 339.11, 'close': 339.13, 'volume': 432, 'datetime': 1637151360000} ], 'symbol': 'MSFT', 'empty': False}

• So then to make it a dataframe I used:

#SELECT THE DICT KEY THAT CONTAINS A LIST AND ASSIGN IT A VARIABLE
#data_dict_list = result['candles']
# CREATE FUNCTION TO SEPERATE THE DICTIONARIES FROM THE LIST

def _seperate_key_value(arg1=dict(),arg2='dict_key',arg3='symbol'):
    data_dict_list = arg1[arg2]
    company = arg1[arg3]
    """
    ==========================================================================
    :param:--> arg1(variable = to a dictionary):--> ['arg2'] is the ('key') in
    arg1's dictionary that contains the values we need returned:-->['arg3']
    returns the symbol for the company whos data it belongs to. MADE FOR:
    dictionary created by using the td ameritrade price history API
    ==========================================================================
    :return: List of dictionaries. used for the TD Ameritrade price history
    API. Each dictionary located in the list will represent one
    (candle) on a "chart".
    ==========================================================================
    :purpose: get price data ready to be sorted in values_list
    ==========================================================================
    """

    candle = list((company, data_dict_list))
    return candle

def _new_values_list(arg1=list()):
    keys = arg1[1]
    sym = arg1[0]
    """
    ==========================================================================
    :param: values returned by the seperate_key_value() in the form of a
    variable. Probably something like candle_data, candleD, etc.
    ==========================================================================
    :returns: a sorted pandas DataFrame with columns for the open, high, low
    close, volume and datetime as index. It also, includes a row with the
    epoch time for easier time series calculations.
    ==========================================================================
    """

    o = []
    for vals in keys: o.append(vals['open'])

    h = []
    for vals in keys: h.append(vals['high'])

    l = []
    for vals in keys: l.append(vals['low'])

    c = []
    for vals in keys: c.append(vals['close'])

    etime = []
    for vals in keys: etime.append(vals['datetime'])

    time = []
    for vals in keys:
        vals = pd.to_datetime(vals['datetime'], unit='ms')
        time.append(vals)

    vol = []
    for vals in keys: vol.append(vals['volume'])

    df = pd.DataFrame({'unix_time':etime, 'datetime': time, 'open': o, 'high': h, 'low': l, 'close': c,'volume': vol})

    return df

rdata = _seperate_key_value(candles,'candles','symbol')
data = _values_list(rdata)
new_data = _new_values_list(rdata)

• So I thought combining the functions into one would make it better and easier to just call one function but no matter what I try I can't get it to work when I pass more than one symbol in the form of a list. So if anyone has any ideas of how to make this work, or can spot what I am doing wrong I would appreciate the help.

Answer 1

It seems the price history api for tdameritrade can only accept 1 symbol at a time.

Answer 2

import time
API_KEY = 'APIKEY given from TD'
stocks = 'AMZN'
Start_date_time = 1661002954000
End_date_time = 1661002964000


def get_one_minute_ph(stocks):
    """
    :param stocks: Chunked list of stocks to be passed to the preset
    price history params that this function will retrieve. 
    
    :returns: Price history data for each company in the chuncked list. 
    """
    #url = BASE + "marketdata/pricehistory"
    url =  "https://api.tdameritrade.com/v1/marketdata/{}/pricehistory".format(stocks)
    params = {
        'apikey': API_KEY, # possible params:
        'symbol': stocks, # DON'T CHANGE, change stocks exchange param in cleaned_symbols func
        #'period': '1', # 1,2,3,4,5,10 
        #'periodType': 'day', # day, week, month, year, ytd
        'frequency': '1', # minute=1,5,10,15,30; daily=1, weekly=1, monthly=1
        'frequencyType': 'minute', # minute, daily, weekly, monthly
        'endDate': End_date_time, # End date as milliseconds since epoch. 
        'startDate': Start_date_time # minute, daily, weekly, monthly
    }
    
    # 10 day 1 minute data request
    request = requests.get(
        url=url,
        params=params
    ).json()
    
    time.sleep(1)
    #SELECT THE DICT KEY THAT CONTAINS A LIST AND ASSIGN IT A VARIABLE
    data_dict_list = result['candles']
    # CREATE FUNCTION TO SEPERATE THE DICTIONARIES FROM THE LIST

def _seperate_key_value(arg1=dict(),arg2='dict_key',arg3='symbol'):
        data_dict_list = arg1[arg2]
        company = arg1[arg3]
        """
        ==========================================================================
        :param:--> arg1(variable = to a dictionary):--> ['arg2'] is the ('key') in
        arg1's dictionary that contains the values we need returned:-->['arg3']
        returns the symbol for the company whos data it belongs to. MADE FOR:
        dictionary created by using the td ameritrade price history API
        ==========================================================================
        :return: List of dictionaries. used for the TD Ameritrade price history
        API. Each dictionary located in the list will represent one
        (candle) on a "chart".
        ==========================================================================
        :purpose: get price data ready to be sorted in values_list
        ==========================================================================
        """

        candle = list((company, data_dict_list))
        return candle
    
#     one_min_df = pd.DataFrame.from_dict(request, orient='index').reset_index(drop='True')
#candles = _seperate_key_value(request)
    
def _new_values_list(arg1=list()):
    keys = arg1[1]
    sym = arg1[0]
    """
        ==========================================================================
        :param: values returned by the seperate_key_value() in the form of a
        variable. Probably something like candle_data, candleD, etc.
        ==========================================================================
        :returns: a sorted pandas DataFrame with columns for the open, high, low
        close, volume and datetime as index. It also, includes a row with the
        epoch time for easier time series calculations.
        ==========================================================================
    """

    o = []
    for vals in keys: o.append(vals['open'])

    h = []
    for vals in keys: h.append(vals['high'])

    l = []
    for vals in keys: l.append(vals['low'])

    c = []
    for vals in keys: c.append(vals['close'])

    etime = []
    for vals in keys: etime.append(vals['datetime'])

    time = []
    for vals in keys:
        vals = pd.to_datetime(vals['datetime'], unit='ms')
        time.append(vals)

    vol = []
    for vals in keys: vol.append(vals['volume'])

    df = pd.DataFrame({'unix_time':etime, 'datetime': time, 'open': o, 'high': h, 'low': l, 'close': c,'volume': vol})

    return df

rdata = _seperate_key_value(candles,'candle','symbol')
data = _values_list(rdata)
new_data = _new_values_list(rdata)

-------------------------------------------------
I get the below error Message
-------------------------------------------------
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Input In [4], in <cell line: 104>()
    100     df = pd.DataFrame({'unix_time':etime, 'datetime': time, 'open': o, 'high': h, 'low': l, 'close': c,'volume': vol})
    102     return df
--> 104 rdata = _seperate_key_value(candles,'candle','symbol')
    105 data = _values_list(rdata)
    106 new_data = _new_values_list(rdata)
enter code here

NameError: name 'candles' is not defined