python selenium: <input> element not interactable

Question

I was trying to crawl down nba player info from https://nba.com/players and click the button "Show Historic" on the webpage nba_webpage_picture

part of the html code for the input button shows below:

<div aria-label="Show Historic Toggle" class="Toggle_switch__2e_90">
    <input type="checkbox" class="Toggle_input__gIiFd" name="showHistoric">
    <span class="Toggle_slider__hCMQQ Toggle_sliderActive__15Jrf Toggle_slidercerulean__1UnnV">
    </span>
</div>

I simply use find_element_by_xpath to locate the input button and click

button_show_historic = driver.find_element_by_xpath("//input[@name='showHistoric']")
button_show_historic.click()

However it says:

Exception has occurred: ElementNotInteractableException
Message: element not interactable
  (Session info: chrome=88.0.4324.192)

Could anyone help on solving this issue? Is this because the input is not visible?

Answer 1

Most probably problem is you don't have any wait code. You should wait until page is loaded. You can use simple python sleep function:

import time 
time.sleep(3) #it will wait 3 seconds
##Do your action

Or You can use explicit wait. Check this page: selenium.dev

Answer 2

Simply wait for the span element not the input element and click.

wait = WebDriverWait(driver, 30)
driver.get('https://www.nba.com/players')
wait.until(EC.element_to_be_clickable((By.XPATH,"//button[.='I Accept']"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH,"//input[@name='showHistoric']/preceding::span[1]"))).click()

Import

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC

Also to find an api just look under Developer tools ->Network->Headers and Response to find if it gets populated.

Answer 3

No need to use selenium when there's an api. Try this:

import requests
import pandas as pd


url = 'https://stats.nba.com/stats/playerindex'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36',
'Referer': 'http://stats.nba.com'}

payload = {
'College': '',
'Country': '',
'DraftPick': '',
'DraftRound': '',
'DraftYear': '',
'Height': '' ,
'Historical': '1',
'LeagueID': '00',
'Season': '2020-21',
'SeasonType': 'Regular Season',
'TeamID': '0',
'Weight': ''}
    
jsonData = requests.get(url, headers=headers, params=payload).json()
cols = jsonData['resultSets'][0]['headers']
data = jsonData['resultSets'][0]['rowSet']
df = pd.DataFrame(data, columns=cols)

Output: [4589 rows x 26 columns]

print(df.head(20).to_string())
    PERSON_ID PLAYER_LAST_NAME PLAYER_FIRST_NAME          PLAYER_SLUG       TEAM_ID  TEAM_SLUG  IS_DEFUNCT      TEAM_CITY      TEAM_NAME TEAM_ABBREVIATION JERSEY_NUMBER POSITION HEIGHT WEIGHT               COLLEGE  COUNTRY  DRAFT_YEAR  DRAFT_ROUND  DRAFT_NUMBER  ROSTER_STATUS   PTS   REB  AST STATS_TIMEFRAME FROM_YEAR TO_YEAR
0       76001        Abdelnaby              Alaa       alaa-abdelnaby  1.610613e+09    blazers           0       Portland  Trail Blazers               POR            30        F   6-10    240                  Duke      USA      1990.0          1.0          25.0            NaN   5.7   3.3  0.3          Career      1990    1994
1       76002       Abdul-Aziz              Zaid      zaid-abdul-aziz  1.610613e+09    rockets           0        Houston        Rockets               HOU            54        C    6-9    235            Iowa State      USA      1968.0          1.0           5.0            NaN   9.0   8.0  1.2          Career      1968    1977
2       76003     Abdul-Jabbar            Kareem  kareem-abdul-jabbar  1.610613e+09     lakers           0    Los Angeles         Lakers               LAL            33        C    7-2    225                  UCLA      USA      1969.0          1.0           1.0            NaN  24.6  11.2  3.6          Career      1969    1988
3          51       Abdul-Rauf           Mahmoud   mahmoud-abdul-rauf  1.610613e+09    nuggets           0         Denver        Nuggets               DEN             1        G    6-1    162       Louisiana State      USA      1990.0          1.0           3.0            NaN  14.6   1.9  3.5          Career      1990    2000
4        1505      Abdul-Wahad             Tariq    tariq-abdul-wahad  1.610613e+09      kings           0     Sacramento          Kings               SAC             9      F-G    6-6    235        San Jose State   France      1997.0          1.0          11.0            NaN   7.8   3.3  1.1          Career      1997    2003
5         949      Abdur-Rahim           Shareef  shareef-abdur-rahim  1.610613e+09  grizzlies           0        Memphis      Grizzlies               MEM             3        F    6-9    245            California      USA      1996.0          1.0           3.0            NaN  18.1   7.5  2.5          Career      1996    2007
6       76005        Abernethy               Tom        tom-abernethy  1.610613e+09   warriors           0   Golden State       Warriors               GSW             5        F    6-7    220               Indiana      USA      1976.0          3.0          43.0            NaN   5.6   3.2  1.2          Career      1976    1980
7       76006             Able            Forest          forest-able  1.610613e+09     sixers           0   Philadelphia          76ers               PHI             6        G    6-3    180      Western Kentucky      USA      1956.0          NaN           NaN            NaN   0.0   1.0  1.0          Career      1956    1956
8       76007        Abramovic              John       john-abramovic  1.610610e+09       None           1     Pittsburgh        Ironmen               PIT          None        F    6-3    195                 Salem      USA         NaN          NaN           NaN            NaN   9.5   NaN  0.7          Career      1946    1947
9      203518          Abrines              Alex         alex-abrines  1.610613e+09    thunder           0  Oklahoma City        Thunder               OKC             8        G    6-6    190          FC Barcelona    Spain      2013.0          2.0          32.0            NaN   5.3   1.4  0.5          Career      2016    2018
10    1630173          Achiuwa          Precious     precious-achiuwa  1.610613e+09       heat           0          Miami           Heat               MIA             5        F    6-8    225               Memphis  Nigeria      2020.0          1.0          20.0            1.0   5.9   3.9  0.6          Season      2020    2020
11     101165            Acker              Alex           alex-acker  1.610613e+09   clippers           0             LA       Clippers               LAC             3        G    6-5    185            Pepperdine      USA      2005.0          2.0          60.0            NaN   2.7   1.0  0.5          Career      2005    2008
12      76008         Ackerman            Donald      donald-ackerman  1.610613e+09     knicks           0       New York         Knicks               NYK                      G    6-0    183  Long Island-Brooklyn      USA      1953.0          2.0           NaN            NaN   1.5   0.5  0.8          Career      1953    1953
13      76009            Acres              Mark           mark-acres  1.610613e+09      magic           0        Orlando          Magic               ORL            42        C   6-11    220          Oral Roberts      USA      1985.0          2.0          40.0            NaN   3.6   4.1  0.5          Career      1987    1992
14      76010            Acton           Charles        charles-acton  1.610613e+09    rockets           0        Houston        Rockets               HOU            24        F    6-6    210             Hillsdale      USA         NaN          NaN           NaN            NaN   3.3   2.0  0.5          Career      1967    1967
15     203112              Acy            Quincy           quincy-acy  1.610613e+09      kings           0     Sacramento          Kings               SAC            13        F    6-7    240                Baylor      USA      2012.0          2.0          37.0            NaN   4.9   3.5  0.6          Career      2012    2018
16      76011            Adams             Alvan          alvan-adams  1.610613e+09       suns           0        Phoenix           Suns               PHX            33        C    6-9    210              Oklahoma      USA      1975.0          1.0           4.0            NaN  14.1   7.0  4.1          Career      1975    1987
17      76012            Adams               Don            don-adams  1.610613e+09    pistons           0        Detroit        Pistons               DET            10        F    6-7    210          Northwestern      USA      1970.0          8.0         120.0            NaN   8.7   5.6  1.8          Career      1970    1976
18     200801            Adams            Hassan         hassan-adams  1.610613e+09       nets           0       Brooklyn           Nets               BKN             8        F    6-4    220               Arizona      USA      2006.0          2.0          54.0            NaN   2.5   1.2  0.2          Career      2006    2008
19    1629121            Adams            Jaylen         jaylen-adams  1.610613e+09      bucks           0      Milwaukee          Bucks               MIL             6        G    6-0    225       St. Bonaventure      USA         NaN          NaN           NaN            1.0   0.3   0.4  0.3          Season      2018    2020