Python and BeautifulSoup : why doen't my if condition work

Question

i'm trying to scrap a web page, using beautifulSoup, and i build a code that gets some informations from a table. here is the code i'm working on but i have a problem with the if condition :

p=soup_tab.find_all('tr')
j=0
for i in p:
 soup_tr = BeautifulSoup(str(i) ,'html.parser')
 if(soup_tr.find('span', 
{"id":"ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl"+str(j)+
 "_reference"})):
       print("enter if 1 =======================")
       cons_intitule_ref= (soup_tr.find('span',
 {"id":"ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl"+str(j)+
 "_reference"})).get_text()
       resultat.append(cons_intitule_ref)

the problem in my code is on the if condition, while executing the program there is no print of ("enter if 1 ========"). and i'm sure that the tag i'm searching for is correct, i think the exact problem is on the condition(if);

any help please, i'm stuck on this problem for hours, and still THANK YOU IN ADVANCE

Answer 1

Primary problem I see: you're starting your loop j=0 at 0 when it should start at 1 in-order to print your desired result.

If this was your html (a condensed version of the actual page), and you're trying to get the text associated with this tag:

html = '''<span class="ref" id="ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl1_reference">01/AMI/RDOE/2017</span>
<span class="ref" id="ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl2_reference">01/ct/2017</span>
<span class="ref" id="ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl3_reference">108/2017/CNSS</span>
<span class="ref" id="ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl4_reference">1/2017</span>
<span class="ref" id="ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl5_reference">09/2017/CZC</span>
<span class="ref" id="ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl6_reference">65/2017/TGR</span>
<span class="ref" id="ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl7_reference">20/2017/DMSPK</span>
<span class="ref" id="ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl8_reference">05/INDH/2017</span>
<span class="ref" id="ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl9_reference">13/CS/2017</span>
<span class="ref" id="ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl10_reference">158/2017/RRA</span>'''

You should use regex like so:

import re
soup = BeautifulSoup(html, 'lxml')
for item in soup.findAll('span', {"id": re.compile(
    "ctl0_CONTENU_PAGE_resultSearch_tableauResultSearch_ctl\d+_reference")}):
    item.get_text()

Returns:

'01/AMI/RDOE/2017'
'01/ct/2017'
'108/2017/CNSS'
'1/2017'
'09/2017/CZC'
'65/2017/TGR'
'20/2017/DMSPK'
'05/INDH/2017'
'13/CS/2017'
'158/2017/RRA'