To access data in web scraping using css select in python using scrapy

Question

I was trying to scrape BHK details from a website using Scrapy with css select in python. Could anyone please let me know on how to access the texts- 3 BHK Residential Apartment,3,4 BHK Independent House/Villa- from the below output.

output:

{'bhk': ['<tr class="NpsrpTuple__subHead"><td>3 BHK Residential Apartment<!-- '
         '--> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Independent '
         'House/Villa<!-- --> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td></td><td class="undefined"><span>Ready To '
         'Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td></td><td class="undefined"><span>Ready To '
         'Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4,5 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->8518<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->4226<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>1,2 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->4700<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->4199<!-- --> / Sq.Ft. </span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->7500<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Independent '
         'House/Villa<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ '
         '<!-- -->4947<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Under Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>4 BHK Residential Apartment<!-- '
         '--> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3,4 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->7229<!-- --> / Sq.Ft. </span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3,4 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->6572<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td></td><td class="undefined"><span>New '
         'Launch</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->6453<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Under Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2 BHK Residential Apartment<!-- '
         '--> </td><td></td><td class="undefined"><span>Ready To '
         'Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Independent '
         'House/Villa<!-- --> </td><td></td><td class="undefined"><span>New '
         'Launch</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>4 BHK Independent '
         'House/Villa<!-- --> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->6618<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Under Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->5900<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Under Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->5233<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr']

I tried to run the code:

bhk = response.css('.NpsrpTuple__subHead').extract()

Answer 1

Not sure if this is something you want to achieve:

from scrapy import Selector

item = {'bhk': ['<tr class="NpsrpTuple__subHead"><td>3 BHK Residential Apartment<!-- '
         '--> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Independent '
         'House/Villa<!-- --> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td></td><td class="undefined"><span>Ready To '
         'Move</span></td></tr>']
    }


items = ""
for i in item['bhk']:
    items+=i

response = Selector(text=items)
for elem in response.css('tr > td::text').getall():
    print(elem)

Output:

3 BHK Residential Apartment
3,4 BHK Independent House/Villa
2,3 BHK Residential Apartment