Search a list based on key word to append specific list contents

Question

Context

I have a list of links I scraped from this site: https://www.ons.gov.uk/economy/economicoutputandproductivity/output/datasets/economicactivityfasterindicatorsuk

This list of links look like this;

['https://twitter.com/ONS',
 'https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2fdecember2019/dataset1.xlsx',
 'https://www.facebook.com/ONS',
 'https://www.ons.gov.uk/peoplepopulationandcommunity/leisureandtourism',
 'https://www.ons.gov.uk/businessindustryandtrade/manufacturingandproductionindustry',
 'https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2ffebruary2020roadsdata/roadstables.xlsx',
 'https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2fjuly2019/economicactivityfasterindicatorsukjuly2019dataset.xlsx',
 'https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2fjanuary2020roadsdata/roadstables.xlsx'...

I want to now use Helium/Selenium to go to them and print them out. Only the list of links has a combination of links I don't need and the excel docs I need to download. I want to be able to append just the links that contain xlsx.

I tried this solution but it did not work. I also tried the .remove function but this is more time consuming. I also tried to collate a list of the links by slicing them but again this is time consuming.

Problem

Is there any easier way to find a string in the list of links to them allow me to append to a list and loop through them via selenium (I can do the latter, just need help with the append).

Answer 1

Use a list comprhension.

linklist = ['https://twitter.com/ONS',
 'https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2fdecember2019/dataset1.xlsx',
 'https://www.facebook.com/ONS',
 'https://www.ons.gov.uk/peoplepopulationandcommunity/leisureandtourism',
 'https://www.ons.gov.uk/businessindustryandtrade/manufacturingandproductionindustry',
 'https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2ffebruary2020roadsdata/roadstables.xlsx',
 'https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2fjuly2019/economicactivityfasterindicatorsukjuly2019dataset.xlsx',
 'https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2fjanuary2020roadsdata/roadstables.xlsx']

relevant_links = [link for link in linklist if ".xlsx" in link]

Will output

['https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2fdecember2019/dataset1.xlsx', 'https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2ffebruary2020roadsdata/roadstables.xlsx', 'https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2fjuly2019/economicactivityfasterindicatorsukjuly2019dataset.xlsx', 'https://www.ons.gov.uk/file?uri=%2feconomy%2feconomicoutputandproductivity%2foutput%2fdatasets%2feconomicactivityfasterindicatorsuk%2fjanuary2020roadsdata/roadstables.xlsx']

Answer 2

Check the string termination:

new_list = [link for link in original_list if link.endswith(".xlsx")]

Then you can open each link in the new_list .