如何只取列表元素的某一部分？

Question

Here's the link of the website :

website

Here's my script :

import requests
from requests import get
from bs4 import BeautifulSoup
import pandas as pd
import numpy as np
import time
from selenium import webdriver
from selenium.webdriver.support.select import Select
from selenium.webdriver.support.ui import WebDriverWait     
from selenium.webdriver.common.by import By     
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys


PATH = "driver\chromedriver.exe"

options = webdriver.ChromeOptions() 
options.add_argument("--disable-gpu")
options.add_argument("--window-size=1200,900")
options.add_argument('enable-logging')

driver = webdriver.Chrome(options=options, executable_path=PATH)

wait = WebDriverWait(driver, 20)

driver.get('https://fr.hotels.com/search.do?destination-id=10398359&q-check-in=2021-06-26&q-check-out=2021-06-27&q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER')
driver.maximize_window()
time.sleep(2)


webdriver.ActionChains(driver).send_keys(Keys.ESCAPE).perform()
time.sleep(2)

wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'button[class="uolsaJ"]'))).click()

links = []

def is_element_visible(xpath):
    wait1 = WebDriverWait(driver, 2)
    try:
        wait1.until(EC.visibility_of_element_located((By.XPATH, xpath)))
        return True
    except Exception:
        return False

while not is_element_visible("//div[@id='20']"):
    my_elems = driver.find_elements_by_xpath('//a[@class="_61P-R0"]')

    links = [my_elem.get_attribute("href") for my_elem in my_elems]


    driver.execute_script("window.scrollBy(0, 1000)")
    time.sleep(5)


print(links)

Here's the output :

['https://fr.hotels.com/ho716157152/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho397103/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho1098309152/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho449686/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho315896/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho1574324896/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho288352/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho748227104/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho225263/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho225250/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho405210/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho547798/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho252584/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho351562/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho714011808/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho424335/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho442661/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho437481/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3']

Those are the URLs of the hotels, I would like to know how to have a specific part.

I would like to have those ids present in each URLs :

'https://fr.hotels.com/ho437481/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3'

-> 437481

Kind of recreate the list but just with those numbers instead of the URLs.

Something like that :

['716157152', '397103', '1098309152' ... , '437481']

Answer 1

You can use regular expressions, but if the structure is always https://fr.hotels.com/ho[your_id]/[...] , split will suffice:

hotel_ids = [link.split('/')[3][2:] for link in links]

split turns the string into a list like ['https:', '', 'fr.hotels.com', 'ho[your_id]'] , so the id will always be in the 4th position (index = 3), and [2:] gets rid of the leading 'ho'.

Answer 2

You can just do this after you have your links

links = [s.split('/')[3][2:] for s in links]

# Output

['716157152', '397103', '1098309152', '449686', '315896', '1574324896', '288352', '748227104', '225263', '225250', '405210', '547798', '252584', '351562', '714011808', '424335', '442661', '437481']

Answer 3

I prefer the other answers, but regex is also a viable option.

import re

in_arr = ['https://fr.hotels.com/ho716157152/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho397103/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho1098309152/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho449686/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho315896/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho1574324896/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho288352/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho748227104/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho225263/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho225250/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho405210/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho547798/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho252584/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho351562/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho714011808/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho424335/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho442661/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho437481/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3']

regex = "(?<=\.com\/ho)[\w]+"
out = map(lambda x: re.findall(regex, x)[0], in_arr)

print(list(out))

Output:

['716157152', '397103', '1098309152', '449686', '315896', '1574324896', '288352', '748227104', '225263', '225250', '405210', '547798', '252584', '351562', '714011808', '424335', '442661', '437481']

Answer 4

Use a regexp against the string:

>>> s = 'https://fr.hotels.com/ho437481/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3'
>>> m = re.match('https://fr.hotels.com/ho(\d+)/',s)
>>> m.group(1)
'437481'

You can put that in a function and use map against the list of URLs, or use a for loop, or even a list comprehension.

Answer 5

#!/usr/bin/env python3

urls = [
    "https://fr.hotels.com/ho716157152/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho397103/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho1098309152/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho449686/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho315896/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho1574324896/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho288352/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho748227104/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho225263/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho225250/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho405210/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho547798/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho252584/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho351562/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho714011808/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho424335/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho442661/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3",
    "https://fr.hotels.com/ho437481/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3"
]

def get_id(url):
    return url.split('/')[3][2:]

ids = [get_id(url) for url in urls]
print(ids)

Answer 6

This can be solved with regex using:

import re
urls = ['https://fr.hotels.com/ho716157152/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3', 'https://fr.hotels.com/ho397103/?q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&ZSX=0&SYE=3']
print([re.findall(r'\d+',url)[0] for url in urls])