如何操作列表中的字符串项？

Question

I am trying to get familiar with web scraping with python and cannot figure out how to manipulate the strings that are part of a list.

below is the code that I am working on to try and extract the movies showing at a local theater and I can get most of the names from the HTML code. what I want to do is iterate through the list and take the first two characters off of the strings in the list because with regex I cannot just extract the name in my attempts.

it throws errors because it sees it as a list object that I am trying to manipulate but its the strings in the list that I want to change.

from urllib.request import urlopen
import re
url = "http://woodburytheatre.com/showtimes"
page = urlopen(url)
page
html_bytes = page.read()
html = html_bytes.decode("utf-8")
#print(html)
span_index = html.find("<div id=\"showtimes_wrapper\">")
start_index = span_index + len("<div id=\"showtimes_wrapper\">")
end_index = html.find("<div id=\"t_comingsoon\">")
#print(span_index)
movie_info = html[start_index:end_index]
movie_list = list()
movie_list2 = list()
movie_list3 = list()
#print(movie_info)
for item in movie_info.split("\n"):
   if "showtimes_movie" in item:
       movie_list.append(item.strip())
#print(movie_list)
for item in movie_list:
   movie_list2.append(re.findall("[0-9]\/[A-z0-9\-]+",str(movie_list)))
#print(movie_list2)
while movie_list2:
   temp = movie_list2.pop()
   print(type(temp))
   print("temp" + str(temp))
   temp2 = temp.lstrip("/")
   print(temp2)
   movie_list3.append(temp2)
print(movie_list3)
print(len(movie_list2))
print(movie_list3)

I know it is very messy and it can be much more efficient but I just want to be able to alter the strings in the list so I can get rid of the number and "/" right before them.

Thanks in advance!

Answer 1

At second for loop:

for item in movie_list:
   movie_list2.append(re.findall("[0-9]\/[A-z0-9\-]+",str(movie_list)))

from python document about re.findall :

def findall(pattern: Pattern[AnyStr],
            string: AnyStr,
            flags: Union[int, RegexFlag] = ...) -> list

re.findall returns list of all matching objects. Therefore, you're appending list to list , and list doesn't have method lstrip , causing error in while loop.

while movie_list2:
   temp = movie_list2.pop()
   print(type(temp))
   print("temp" + str(temp))
   temp2 = temp.lstrip("/")  # << here
   print(temp2)
   movie_list3.append(temp2)

---

Ultimately what you've trying to archive can be done with one-line:

from urllib.request import urlopen

page = urlopen("http://woodburytheatre.com/showtimes")
html_bytes = page.read()
html = html_bytes.decode("utf-8")

span_index = html.find("<div id=\"showtimes_wrapper\">")
start_index = span_index + len("<div id=\"showtimes_wrapper\">")
end_index = html.find("<div id=\"t_comingsoon\">")

movie_info = html[start_index:end_index]

# this line below
movie_list = [i.split('/')[-1].rstrip(r"'>") for i in movie_info.split("\n") if 'showtimes_movie' in i]
print(movie_list)

outputs:

['Bill-and-Ted-Face-The-Music', 'The-New-Mutants', 'The-Personal-History-of-David-Copperfield', 'Inception-10th-Anniversary', 'Unhinged', 'Made-in-Italy', 'The-Rental', 'Trolls-World-Tour', 'Indiana-Jones-and-the-Temple-of-Doom', 'Indiana-Jones-and-the-Raiders-of-the-Lost-Ark']

With this you can simply replace hyphen with use of .replace function.

---

To dive in details how that one line generator-expression - aka genexp - works, lets see this example:

>>> output = [i for i in range(4) if i != 3]

You can put if condition check in genexp to filter out items when creating list. This does exact same thing with following for loop but does job quicker:

>>> output = []
>>> for i in range(4):
...     if i != 3:
...         output.append(i)

And you can use expressions again to item you're adding to list.
Instead of saving i of for loop, you can even just add absolutely unrelated things:

>>> [[i for i in 'Nope.'] for i in range(2)]
[['N', 'o', 'p', 'e', '.'], ['N', 'o', 'p', 'e', '.']]

>>> ['Nope.' for i in range(4)]
['Nope.', 'Nope.', 'Nope.', 'Nope.']

---

With this combined, what this means is:

movie_list = [i.split('/')[-1].rstrip(r"'>") for i in movie_info.split("\n") if 'showtimes_movie' in i]

movie_list = []
for i in movie_info.split("\n"):
    if 'showtimes_movie' in i:
        split_strings_list = i.split('/')
        last_item_in_list = split_strings_list[-1]  # python can do backward-indexing
        strip_text_from_right = last_item_in_list.rstrip("'>")  # remove '> from behind of string
        
        movie_list.append(strip_text_from_right)

Answer 2

By using enumerate you can get both the index and item of a list:

for index, item in enumerate(lst):
    # Strip the first two letters
    list[index] = lst[index][2:]

Do note however that using regex on HTML is a bad idea. I highly suggest using HTML parsing libraries, the famous of them being beautifulsoup4 .