Python For-Loop Output Column Patterns

Question

I am trying to output data in a columned format.

I've tried two methods, the first, I used for testing purposes; to call identified values in the data sets (limiting run-time during testing). The second method utilizes a for-loop to pull all the data from the data sets. However, the second method output a different column pattern than I expected.

Each HTML page I am pulling data from has an unpredictable number of items that would be appended to the list, so numerating them out individually (as in method 1) won't work.

Here is an overview of the code and a display of its results:

Method 1 Code:

for num in range(1, 3):
    url = "https://test.com/testid={}".format(num)
    # [. . . ]
    set = soup.find_all('td') # call to pull data
    a = set[0]
    b = set[1]
    c = set[2]
    info = [a,b,c]
    print(info)

Method 1 Output:

Column 1: a, b, c

Column 2: a, b, c

Method 2 Code:

for num in range(1, 3):
    url = "https://test.com/testid={}".format(num)
    # [. . . ]
    set = soup.find_all('td') # call to pull data
    info = []
    for data in set:    
        info.append(data)
    print(info)

Method 2 Output:

Column 1: a, b, c, a, b, c

Does someone know why method 2 isn't producing the same output column pattern or what I might do about it? Thank you

Answer 1

Try to first get the required table , then the table rows (tr) and then the table data (td).

table = soup.find("table") # get the table
table_data = table.tbody.find_all("tr")  # get the table rows (tr)

data = []
for i in table_data[0].find_all("td"): # get the table data (td)
    data.append(i.text)

print(data)

Answer 2

I know this isn't cool but I figured it out first thing the next morning, I guess I just needed a fresh look at it. I needed to segment the data sets into list items as in method 1 and call to all items as in method 2. After some testing I found that the issue lied with the.append call (I'm not sure why its properties are such that it isn't segmenting the data in the same way as manually adding the values to a list?) So, I made a list that spans from beginning to end automatically to bypass the.append call. Like so:

for num in range(1, 3):
    url = "https://test.com/testid={}".format(num)
    # [. . . ]
    set = soup.find_all('td') # call to pull data
    info = td[0:-1] #segments list items but also calls the full list
    print(info)