File output different to terminal output

Question

I'm trying to write a program that reads an online calendar (for Coles Employees) and writes the output to an ICS file for calendar applications.

I've gotten to the stage where I want to read the page source and sift through it to find the shifts that are rostered.

My only problem is when trying to output these to a file, the output that shows in my terminal when printing (which is correct) is different to the output that gets written to my output file.

# this is how i collect the page source #
from webbot import Browser

web =  Browser()
web.go_to('https://login.colesgroup.com.au/nidp/saml2/sso?sid=0&option=credential')
web.type('********') #username
web.press(web.Key.TAB)
web.type('********')#password
web.click(id = 'button')
web.click(id = 'a.actionn-item')
web.go_to("https://colesgroup.sharepoint.com/sites/mycoles/Pages/redirect.aspx?feature=myhours")
web.click('Shift Details View')

data = web.get_page_source()



with open('pagesource.txt', 'w') as file:

    file.write(data)

#in another file #
with open("pagesource.txt", 'r') as file:
    for line in file:
        if 'roster-timeblock-home' in line:
            print(line)
            output = line

with open("shifts.txt", 'w') as file:
                file.write(output) # The output that gets written in shifts.txt is different to the terminal output#

The output in shifts.txt should be the same as the terminal output, here

<div class="roster-timeblock roster-timeblock-home" data-date="2019-16-07" style="top: 66.6667%; height: 12.5%; opacity: 1;"><div class="roster-timeblock-time-wrapper"><span class="roster-timeblock-time">16:00<br />19:00</span></div></div></div>

<div class="roster-timeblock roster-timeblock-home" data-date="2019-21-07" style="top: 50%; height: 33.3333%; opacity: 1;"><div class="roster-timeblock-time-wrapper"><span class="roster-timeblock-time">12:00<br />20:00</span></div></div></div>

However, this is what I get

<div class="roster-timeblock roster-timeblock-home" data-date="2019-21-07" style="top: 50%; height: 33.3333%; opacity: 1;"><div class="roster-timeblock-time-wrapper"><span class="roster-timeblock-time">12:00<br />20:00</span></div></div></div>

Only the second output... Can I get some help? Where am I going wrong?

Answer 1

You can see below a possible solution:

with open("pagesource.txt", 'r') as file:
    results = []
    for line in file:
        if 'roster-timeblock-home' in line:
            print(line)
            results.append(line)  # Here if you use a simple variable, it will be overwritten in case of every new match.

with open("shifts.txt", 'w') as file:
                file.write("\n".join(results))