Python overwrite to file in wrong order

Question

When I run my program with ./program.py >temp.out

I get all the unit test output first then the prints I am putting in in the python. Is there anyway I can get this to appear in this file the way it does on screen?

Test Results Suite "curl"
                               Name:     Checks  Failures   Time (s)
                        couch check:          -         -   Disabled
                   couch check fail:          -         -   Disabled
                database check fail:          -         -   Disabled
                    create database:          -         -   Disabled
                     database check:          -         -   Disabled
                  upload design doc:          -         -   Disabled
                     remove_witness:          -         -   Disabled
====================================================================
                              Total:          0         0

Passed
+=================================================================+
| Running: hba_test                                               |
| Skipping:abort/"Basic Sanity" delayedabort/"Abort Control List" |
+=================================================================+
+====================+
| Skipping: sdt_test |
+====================+
+======================+
| Skipping: dtd_tester |
+======================+
+===============+
| Running: pssm |
+===============+
+==============+
| Running: psm |
+==============+

This is the code which executes each unit test and prints seperate headers around them

#calculate lengths to make sure header is correct length
        l1 = len(x)
        l2 = len(y)
        #if entire test suite is to be disabled
        if disable:
            headerBreak ="+" + "="*(l1+12) + "+"
            print headerBreak
            print "| Skipping: %s |" % x
        #if the test suite will be executed
        else:
            headerBreak =  "+" + "="*(max(l1,l2)+11) + "+"
            print headerBreak
            print "| Running: %s" % x, ' '*(l2-l1)+ '|'
            #if some suites are disabled but some are still running
            if 'disable=' in test:
               print "| Skipping:%s |" % y 
        print headerBreak
        #bitshift right to obtain correct return value, execution of each test.
        returnValue = os.system(path) >> 8
        #running total of failures in the program.
        failures += returnValue

Last bit of code to run the method

#execute tests failures = execTests(path, testList)
#exit program with returncode as number of failures sys.exit(failures)

It should appear like this:

+==============+
| Running: ssm |
+==============+

Test Results Suite "Secondary Set Manager Tests"
                               Name:     Checks  Failures   Time (s)
              SSM_1 validate checks:         30         0      0.002
                 SSM_2 group create:          6         0      0.001
           SSM_3 rcvd invalid group:          3         0      0.001
            SSM_4 rcvd invalid data:          9         0      0.001
               SSM_5 aborted subset:          7         0      0.000
          SSM_6 pri node down abort:         14         0      0.000
        SSM_7 excess ios in subsets:          6         0      0.000
              SSM_8 all ss received:         11         0      0.000
                  SSM_9 applying ss:         12         0      0.000
               SSM_10 applying ss 2:         18         0      0.000
            SSM_11 subsets complete:         32         0      0.001
     SSM_12 subsets complete errors:         19         0      0.000
           SSM_13 apply waiting set:         40         0      0.000
                 SSM_14 extend test:         14         0      0.000
               SSM_15 group destroy:          6         0      0.000
                 SSM_16 null params:          2         0      0.001
                  SSM_17 stop group:         26         0      0.001
                SSM_18 dupe receive:          6         0      0.000
           SSM_19 apply waiting set:         36         0      0.001
====================================================================
                              Total:        297         0

Test Results Suite "Secondary Subset Manager Tests"
                               Name:     Checks  Failures   Time (s)
             SSSM_1 Validate Checks:         14         0      0.001
                SSSM_2 Steady State:         92        42      0.001
              SSSM_3 Multi Sequence:        417       227      0.003
                  SSSM_4 Test Abort:         69         6      0.001
      SSSM_5 Test Inconsistent Mreq:         10         0      0.001
                 SSSM_6 Test extend:         37         1      0.001
          SSSM_7 Test unexpected IO:         11         0      0.001
            SSSM_8 test null params:          5         0      0.000
                  SSSM_9 exceptions:          0         1      0.001
            SSSM_10 done_incomplete:         20         0      0.001
                  SSSM_11 failed io:         92        42      0.001
====================================================================
                              Total:        767       319
Failed Cases [ssm]:
   Secondary Subset Manager Tests/"SSSM_2 Steady State"
   Secondary Subset Manager Tests/"SSSM_3 Multi Sequence"
   Secondary Subset Manager Tests/"SSSM_4 Test Abort"
   Secondary Subset Manager Tests/"SSSM_6 Test extend"
   Secondary Subset Manager Tests/"SSSM_9 exceptions"
   Secondary Subset Manager Tests/"SSSM_11 failed io"


Overall Failures: 319

Answer 1

When you redirect output from the terminal to a file, the buffering mode changes from line buffered to using a fixed-size buffer instead. This means that newlines no longer trigger a flush.

Your print outputs are therefore being buffered, but the tests you are running with os.system() have their buffers flushed when the process completes.

The solution is to explicitly flush after your print statements, just before you run os.system() :

import sys

# ....
print headerBreak
sys.stdout.flush()
returnValue = os.system(path) >> 8