Saving shell command output in python script to text file

Question

I manage to call and use a terminal command in my python script, in which it is working.

But currently I am trying to save the 'output' result from this command into a text file but I am getting errors.

This is my initial code in which it is doing correctly:

import os
os.system("someCmds --proj ARCH --all")

The code that I used when I tried to save the outputs into a text file:

import os
import sys
sys.stdout=open("output.txt","w")
command = os.system("someCmds --proj ARCH --all")
print command
sys.stdout.close()

However I got the following error - ValueError: I/O operation on closed file

I did close the file as I found this online. So where am I wrong?

Answer 1

The Python programming part of this is more appropriate for stackoverflow.com. However, there's a Unix-oriented component as well.

Every process has three well-known file descriptors . While they have names—stdin, stdout, and stderr—they are actually known by their file numbers , which are 0, 1, and 2 respectively. To run a program and capture its stdout—its file-number-1 output—you must connect that process's file-descriptor-1 to a file or pipe.

In the command-line shells, there is syntax for this:

prog > file

runs program prog , in a process whose stdout is connected to an open file descriptor that has been moved into descriptor-slot-1. Actually making all that happen at the system-call level is complicated:

1. Open the file. This results in a open file descriptor, whose number is probably higher than 2 since zero-through-two are your own stdin, stdout, and stderr.
2. Use the fork system call or one of its variants: this makes a clone of yourself.
3. In the clone (but not in the original), move the file descriptor from wherever it is now, to file descriptor 1. Then close the original higher-numbered descriptor (and any others that you have open that you do not wish to pass on).
4. Use one of the the exec family of calls to terminate yourself (the clone) but in the process, replace everything with the program prog . This maintains all your open file descriptors, including the one pointing to the file or pipe you moved in step 3. Once the exec succeeds, you no longer exist and cannot do anything. (If the exec fails , report the failure and exit.)
5. Back in the original ( parent ) process, wait for the clone-child to exec and then finish, or to fail, or whatever happens.

Python being what it is, this multi-step sequence is all wrapped up for you, with fancy error checking, in the subprocess module. Instead of using os.system , you can use subprocess.Popen . It's designed to work with pipes—which are actually more difficult than files—so if you really want to redirect to a file , rather than simply reading the program's output through a pipe, you will want to open the file first, then invoke subprocess.Popen .

In any case, altering your Python process's sys.stdout is not helpful, as sys.stdout is a Python data structure quite independent of the underlying file descriptor system. By open ing a Python stream, you do obtain a file descriptor (as well as a Python data structure), but it is not file-descriptor-number-1. The underlying file descriptor number, whatever it is, has to be moved to the slot-1 position after the fork call, in the clone. Even if you use subprocess.Popen , the only descriptor Python will move, post-fork, is the one you pass as the stdout= argument.

(Subprocess's Popen accepts any of these:

• a Python stream: Python uses stream.fileno() to get the descriptor number, or
• an integer: Python assumes that this represents an open file descriptor that should be moved to fd 0, 1, or 2 depending on whether this is an argument to stdin= , stdout= , or stderr= , or
• the special values subprocess.PIPE or, for stderr= , subprocess.STDOUT : these tell it that Python should create a pipe, or re-use the previously-created stdout pipe for the special stderr=subprocess.STDOUT case.

The library is pretty fancy and knows how to report, with a Python traceback, a failure to exec , or various other failures that occur in the child. It does this using another auxiliary pipe with close-on-exec. EOF on this pipe means the exec succeeded; otherwise the data arriving on this extra pipe include the failure, converted to a byte stream using the pickle module.)

Answer 2

For some reason I am unable to post my question so I thought I comment here. I need a little help on output to text in python.

I trained a language model using kaldi ASR and now I am trying to link it to vosk API to make a speech to text software. The back end engine is kaldi and front end working involves python. The python script is as follows:

from vosk import Model, KaldiRecognizer, SetLogLevel
import os
import wave
import json

fname =  input ( "Enter audio file name: " ) 
wav = ".wav"
txt = ".txt"
transcript = fname + txt
audiofilename = fname + wav

#wav = input( "enter audio filename with extension: " )
wf = wave.open(audiofilename, "rb")
model = Model(".")
rec = KaldiRecognizer(model, wf.getframerate())

results = []
# recognize speech using vosk model

while True:
    data = wf.readframes(4000)

    if len(data) == 0:
        break

    if rec.AcceptWaveform(data):
        part_result = json.loads(rec.Result())
        results.append(part_result)
        print(rec.Result())

    else:
        print(rec.PartialResult())
part_result = json.loads(rec.FinalResult())
results.append(part_result)

# forming a final string from the words
text = ''
for r in results:
    text += r['text'] + ' '

print(f"Vosk thinks you said:\n {text}")         
#print(rec.FinalResult())

# convert list of JSON dictionaries to list of 'Word' objects
list_of_Words = []
for sentence in results:
    if len(sentence) == 1:
        # sometimes there are bugs in recognition 
        # and it returns an empty dictionary
        # {'text': ''}
        continue
    for obj in sentence['result']:
        w = custom_Word.Word(obj)  # create custom Word object
        list_of_Words.append(w)  # and add it to list

wf.close()  # close audiofile

# output to the screen
for word in list_of_words:
    print(word.to_string())``` 

The output is working correctly. Info displayed on Shell is fine.

I cant figure out how to use f.write or similar command to save the shell output (all of it or any particular output I need from the code) and save it to a text file.