add text in a file with python (without replacing it)

Question

I have a file with ids and information, like this:

1oMZgkoaz3o 2011-12-29T01:23:00.000Z 9 503 ApolloIsMyCoPilot
nUW1TomCSQg 2011-12-29T01:23:15.000Z 9 348 grea7stuff
tJuLnRrAcs0 2011-12-29T01:26:20.000Z 9 123 AdelGaming
tyi5g0mnPIs 2011-12-29T01:28:07.000Z 9 703 PreferredGaming

and I want to add a flag on some of the line, so if I have a dictionary

flags = {'1oMZgkoaz3o': flag1, 'tJuLnRrAcs0': flag2}

the result I want is

1oMZgkoaz3o 2011-12-29T01:23:00.000Z 9 503 ApolloIsMyCoPilot flag1
nUW1TomCSQg 2011-12-29T01:23:15.000Z 9 348 grea7stuff
tJuLnRrAcs0 2011-12-29T01:26:20.000Z 9 123 AdelGaming flag2
tyi5g0mnPIs 2011-12-29T01:28:07.000Z 9 703 PreferredGaming

So I made this code

l = True
while l is True:
    a = f.readline()
    try a.split(' ')[0] in flags.iterkeys():
        f.seek(-1,1)
        f.write(' '+str(flags[a.split(' ')[0]])+'\n')
        del flags[a.split(' ')[0]]
    except IndexError:
        l = False

so, my Python code could be poor, but the problem is that with this code I'm replacing text, so the file is all messed up. How can I write without replacing? and if you have better ideas for the code, you are welcome...

Answer 1

You can't write to the file and "insert". Best approach would be to read your file and write out the contents with modifications to a new file and then rename as necessary.

Answer 2

I see two problems here:

Reading and writing from/to the same file

This doesn't work too well. It would be better to read from one file and write to another one (this way, you also won't lose data if something goes wrong in your program). Example:

input_file = open('infile.txt', 'r')
output_file = open('outfile.txt', 'w')
for line in input_file:
    line += "transformed"
    output_file.write(line)

Syntactic/semantic errors

You have a syntactic error in your code snippet, the line

try a.split(' ')[0] in flags.iterkeys():

is not valid (and Python should complain about that!).

Some other things to note:

• in flags.iterkeys() is semantically equivalent to in flags
• Also, you can just use while l instead of while l is True . Even better, you could drop the flag variable l completely and jump out of the loop with break if an error occurs.

My attempt

input_file = open('infile.txt', 'r')
output_file = open('outfile.txt', 'w')
flags = { ... }

for line in input_file:
    parts = line.strip().split()
    if parts[0] in flags:
        line = line + ' ' + flags[parts[0]]
    output_file.write(line + "\n")

If you know how to use a shell, you could make your life easier if you just use STDIN/STDOUT for data in- and output. You save yourself the file handling then and leave the user more flexibility in how he can use your script.

Answer 3

I'd like use stdin/stdout redirection:

#!/usr/bin/env python3
import sys

flags = {'1oMZgkoaz3o': 'flag1', 'tJuLnRrAcs0': 'flag2'}

for line in sys.stdin:
    line = line.rstrip()
    k = line.split()[0]
    if k in flags:
        print(line, flags[k])
    else:
        print(line)

---

$ python3 script.py <input.txt >output.txt
$ cat output.txt
1oMZgkoaz3o 2011-12-29T01:23:00.000Z 9 503 ApolloIsMyCoPilot flag1
nUW1TomCSQg 2011-12-29T01:23:15.000Z 9 348 grea7stuff
tJuLnRrAcs0 2011-12-29T01:26:20.000Z 9 123 AdelGaming flag2
tyi5g0mnPIs 2011-12-29T01:28:07.000Z 9 703 PreferredGaming

Answer 4

Modifying a file in-place is possible using the fileinput module:

from fileinput import FileInput

f = FileInput(the_filename, inplace=True)
for line in f:
    line = modify_line() # do whatever modifications you need to do
    print line # this writes the line to the file
f.close()

Answer 5

Firstly, let's clean up the script a bit:

for line in f.readlines():
    line = line.strip()
    parts = line.split()
    if parts[0] in flags:
        f.write(line + flags[parts[0]] + "\n");
    else:
        f.write(line + "\n");

It's been a while since I last mucked around with reading and writing in the same file, so I might be a bit off.

Answer 6

The root of your problem is that you're trying to read and write to the same file object. You need to create a new file instead.

However, there are a few other things that you should consider cleaning up first...

First off, you can simplify things by just iterating over the file object directly instead of using a while loop. Eg:

flags = {'1oMZgkoaz3o': 'flag1', 'tJuLnRrAcs0': 'flag2'}

# The "with" statement automatically closes the file when we're done with it
with open('test.txt', 'r') as infile:
    # If we just iterate over the open file, we're iterating over the lines in it
    for line in infile:
        line = line.strip().split()
        key = line[0]
        # I'm using "flags.get" with a default arugment here. If "key" isn't in
        # "flags", then an empty string will be returned.
        line.append(flags.get(key, ''))
        print ' '.join(line)

In this example, we're just printing the output we want. If the file is small, then we could easily do something like this

flags = {'1oMZgkoaz3o': 'flag1', 'tJuLnRrAcs0': 'flag2'}

with open('test.txt', 'r') as infile:
    # Load the entire contents of the file into memory...
    lines = infile.readlines()
with open('test.txt', 'w') as outfile:
    for line in lines:
        print line
        line = line.strip().split()
        line.append(flags.get(line[0], ''))
        outfile.write(' '.join(line) + '\n')

However, if it's a large file, we may not want to read an entire copy into memory.

In that case, we'd want to do iterate over the original file and write to a different file. Then we'd need to rename the new file to the original file's name.

If we want to be very careful, we'd do something like the following:

import os 

flags = {'1oMZgkoaz3o': 'flag1', 'tJuLnRrAcs0': 'flag2'}

infile = open('test.txt', 'r')
outfile = open('test2.txt', 'w')

try:
    # Try to do this...
    for line in infile:
        line = line.strip().split()
        line.append(flags.get(line[0], ''))
        outfile.write(' '.join(line) + '\n')
finally:
    # Do this no matter what...
    infile.close()
    outfile.close()

# If nothing goes wrong, do this...
os.remove('test.txt')
os.rename('test2.txt', 'test.txt')

The try:... finally:... part is essentially manually doing what a with statement does for a file object. It's arguably a bit cleaner than nesting two with statements in this particular case, but I'm mostly using it to show the alternate (older) syntax for doing this. Ideally, you'd probably write that piece of code similar to this:

import os 

def main():
    flags = {'1oMZgkoaz3o': 'flag1', 'tJuLnRrAcs0': 'flag2'}
    with open('test.txt', 'r') as infile:
        with open('test2.txt', 'w') as outfile:
            append_flags(infile, outfile, flags)
    os.remove('test.txt')
    os.rename('test2.txt', 'test.txt')

def append_flags(infile, outfile, flags):
    for line in infile:
        line = line.strip().split()
        line.append(flags.get(line[0], ''))
        outfile.write(' '.join(line) + '\n')

main()

However, we're clearly getting more complex the further we go with this.

In your case, the second example (reading the entire file into memory and then writing over the original file) is probably what you want.