Editing temporary file with Vim in Python subprocess not working as expected on Mac OS

Question

My initial goal was to get user input via a command-line text-editor from within a Python script. More specifically, my plan was to create a temporary file and populate it with some pre-written text, open the file with a text-editor and allow the user to modify the file content, read the data from the file after the user exits the editor, then finally delete the file after it's all over.

I seem to have found a way to do this that is working for me, but along the way I tried a couple of approaches that did not work and I'd like to understand exactly why.

Consider the following Python script (a slightly modified version of the script taken from this post ):

#!/usr/bin/env python2
# -*- encoding: ascii -*-
"""callvim.py

Demonstrates calling a text-editor (e.g. Vim) from within a Python script,
including passing input to the editor and reading output from the editor.
"""

import tempfile
import os
from subprocess import call

# Get the text editor from the shell, otherwise default to Vim
EDITOR = os.environ.get('EDITOR','vim')

# Set initial input with which to populate the buffer
initial_message = "Hello world!"

# Open a temporary file to communicate through
with tempfile.NamedTemporaryFile(suffix=".tmp") as tf:

    # Write the initial content to the file I/O buffer
    tf.write(initial_message)

    # Flush the I/O buffer to make sure the data is written to the file
    tf.flush()

    # Open the file with the text editor
    call([EDITOR, tf.name])

    # Rewind the file offset to the beginning of the file
    tf.seek(0)

    # Read the file data into a variable
    edited_message = tf.read()

    # Output the data
    print(edited_message)

I've tried running this script in two different environments so far: on a macOS computer (running macOS 10.12) and on a Debian computer (running Debian 8.8). Both computers have the same (minor) version of Vim installed (Vim 7.4).

When I run this script with EDITOR=vim on my Debian 8 (Jessie) machine it works as expected. I'm prompted with Vim and a buffer containing the string "Hello world!". After editing the buffer to contain the string "Goodbye world!", saving the file, and exiting Vim, I see the string "Goodbye world!" printed to the console.

When I run the same script on my macOS 10.12 (Sierra) machine it does not seem to work. The same procedure results in "Hello world!" being displayed on-screen - as if the file is being read before it is edited.

However if run the script on my Mac with EDITOR=nano then once again everything seems to work as expected.

I tried a few variations on this script using different methods from the tempfile module (eg using tempfile.TemporaryFile() and tempfile.mkstemp() ) with the same results.

Now consider the following alternative script:

#!/usr/bin/env python2
# -*- encoding: ascii -*-
"""callvim.py

Demonstrates calling a text-editor (e.g. Vim) from within a Python script,
including passing input to the editor and reading output from the editor.
"""

import subprocess
import os

# Create a temporary file and write some default text
file_path = "tempfile"
file_handle = open(file_path, "w")
file_handle.write("Hello world!")
file_handle.close()

# Open the file with Vim
subprocess.call(["vim", file_path])

# Rewind to the beginning of the file
file_handle = open(file_path, 'r')

# Read the data from the file
data = file_handle.read()

# Close the temporary file
file_handle.close()

# Delete the temporary file
os.remove(file_path)

# Print the data
print(data)

This script, which avoids using the tempfile module, appears to be working consistently across both platforms.

So it seems that this script may be failing for some reason having to do with how Vim and the tempfile Python module interact on macOS. What's going on here?

Answer 1

This is happening because your second script closes the file handle before invoking vim , then opens a new one afterwards, whereas the first script doesn't. It has nothing to do with the tempfile module per se. This code works as expected:

import tempfile, os
from subprocess import call

initial_message = "Hello world!"

tf = tempfile.NamedTemporaryFile(suffix=".tmp", delete=False)
tf.write(initial_message)
tf.close()

call(['vim', tf.name])

tf = open(tf.name, 'r')
edited_message = tf.read()
tf.close()

os.unlink(tf.name)

print(edited_message)

Note the delete=False in the call to NamedTemporaryFile , which ensures that the file isn't deleted when we close it the first time (we have to delete it manually with os.unlink later).

I think what's going on here is that vim isn't editing your file in-place, it's writing to a swap file. When you save and quit, vim replaces the original file with the edited one, leaving your file handle pointing to the old, unedited one. There are ways to prevent vim from using swap files (see, eg here ), but I wouldn't recommend them. Just remember to update your file handles after vim has done its business.