Py2 to Py3: Add future imports

Question

. 。 Answers to this question are eligible for a +500 reputation bounty. guettli wants to to this question. 。

I need to make a old code base compatible with Python3. The code needs to support Python2.7 and Python3 for some months.

I would like to add this in very file:

# -*- coding: utf-8 -*-
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals

I tried this:

futurize --both-stages --unicode-literals --write --nobackups .

But this adds the unicode-literals future import only. Not the other future imports.

I would like to avoid to write my own script which adds this, since adding this blindly does not work, since some files already have this header.

Answer 1

If all you need is to ensure that the Source Code Encoding Heading and the 4 import statements are present, then the following script will recursively descend through a directory modifying all.py files ensuring that these the required statements are present. The updates will be done in place, so it could be wise to make sure that you have a backup in case of a hardware failure in the middle of rewriting one of the files. Even if the code is rewritten to write to a temporary file and then do a final "move" to replace the old with the new, the same problem remains.

You should try this out on experimentally first in a test directory, and as always, USE AT YOUR OWN RISK.

from pathlib import Path
import re

root_path = 'absolute_or_releative_path_to_directory'

encoding_re = re.compile(r'^# -\*- coding: utf-8 -\*-(\r|\r?\n)')
import_re = re.compile(r'\bfrom\s+__future__\s+import\s+((?:absolute_import|division|print_function|unicode_literals))\b')
all_imports = ['absolute_import', 'division', 'print_function', 'unicode_literals']
all_imports_set = set(all_imports)

for path in Path(root_path).glob('**/*.py'):
    with open(path, 'r', encoding='utf-8') as f:
        source = f.read()

    # Look for source encoding header:
    m = encoding_re.search(source)
    # Look for the 4 imports:
    found_imports = set(import_re.findall(source))
    if m and len(found_imports) == 4:
        # Found encoding line and all 4 imports,
        # so there is nothing to do:
        continue

    # Else we need to write out a replacement file:
    with open(path, 'w', encoding='utf-8') as f:
        if not m or len(found_imports) < 4:
            # Did not find encoding line or we must write out
            # at least one import. In either case we must write
            # the encoding line to ensure it is the first line:
            print('# -*- coding: utf-8 -*-', file=f)

        if not found_imports:
            # Found no imports so write out all 4:
            missing_imports = all_imports
        else:
            # Found 1 to 4 import statement; compute the missing ones:
            missing_imports = all_imports_set - found_imports
        for import_name in missing_imports:
            print(f'from __future__ import {import_name}', file=f)

        # Print remaining source:
        print(source, end='', file=f)

Answer 2

From the docs :

Only those __future__ imports deemed necessary will be added unless the --all-imports command-line option is passed to futurize , in which case they are all added.

If you want futurize to add all those imports unconditionally, you need to pass it the --all-imports flag.

Answer 3

First define a list of the imports you want to be each file.

You can use the glob.glob() method with the recursive keyword argument set to True so that it recursively searches for .py files within a given path.

For each filename returned, after reading in its content, you can use a filter to filter out the imports that already exist in the file, and only write those that don't:

import glob

lines = [
    "from __future__ import absolute_import",
    "from __future__ import division",
    "from __future__ import print_function",
    "from __future__ import unicode_literals"
]

for file in glob.glob('/project/**/*.py', recursive=True): # replace project with the path
    with open(file, 'r+') as f:
        content = f.read().splitlines() # Read into a list of lines
        f.seek(0, 0) # To the top of the file
        lines2 = [line for line in lines if line not in content] # Filter out existing imports
        f.write("\n".join(lines2 + content))