当 python ttp 模块就位时使用 pyinstaller exe 时出错

Question

I am trying convert my.py file to an exe file using pyinstaller. The.py file perfectly work fine, however, I am facing an issue after the program is converted to.exe file. The problem is shared right below. ttp.lazy_import_functions: failed to save problem with File not found indication.

I did a search in google if any similar error, it looks there is only one similar discussion in github which is not the %100 same problem. Because I am facing an issue when using.exe file. See https://github.com/dmulyalin/ttp/issues/54

However, I have checked ttp/ttp.py file, I can see following lazy_import_functions with the path_to_cache.

log.info("ttp.lazy_import_functions: starting functions lazy import")

# try to load previously pickled/cached _ttp_ dictionary
path_to_cache = os.getenv("TTPCACHEFOLDER", os.path.dirname(__file__))
cache_file = os.path.join(path_to_cache, "ttp_dict_cache.pickle")

As it is also shown above picture, it looks that.exe file trying to find ttp/ttp.py file under _MEIXXXX cache file.

I have actually created a the following patch with the following changes in my ttp.py file to make.exe file work, however I have a few concerns here if someone explain it, I appricated it.

Changes in my ttp.py:

print(path_to_python_3x)
if path_to_python_3x:
    os.startfile(f"{path_to_python_3x}\\patch.py")

def lazy_import_functions():
    """function to collect a list of all files/directories within ttp module,
    parse .py files using ast and extract information about all functions
    to cache them within _ttp_ dictionary
    """
    _ttp_ = {
        "macro": {},
        "python_major_version": version_info.major,
        "global_vars": {},
        "template_obj": {},
        "vars": {},
    }
    log.info("ttp.lazy_import_functions: starting functions lazy import")

    # try to load previously pickled/cached _ttp_ dictionary
    path_to_temp_file = tempfile.gettempdir()
    _MEI_regex = "_MEI.*"
    for temp_file in os.listdir(path_to_temp_file):
        if re.search(_MEI_regex, temp_file):
            path_to_temp_mei = path_to_temp_file +f"\\{temp_file}"
            path_to_temp_ttp = f"{path_to_temp_mei}" + "\\ttp"
            path_to_cache = os.getenv("TTPCACHEFOLDER", path_to_temp_ttp)
            cache_file = os.path.join(path_to_cache, "ttp_dict_cache.pickle")
        else:
            path_to_cache = os.getenv("TTPCACHEFOLDER", os.path.dirname(__file__))
            #print(path_to_cache)
            cache_file = os.path.join(path_to_cache, "ttp_dict_cache.pickle")

With this patch file I am copying ttp/ folder includes ttp.py into _IMEXXXX cache file, so that.exe file finds the path, and worked fine, thankfully.

import os.path
import os
import sys
import tempfile
import shutil
import re

path_to_python_3x = os.path.dirname(sys.executable)
# print(path_to_python_3x)
# print(os.getcwd())

path_to_site_packages = path_to_python_3x + "\\Lib\\site-packages"
#print(path_to_site_packages)
path_to_site_ttp = path_to_site_packages +"\\ttp"
#print(path_to_site_ttp)

_MEI_regex = "_MEI.*"
_MEI_regex_a_list = []
while True:
    path_to_temp_file = tempfile.gettempdir()
    for temp_file in os.listdir(path_to_temp_file):
        if re.search(_MEI_regex, temp_file):
            path_to_temp_mei = path_to_temp_file +f"\\{temp_file}"
            _MEI_regex_a_list.append(path_to_temp_mei)
            path_to_temp_ttp = os.path.join(path_to_temp_mei, "ttp")
            try:
                if "ttp" not in os.listdir(path_to_temp_mei):
                    shutil.copytree(path_to_site_ttp, path_to_temp_ttp)
            except Exception as e:
                print(e)

My queries here are that:

1. Why the program does not work when installing with pyinstaller?
2. Why it checks /ttp/ttp.py file under under Temp?
3. Any way to change cache directory when converting with pyinstaller?
4. As you can see, I have a workaround for now. However, I won't work if cache file started to be kept other than Temp/_IMEXXXX. Because my regex string chooses the files startswidth _IME. Not sure if any possiblity here as well.

Answer 1

From what i see, the ttp module tries to access its files and has references to the installation path for ttp which it cannot get to using the os module after its bundled by pyinstaller.

One simpler workaround than changing the module files and applying the patch file that you did, would be to just copy the installation folders of the ttp module to the bundle output folder using pyinstaller itself or do it manually. This way it would find all the ttp module files which it was not after the bundle process using pyinstaller.

I was facing the same issue and it fixed it for me.