open files from another directory from python in linux

Question

I have a file in Linux called testing.txt .

Below is my project structure in Linux

main_project
├── base_dir
│   └── helper
│       └── file_helper.py
│       └── __init__.py
    └── jobs
        └── adhoc_job.py
        └── __init__.py

        └── test_job.py
        └── __init__.py     

├── share_dir
│   └── testing.txt     

├── scripts
│   └── python_wrapper.sh       

Run mechanism

1) Using python_wrapper.sh I will run the `jobs/test_job.py` script. 
2) In jobs/test_job.py I have from helper.file_helper import *
3) In helper/file_helper.py I will read the testing.txt file

helper/file_helper.py contents

print(os.getcwd())

with open("main_project/share_dir/testing.txt") as f:
    data = {}
    for line in f:
        key,value = line.strip().split('#')
        data[key] = value
        

When I call the python_wrapper.sh like below I do not have any issues

sh /home/$USER/main_project/scripts/python_wrapper.sh test_job

When I call the python_wrapper.sh like below

# change directory to main_project(this is not in python_wrapper.sh)
cd main_project

# run the scripts
sh scripts/python_wrapper.sh test_job

Then I am getting NO file or directory main_project/share_dir/testing.txt error

I want to read the testing.txt file without any issues when calling the python_wrapper.sh from any directory

How can I resolve this issue

Answer 1

This starts from the file's own location (inside helper) and constructs the actual location of main_project :

mainproj = os.path.realpath( os.path.dirname(__file__)+"/../.." )
with open( mainproj + "/share_dir/testing.text") as f: