Python: Program will not run in other directory

Question

I am running ubuntu 12.04 and running programs through the terminal. I have a file that compiles and runs without any issues when I am in the current directory. Example below,

    david@block-ubuntu:~/Documents/BudgetAutomation/BillList$ pwd
    /home/david/Documents/BudgetAutomation/BillList
    david@block-ubuntu:~/Documents/BudgetAutomation/BillList$ python3.4 bill.py
    ./otherlisted.txt
    ./monthlisted.txt
    david@block-ubuntu:~/Documents/BudgetAutomation/BillList$

Now when I go back one directory and try running the same piece of code, I get an error message, ValueError: need more than 1 value to unpack . Below is what happens when I run the sample code one folder back and then the sample code below that.

    david@block-ubuntu:~/Documents/BudgetAutomation$ python3.4 /home/david/Documents/BudgetAutomation/BillList/bill.py 
    Traceback (most recent call last):
    File "/home/david/Documents/BudgetAutomation/BillList/bill.py", line 22, in <module>
        bill_no, bill_name, trash = line.split('|', 2)
    ValueError: need more than 1 value to unpack

The code, bill.py , below. This program reads two text files from the folder that it is located in and parses the lines into variables.

#!/usr/bin/env python
import glob

# gather all txt files in directory
arr = glob.glob('./*.txt')
arrlen = int(len(arr))

# create array to store list of bill numbers and names
list_num = []
list_name = []

# for loop that parses lines into appropriate variables
for i in range(arrlen):
    with open(arr[i]) as input:
        w = 0 ## iterative variable for arrays
        for line in input:
            list_num.append(1) ## initialize arrays
            list_name.append(1)

            # split line into variables.. trash is rest of line that has no use
            bill_no, bill_name, trash = line.split('|', 2)

            # stores values in array
            list_num[w] = bill_no
            list_name[w] = bill_name

            w += 1

What is going on here? Am I not running the compile and run command in the terminal correctly? Another note to know is that I eventually call this code from another file and it will not run the for loop, I am assuming since it doesn't run unless its called from its own folder/directory?

Answer 1

Your problem starts in line 5:

arr = glob.glob('./*.txt')

You are telling glob to look in the local directory for all .txt files. Since you are one directory up you do not have these files.

You are getting a ValueError because the line variable is empty.

As it is written you will need to run it from that directory.

Edit: The way I see it you have three separate options.

1. You could simply run script with the full path (assuming it is executable)

   ~/Documents/BudgetAutomation/BillList/bill.py

2. You could put the full path into the file (although not very Pythonic)

   arr = glob.glob('/home/[username]/Documents/BudgetAutomation/BillList/*.txt')

3. You could use sys.argv to pass the path in the file. This would be my personal preferred way. Use os.path.join to put the correct slashes.

   arr = glob.glob(os.path.join(sys.argv 1 , '*.txt'))

Answer 2

As eomer explains , the problem is that './*.txt' is a relative path—relative to the current working directory. If you're not running from the directory that all those *.txt files are in, you won't find anything.

If the *.txt files are supposed to be in the same directory as the script , use the same directory as the script , not the current working directory .

The standard way of doing that is to put code like this at the top of your script:

import os
import sys

scriptdir = os.path.abspath(os.path.dirname(sys.argv[0]))

• argv[0] gets the path to the script itself. So, if you ran the script as python BillList/bill.py , this will be 'BillList/bill.py' . ^*

• dirname just takes a path to a file, and gives you the path to the directory the file was in. So, in this case, BillList .

• abspath normalizes and absolutizes the path. ^** So, you'll get /home/david/Documents/BudgetAutomation/BillList/ . And that's the directory the *.txt files are in.

Then, instead of this:

glob.glob('./*.txt')

… you do this:

glob.glob(os.path.join(scriptdir, '*.txt'))

---

_{* Actually, on some platforms you will get an absolute path here, rather than relative, meaning the later abspath is unnecessary. But for portability, it's worth doing. A bigger problem is that in some cases you will get just bill.py , with no path. There used to be cases where it was worth checking for that and trying __file__ instead, but as far as I know that isn't true on any modern platform—and there are cases where __file__ is wrong but argv[0] is right.}

_{** For a relative path, it absolutizes it relative to the current working directory. That's why it's important to do this at the top of the script—in case someone does an os.chdir later.}

Answer 3

You don't need to create that range object to iterate over the glob result. You can just do it like this:

for file_path in arr:
    with open(file_path) as text_file:
        #...code below...

The reason of why that exception is raised, I guess, is there exist text files contain content not conforming with your need. You read a line from that file, which is something may be like "foo|bar", then the splitting result of it is ["foo", "bar"].

If you want to avoid this exception, you could just catch it:

try:
    bill_no, bill_name, trash = line.split('|', 2)
except ValueError:
    # You can do something more meaningful but just "pass"
    pass

Answer 4

You must use absolute path arr = glob.glob('./*.txt') here.

Do something like arr = glob.glob('/home/abc/stack_overflow/*.txt')

If possible use below code

dir_name = "your directory name" # /home/abc/stack_overflow
[file for file in os.listdir(dir_name) if file.endswith('txt')]

This will provide you with list of files that you want to glob with