How to loop through the list of .tar.gz files using linux command in python

Question

Using python 2.7

I have a list of *.tat.gz files on a linux box. Using python, I want to loop through the files and extract those files in a different location, under their respective folders.

For example: if my file name is ~/TargetData/zip/1440198002317590001.tar.gz
then I want to untar and ungzip this file in a different location under its respective folder name ie ~/TargetData/unzip/1440198002317590001.

I have written some code but I am not able to loop through the files. In a command line I am able to untar using $ tar -czf 1440198002317590001.tar.gz 1440198002317590001 command. But I want to be able to loop through the .tar.gz files. The code is mentioned below. Here, I'm not able to loop just the files Or print only the files. Can you please help?

    import os
    inF = []
    inF = str(os.system('ls ~/TargetData/zip/*.tar.gz'))
    #print(inF)
    if inF is not None:
        for files in inF[:-1]:
        print files
    """
    os.system('tar -czf files /unzip/files[:-7]')
    # This is what i am expecting here files = "1440198002317590001.tar.gz" and files[:-7]= "1440198002317590001"
    """

Have you ever worked on this type of use case? Your help is greatly appreciated!! Thank you!

Answer 1

I think you misunderstood the meaning of os.system() , that will do the job, but its return value was not expected by you, it returns 0 for successful done, you can not directly assign its output to a variable. You may consider the module [subprocess] , see doc here . However, I DO NOT recommend that way to list files (actually, it returns string instead of list, see doc find the detail by yourself).

The best way I think would be glob module, see doc here . Use glob.glob(pattern) , you can put all files match the pattern in a list, then you can loop it easily.

Of course, if you are familiar with os module, you also can use os.listdir() , os.path.join() , or even os.paht.expanduser() to do this. (Unlike glob , it only put filenames without fully path into a list, you need to reconstruct file path).

By the way, for you purpose here, there is no need to declare an empty list first (ie inF = [] )

For unzip file part, you can do it by os.system , but I also recommend to use subprocess module instead of os.system , you will find the reason in the doc of subprocess .

---

DO NOT see the following code, ONLY see them after you really can not solve this by yourself.

import os
import glob

inF = glob.glob('~/TargetData/zip/*.tar.gz')
if inF:
    for files in inF:
    # consider subprocess.call() instead of os.system
    unzip_name = files.replace('zip', 'unzip')[:-7]
    # get directory name and make sure it exists, otherwise create it
    unzip_dir = os.path.dirname(unzip_name)
    if not os.path.exists(unzip_dir):
        os.mkdir(unzip_dir)
    subprocess.call(['tar -xzf', files, '-C', unzip_name])
    # os.system('tar -czf files /unzip/files[:-7]')