python新手。 為什么我的腳本沒有上傳文件？

Question

我是python的新手，並嘗試遞歸將目錄中所有tar文件的gz內容及其內部目錄上傳到s3。 總共大約有1,350,000個tar文件。

我沒有空間立即解壓縮所有文件，因此我一次只能處理其中一個。

本來我的腳本可以運行，但是一旦遇到錯誤（tar文件已損壞），我的腳本就會結束。 我添加了一堆tryexcept子句來嘗試記錄並繼續解決這些錯誤，即使我得到如下輸出，我的腳本似乎也沒有上傳文件：

('iter: ', '/srv/nfs/storage/bucket/2015/11/20/KFWS/NWS_NEXRAD_NXL2DP_KFWS_20151120130000_20151120135959.tar')
KFWS20151120_130030_V06.gz
('singlepart', <Bucket: bucket>, '/srv/nfs/storage/bucket/2015/11/20/KFWS/KFWS20151120_130030_V06.gz', '2015/11/20/KFWS/KFWS20151120_130030_V06.gz')
('single_part: ', '2015/11/20/KFWS/KFWS20151120_130030_V06.gz', '/srv/nfs/storage/bucket/2015/11/20/KFWS/KFWS20151120_130030_V06.gz')
KFWS20151120_131000_V06.gz
('iter: ', '/srv/nfs/storage/bucket/2015/11/20/KFWS/NWS_NEXRAD_NXL2DP_KFWS_20151120110000_20151120115959.tar')
KFWS20151120_110630_V06.gz
('singlepart', <Bucket: bucket>, '/srv/nfs/storage/bucket/2015/11/20/KFWS/KFWS20151120_110630_V06.gz', '2015/11/20/KFWS/KFWS20151120_110630_V06.gz')
('single_part: ', '2015/11/20/KFWS/KFWS20151120_110630_V06.gz', '/srv/nfs/storage/bucket/2015/11/20/KFWS/KFWS20151120_110630_V06.gz')
KFWS20151120_111601_V06.gz

它表明它正在進入single_part，對我而言，這意味着至少正在運行singlept函數並嘗試上載對象，但從未創建Zimport_errors.list或Znoaa_nexrad_files.list，並且在其中看不到任何新對象桶。

下面的代碼：（對不起，它的總粗細。我試圖自學python，而且只有幾個星期。）

http://pastebin.com/X56FHDaa

這是主要的街區

def singlept(bucket, keyname, local_file):
    retries = 0
    key_size = 0
    local_size = os.path.getsize(local_file)
    while retries <= 4 and local_size != key_size:
        local_md5 = md5file(local_file=local_file)
        print('single_part: ', keyname, local_file)
        try:
            key = bucket.new_key(keyname)
        except Exception:
            print('couldn\'t create key: ', keyname)
            pass
        try:
            key.set_contents_from_filename(local_file)
            key_size = key.size
            with open(successfile, 'ab') as f:
                f.write('\n')
                f.write(str(local_file + ',' + keyname + ',' + str(key_size) + ',' + str(local_size)))
        except Exception:
            print('couldn\'t upload file: ', local_file, ' as key: ', keyname)
            with open(errorfile, 'ab') as f:
                f.write('\n')
                f.write(str(local_file + ',' + keyname + ',' + str(key_size) + ',' + str(local_size)))
            pass


for dir, subdir, files in os.walk(local_bucket):
    s3path = "/".join(str(dir).split('/')[5:])
    local_path = str(local_bucket + '/' + s3path)
    for fname in files:
        if fname.endswith("tar"):
            fullpath = local_path + '/' + fname
            if (debug):
                print('iter: ',fullpath)
            with tarfile.open(fullpath, 'r') as tarball:
                zips = tarball.getmembers()
                try:
                    tarball.extractall(path=local_path)
                except Exception:
                    with open(errorfile, 'ab') as f:
                        f.write('\n')
                        f.write(str(fullpath + ',' + str(os.path.getsize(fullpath))))
                    continue
            for zip in zips:
                if (debug):
                    print(zip.name)
                local_file = local_path + '/' + zip.name
                keyname = s3path + '/' + zip.name
                try:
                    if zip.size >= 1073741824:
                        if (debug):
                            print('multipart',bucket, local_file, keyname)
                        multipt(bucket, local_file, keyname)
                    else:
                        if (debug):
                            print('singlepart',bucket, local_file, keyname)
                        singlept(bucket, keyname, local_file)
                except Exception:
                    with open(errorfile, 'ab') as f:
                        f.write('\n')
                        f.write(str(local_file + "," + keyname))
                    continue
                if local_file.endswith("gz"):
                    try:
                        os.remove(local_file)
                    except Exception:
                        print('couldn\'t remove file: ', local_file)
                        continue

提前非常感謝您的幫助！ 我正在拔頭發！

編輯-直接添加代碼並希望修復縮進！ 它在Atom中看起來正確，但不能正確粘貼。 ：-/

Answer 1

except Exception僅捕獲except Exception類型的Exception -請參閱https://stackoverflow.com/a/18982726/264822 。 你應該試試：

   try:
        key = bucket.new_key(keyname)
    except:
        print('couldn\'t create key: ', keyname)
        pass