如何從 python 的循環中保存 CSV 文件中的多個值？

Question

我有大約 20000 個文件，我想根據閾值（我有 10 個閾值）計算每個文件的峰值，然后將其保存在 csv 文件中。 我很困惑如何根據 csv 文件中的閾值保存文件的值。

for threshold in np.arange(1,10,1):
threshold_p=calculate_th(n,m, threshold)
for root, dirs, files in os.walk(dir_path, topdown=False):
    for name in files:
        allfiles.append(os.path.join(root, name))
    for filename in tqdm.tqdm(allfiles, desc= "files progress"):
        output_g = np.load(filename)
        filtered=np.sum(output_g > threshold_p)
        result= [filename,filtered, threshold_p, threshold]

我想將“結果”值保存為具有 4 列的 csv 文件，但我無法在不重寫它們的情況下將它們保存為 csv 文件。

Answer 1

你沒有展示你是怎么寫的，但是可以有不同的方法來做。

---

第一的。 有一個標准規則：如果你使用for -loop 然后使用list來保留所有結果。

這樣你就可以在循環之前創建空列表，append 結果到這個循環內的列表中，並在循環之后寫入所有結果。

為了使所有版本相似，我不會將with open(..) as fh - 但您可以嘗試使用它。

# --- before loop ---

all_results = []  # <-- list for all results

# --- loop ---

for threshold in np.arange(1,10):
    threshold_p = calculate_th(n, m, threshold)
    
    for root, dirs, files in os.walk(dir_path, topdown=False):
        for name in tqdm.tqdm(files, desc="files progress"):
            filename = os.path.join(root, name)
            output_g = np.load(filename)
            filtered = np.sum(output_g > threshold_p)
            result = [filename, filtered, threshold_p, threshold]
     
            all_results.append( result )  # <-- keep result           

# --- after loop ---

fh = open('output.csv', 'w')
cvs_writer = cvs.writer(fh)

# write one row with headers (using `writerow` without `s` at the end)
cvs_writer.writerow(["filename", "filtered", "threshold_p", "threshold"]    cvs_writer.writerow(["filename", "filtered", "threshold_p", "threshold"]

# write many rows with results (using `writerows` with `s` at the end)
cvs_writer.writerows(all_results)
                                                                                                
fh.close()

---

第二。 您應該在循環前打開文件，在循環內寫入行，並在循環后關閉文件。

# --- before loop ---

fh = open('output.csv', 'w')
cvs_writer = cvs.writer(fh)

# write one row with headers (using `writerow` without `s` at the end)
cvs_writer.writerow(["filename", "filtered", "threshold_p", "threshold"]    cvs_writer.writerow(["filename", "filtered", "threshold_p", "threshold"]

# --- loop ---

for threshold in np.arange(1,10):
    threshold_p = calculate_th(n, m, threshold)
    
    for root, dirs, files in os.walk(dir_path, topdown=False):
        for name in tqdm.tqdm(files, desc="files progress"):
            filename = os.path.join(root, name)
            output_g = np.load(filename)
            filtered = np.sum(output_g > threshold_p)
            result = [filename, filtered, threshold_p, threshold]
     
            # write row row with result (using `writerow` without `s` at the end)
            cvs_writer.writerow(result)

# --- after loop ---
                                                                                                
fh.close()

---

第三。 在循環之前只創建帶有標題的文件 - 並關閉它。 在append mode循環打開文件以在文件末尾添加新行。

如果代碼崩潰，此方法將保留所有結果。

# --- before loop ---

fh = open('output.csv', 'w')
cvs_writer = cvs.writer(fh)

# write one row with headers (using `writerow` without `s` at the end)
cvs_writer.writerow(["filename", "filtered", "threshold_p", "threshold"]    cvs_writer.writerow(["filename", "filtered", "threshold_p", "threshold"]
                                                                                                
fh.close()
                                                                                                
# --- loop ---

for threshold in np.arange(1,10):
    threshold_p = calculate_th(n, m, threshold)
    
    for root, dirs, files in os.walk(dir_path, topdown=False):
        for name in tqdm.tqdm(files, desc="files progress"):
            filename = os.path.join(root, name)
            output_g = np.load(filename)
            filtered = np.sum(output_g > threshold_p)
            result = [filename, filtered, threshold_p, threshold]

            fh = open('output.csv', 'a')  # `a` for `append mode`
            cvs_writer = cvs.writer(fh)
     
            # write row row with result (using `writerow` without `s` at the end)
            cvs_writer.writerow(result)

            fh.close()

# --- after loop ---

# nothing