Improving speed of python script

Question

Is there anyway to improve the speed of the following. The code inserts a modifier to a http request. The modfiers are 5 digit email IDs. The goal is to apend each result into a JSON file.

I know getting rid of the Progessbar() will help, but i find the slow down worth the usefullness of the function. The Try Except Else section slowed it down again quit a lot, but idents has 4500 entries. Any ideas?

import urllib3
import json
import csv
from progressbar import ProgressBar
pbar = ProgressBar()

with open('blim2.csv', newline='') as csvfile:
    idents = csv.reader(csvfile, delimiter=' ', quotechar='|')
    json_arr = []
    while True:
        try:
             for x in pbar(idents):
                http = urllib3.PoolManager()
                r = http.request('GET', 'https://api.pipedrive.com/v1/mailbox/mailMessages/'+"".join(x)+'?include_body=1&api_token=token')
                mails = json.loads(r.data.decode('utf-8'))
                json_arr.append(mails)
                with open('test2.json', 'w') as outfile:
                    json.dump(json_arr, outfile)             
        except:
            continue
        else:
            break

Answer 1

Try moving the while True loop all the way to the inside to keep retrying a single request only. Also, write the file only once, at the end.

import urllib3
import json
import csv
from progressbar import ProgressBar
import time

pbar = ProgressBar()
base_url = 'https://api.pipedrive.com/v1/mailbox/mailMessages/'
fields = {"include_body": "1", "api_token": "token"}

json_arr = []
http = urllib3.PoolManager()
with open('blim2.csv', newline='') as csvfile:
    for x in pbar(csv.reader(csvfile, delimiter=' ', quotechar='|')):
        while True:
            try:
                r = http.request('GET', base_url + "".join(x), fields=fields)
                break
            except Exception:
                # wait a bit before trying again
                time.sleep(1)  # seconds
        mails = json.loads(r.data.decode('utf-8'))
        json_arr.append(mails)
with open('test2.json', 'w') as outfile:
    json.dump(json_arr, outfile)

You should also look up what exception is actually being raised in your case and except only that. Otherwise you could be masking other problems. I already replaced your bare except with a except Exception , which at least allows the user to press Ctrl+C to abort the program.