Python asyncio not running asynchronously

Question

In Python 3.10, I am submitting a get request to an endpoint running in an asynchronous manner so that, while a get call is made and I'm waiting for the response, other get requests run asynchronously. The code is currently running without exception or error but there is no asynchronous activity happening. Instead the code is still running sequentially (one request call at a time from the beginning of the range to the end). Can anyone spot what I might be doing wrong? Not sure where I've gone wrong.

import json
from src.services import data_transformation_service as dts, connectors_service as cs
import os
import requests as rq
import logging
import sqlalchemy.exc as saexc
import asyncio
import time

# Set globals, get config, set logging level
logging.basicConfig(level=logging.ERROR)
num_movies_to_import = 5
region = 'us-east-1'
config = cs.Config(cs.get_config(os.getenv('config_s3_bucket'), os.getenv('config_s3_file')), region)


# ----------------- Execute API request(s) & Load into Database -----------------
async def get_endpoint_data(endpoint):
    return json.loads(rq.get(endpoint).text.replace('\'', ''))


async def retrieve_data(connection, movie):
    try:
        print(f"{movie}")
        endpoint = config.get_movie_db_api_url('movies', movie)
        ar = await get_endpoint_data(endpoint)
        if sql := dts.sql_insert_creation_movie_api(config.get_movie_table(), ar):
            connection.execute(sql)  # run insert into database
            print(f"Merged Record: {sql}")
        else:
            print("Skipping record insert.")
    except saexc.StatementError:
        print(f"Error with record id {movie} most likely missing data or mal structured, skipping to next.")
    # except TypeError:
    #     print("JSON returned by API not valid, skipping to next.")
    return True


async def download_movie_data():
    with cs.DatabaseConnection('postgres', config.get_config_env(), 'evandb', region) as saq:
        results = await asyncio.gather(*[retrieve_data(saq, movie) for movie in range(500, num_movies_to_import + 500)])
        print(f'{results}')


# list(range(500, num_movies_to_import + 500))

if __name__ == "__main__":
    start = time.time()
    asyncio.run(download_movie_data())  # Create the asynch event loop
    end = time.time()
    print(f"Total Time: {end - start} seconds")

Answer 1

I found my issue... to help others new to asynch - the regular requests library doesn't support asynchronous processing. I decided to switch to aiohttp instead. For example:

async def get_endpoint_data(endpoint):
    async with aiohttp.ClientSession() as session:
        async with session.get(endpoint) as response:
            response = await response.json()
            return response