Python asyncio: blocking and race conditions

Question

I'm trying to understand python's asyncio library, and am somewhat surprised by the behavior of the following example:

import asyncio
import time

from aiohttp import web


_lock_table = {'locked': False}


async def process(text):
    # wait while we're blocked
    while _lock_table['locked']:
        await asyncio.sleep(1)

    _lock_table['locked'] = True

    # hold the lock for three seconds
    # simulates "processing something"
    await asyncio.sleep(3)

    # release the lock
    _lock_table['locked'] = False


async def request_handler(request):
    text = request.query.get('text')

    start_time = time.time()
    await process(text)
    end_time = time.time()

    return web.Response(
        body='text: {}, request time: {}'.format(text, end_time - start_time),
        content_type='text/html'
    )


app = web.Application()
app.router.add_route('GET', '/', request_handler)

web.run_app(app)

If I start the above program and open up two requests to http://0.0.0.0:8080 I see that both requests take around three seconds to complete. Why is this? Shouldn't the second request be blocked by the first one that acquires the lock?

Answer 1

I can't reproduce the problem you are describing. aiohttp discourages the use of global variables and suggests to use the Application object in favor of globals. But they don't seem to mention that threads are involved here. It rather seems to be a general hint for cleaner code than a necessity - so your code should be fine...

I tested your code with this example bash script:

#!/bin/bash

pids=()
range=$(seq 1 10)
url='http://127.0.0.1:8080/?text='
for i in $range; do
    curl -o $i.txt "${url}$(date +%H%M%S)" 2>/dev/null &
    pids[$i]=$!
done

wait ${pids[$@]}

for i in $range; do
    echo "$i $(cat $i.txt)"
done

the output looks something like this:

1 text: 222331, request time: 24.153167724609375
2 text: 222331, request time: 15.077209711074829
3 text: 222331, request time: 27.17816162109375
4 text: 222331, request time: 6.0150146484375
5 text: 222331, request time: 18.09979248046875
6 text: 222331, request time: 21.1268310546875
7 text: 222331, request time: 3.004944086074829
8 text: 222331, request time: 9.03411865234375
9 text: 222331, request time: 12.050049066543579
10 text: 222331, request time: 30.205841302871704

You might want to make sure that the actual start time is similar.