In pytest, how to write a fixture that returns another function's AsyncGenerator?

Question

The following works; it's a test that successfully uses an async generator as a fixture.

from collections.abc import AsyncGenerator

import pytest

@pytest.fixture()
async def fixture() -> AsyncGenerator[str, None]:
    yield "a"

@pytest.mark.asyncio
async def test(fixture: str):
    assert fixture[0] == "a"

However, let's say I want fixture() to return an AsyncGenerator produced by some other function. In that case, I get an error:

from collections.abc import AsyncGenerator

import pytest

async def _fixture() -> AsyncGenerator[str, None]:
    yield "a"

@pytest.fixture()
async def fixture() -> AsyncGenerator[str, None]:
    return _fixture()

@pytest.mark.asyncio
async def test(fixture: str):
    assert fixture[0] == "a"

And the error is:

>     assert fixture[0] == "a"
E     TypeError: 'async_generator' object is not subscriptable

What am I missing?

Answer 1

First of all, the example code that supposedly works, should not (and does not for me). It produces the same TypeError about fixture being an async generator. According to the (rather poor) documentation of pytest-asyncio :

Asynchronous fixtures are defined just like ordinary pytest fixtures, except they should be decorated with @pytest_asyncio.fixture .

So, unless you specifically configure asyncio_mode = auto , to get the first test working, you would need to change the code to this:

from collections.abc import AsyncGenerator

import pytest
import pytest_asyncio


@pytest_asyncio.fixture
async def fixture() -> AsyncGenerator[str, None]:
    yield "abcdef"


@pytest.mark.asyncio
async def test(fixture: str):
    assert fixture[0] == "a"

The same goes for fixture in the second example.

---

Secondly, _fixture is just a regular old asynchronous generator function. Without the weird and obscure pytest magic, calling _fixture() just returns an async genrator object (as correctly indicated by the return type annotation AsyncGenerator ).

You are just returning it from your fixture function instead of performing a composition. With normal (non- async ) generators, you would do yield from in that situation. This does not work ( PEP 525 ) with asynchronous generators though, so you'll need to do an async for loop instead and yield the elements instead:

from collections.abc import AsyncGenerator

import pytest
import pytest_asyncio


async def _fixture() -> AsyncGenerator[str, None]:
    yield "a"


@pytest_asyncio.fixture
async def fixture() -> AsyncGenerator[str, None]:
    async for item in _fixture():
        yield item


@pytest.mark.asyncio
async def test(fixture: str):
    assert fixture[0] == "a"

---

Side rant: This is one of the reasons I don't like pytest very much. This strange insistence on obscuring logic sometimes.

Regular fixtures are just run and their return value is passed to the test function requesting that fixture. But a generator is apparently not good enough, so instead of passing the generator itself to the test function, we get the first item from it and then the rest is the "finalizer", I guess. (see yield fixtures )

I suppose, if I wanted an actual generator in my test function, I would have to do exactly what you (mistakenly) did above, namely defining it separately and then returning it from a fixture function.

Thus, you could also get your original code to work, if you changed the test function to consume the async generator it was passed. Since it only yields a single time, putting the contents into a list would give you "a" as its first element:

...

@pytest_asyncio.fixture
async def fixture() -> AsyncGenerator[str, None]:
    return _fixture()

@pytest.mark.asyncio
async def test(fixture: AsyncGenerator[str, None]):
    items = [item async for item in fixture]
    assert items[0] == "a"