Assertion error parsing response to POST request from DRF viewset with DRF APIClient in tests

Question

I've run into a strange issue with Django Rest Framework testing engine. The weird thing is that everything used to work fine with Django 3 and this issue turned up after I migrated to Django 4. Apart from testing, everything works well, and responds to queries as expected.

The problem

I'm using DRF APIClient to make queries for unit tests. While GET requests perform predictably, I fail to make POST requests work.

Here is some minimalistic example code I created to figure out the issue. The versions I'm using:

Python 3.9
Django==4.0.3
djangorestframework==3.13.1

from django.db import models
from django.urls import include, path
from django.utils import timezone
from rest_framework import routers, serializers, viewsets

router = routers.DefaultRouter()


# models.py

class SomeThing(models.Model):
    created_at = models.DateTimeField(default=timezone.now)
    title = models.CharField(max_length=100, null=True, blank=True)


# serializers.py

class SomeThingSerializer(serializers.ModelSerializer):
    class Meta:
        fields = "__all__"
        model = SomeThing


# views.py

class SomeThingViewSet(viewsets.ModelViewSet):
    queryset = SomeThing.objects.all().order_by('id')
    serializer_class = SomeThingSerializer


# urls.py

router.register("some-things", SomeThingViewSet, basename="some_thing")

app_name = 'question'
urlpatterns = (
    path('', include(router.urls)),
)

Here is my test case:

import json

from django.contrib.auth import get_user_model
from rest_framework import status
from rest_framework.test import APITestCase, APIClient


class TestUserView(APITestCase):
        self.some_user = get_user_model().objects.create(login="some_user@test.ru")

    @staticmethod
    def get_client(user):
        client = APIClient()
        client.force_authenticate(user=user)
        return client


    def test_do_something(self):
        client = self.get_client(self.compliance_chief)
        url = reverse('question:some_things-list')
        resp = client.post(
            path=url,
            data=json.dumps({"title": "Created Something"}),
            content_type="application/json",
        )
        assert resp.status_code == status.HTTP_201_OK

(Yes, I have to use some authentication to get access to the data, but I don't think it is relevant to the problem.) To which I receive a lengthy traceback, ending with an assertion error:

  File "/****/****/****/venv/lib/python3.9/site-packages/django/test/client.py", line 82, in read
    assert (
AssertionError: Cannot read more than the available bytes from the HTTP incoming data.

As it is really fairly long, I'll leave it just in case in a gist without posting it here.

Steps to fix

The problem clearly happens after the correct response is returned by the viewset. To make sure the response is correct I made a slight customisation in the create method to print out the response before it is returned, like so:

class SomeThingViewSet(viewsets.ModelViewSet):
    queryset = SomeThing.objects.all().order_by('id')
    serializer_class = SomeThingSerializer

    def create(self, request, *args, **kwargs):
        response = super().create(request, *args, **kwargs)
        print("THIS IS THE RESPONSE FROM THE VIEWSET", response)
        return response

And, sure enough, the result is correct:

THIS IS THE RESPONSE FROM THE VIEWSET <Response status_code=201, "text/html; charset=utf-8">

Which makes me think something goes wrong at the parsing stage (actually, the traceback implies the same). I tried to tweak the way I build the query, namely:

• using format instead of content type like so: resp = client.post(path=url, data={"title": "Created Something"}, format="json")
• using the.generic method instead of.post like so: resp = client.generic(method="POST", path=url, data=json.dumps({"title": "Created Something"}), content_type="application/json")

The result is the same.

From googling I found out that this error indeed has occasionally occurred in connection with DRF APIClient and Django, but really long ago (likethis discussion , which claims that the issue was fixed in the later versions of Django).

I'm sure the reason for this behaviour is rather obvious (some stupid mistake most likely) and the solution must be very simple, but so far I've failed to find it. I would be very grateful if somebody shared their experience, if there is any, of dealing with such an issue, or their considerations as to where to move from this deadlock.

Answer 1

Alright, the mystery's been resolved and I'm going to share it here in case somebody runs into something similar, although it would take quite a coincidence, so it is unlikely.

Long story short: I messed up the source code of my Django4.0.3. installed in this project.

Now, how it happened. While I was testing some stuff, I ran into an error, which I failed to locate, so I went along the whole chain of events checking if the output was what I expected it to be. Soon enough I found myself checking the output from functions in the libraries installed under my virtual environment. I realise it's a malpractice to directly modify their code, but as I was working in my local environment with an option to reinstall everything at any moment, I decided it was fine to play with them. As it resulted in nothing, I removed all the code I had added (or so I thought).

After a while I realised what caused the initial error (an overlooked condition in my testing setup), fixed it and tried to run the test. That's when the problem in question showed up.

Later I found out that the same very test performs correctly in an identical environment. Then I suspected that I broke something in my local library code. Next I simply compared the code I had dealt with in my local environment with the code from the official source and soon enough I established the offending line. It happened to be in django/test/client.py, in the definition of the RequestFactory.generic method. Something like this:

...
        if not r.get("QUERY_STRING"):
            # WSGI requires latin-1 encoded strings. See get_path_info().
            query_string = parsed[4].encode().decode("iso-8859-1")
            r["QUERY_STRING"] = query_string
        req = self.request(**r)
        return self.request(**r)
...

The offending line (which I added and forgot to remove) was req = self.request(**r) . After I deleted it, everything returned back to normal.