Django REST Framework: TestCase is not returning correct queryset

Question

I have created an API, using DRF , for products in an inventory that can be accessed by the following endpoint url(r'products/$', views.InventoryList.as_view(), name='product-list') .

When issuing a GET request via postman, I get the correct queryset back, which is a total of 11 products:

[
    {
        "id": 1,
        "name": "Biscuits",
        "description": "Papadopoulou Biscuits",
        "price": "2.52",
        "comments": [
            {
                "id": 1,
                "title": "First comments for this",
                "comments": "Very tasty",
                "rating": 8,
                "created_by": "xx"
            }
        ]
    },
    {
        "id": 2,
        "name": "Rice",
        "description": "Agrino Rice",
        "price": "3.45",
        "comments": []
    },
    {
        "id": 3,
        "name": "Spaghetti",
        "description": "Barilla",
        "price": "2.10",
        "comments": []
    },
    {
        "id": 4,
        "name": "Canned Tomatoes",
        "description": "Kyknos",
        "price": "3.40",
        "comments": []
    },
    {
        "id": 5,
        "name": "Bacon",
        "description": "Nikas Bacon",
        "price": "2.85",
        "comments": []
    },
    {
        "id": 6,
        "name": "Croissants",
        "description": "Molto",
        "price": "3.50",
        "comments": []
    },
    {
        "id": 7,
        "name": "Beef",
        "description": "Ground",
        "price": "12.50",
        "comments": []
    },
    {
        "id": 8,
        "name": "Flour",
        "description": "Traditional Flour",
        "price": "3.50",
        "comments": []
    },
    {
        "id": 9,
        "name": "Oregano",
        "description": "Traditional oregano",
        "price": "0.70",
        "comments": []
    },
    {
        "id": 10,
        "name": "Tortellini",
        "description": "Authentic tortellini",
        "price": "4.22",
        "comments": []
    },
    {
        "id": 11,
        "name": "Milk",
        "description": "Delta",
        "price": "1.10",
        "comments": []
    }
]

I wrote then a test (using pytest ) to test this endpoint:

import pytest
import pytest_django
from django.urls import reverse
from rest_framework import status
from rest_framework.test import APITestCase

class TestInventoryList(APITestCase):
    @pytest.mark.django_db
    def test_get_product_list(self):
        url = reverse('product-list')
        response = self.client.get(url)
        print(response.json())
        self.assertEqual(response.status_code, status.HTTP_200_OK)
        self.assertEqual(len(response.json()), 11) # <-- TC fails here

but it fails since response.json() returns only the first 9 objects:

[{
    'id': 1,
    'name': 'Biscuits',
    'description': 'Papadopoulou Biscuits',
    'comments': [],
    'price': '2.52'
}, {
    'id': 2,
    'name': 'Rice',
    'description': 'Agrino Rice',
    'comments': [],
    'price': '3.45'
}, {
    'id': 3,
    'name': 'Spaghetti',
    'description': 'Barilla',
    'comments': [],
    'price': '2.10'
}, {
    'id': 4,
    'name': 'Canned Tomatoes',
    'description': 'Kyknos',
    'comments': [],
    'price': '3.40'
}, {
    'id': 5,
    'name': 'Bacon',
    'description': 'Nikas Bacon',
    'comments': [],
    'price': '2.85'
}, {
    'id': 6,
    'name': 'Croissants',
    'description': 'Molto',
    'comments': [],
    'price': '3.50'
}, {
    'id': 7,
    'name': 'Beef',
    'description': 'Ground',
    'comments': [],
    'price': '12.50'
}, {
    'id': 8,
    'name': 'Flour',
    'description': 'Traditional Flour',
    'comments': [],
    'price': '3.50'
}, {
    'id': 9,
    'name': 'Oregano',
    'description': 'Traditional oregano',
    'comments': [],
    'price': '0.70'
}]

A couple of observations here:

1. The queryset returned in my test case does not contain the comments for my first product even-though when accessed via postman I can see the comments. Comments is a different django model which is accessed through this nested endpoint: url(r'^products/(?P<product_id>[0-9]+)/comments/$', views.CommentsList.as_view())
2. I inserted the last two products as well as the comment for my first product (none of which is returned in the latter queryset) using POST and an API auth token. Is this an information that I should somehow include in my test case?

---

EDIT

models.py

    from django.db import models
from django.contrib.auth.models import User

class Product(models.Model):
    name = models.CharField(max_length=255)
    description = models.TextField()
    price = models.DecimalField(decimal_places=2, max_digits=20)


class Comments(models.Model):
    product = models.ForeignKey(Product, related_name='comments')
    title = models.CharField(max_length=255)
    comments = models.TextField()
    rating = models.IntegerField()
    created_by = models.ForeignKey(User)

urls.py

from django.conf.urls import url
from . import views

urlpatterns = [
    url(r'products/$', views.InventoryList.as_view(), name='product-list'),
    url(r'^products/(?P<product_id>[0-9]+)/$', views.InventoryDetail.as_view()),
    url(r'^products/(?P<product_id>[0-9]+)/comments/$', views.CommentsList.as_view()),
    url(r'^products/(?P<product_id>[0-9]+)/comments/(?P<comment_id>[0-9]+)/$', views.CommentsDetail.as_view()),
]

views.py

from rest_framework import generics
from rest_framework.permissions import IsAuthenticatedOrReadOnly
from .models import Product, Comments
from .serializers import ProductSerializer, CommentSerializer
from .permissions import IsAdminOrReadOnly, IsOwnerOrReadOnly


class InventoryList(generics.ListCreateAPIView):
    queryset = Product.objects.all()
    serializer_class = ProductSerializer
    permission_classes = (IsAdminOrReadOnly, )
    lookup_url_kwarg = 'product_id'


class InventoryDetail(generics.RetrieveUpdateAPIView):
    queryset = Product.objects.all()
    serializer_class = ProductSerializer
    permission_classes = (IsAdminOrReadOnly, )
    lookup_url_kwarg = 'product_id'


class CommentsList(generics.ListCreateAPIView):
    serializer_class = CommentSerializer
    permission_classes = (IsAuthenticatedOrReadOnly, )
    lookup_url_kwarg = 'product_id'

    def perform_create(self, serializer):
        serializer.save(created_by=self.request.user, product_id=self.kwargs['product_id'])

    def get_queryset(self):
        product = self.kwargs['product_id']
        return Comments.objects.filter(product__id=product)


class CommentsDetail(generics.RetrieveUpdateDestroyAPIView):
    serializer_class = CommentSerializer
    permission_classes = (IsAuthenticatedOrReadOnly, IsOwnerOrReadOnly)
    lookup_url_kwarg = 'comment_id'

    def get_queryset(self):
        comment = self.kwargs['comment_id']
        return Comments.objects.filter(id=comment)

permissions.py

from rest_framework.permissions import BasePermission, SAFE_METHODS


class IsAdminOrReadOnly(BasePermission):
    def has_permission(self, request, view):
        if request.method in SAFE_METHODS:
            return True
        else:
            return request.user.is_staff


class IsOwnerOrReadOnly(BasePermission):
    def has_object_permission(self, request, view, obj):
        if request.method in SAFE_METHODS:
            return True

        return obj.created_by == request.user

Answer 1

I suspect, (without having your product model at hand) that you are not getting all the elements from the products table, for the following reasons:

• You created your first 9 elements manually, without registering them to a specific user.
• Afterward, you added an authentication method ( TokenAuthentication ) and create some users with access tokens.
• Because you added an authentication method, you probably added @permission_classes((IsAuthenticated,)) / permission_classes=(IsAuthenticated,) to your product-list view.
  That restricts any unauthenticated user from accessing the product-list .
  The unauthenticated-anonymous users will only see the anonymous elements of your database.
• You added the next 2 elements and the comment with one of the registered users, which in turn registered those elements to the user-creator, therefore you cannot access them without an authenticated user.

To access resources that need authentication from the DRF's test client, you need to authenticate your user first.
You can use force_authenticate method:

class TestInventoryList(APITestCase):
    def setUp(self):
        self.req_factory = APIRequestFactory()
        self.view =  views.InventoryList.as_view({'get': 'list',})

    @pytest.mark.django_db
    def test_get_product_list(self):
        url = reverse('product-list')
        request = self.client.get(url)
        force_authenticate(request, user=YOUR_USER)
        response = self.view(request)
        print(response.json())
        self.assertEqual(response.status_code, status.HTTP_200_OK)
        self.assertEqual(len(response.json()), 11)

This test assumes that you list method returns Products.objects.all()

---

As @cezar points out, testing a view against real data is prone to fail (for example, when you add a new element, the self.assertEqual(len(response.json()), 11) will fail)

You should consider mocking your responses to create an isolated environment.

I tend to use a combination of factory_boy and django-nose ( pytest works as well).