Unit Testing with django-pipeline

Question

I have problems unit testing views of an application that uses django-pipeline? Whenever I perform a client.get() on any URL, it produces the following exception:

ValueError: The file 'css/bootstrap.css' could not be found with <pipeline.storage.PipelineCachedStorage object at 0x10d544950>.

The fact that it is bootstrap.css is of course not important, but that I'm unable to execute view rendering due to this exception.

Any guide / tips are welcome!

Answer 1

I ran into a similar problem. However, setting

STATICFILES_STORAGE='pipeline.storage.NonPackagingPipelineStorage'

when running the tests only partly solved my issue. I also had to disable the pipeline completely if you want to run LiverServerTestCase tests without having to calling 'collecstatic' before running the tests:

PIPELINE_ENABLED=False

Since django 1.4 it's fairly easy to modify settings for tests - there is a handy decorator that works for methods or TestCase classes:

https://docs.djangoproject.com/en/1.6/topics/testing/tools/#overriding-settings

eg

from django.test.utils import override_settings

@override_settings(STATICFILES_STORAGE='pipeline.storage.NonPackagingPipelineStorage', PIPELINE_ENABLED=False)
class BaseTestCase(LiveServerTestCase):
    """
    A base test case for Selenium
    """

    def setUp(self):
        ...

However this produced inconsistent results as @jrothenbuhler describes in his answer. Regardless, this is less than ideal if you are running integration tests - you should mimic production as much as possible to catch any potential issues. It appears django 1.7 has a solution for this in the form of a new test case "StaticLiveServerTestCase". From the docs: https://docs.djangoproject.com/en/dev/ref/contrib/staticfiles/#django.contrib.staticfiles.testing.StaticLiveServerCase

This unittest TestCase subclass extends django.test.LiveServerTestCase.

Just like its parent, you can use it to write tests that involve running the code under test and consuming it with testing tools through HTTP (eg Selenium, PhantomJS, etc.), because of which it's needed that the static assets are also published.

I haven't tested this, but sounds promising. For now I'm doing what @jrothenbuhler in his solution using a custom test runner, which doesn't require you to run collectstatic. If you really, really wanted it to run collectstatic you could do something like this:

from django.conf import settings
from django.test.simple import DjangoTestSuiteRunner
from django.core.management import call_command

class CustomTestRunner(DjangoTestSuiteRunner):
    """
    Custom test runner to get around pipeline and static file issues
    """

    def setup_test_environment(self):
        super(CustomTestRunner, self).setup_test_environment()
        settings.STATICFILES_STORAGE = 'pipeline.storage.NonPackagingPipelineStorage'
        call_command('collectstatic', interactive=False)

In settings.py

TEST_RUNNER = 'path.to.CustomTestRunner'

Answer 2

I've been running into the same problem. I tackled it using a custom test runner:

from django.conf import settings
from django.test.simple import DjangoTestSuiteRunner

from pipeline.conf import settings as pipeline_settings

class PipelineOverrideRunner(DjangoTestSuiteRunner):

    def setup_test_environment(self):
        '''Override STATICFILES_STORAGE and pipeline DEBUG.'''
        super(PipelineOverrideRunner, self).setup_test_environment()
        settings.STATICFILES_STORAGE = 'pipeline.storage.PipelineFinderStorage'
        pipeline_settings.DEBUG = True

Then in your settings.py:

TEST_RUNNER = 'path.to.PipelineOverrideRunner'

Setting pipeline's DEBUG setting to True ensures that the static files are not packaged. This prevents the need to run collectstatic before running the tests. Note that it's pipeline's DEBUG setting, not Django's, which is overridden here. The reason for this is that you want Django's DEBUG to be False when testing to best simulate the production environment.

Setting STATICFILES_STORAGE to PipelineFinderStorage makes it so that the static files are found when Django's DEBUG setting is set to False, as it is when running tests.

The reason I decided to override these settings in a custom test runner instead of in a custom TestCase is because certain things, such as the django.contrib.staticfiles.storage.staticfiles_storage object, get set up once based on these and other settings. When using a custom TestCase, I was running into problems where tests would pass and fail inconsistently depending on whether the override happened to be in effect when modules such as django.contrib.staticfiles.storage were loaded.

Answer 3

I ran into the same problem. I managed to get around it by using a different STATIC_FILES_STORAGE when I'm testing:

STATICFILES_STORAGE = 'pipeline.storage.NonPackagingPipelineStorage'

I have separate settings files for production and testing, so I just put it in my test version, but if you don't, you could probably wrap it in if DEBUG .

--EDIT

It took a little more effort, because this can only present during the unittesting. To address that, I used the snippet at http://djangosnippets.org/snippets/1011/ and created a UITestCase class:

class UITestCase(SettingsTestCase):
    '''
    UITestCase handles setting the Pipeline settings correctly.
    '''
    def __init__(self, *args, **kwargs):
        super(UITestCase, self).__init__(*args, **kwargs)

    def setUp(self):
        self.settings_manager.set(
            STATICFILES_STORAGE='pipeline.storage.NonPackagingPipelineStorage')

Now all of my tests that need to render UI that incude compressed_css tags use UITestCase instead of django.test.TestCase.

Answer 4

I ran into the same problem, and it turned out to be because I had

TEST_RUNNER = 'djcelery.contrib.test_runner.CeleryTestSuiteRunner'

I don't understand how, but it must somehow have interacted with Pipeline. Once I removed that setting, the problem went away.

I still needed to force Celery to be eager during testing, so I used override_settings for the tests that needed it:

from django.test.utils import override_settings

…

class RegisterTestCase(TestCase):

    @override_settings(CELERY_EAGER_PROPAGATES_EXCEPTIONS=True,
                       CELERY_ALWAYS_EAGER=True,
                       BROKER_BACKEND='memory')
    def test_new(self):
        …

Answer 5

Same here. Refers to this issues: https://github.com/cyberdelia/django-pipeline/issues/277

As I use py.test, I put this in conftest.py as a workaround:

import pytest
from django.conf import settings

def pytest_configure():
    # workaround to avoid django pipeline issue
    # refers to 
    settings.STATICFILES_STORAGE = 'pipeline.storage.PipelineStorage'