lxml + django + uwsgi 无法生成正确格式的excel文件？

Question

The excel file generated cannot be open when my project started with uwsgi when lxml installed in my environment as it can be opened successfully with django manage.py runserver and gunicorn

My main codes like below:

── test_excel
           ├── urls.py
           ├── wsgi.py
           └── settings.py

── manage.py

urls.py

from django.contrib import admin
from django.urls import path
from django.views import View
from openpyxl.writer.excel import save_virtual_workbook
import pandas as pd
from django.http import HttpResponse, StreamingHttpResponse


import xlrd
import openpyxl
from openpyxl import Workbook
from openpyxl.styles import Font, Alignment
from openpyxl.writer.excel import save_virtual_workbook
from openpyxl.cell.cell import ILLEGAL_CHARACTERS_RE



class TestExcelView(View):
    def get(self, request):
        # indexs = {0: ['cost', '2020-01', 'testing'],
        #           1: ['cost', '2020-01', '360 Limited'],
        #           2: ['cost', '2020-02', 'Korea Co.,LTD'],
        #           3: ['cost', '2020-02', 'ADS4EACH HK TECH LIMITED']}
        # columns = [['1'], ['amy'], ['tom'], ['zara'], ['jay'], ['Lee'], ['Uzi'], ['Zome'], ['Qoe'], ['Aoi'], ['Yeezy'],
        #            ['Hazy'], ['Wash'], ['pany'], ['zoey'], ['Moe'], ['total']]
        # datas = {
        #     0: [0.0, 0.0, 0.0, 0.0, 0.0, 7.85, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 7.85],
        #     1: [0.0, 0.0, 0.0, 0.0, 0.0, 7.85, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 7.85],
        #     2: [0.0, 0.0, 0.0, 0.0, 0.0, 7.85, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 7.85],
        #     3: [0.0, 0.0, 0.0, 0.0, 0.0, 7.85, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 7.85]
        # }
        #
        # index = pd.MultiIndex.from_tuples(indexs.values())
        # column = pd.MultiIndex.from_tuples(columns)
        # data = datas.values()
        #
        # df = pd.DataFrame(data, index=index, columns=column)
        #
        # # 1 saved as excel
        #
        # excel_writer = pd.ExcelWriter(path='download.xlsx', engine='openpyxl')
        # df.to_excel(excel_writer)
        # wb = excel_writer.book
        #
        # response = HttpResponse(save_virtual_workbook(wb))
        # response["Content-Type"] = 'application/vnd.ms-excel'
        # response['Content-Disposition'] = 'attachment; filename={}.xlsx'.format("for_test")
        # return response

        wb = Workbook()
        ws = wb.active
        ws.merge_cells('A1:B1')
        a1 = ws["A1"]
        ws["A1"] = "reason"
        ws.column_dimensions["A"].width = 100
        ali = Alignment(horizontal='center', vertical='center')

        a1.alignment = ali

        ws["A2"] = "request_id"
        ws.column_dimensions["A"].width = 50
        ws["B2"] = "rebate_reason"
        ws.column_dimensions["B"].width = 50

        for item in ws["A2:B2"][0]:
            item.font = Font(color='FF0000')
            item.alignment = ali

        response = HttpResponse(save_virtual_workbook(wb))
        response["Content-Type"] = 'application/vnd.ms-excel'
        response['Content-Disposition'] = 'attachment; filename={}.xlsx'.format("test_excel")
        return response


urlpatterns = [
    path('admin/', admin.site.urls),
    path('test/', TestExcelView.as_view()),
]

wsgi.py

import os

from django.core.wsgi import get_wsgi_application

#os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'test_excel.settings')

application = get_wsgi_application()

test_excel.ini

[uwsgi]
pythonpath=/home/bluev/work/bv_crm_bak/test_excel
pythonpath=/home/bluev/work/bv_crm_bak/test_excel
env=DJANGO_SETTINGS_MODULE=test_excel.settings
module=test_excel.wsgi
master=True
pidfile=logs/test_excel.pid
vacuum=True
max-requests=100000
enable-threads=true
processes = 4
threads=8
listen=64
daemonize=logs/test_excel.log
log-slow=3000
python-autoreload=1
http=0.0.0.0:8876

Then if i start the project like uwsgi --ini test_excel.ini as lxml installed, and request localhost:8876/test/ , i would get a failed excel file. But when I uninstalled lxml the result is right.

I don't understand why lxml affects the excel response of django with uwsgi

How can I get the right excel file response with uwsgi in this situation?

Thanks.

python version: Python 3.6.8

package list:

asgiref==3.2.7
Django==3.0.6
et-xmlfile==1.0.1
jdcal==1.4.1
lxml==4.5.1
mysqlclient==1.4.6
numpy==1.18.4
openpyxl==3.0.3
pandas==1.0.3
python-dateutil==2.8.1
pytz==2020.1
six==1.14.0
sqlparse==0.3.1
uWSGI==2.0.18
xlrd==1.2.0

• Updated different condition detail

environ condition             excel_file(if excel file can be opened)

gunicorn + lxml  + django                      yes

gunicorn + django                              yes

django runserver + lxml                        yes

django runserver                               yes

uwsgi + lxml  + django                         no

uwsgi + django                                 yes

Answer 1

I can't answer as to what exactly caused the error you had, but if your question is why the presence of lxml is affecting your output, then it is because openpyxl uses lxml when it is installed, but the standard library's xml package when it is not installed.

From the openpyxl docs :

There is support for the popular lxml library which will be used if it is installed. This is particular useful when creating large files.

Here is where you can see in the openpyxl source that it checks whether lxml is installed or not before deciding which function to call.