snapcraft gives 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128)

Question

I try to compile Qt 5.13 in a snap package, but I get the following error when priming it:

'NoneType' object has no attribute 'decode'

I thought that this is related to some internationalization issues in the underlying python code, so tried to change the build order, did a lot of other things, but every time we get back the error above.

The following is the snapcraft file:

name: vcs
version: '2.0'
summary: VCS GUI
description: |
  Maritime Robotics Vehicle Control System GUI
confinement: devmode
base: core18
layout:
   /usr/lib/vcs-gui/plugins:
      bind: $SNAP/usr/lib/vcs-gui/plugins

parts:

  desktop-qt5:
    source: https://download.qt.io/archive/qt/5.13/5.13.2/single/qt-everywhere-src-5.13.2.tar.xz
    plugin: dump
    override-build: |
      snapcraftctl build
      ./configure -opensource -confirm-license -debug -nomake examples  -nomake tests
      make
      make install
    override-prime: |
      locale-gen "en_US.UTF-8"
      update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
      snapcraftctl prime
    build-packages:
      - python
      - locales

  protobuf:
    source: https://github.com/protocolbuffers/protobuf/releases/download/v3.8.0/protobuf-all-3.8.0.tar.gz
    plugin: autotools

  cmake:
    source: https://github.com/Kitware/CMake/releases/download/v3.15.0-rc1/cmake-3.15.0-rc1.tar.gz
    plugin: autotools

  libproj:
    source: http://download.osgeo.org/proj/proj-6.0.0.tar.gz
    plugin: autotools
    build-packages: 
      - libsqlite3-dev
      - sqlite3
      - libgl1-mesa-dev
      - libglu1-mesa-dev

  gdal:
    source: https://github.com/OSGeo/gdal/releases/download/v3.0.0/gdal-3.0.0.tar.gz
    plugin: autotools
    after: [libproj]

  ecc:
    source: https://confluence.ecmwf.int/download/attachments/45757960/eccodes-2.12.5-Source.tar.gz
    plugin: cmake
    configflags: [-DENABLE_FORTRAN=OFF]

  vcs:
    plugin: cmake
    configflags: [-DQt5Core_DIR=/usr/local/Qt-5.13.2/lib/cmake/Qt5Core,-DQt5_DIR=/usr/local/Qt-5.13.2/lib/cmake/Qt5,-DQT_QMAKE_EXECUTABLE=/usr/local/Qt-5.13.2/bin/qmake]
    source: https://github.com/fritzone/Qt-CMake-HelloWorld.git
    after: [protobuf,cmake,libproj,gdal,ecc,desktop-qt5]
    build-packages:
      - git
      - g++
      - make
      - libkml-dev
      - libarmadillo-dev
      - libgeographic-dev
      - libssl-dev
      - libconfig++-dev
      - libxml2-dev
      - libmodbus-dev
      - libev-dev
      - libudev-dev
      - libexiv2-dev
      - libv4l-dev
      - doxygen
      - graphviz
      - libgeotiff-dev
      - libgeos-dev 
      - libpng-dev
      - libbotan-2-dev
    stage-packages:
      - libbotan-2-4
      - libtspi1
      - libkmlconvenience1
      - libkmlbase1
      - libkmlengine1
      - libkmldom1
      - libssl1.0.0
      - libconfig++9v5
      - libxml2
      - libmodbus5
      - libev4
      - libaec0
      - libhdf4-0-alt
      - libsz2
      - libexiv2-14
      - libv4l-0
      - libgeotiff2
      - libsdl2-2.0-0
      - libxcb-xinerama0
      - libarmadillo8
      - libarpack2
      - libsuperlu5
      - libgeos-3.6.2
      - libgeos-c1v5

apps:
  vcs:
    command: bin/desktop-launch bin/vcs
    adapter: full
    command-chain: 
      - bin/desktop-launch
      - bin/vcs
    common-id: vcs-gui.desktop
    desktop: usr/share/applications/vcs-gui.desktop
    environment:
      "DISABLE_WAYLAND": "0"
    plugs: [x11, wayland, desktop, desktop-legacy, opengl, network, home]

Here is the output I get from it when I run with:

SNAPCRAFT_ENABLE_DEVELOPER_DEBUG=yes SNAPCRAFT_BUILD_ENVIRONMENT_MEMORY=8G snapcraft --debug

(Yes, it needs circa 8GB of memory due to the compiler crashing while building Qt)

.
. // everything is fine till this point
.
Priming desktop-qt5 
Generating locales (this might take a while)...
  en_US.UTF-8... done
Generation complete.
'NoneType' object has no attribute 'decode'
We would appreciate it if you anonymously reported this issue.
No other data than the traceback and the version of snapcraft in use will be sent.

And here is the stacktrace of the python script which tries to build the package:

'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128)
We would appreciate it if you anonymously reported this issue.
No other data than the traceback and the version of snapcraft in use will be sent.
Would you like to send this error data? (Yes/No/Always/View) [no]: View
Traceback (most recent call last):
  File "/snap/snapcraft/3440/bin/snapcraft", line 11, in <module>
    load_entry_point('snapcraft==3.8', 'console_scripts', 'snapcraft')()
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/click/core.py", line 1114, in invoke
    return Command.invoke(self, ctx)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/cli/_runner.py", line 103, in run
    snap_command.invoke(ctx)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/cli/_command.py", line 87, in invoke
    return super().invoke(ctx)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/cli/lifecycle.py", line 261, in snap
    _execute(steps.PRIME, parts=[], pack_project=True, output=output, **kwargs)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/cli/lifecycle.py", line 66, in _execute
    lifecycle.execute(step, project_config, parts)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_runner.py", line 94, in execute
    executor.run(step, part_names)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_runner.py", line 148, in run
    self._handle_step(part_names, part, step, current_step, cli_config)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_runner.py", line 162, in _handle_step
    getattr(self, "_run_{}".format(current_step.name))(part)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_runner.py", line 237, in _run_prime
    self._run_step(step=steps.PRIME, part=part, progress="Priming")
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_runner.py", line 281, in _run_step
    getattr(part, step.name)()
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 795, in prime
    self._do_runner_step(steps.PRIME)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 242, in _do_runner_step
    return getattr(self._runner, "{}".format(step.name))()
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/_runner.py", line 91, in prime
    "override-prime", self._override_prime_scriptlet, self._primedir
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/_runner.py", line 137, in _run_scriptlet
    scriptlet_name, function_call.strip()
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/_runner.py", line 193, in _handle_builtin_function
    function(**function_args)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 807, in _do_prime
    dependency_paths = self._handle_elf(snap_files)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 814, in _handle_elf
    elf_files = elf.get_elf_files(self.primedir, snap_files)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/elf.py", line 576, in get_elf_files
    elf_file = ElfFile(path=path)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/elf.py", line 219, in __init__
    elf_data = self._extract(path)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/snapcraft/internal/elf.py", line 252, in _extract
    interp_section = elf.get_section_by_name(_INTERP)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/elftools/elf/elffile.py", line 94, in get_section_by_name
    for i, sec in enumerate(self.iter_sections()):
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/elftools/elf/elffile.py", line 103, in iter_sections
    yield self.get_section(i)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/elftools/elf/elffile.py", line 83, in get_section
    return self._make_section(section_header)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/elftools/elf/elffile.py", line 288, in _make_section
    name = self._get_section_name(section_header)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/elftools/elf/elffile.py", line 283, in _get_section_name
    return self._file_stringtable_section.get_string(name_offset)
  File "/snap/snapcraft/3440/lib/python3.5/site-packages/elftools/elf/sections.py", line 70, in get_string
    return s.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128)
We would appreciate it if you anonymously reported this issue.

Does anyone have any ideas on how to make it work? Please note, using the Qt which comes with default base core18 (5.9) is not good for our application due to some bugs that were fixed in 5.13, so we need at least 5.13.

The application this tries to build is just a helloworld app, it's not relevant to the error we get, but it's required by snapcraft to reach the stage of the error.

(Be careful if you try to run snapcraft against this file, Qt takes 4 - 5 hours on my computer to compile)

Answer 1

In the traceback, a get_string function is raising a UnicodeDecodeError because it can't decode some text from ASCII.

In the current source for elftools, this line has been replaced by

return s.decode('UTF-8') is else ''

So you could try upgrading the version of pyelftools bundled with snap

<the python interpreter bundled with snap> -m pip install --upgrade pyelftools 

(ensure you can roll back, in case this upgrade actually makes things worse)