Unable to Install Sndfile with Amazon EC2 Instance due to unavailability

Question

I am trying to deploy a flask application that uses TensorFlow and Librosa. So as a dependency of Librosa I need the sndfile package.

When I run my Flask app I get this error:

import librosa
  File "/home/ec2-user/.local/lib/

python3.7/site-packages/librosa/__init__.py", line 211, in <module>
    from . import core
  File "/home/ec2-user/.local/lib/python3.7/site-packages/librosa/core/__init__.py", line 6, in <module>
    from .audio import *  # pylint: disable=wildcard-import
  File "/home/ec2-user/.local/lib/python3.7/site-packages/librosa/core/audio.py", line 8, in <module>
    import soundfile as sf
  File "/home/ec2-user/.local/lib/python3.7/site-packages/soundfile.py", line 142, in <module>
    raise OSError('sndfile library not found')
OSError: sndfile library not found
[ec2-user@ip-172-31-90-51 CE_Ayush_Deployment]$ python3 app.py
Traceback (most recent call last):
  File "app.py", line 2, in <module>
    from model_pipeline import Corona_Diagnoser
  File "/home/ec2-user/CE_Ayush_Deployment/model_pipeline.py", line 5, in <module>
    import librosa
  File "/home/ec2-user/.local/lib/python3.7/site-packages/librosa/__init__.py", line 211, in <module>
    from . import core
  File "/home/ec2-user/.local/lib/python3.7/site-packages/librosa/core/__init__.py", line 6, in <module>
    from .audio import *  # pylint: disable=wildcard-import
  File "/home/ec2-user/.local/lib/python3.7/site-packages/librosa/core/audio.py", line 8, in <module>
    import soundfile as sf
  File "/home/ec2-user/.local/lib/python3.7/site-packages/soundfile.py", line 142, in <module>
    raise OSError('sndfile library not found')
OSError: sndfile library not found

I tried:

sudo yum install autoconf autogen automake build-essential libasound2-dev \
   libflac-dev libogg-dev libtool libvorbis-dev libopus-dev libmp3lame-dev \
   libmpg123-dev pkg-config python

I then get this message:

Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
No package build-essential available.
No package libasound2-dev available.
No package libflac-dev available.
No package libogg-dev available.
No package libvorbis-dev available.
No package libopus-dev available.
No package libmp3lame-dev available.
No package libmpg123-dev available.
No package pkg-config available.

I know the package is not available in the Amazon EC2 instance. Could anyone guide me on how to install the sndfile packages to the system?

Answer 1

Amazon Linux 2

I am on Amazon Linux 2 , which you can verify by running the following on your EC2 server.

[ec2-user@fake-ip ~]$ cat /etc/system-release
Amazon Linux release 2 (Karoo)

The following packages are available from the Amazon package repository : libsndfile-devel , libsndfile-utils , libsndfile .

 [ec2-user@fakeip ~]$ yum search libsndfile
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
=============================================== N/S matched: libsndfile ================================================
libsndfile-devel.x86_64 : Development files for libsndfile
libsndfile-utils.x86_64 : Command Line Utilities for libsndfile
libsndfile.x86_64 : Library for reading and writing sound files
libsndfile.i686 : Library for reading and writing sound files

Amazon Linux AMI

According to Amazon Linux AMI package list , this earlier version of Amazon Linux does not have libsndfile. Although you can enable the Extra Packages for Enterprise Linux (EPEL), libsndfile does not appear to be there either.

[ec2-user@fakeip ~]$ cat /etc/system-release
Amazon Linux AMI release 2018.03
[ec2-user@fakeip ~]$ sudo yum-config-manager --enable epel
... skipping lots of output
mirrorlist = https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=x86_64
...
[ec2-user@fakeip ~]$ yum search libsndfile
Loaded plugins: priorities, update-motd, upgrade-helper
epel/x86_64/metalink                         | 3.8 kB  00:00:00
epel                                         | 4.7 kB  00:00:00
(1/3): epel/x86_64/group_gz                  |  74 kB  00:00:00
(2/3): epel/x86_64/updateinfo                | 788 kB  00:00:00
(3/3): epel/x86_64/primary_db                | 6.1 MB  00:00:00
1067 packages excluded due to repository priority protections
Warning: No matches found for: libsndfile
No matches found

The libsndfile docs state that it has been compile on Debian/GNU Linux , but Amazon Linux AMI is not based on Debian. The build dependencies for libsndfile ( autoconf autogen automake build-essential libasound2-dev libflac-dev libogg-dev libtool libvorbis-dev libopus-dev libmp3lame-dev libmpg123-dev pkg-config python ) all refer to Debian packages, which do not have the same name for other Linux flavors. In particular, build-essential , libasound2-dev , libflac-dev , libopus-dev , libmp3lame-dev and libmpg123-dev are not on EPEL.

[ec2-user@fakeip ~]$ yum search autoconf autogen automake build-essential \
    libasound2-dev   libflac-dev libogg-dev libtool libvorbis-dev libopus-dev \
    libmp3lame-dev   libmpg123-dev pkg-config python
... skipping output
Warning: No matches found for: build-essential
Warning: No matches found for: libasound2-dev
Warning: No matches found for: libflac-dev
Warning: No matches found for: libopus-dev
Warning: No matches found for: libmp3lame-dev
Warning: No matches found for: libmpg123-dev

libmpelame-dev may be lame-devel , lame-libs , lame , and/or lame-mp3x .

[ec2-user@fakeip ~]$ yum search lame
Loaded plugins: priorities, update-motd, upgrade-helper
1067 packages excluded due to repository priority protections
================================================== N/S matched: lame ===================================================
lame-devel.i686 : Development files for lame
lame-devel.x86_64 : Development files for lame
lame-libs.i686 : LAME MP3 encoding library
lame-libs.x86_64 : LAME MP3 encoding library
flamerobin.x86_64 : Graphical client for Firebird
lame.x86_64 : Free MP3 audio compressor
lame-mp3x.x86_64 : MP3 frame analyzer

libflac-dev may be flac-devel

[ec2-user@fakeip ~]$ yum search flac
Loaded plugins: priorities, update-motd, upgrade-helper
1067 packages excluded due to repository priority protections
================================================== N/S matched: flac ===================================================
flac-devel.x86_64 : Development libraries and header files from FLAC
easytag.x86_64 : Tag editor for mp3, ogg, flac and other music files
flac.i686 : An encoder/decoder for the Free Lossless Audio Codec
flac.x86_64 : An encoder/decoder for the Free Lossless Audio Codec
mp3fs.x86_64 : FUSE filesystem to transcode FLAC to MP3 on the fly

libopus-dev may be opus-devel

[ec2-user@fakeip ~]$ yum search opus
Loaded plugins: priorities, update-motd, upgrade-helper
1067 packages excluded due to repository priority protections
================================================== N/S matched: opus ===================================================
opus-devel.i686 : Development package for opus
opus-devel.x86_64 : Development package for opus
opus.i686 : An audio codec for use in low-delay speech and audio communication
opus.x86_64 : An audio codec for use in low-delay speech and audio communication

libmpg123-dev is https://www.mpg123.org/

Recommendations

• Switch your EC2 to Amazon Linux 2 or Ubuntu
• Run your app in a Docker image on the EC2

Answer 2

You need to install libsndfile and it's development files. Unfortunately it's not packaged up in a repository, but you can get it installed by downloading the RPMs directly

wget http://ftp.altlinux.org/pub/distributions/ALTLinux/Sisyphus/x86_64/RPMS.classic//libsndfile-1.0.31-alt1.x86_64.rpm
wget http://ftp.altlinux.org/pub/distributions/ALTLinux/Sisyphus/x86_64/RPMS.classic//libsndfile-devel-1.0.28-alt1.x86_64.rpm

sudo yum localinstall libsndfile-devel-1.0.31-alt1.x86_64.rpm

You can also install libsndfile from source.

Another option is to switch to Ubuntu, which has packages for libsndfile and libsndfile-dev . If you're looking to deploy tensorflow specifically then you should start with the AWS Deep Learning AMI . These AMIs are built by AWS and come with the drivers and a bunch of frameworks already installed. These AMis have Ubuntu flavored versions.