Importing python libraries from Github

Question

I have written some libraries in Python for use in my project. I have stored them locally on my system and also remotely on Github. Now every time I write some code I use sys.path.append() in the beginning to help import my libraries from the directory in my system. I was wondering that if there is anyway to import these files directly from my Github repository

The link to my repo is this - Quacpy

Answer 1

If you want to use a repo which has to be installed, I'm not sure how you would want to automate installation inside another python script (also what to do if the installation fails).

However, if you just want to use some methods from another file, you could download that file and then import it:

import urllib2

def download(url):
    filename = url.split('/')[-1]
    print 'Downloading', filename
    f = urllib2.urlopen(url)
    data = f.read()
    f.close()
    with open(filename, 'w') as myfile:
        myfile.write(data)

# get repository
download('https://raw.githubusercontent.com/biryani/Quacpy/master/auxfun.py')

# try to import something from it
from auxfun import qregnorm
q = qregnorm([0, 1, 2])
print 'Success! q =', q

Maybe you could even download the whole zip, unzip it and then import the files.

Answer 2

Assuming you have a valid setup.py file, pip supports git-based installation. See https://pip.pypa.io/en/latest/reference/pip_install.html#git for details

Spoiler: Because you don't have a setup.py file, you'll see the following error if you try using pip currently:

pip install -e git+https://github.com/biryani/Quacpy.git#egg=quacpy
Obtaining quacpy from git+https://github.com/biryani/Quacpy.git#egg=quacpy
  Cloning https://github.com/biryani/Quacpy.git to /.../quacpy
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 18, in <module>
    IOError: [Errno 2] No such file or directory: '/.../quacpy/setup.py'

    ----------------------------------------
    Command "python setup.py egg_info" failed with error code 1 in /.../quacpy

Answer 3

This imports the whole repository as a module, Python 3:

import sys
import urllib.request # python 3
import zipfile
import os

REPOSITORY_ZIP_URL = 'https://github.com/biryani/Quacpy/archive/master.zip'

filename, headers = urllib.request.urlretrieve(REPOSITORY_ZIP_URL)

zip = zipfile.ZipFile(filename)

directory = filename + '_dir'

zip.extractall(directory)

module_directory_from_zip = os.listdir(directory)[0]
module_directory = 'Quacpy'
os.rename(os.path.join(directory, module_directory_from_zip),
          os.path.join(directory, module_directory))

sys.path.append(directory)

import Quacpy

Answer 4

This feels a bit off the wall but might work for you (if any of your libraries depend on each other, you'll have to change those imports to githubimports too!?):

import requests
def githubimport(user, repo, module):
   d = {}
   url = 'https://raw.githubusercontent.com/{}/{}/master/{}.py'.format(user, repo, module)
   r = requests.get(url).text
   exec(r, d)
   return d

qoperator = githubimport('biryani', 'Quacpy', 'qoperator')

Answer 5

One workaround I use in Colab is copying the file to Colab's folder and then importing it in Python's interpreter. With your library this would be:

!wget -q https://github.com/biryani/Quacpy/__init__.py -O __init__.py > txt.log
from __init__ import *

There should be a way to download the whole repo, unzip if necessary, and then import.