bundler: fetch partial git repo?

Question

I have a Rails project that contains a my_gem folder at root:

my_repo
├── app
├── config
├── Gemfile
├── Gemfile.lock
├── db
├── lib
├── log
├── my_gem <-- the gem folder
├── public
├── Rakefile
├── README.md
├── script
├── test
├── tmp
└── vendor

I use this gem in another project by adding these lines to it's Gemfile:

git 'git@gitlab.mydomain.com:qosenergy/my_repo.git', :branch => 'master' do
  gem 'my_gem'
end

It works perfectly except that all the repo is fetched, that represents hundreds of megabytes (vs less than 1Mo for the only my_gem folder) and this is problematic when I build docker images from the repo that uses the gem: because more than 60% of the image size is filled by the full rails app contained in my_repo .

How could I keep the advantage of this behavior but keep only the my_gem folder instead of the complete repo ?

Answer 1

According to the bundler documentation it seems that you can fetch one or more gems inside a given repository.

Eg The Rails repo contains several gems: actioncable, actionmailer, actionpack ...

Specify that a git repository containing multiple .gemspec files should be treated as a gem source

 git 'https://github.com/rails/rails.git' do gem 'railties' gem 'action_pack' gem 'active_model' end 

If your my_gem contained in the project has a given .gemspec , you should be able to install it without downloading the full application where it is contained. If you have done it this way, you might consider other options.

Git sparse checkout

Git provides the feature of sparse checkout for cloning only a given path inside the repo. You can read this blog for a few use cases:

$ mkdir pcl-examples
$ cd pcl-examples                               #make a directory we want to copy folders to
$ git init                                      #initialize the empty local repo
$ git remote add origin -f https://github.com/PointCloudLibrary/pcl.git     #add the remote origin
$ git config core.sparsecheckout true           #very crucial. this is where we tell git we are checking out specifics
$ echo "examples/*" >> .git/info/sparse-checkout" #recursively checkout examples folder
$ git pull --depth=2 origin master          #go only 2 depths down the examples directory

This answer is probably making this concept even clearer but I did not figure out how to take advantage of it inside a Gemfile .

Other debugging

Since you are building a Docker image, you can refer to this checklist about rails errors. However it seems that you are not experiencing any error, but only an image size issue.

A couple of clean solution

I guess the cleanest solution to your issue would be to extract the gem code inside a new repository and install it as you were trying to do. I would personally go for this solution.

If you really don't want to take it out, you could give a try to submodules inside your original project. This approach can be very tricky and even Github is not really encouraging it.