git lfs push to github failure on Ubuntu 18.04

Question

I have a local git repository where sync'ed with remote GitHub repository. On a feature branch, I needed to add/commit an large binary.pt file (236Mb) and then push it to remote origin in GitHub. Initially I added the file normally (git add), committed the file (git commit)and then tried to push (git push). The push to GitHub failed due to size of file and suggesting to use git-lfs. Following this error, my colleague pushed a.gitattributes file to remote master branch on GitHub with this content:

*.pt filter=lfs diff=lfs merge=lfs -text

I then rebased my feature branch with master to get this file into my feature branch. I installed git-lfs using sudo apt install git-lfs Then I tried to unstage the file using

$ git rm --cached bigFile.pt
rm 'bigFile.pt'

by running git status I get

$ git status
On branch feature_branch
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    deleted:    bigFile.pt

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    bigFile.pt

I then tried to add it, commit and push it which failed again with similar push failure error message due to big size. I then googled and then found this command ( git-lfs.github.com ) and ran it in the folder inside branch:

$git lfs install
Updated git hooks.
Git LFS initialized.

I then followed the instruction and ran:

git lfs track "*.pt"

which seems to update.gitattributes which already updated by rebase from master branch.

I then tried to unstage the file using:

git rm --cached bigFile.pt

The run the git lfs ls-files

$ git lfs ls-files
e7e59a98b5 - bigFile.pt

Then:

$ git lfs status
On branch feature_branch

Objects to be committed:

    bigFile.pt (LFS: e7e59a9 -> File: e7e59a9)

Objects not staged for commit:

I then added the file:

$ git add bigFile.pt

And then checked:

$ git lfs status
On branch feature_branch

Objects to be committed:


Objects not staged for commit:

$ git lfs ls-files
e7e59a98b5 - bigFile.pt

$ git status
On branch feature_branch
nothing to commit, working tree clean

But at the end push fails as follows?::

$ git push --set-upstream origin feature_branch
Uploading LFS objects: 100% (1/1), 247 MB | 0 B/s, done.                                                                                                                                                    
Counting objects: 16, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (14/14), done.
Writing objects: 100% (16/16), 218.00 MiB | 1.08 MiB/s, done.
Total 16 (delta 9), reused 0 (delta 0)
remote: Resolving deltas: 100% (9/9), completed with 6 local objects.
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
remote: error: Trace: 259f0ded862988234bfa0bf1094ba2da8a3b59f4a9a0ea91f8c48ef330598dc4
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File bigFile.pt is 235.15 MB; this exceeds GitHub's file size limit of 100.00 MB
To github.com:reporName.git
 ! [remote rejected] feature_branch -> feature_branch (pre-receive hook declined)
error: failed to push some refs to 'git@github.com:repoName.git'

BTW, this repository has been added as git submodule to another repository. I am running out of ideas to test and verify here? Any help how to verify and resolve this issue so that I can push this branch is appreciated?

Thanks

Answer 1

It looks like you made some commits that have the big file, then you added some more commits in which you've replaced the big file with the LFS-indirection file. What this means is that you need to remove the commits that have the big files.

Remember that when you use Git-LFS, you're using two external web sites to store your data. First, you send the big files to some LFS storage site. These big files are not (yet) committed. Then, with the real files safely stored elsewhere, you create and push commits that store only the information needed to retrieve the big files from the LFS storage site. So the commits , in the form of a Git repository, exist somewhere (eg, on GitHub) but don't have any big files in any of them. Instead, they have small files that have instructions that say don't use this file, use a big file you get with the LFS big-file-swapper-replacer-trick, here's how to get the big file . The LFS wrappers around Git intercept Git's attempt to put the small replacement file into your working tree, and put the big file there instead.

(Note that this means that anyone who clones your repository with regular Git gets, and sees, these weird small indirection files. They must set up Git-LFS themselves so that they can have the same wrapper program intercept the attempt to put the small files out for viewing or editing.)

But if you do this in the wrong order , you first commit the big files . Then you send the big files to a second site , then you remove the big files and put in the small "here's how to get the real file" files, and make another commit that contains these small files. You still have the commit with the big files in it!

Git is designed to add new commits , and not to take old commits away . This makes it very hard to remove any commit that has a big file in it. To do that automatically requires specialized tools. Fortunately, GitHub already have instructions and tools for you, to tell you how to do this. See them here . The short version is to use git lfs migrate , if you have a modern Git-LFS. See also this other GitHub page .