Bitbucket git repo has a size limit of 2GB, and now I have one repo ( let's call it bigsize repo) that is already dangerously close to that limit, due to a lot of binary files ( files with extensions of dll
and msm
). It's so close to the limit that I'm scared anymore commit operation involving binary files will tip the size to over 2GB, and hence the commit will fail and unable to proceed.
Now, how to best go by to reduce the bigsize repo size?
I'm thinking about using the LFS feature , but then, there is a 1GB limit on the LFS space, which I afraid will not be sufficient for the usage of this repo ( as the majority of the repo size is coming from the binary files that I want to store in LFS).
So I'm thinking about just removing all the binary files ( I don't mind losing them from source control, as I have them on my local drive) from the repo and the history, how to best do this, considering the current size of my bigsize repo?
The attack plan that I have:
*.dll
in gitignore and use git rm -rf -cached
command ). This is needed because " By default the BFG doesn't modify the contents of your latest commit on your master (or 'HEAD') branch, even though it will clean all the commits before it. " Does the approach work for a repo that is very closed to 2GB? I afraid that at step 1, when I use git rm
, it will add to the history and push the repo size to over 2GB, and hence fail.
Important details:
git filter-branch
or BFG are obsolete
With Git 2.22 or more, use git filter-repo
:
git filter-repo --path your/big/file --invert-path
Or:
git filter-repo --strip-blobs-bigger-than 10M
Here's how I solve the problem using BFG repo cleaner . I write it down so that it will be helpful to those in the future who might face the same situation as I do.
Before one even gets started, make sure that all your branches are "clear", ie: all changes that you desired are pushed into their respective branch. Let's name our gigantic repo as big repo, and assume that it is located at D:\MyCompany\big
.
Also you might want to try this on a toy repo before actually working on the production repo.
git clone --mirror git@bitbucket.org:YourUserName/big.git
), make sure that this new folder is outside of your original big repo folder. You need to be very sure that this is a completely separate thing from D:\MyCompany\big
( and hence you can experiment on it without fear). Assuming that this folder is D:\big_work
, you will be able to find D:\big_work\big.git
folder after doing the cloning.big.git
folder to another place, let's put it in D:\backup
. determine that your binary files are really out of the LFS limit provided by bitbucket ( or your repo provider). In my case here, I need to ascertain that my binary file size is less than 1GB. To do this, convert the git of big.git
to LFS via BFG. At command line:
3.1. cd D:\big_work
3.2. java -jar <path to>bfg-xxxjar --convert-to-git-lfs "*.{dll, msm}" --no-blob-protection big.git
( make sure that you do this while you are inside the D:\big_work
folder)
Now check the size of D:\big_work\big.git\lfs
folder, is it more than 1GB? If no, you can proceed to convert the binary files to lfs
by continuing from step 4 here . No need to read the rest.
lfs
folder is more than 1GB, then you have to delete binary files from your repo and rewrite the history.big.git
folder from D:big_work
, and restore it from D:\backup
( the backup that you did in step 2). Here comes the most important step: the actual deletion . A lot of guides talk about why it's important to first make sure that the commits are clean (meaning, there are no more binary files in current state of repo) before carrying out this step. But I do find advice such as this unnecessary and distracting. Since we already know very well what we want-- namely, remove all of the binary files from our repo, present and past, from all branches, merged or unmerged, and the repo should look like as if it was never contaminated by binary files at any one point of time -- we should just disregard this warning. We bring on our nuclear weapon no-blob-protection
for this purpose.
7.1. cd D:\big_work
7.2. java -jar <path to>bfg-xxxjar --delete-files "*.{dll,msm}" --no-blob-protection big.git
cd D:\big_work\big.git
git reflog expire --expire=now --all && git gc --prune=now --aggressive
git push
D:\MyCompany\big
and you have your sourcetree bookmark pointed to this folder, so you will need to first remove the sourcetree bookmark, and rename the D:\MyCompany\big
to D:\MyCompany\big_old
. Don't delete big_old
folder yet because even though you don't want the binary files to locate inside the repo, but you still want them to locate on your hard disk, and big_old
folder provides a natural backup. You can delete big_old
folder some point in the future, after you are sure that it is really of no value.D:\MyCompany\big
folder to avoid any breaking changes.D:\MyCompany\big
folder.big_old
to big
folder on need basis, from time to time.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.