GIT > 20k new files chokes VSCode UI, "Too many active changes"

Question

I am trying to add a folder that has to be tracked, however it is > 1GB / > 20k files.

As soon I add it, I will get warning about too many active changes and Git simply will refuse to work correctly (on VS Code after the warning I cannot commit anything is it says there is no changes (yet left side button shows that there are changes)).

Tried to use command line but simply doesn't push new folder :/

Any advice how to deal with lots of new files ? I will have ~ 5-6x folders between 1GB - 5GB and from 20k - 200k filer per folder.

Answer 1

I've ran into similar problems with VS Code. If you are currently stuck where VS Code git extension keep repeating this message, put the offending folder in .gitignore then delete the git object cache for it.

git rm -r --cached path/to/folder

A good strategy for large amount of files in VS Code is to compartmentalize each folder and commit separately as stated by Lasse.

Something like in .gitignore structure of splitting the folder like this

folder/folderA
folder/folderB
folder/folderC

Remove one at a time and commit, and be as granular as required. Rinse and repeat. And if you have phpstorm use that instead for these large commits.

Answer 2

Should you really be pushing this many files to a git repo? Adding multiple gigabytes of files to a git repo is considered a bad practice.

For example Github.com requests that git repos be less than 1GB

Consider the following approaches

Git LFS

https://git-lfs.github.com/
Git LFS is a plugin that is designed to store binary files. They are managed with git, but are stored externally. This is considered the best approach for when you must have large files in a git repo.

Dependency Resolver

If the files that are being added to the git repo are external dependencies, consider looking at a dependency resolver to fetch the dependencies on demand.

Javascript/Node - npm
Ruby - bundler