Pass file list to Mercurial 'hg add' command from text file

Question

I'm attempting to add about 40k files to Mercurial using 'hg add' on Win 10. Due to downstream constraints I need to split this up into chunks of 10k files for each add/commit, therefore I need to pass large lists of files to hg add. However I'm hitting Window's commandline argument character limits .

Is there a way to pass file lists stored in a file to hg add? AFAIK can't just use pipes as this is just reading the file and passing it as arguments. I think hg would need to directly read the text file.

Answer 1

You can solve this by using Mercurial's extensive pattern functionality and possibly file sets (see hg help patterns and hg help filesets ).

One type of pattern that you can specify is the listfile: pattern, which takes a filename as an argument, which must contain a newline-separated list of patterns (which of course can be filenames).

So, you would first produce lists of the files in question, split up across several files. Let's assume they're called files1.txt , files2.txt , ...

Then you'd do the following:

hg add listfile:files1.txt
hg commit
hg add listfile:files2.txt
hg commit
...

If that by itself isn't sufficient, you can use file sets to further modify what files get added.

Answer 2

(This is not a direct answer to how to get hg to read from a list of files - but an alternative idea).

My thought is to use hg addremove instead of add since it already is designed to work in "batch".

From hg addremove --help :

hg addremove [OPTION]... [FILE]...

add all new files, delete all missing files

...

options ([+] can be repeated):

 -I --include PATTERN [+] include names matching the given patterns... -n --dry-run do not perform actions, just print output

(some details omitted)

So a command like the following might work:

hg addremove --include path\to\files\prefix*.*

(The question doesn't have any specifics about filenames or paths to know more specifically how to structure the include details.)

This only marks the files for addition; it doesn't automatically commit. So you would be able to run this multiple times, varying the inclusion criteria, committing each time in multiple batches.

Also note that hg commit accepts a similar --include option. So alternately you could addremove 100% of your files in one go, and then issue multiple commits for only a portion of all the files each. For instance, just commit one directory at a time, or some other scheme that is easy to do and get under the 10K file limit.