How do I restore all deleted files in a particular directory of a git repo?

Question

We would like to recover deleted files in a particular directory, and all of its subdirectories, of a git repo.

The directory contains our database migrations. The names and contents can change (renames, deletions, additions), and have done so over the last 5 or years.

We realise now that we haven't got a full list of all the migrations, and would like to restore them from the repo.

What we would like to achieve is a restore of all the files to their last form, prior to deletion.

As I understand it, this may not be particularly effect for all renamed files, but we can sort that out manually.

Each file's name has 2 parts, a timestamp and a migration name - the name remains the same, but the timestamp part changes during the rename.

For example: 20150101010101_inital_migration.sql could have been renamed to 20150101010102_inital_migration.sql.

The content MAY have changed slightly, but the 'initial_migration' part of the name would not have changed. Not sure if this is useful in any way, but would be how we would detect the rename if there was no way to do it within git itself.

If the file change was so different that it was recorded as an actual deletion and a new file, then the file prior to the deletion will do. We can go through the files afterwards once all the final forms of the files are present.

EDIT 1 - Got a list of deleted files but can't process them.

So I've got SOME way further on : git log --diff-filter=D --summary migrations/

Will show all the files that have been deleted (10,190 ... ouch!!).

Running : git log --diff-filter=D --summary migrations/ | grep 'delete mode' | grep -o migr.* git log --diff-filter=D --summary migrations/ | grep 'delete mode' | grep -o migr.* git log --diff-filter=D --summary migrations/ | grep 'delete mode' | grep -o migr.* will show me all the files that have been deleted.

But attempting to run that via xargs (Slow? Don't care! This is a one off) : git log --diff-filter=D --summary migrations/ | grep 'delete mode' | grep -o migr.* | xargs -I {} git checkout $commit~1 "{}" git log --diff-filter=D --summary migrations/ | grep 'delete mode' | grep -o migr.* | xargs -I {} git checkout $commit~1 "{}" git log --diff-filter=D --summary migrations/ | grep 'delete mode' | grep -o migr.* | xargs -I {} git checkout $commit~1 "{}" ...

$ git log --diff-filter=D --summary migrations/ | grep 'delete mode' | grep -o migr.* | xargs -I {} git checkout $commit~1 "{}"
error: pathspec '~1' did not match any file(s) known to git
error: pathspec 'migrations/20190727100901_in651_fix_item_id_on_order_items_table.php' did not match any file(s) known to git
error: pathspec '~1' did not match any file(s) known to git
error: pathspec 'migrations/20190130101201_bac1005_add_paysera_payment_gateway_driver.php' did not match any file(s) known to git
error: pathspec '~1' did not match any file(s) known to git
error: pathspec 'migrations/20190210174730_truncate_phinx_log_20190210174730.php' did not match any file(s) known to git
error: pathspec '~1' did not match any file(s) known to git
error: pathspec 'migrations/20190212110201_fix_end_times_in_sessions.php' did not match any file(s) known to git
error: pathspec '~1' did not match any file(s) known to git
error: pathspec 'migrations/20190212171608_bac1611_delete_orders.php' did not match any file(s) known to git
error: pathspec '~1' did not match any file(s) known to git
error: pathspec 'migrations/20190212174401_bac1610_make_contact_pseudonymisation_not_nullable.php' did not match any file(s) known to git
...

EDIT 2 - The $commit part is supposed to be the commit hash, not a literal $commit . Doh!

EDIT 3 - So, having determined that the hash and the deleted file are presented separately, I've written a small filter to process the output from the git log command and generate git checkout commands.

<?php

$lastHash = null;

while (false !== ($line = fgets(STDIN))) {
    // Is the line a hash?
    if (preg_match('`(?P<hash>[a-f0-9]{10}) `', $line, $matches)){
        $lastHash = $matches['hash'];
    } else {
        preg_match('`(?P<file>migrations/.*$)`', $line, $matches);
        echo sprintf('git checkout %s~1 "%s"', $lastHash, $matches['file']), PHP_EOL;
    }
}

So my final command is git log --diff-filter=D --summary --oneline migrations/ | php ../ugd.php | bash git log --diff-filter=D --summary --oneline migrations/ | php ../ugd.php | bash

Answer 1

As with so many things, there are always other ways to think about the problem.

So, the git log command can show me the commit hash that deleted files from a particular directory.

git log --diff-filter=D --summary --oneline migrations/

Processing this output requires reading 2 different types of line and using values from both lines.

The first line has the hash, the other line has the filename.

So, using a PHP based filter to convert the log output into a list of commands to restore the files was quite simple.

<?php

$lastHash = null;

while (false !== ($line = fgets(STDIN))) {
    // Is the line a hash?
    if (preg_match('`(?P<hash>[a-f0-9]{10}) `', $line, $matches)){
        $lastHash = $matches['hash'];
    } else {
        preg_match('`(?P<file>migrations/.*$)`', $line, $matches);
        echo sprintf('git checkout %s~1 "%s"', $lastHash, $matches['file']), PHP_EOL;
    }
}

This will generate output like

git checkout a653b7f95a~1 "migrations/20140514122900_populate_dimensions.php"
git checkout 75a2989b5b~1 "migrations/20140116173610_drop_xusers_copy.php"
git checkout b0de683071~1 "migrations/20140114163419_initial_migration.php"

And then, running the output of this via bash will restore all the files that have been deleted.