Mercurial: Permanently remove sensitive data from HG repo?

Question

I know that in Mercurial, "history is sacred".

But let's say someone accidentally commits something they shouldn't, like a settings file containing a password or something. Let's even say some time passes before anyone realizes it, so its been hanging around for several commits. Obviously, the discoverer then removes the sensitive data from the repository.

Is there any way to permanently scrub that file or the sensitive data from the commit history, as though it never existed? Or would that sensitive data just be a permanent part of the repo forever and ever?

Answer 1

There are a few methods to accomplish this. All of them require the cooperation of everybody who has cloned your repository or pulled changesets from it after the change was introduced.

Which method to use depends on the exact nature of the committed data and where it is in the history. All of them require the use of Mercurial extensions and cannot be accomplished with core Mercurial. Luckily, all the required extensions are shipped by default with Mercurial and simply have to be enabled.

I'm not going into detail about the methods here as there are several answers that give different methods in the question this is a dupe of. I just want to be clear that the accepted answer in that question is technically correct, but not useful. It is actually possible.

Answer 2

I haven't investigated the details of how hooks work so this idea may not fully play out. It may be possible to set up hooks to disallow the committing, pushing and pulling of your sensitive files. There are hooks that will run before committing or pushing ( precommit and preoutgoing ). Protecting the hooks so they are not circumvented is another issue that Mercurial: The Definitive Guide also discusses.

Answer 3

No. Not really. If you can convince everyone who had access to delete and reclone, you can remove the file from future access.

But if you, eg, pushed your root password to a public Bitbucket repo - ? You should change it. Your information is now public and leaked and should be considered as such. Sorry.

Answer 4

If, and only if, this repository hasn't escaped into the wild , you can remove a file from history by essentially cloning to a new repository while filtering the sensitive file in the process, using hg convert Hg Convert Extension doc here

Commonly, we find something when we audit the repository prior to publishing or delivering to a client, such as a web.config or ini file with a password.

The extension isn't enabled by default, but is included with all clients I use, you need to enable it before Mercurial will recognize the convert command.

If using Tortoise Hg or Kiln, for example:

1. Open Tortoise Hg -> Global Settings -> Extensions
2. Check the box beside "Convert"
3. Click Ok

Or edit Mercurial.ini directly:

[extensions]
convert = 

Go to the directory above your repository (in my example, my repos is HelloApp):

1. Create a file named filemap.txt

2. Add a line with the full path to the filename you want to exclude.

    exclude HelloApp/sensitive.config 

3. Open a command prompt, cd to the same directory, containing your filemap.txt, and run the hg convert

    cd C:\\projects hg convert --filemap filemap.txt HelloApp HelloApp_clean 

4. Then get latest working copy:

    cd HelloApp_clean hg update 

You will need to create a fresh clone on your server with your clean copy.