从 Protonmail 导入邮件到 Mailcow

Question

I'm attempting to import emails from Protonmail to Mailcow.

Mailcow, of course, has the sync jobs functionality so my first approach was to create one such job for pulling emails from ProtonMail Bridge. The Bridge runs locally at 127.0.0.1 with port 1143 for IMAP and port 1025 for SMTP. I tried exposing it with Ngrok, by typing the following:

./ngrok http https://127.0.0.1:1143

Then, opening a sync job in Mailcow's UI (Mail Setup --> Sync jobs) with the URL provided by Ngrok and 1143 as port got me this:

Command line used, run by /usr/bin/perl:
/usr/local/bin/imapsync --tmpdir /tmp --nofoldersizes --exclude (?i)spam|(?i)junk --subfolder2 External --delete2duplicates --subscribeall --automap --tls1 --host1 https://sdfsdf45.ngrok.io --user1 info@email.net --passfile1 /tmp/imapsync.XZ1EqMy --port1 1143 --host2 localhost --user2 info@email.net*my-password@mailcow.local --passfile2 /tmp/imapsync.yn2Arhj --no-modulesversion --noreleasecheck
Temp directory is /tmp ( to change it use --tmpdir dirpath )
kill -QUIT 27012 # special behavior
kill -TERM 27012 # special behavior
kill -INT 27012 # special behavior
kill -HUP 27012 # special behavior
No variable pid_filename
PID file is unset ( to set it, use --pidfile filepath ; to avoid it use --pidfile "" )
Info: will resync flags for already transferred messages. Use --noresyncflags to not resync flags.
Host2: probing ssl on port 993 ( use --nosslcheck to avoid this ssl probe ) 
Host2: sslcheck detected open ssl port 993 so turning ssl on (use --nossl2 --notls2 to turn off SSL and TLS wizardry)
SSL debug mode level is --debugssl 1 (can be set from 0 meaning no debug to 4 meaning max debug)
Host2: SSL default mode is like --sslargs2 "SSL_verify_mode=0", meaning for host2 SSL_VERIFY_NONE, ie, do not check the certificate server.
Host2: Use --sslargs2 SSL_verify_mode=1 to have SSL_VERIFY_PEER, ie, check the certificate server of host2
Info: will act as --uidexpunge2
Info: turned ON syncinternaldates, will set the internal dates (arrival dates) on host2 same as host1.
Host1: will try to use LOGIN authentication on host1
Host2: will try to use LOGIN authentication on host2
Host1: imap connection timeout is 120 seconds
Host2: imap connection timeout is 120 seconds
Host1: IMAP server [https://sdfsdf45.ngrok.io] port [1143] user [info@email.net]
Host2: IMAP server [localhost] port [993] user [info@email.net*my-password@mailcow.local]
Host1: connecting and login on host1 [https://sdfsdf45.ngrok.io] port [1143] with user [info@email.net]
Host1 failure: can not open imap connection on host1 [https://sdfsdf45.ngrok.io] with user [info@email.net]: Unable to connect to https://sdfsdf45.ngrok.io:  Invalid argument
Exiting with return value 10

I tried several variations for the port, while also trying tls instead of http , all without success:

./ngrok http https://127.0.0.1:1143
./ngrok tls -subdomain=encrypted 1143
./ngrok tls 1143

I also tried accessing the ProtonMail Bridge through the tcp transport layer:

./ngrok tcp 1143

The tcp protocol allowed me to insert a port in the sync job different from that of the ProtonMail Bridge (15926 in the example below, as opposed to 1143 for IMAP):

ngrok by @inconshreveable                                                                      (Ctrl+C to quit)
                                                                                                               
Session Status                online                                                                           
Account                       John Doe (Plan: Pro)                                                         
Version                       2.3.35                                                                           
Region                        United States (us)                                                               
Web Interface                 http://127.0.0.1:4040                                                            
Forwarding                    tcp://0.tcp.ngrok.io:15926 -> localhost:1143                                     
                                                                                                               
Connections                   ttl     opn     rt1     rt5     p50     p90                                      
                              0       0       0.00    0.00    0.00    0.00   

Here's the relevant portion of the sync job's log:

Command line used, run by /usr/bin/perl:
/usr/local/bin/imapsync --tmpdir /tmp --nofoldersizes --exclude (?i)spam|(?i)junk --subfolder2 External --delete2duplicates --subscribeall --automap --host1 tcp://0.tcp.ngrok.io --user1 info@email.net --passfile1 /tmp/imapsync._aWrUsG --port1 15926 --host2 localhost --user2 info@email.net*my-password@mailcow.local --passfile2 /tmp/imapsync.r4EJIwe --no-modulesversion --noreleasecheck
Temp directory is /tmp ( to change it use --tmpdir dirpath )
kill -QUIT 17173 # special behavior
kill -TERM 17173 # special behavior
kill -INT 17173 # special behavior
kill -HUP 17173 # special behavior
No variable pid_filename
PID file is unset ( to set it, use --pidfile filepath ; to avoid it use --pidfile "" )
Info: will resync flags for already transferred messages. Use --noresyncflags to not resync flags.
Host2: probing ssl on port 993 ( use --nosslcheck to avoid this ssl probe ) 
Host2: sslcheck detected open ssl port 993 so turning ssl on (use --nossl2 --notls2 to turn off SSL and TLS wizardry)
SSL debug mode level is --debugssl 1 (can be set from 0 meaning no debug to 4 meaning max debug)
Host2: SSL default mode is like --sslargs2 "SSL_verify_mode=0", meaning for host2 SSL_VERIFY_NONE, ie, do not check the certificate server.
Host2: Use --sslargs2 SSL_verify_mode=1 to have SSL_VERIFY_PEER, ie, check the certificate server of host2
Info: will act as --uidexpunge2
Info: turned ON syncinternaldates, will set the internal dates (arrival dates) on host2 same as host1.
Host1: will try to use LOGIN authentication on host1
Host2: will try to use LOGIN authentication on host2
Host1: imap connection timeout is 120 seconds
Host2: imap connection timeout is 120 seconds
Host1: IMAP server [tcp://0.tcp.ngrok.io] port [15926] user [info@email.net]
Host2: IMAP server [localhost] port [993] user [info@email.net*my-password@mailcow.local]
Host1: connecting and login on host1 [tcp://0.tcp.ngrok.io] port [15926] with user [info@email.net]
Host1 failure: can not open imap connection on host1 [tcp://0.tcp.ngrok.io] with user [info@email.net]: Unable to connect to tcp://0.tcp.ngrok.io:  Invalid argument
Exiting with return value 10

Answer 1

I got around the failure to import emails from ProtonMail (through Bridge) to Mailcow directly (using the sync job functionality, which implements imapsync ) with isync/mbsync installed locally on my computer. There are several steps to this:

1) The first step is to download all your email from ProtonMail by typing mbsync -Va in the command line. You will need a .mbsyncrc file properly configured first. The following configuration worked for me:

IMAPAccount my-name
Host 127.0.0.1
Port 1143
User info@email.net
Pass protonmailbridgepassword
SSLType STARTTLS
Timeout 300

IMAPStore my-name-remote
Account my-name

MaildirStore my-name-local
Path ~/path/to/MAIL/
Inbox ~/path/to/MAIL/Inbox
SubFolders Verbatim
Flatten .

Channel my-name
Master :my-name-remote:
Slave :my-name-local:
Patterns INBOX Sent
Create Both
Expunge Both
SyncState *

You should get something along these lines:

Reading configuration file /home/my-name/.mbsyncrc
C: 0/1  B: 0/0  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Channel my-name
Opening master store my-name-remote...
Resolving 127.0.0.1... ok
Connecting to 127.0.0.1 (127.0.0.1:1143)... 
Opening slave store my-name-local...
Connection is now encrypted
Logging in...
Authenticating with SASL mechanism PLAIN...
C: 0/1  B: 0/2  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box INBOX...
Opening slave box INBOX...
Maildir notice: no UIDVALIDITY, creating new.
Loading master...
Loading slave...
slave: 0 messages, 0 recent
master: 363 messages, 0 recent
Synchronizing...
C: 0/1  B: 1/2  M: +0/0 *0/0 #0/0  S: +363/363 *0/0 #0/0
Error: channel :my-name-remote:Sent-:my-name-local:Sent is locked
C: 1/1  B: 2/2  M: +0/0 *0/0 #0/0  S: +363/363 *0/0 #0/0

2) Notice the line before the last one, which reads Error: channel :my-name-remote:Sent-:my-name-local:Sent is locked . If you get this error, it means that you need to delete the folder /path/to/MAIL/Sent/ and all its contents then type mbsync -Va once again without touching the .mbsyncrc file just yet. Then fire mbsync -Va to get all your sent emails from ProtonMail (you may need to perform this action for each folder that you want to import from ProtonMail). You should get something along these lines now:

Reading configuration file /home/my-name/.mbsyncrc
C: 0/1  B: 0/0  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Channel my-name
Opening master store my-name-remote...
Resolving 127.0.0.1... ok
Connecting to 127.0.0.1 (127.0.0.1:1143)... 
Opening slave store my-name-local...
Connection is now encrypted
Logging in...
Authenticating with SASL mechanism PLAIN...
C: 0/1  B: 0/2  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box INBOX...
Opening slave box INBOX...
Loading master...
Loading slave...
slave: 364 messages, 1 recent
master: 364 messages, 0 recent
Synchronizing...
C: 0/1  B: 1/2  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent...
Opening slave box Sent...
Creating slave Sent...
Maildir notice: no UIDVALIDITY, creating new.
Loading master...
Loading slave...
slave: 0 messages, 0 recent
master: 176 messages, 0 recent
Synchronizing...
C: 1/1  B: 2/2  M: +0/0 *0/0 #0/0  S: +176/176 *0/0 #0/0

3) Now that you got all your old emails from ProtonMail, go ahead and update mbsync's configuration file .mbsyncrc with your Mailcow's credentials:

IMAPAccount my-name
Host mail.my-domain.com
Port 993
User mailcowuser
Pass mailcowuserpassword
SSLType IMAPS
SSLVersions TLSv1.2
Timeout 300

IMAPStore my-name-remote
Account my-name

MaildirStore my-name-local
Path ~/path/to/MAIL/
Inbox ~/path/to/MAIL/Inbox
SubFolders Verbatim
Flatten .

Channel my-name
Master :my-name-remote:
Slave :my-name-local:
Patterns INBOX Sent
Create Both
Expunge Both
SyncState *

4) Before running mbsync once more, delete everything but the folders /path/to/MAIL/Inbox/cur/ and /path/to/MAIL/Sent/cur/ . You need to do this or you will get an UIDVALIDITY error similar to this:

Error: channel my-name, master INBOX: UIDVALIDITY genuinely changed (at UID 103).

5) Fire mbsync one last time (you may want to add a cron job or systemd service/timer to run mbsync once in a while). It will download any new email you may have upstream in your Mailcow server while uploading the emails you have locally on your computer after downloading them from ProtonMail. You should now have everything in sync, your old and new email, both in your account in your own Mailcow's server and in your local computer. You should get this, adjusted to the number of messages you are uploading to your Mailcow server:

Reading configuration file /home/my-name/.mbsyncrc
C: 0/1  B: 0/0  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Channel my-name
Opening master store my-name-remote...
Resolving mail.my-domain.com... ok
Connecting to mail.my-domain.com (12.345.67.89:993)... 
Opening slave store my-name-local...
Connection is now encrypted
Logging in...
Authenticating with SASL mechanism PLAIN...
C: 0/1  B: 0/2  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box INBOX...
Opening slave box INBOX...
Maildir notice: no UIDVALIDITY, creating new.
Loading master...
Loading slave...
slave: 363 messages, 0 recent
master: 2 messages, 0 recent
Synchronizing...
C: 0/1  B: 1/2  M: +363/363 *0/0 #0/0  S: +2/2 *0/0 #0/0
Error: channel :my-name-remote:Sent-:my-name-local:Sent is locked
C: 1/1  B: 2/2  M: +363/363 *0/0 #0/0  S: +2/2 *0/0 #0/0

The first thing to notice is the same error in the line before the last one. You may need to run mbsync -Va again without deleting anything in your mail folders. If you do that, the new log should be clean of errors:

Reading configuration file /home/my-name/.mbsyncrc
C: 0/1  B: 0/0  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Channel my-name
Opening master store my-name-remote...
Resolving mail.my-domain.com... ok
Connecting to mail.my-domain.com (12.345.67.89:993)... 
Opening slave store my-name-local...
Connection is now encrypted
Logging in...
Authenticating with SASL mechanism PLAIN...
C: 0/1  B: 0/2  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box INBOX...
Opening slave box INBOX...
Loading master...
Loading slave...
slave: 365 messages, 0 recent
master: 365 messages, 0 recent
Synchronizing...
C: 0/1  B: 1/2  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent...
Opening slave box Sent...
Loading master...
Loading slave...
slave: 176 messages, 0 recent
master: 176 messages, 0 recent
Synchronizing...
C: 1/1  B: 2/2  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0