[英]Recursively PUT files to a remote server using FTP
I'm currently in a situation where I have very limited access to a server, but need to upload and download a significant amount of files contained within a single directory structure.我目前处于一种情况,我对服务器的访问非常有限,但需要上传和下载包含在单个目录结构中的大量文件。 I don't have SSH access, so I can't use SCP - and rsync isn't an option either unfortunately.
我没有 SSH 访问权限,所以我不能使用 SCP - 不幸的是,rsync 也不是一个选项。
I'm currently using ncftpput, which is great but seems to be quite slow (in spite of a fast connection).我目前正在使用 ncftpput,它很棒但似乎很慢(尽管连接速度很快)。
Is there an alternative / better method I could look into?有没有我可以研究的替代/更好的方法?
(Please accept my apologies if this has been covered, I did a quick search prior to posting but didn't find anything that specifically answered my question) (如果这已被覆盖,请接受我的歉意,我在发帖之前进行了快速搜索,但没有找到任何专门回答我的问题的内容)
Try using LFTP: http://lftp.yar.ru/尝试使用 LFTP: http ://lftp.yar.ru/
or YAFC: http://yafc.sourceforge.net/index.php或YAFC: http://yafc.sourceforge.net/index.php
I'm not familiar with ncftpput
.我不熟悉
ncftpput
。 For non-interactive FTP, I've always used the Perl Net::FTP module -- http://perldoc.perl.org/Net/FTP.html对于非交互式 FTP,我一直使用 Perl Net::FTP 模块 -- http://perldoc.perl.org/Net/FTP.html
This will be faster because you can login, then do all the transfers at once (it seems from a cursory glance that you execute ncftpput
once for each file get/put).这会更快,因为您可以登录,然后立即进行所有传输(粗略地看,您似乎为每个文件 get/put 执行一次
ncftpput
)。
Just remember to NEVER use ASCII mangling!请记住永远不要使用 ASCII 重整! This is the default, so use:
这是默认设置,因此请使用:
$ftp->binary
ASCII mangling needs to die in the same fire with MySQL automatic-timezone-interpreting. ASCII 重整需要与 MySQL 自动时区解释同归于尽。
Since I always end up having a problem with this, I'll post my notes here:由于我总是最终遇到此问题,因此我将在此处发布我的笔记:
One thing I always get to confuse is the syntax;我总是混淆的一件事是语法; so below there is a
bash
tester script which creates some temporary directories, then starts a temporary ftp server, and compares rsync
(in plain local file mode, as it doesn't support ftp) with lftp
and ftpsync
.所以下面有一个
bash
测试脚本,它创建一些临时目录,然后启动一个临时 ftp 服务器,并将rsync
(在纯本地文件模式下,因为它不支持 ftp)与lftp
和ftpsync
。
The thing is - you can use rsync /path/to/local /path/to/remote/
, and rsync will automatically figure out, that you want a local
subdirectory created under remote
;问题是 - 您可以使用
rsync /path/to/local /path/to/remote/
,并且 rsync 会自动确定您想要在remote
下创建local
子目录; however, for lftp
or ftpsync
you have to specify the target directory manually, as in ... /path/to/local /path/to/remote/local
(if it doesn't exist it will be created).但是,对于
lftp
或ftpsync
您必须手动指定目标目录,如... /path/to/local /path/to/remote/local
(如果它不存在,它将被创建)。
You can find the ftpserver-cli.py
in How do I temporarily run an FTP server?您可以在如何临时运行 FTP 服务器中找到
ftpserver-cli.py
。 - Ask Ubuntu ; - 询问 Ubuntu ; and
ftpsync
is here: FTPsync (however, note it is buggy; see also Search/grep ftp remote filenames - Unix & Linux Stack Exchange );和
ftpsync
在这里: FTPsync (但是,请注意它有问题;另请参阅Search/grep ftp 远程文件名 - Unix & Linux Stack Exchange );
Here is a shortened output of the puttest.sh
script, showing the recursive put behavior in different cases:这是
puttest.sh
脚本的简短输出,显示了不同情况下的递归放置行为:
$ bash puttest.sh
Recreate directories; populate loctest, keep srvtest empty:
show dirs:
+ tree --noreport -a /tmp/srvtest /tmp/loctest
/tmp/srvtest
/tmp/loctest
├── .git
│ └── tempa2.txt
└── tempa1.txt
*NOTE, rsync can automatically figure out parent dir:
+ rsync -a --exclude '*.git*' /tmp/loctest /tmp/srvtest/
show dirs:
+ tree --noreport -a /tmp/srvtest /tmp/loctest
/tmp/srvtest
└── loctest
└── tempa1.txt
/tmp/loctest
├── .git
│ └── tempa2.txt
└── tempa1.txt
cleanup:
+ rm -rf /tmp/srvtest/loctest
Start a temporary ftp server:
+ sudo bash -c 'python /path/to/pyftpdlib/ftpserver-cli.py --username=user --password=12345 --directory=/tmp/srvtest &'
+ sleep 1
Using: user: user pass: 12345 port: 21 dir: /tmp/srvtest
[I 14-03-02 23:24:01] >>> starting FTP server on 127.0.0.1:21, pid=21549 <<<
[I 14-03-02 23:24:01] poller: <class 'pyftpdlib.ioloop.Epoll'>
[I 14-03-02 23:24:01] masquerade (NAT) address: None
[I 14-03-02 23:24:01] passive ports: None
[I 14-03-02 23:24:01] use sendfile(2): False
test with lftp:
*NOTE, lftp syncs *contents* of local dir (rsync-like syntax doesn't create target dir):
+ lftp -e 'mirror -R -x ".*\.git.*" /tmp/loctest / ; exit' -u user,12345 127.0.0.1
show dirs:
+ tree --noreport -a /tmp/srvtest /tmp/loctest
/tmp/srvtest
└── tempa1.txt
/tmp/loctest
├── .git
│ └── tempa2.txt
└── tempa1.txt
cleanup:
+ rm -rf /tmp/srvtest/tempa1.txt
*NOTE, specify lftp target dir explicitly (will be autocreated):
+ lftp -e 'mirror -R -x ".*\.git.*" /tmp/loctest /loctest ; exit' -u user,12345 127.0.0.1
show dirs:
+ tree --noreport -a /tmp/srvtest /tmp/loctest
/tmp/srvtest
└── loctest
└── tempa1.txt
/tmp/loctest
├── .git
│ └── tempa2.txt
└── tempa1.txt
cleanup:
+ sudo rm -rf /tmp/srvtest/loctest
*NOTE, ftpsync syncs *contents* of local dir (rsync-like syntax doesn't create target dir); also info mode -i is buggy (it puts, although it shouldn't):
*NOTE, ftpsync --ignoremask is for older unused code; use --exclude instead (but it is buggy; need to change in source)
+ /path/to/ftpsync/ftpsync -i -d '--exclude=.*\.git.*' /tmp/loctest ftp://user:12345@127.0.0.1/
show dirs:
+ tree --noreport -a /tmp/srvtest /tmp/loctest
/tmp/srvtest
└── tempa1.txt
/tmp/loctest
├── .git
│ └── tempa2.txt
└── tempa1.txt
cleanup:
+ sudo rm -rf /tmp/srvtest/tempa1.txt
*NOTE, specify ftpsync target dir explicitly (will be autocreated):
+ /path/to/ftpsync/ftpsync -i -d '--exclude=.*\.git.*' /tmp/loctest ftp://user:12345@127.0.0.1/loctest
show dirs:
+ tree --noreport -a /tmp/srvtest /tmp/loctest
/tmp/srvtest
└── loctest
└── tempa1.txt
/tmp/loctest
├── .git
│ └── tempa2.txt
└── tempa1.txt
cleanup:
+ sudo rm -rf /tmp/srvtest/loctest
+ sudo pkill -f ftpserver-cli.py
And, here is the puttest.sh
script:而且,这是
puttest.sh
脚本:
#!/usr/bin/env bash
set -x
# change these to match your installations:
FTPSRVCLIPATH="/path/to/pyftpdlib"
FTPSYNCPATH="/path/to/ftpsync"
{ echo "Recreate directories; populate loctest, keep srvtest empty:"; } 2>/dev/null
sudo rm -rf /tmp/srvtest /tmp/loctest
mkdir /tmp/srvtest
mkdir -p /tmp/loctest/.git
echo aaa > /tmp/loctest/tempa1.txt
echo aaa > /tmp/loctest/.git/tempa2.txt
{ echo "show dirs:"; } 2>/dev/null
tree --noreport -a /tmp/srvtest /tmp/loctest
{ echo -e "\n*NOTE, rsync can automatically figure out parent dir:"; } 2>/dev/null
rsync -a --exclude '*.git*' /tmp/loctest /tmp/srvtest/
{ echo "show dirs:"; } 2>/dev/null
tree --noreport -a /tmp/srvtest /tmp/loctest
{ echo "cleanup:"; } 2>/dev/null
rm -rf /tmp/srvtest/*
{ echo -e "\nStart a temporary ftp server:"; } 2>/dev/null
# https://askubuntu.com/questions/17084/how-do-i-temporarily-run-an-ftp-server
sudo bash -c "python $FTPSRVCLIPATH/ftpserver-cli.py --username=user --password=12345 --directory=/tmp/srvtest &"
sleep 1
{ echo "test with lftp:"; } 2>/dev/null
# see http://russbrooks.com/2010/11/19/lftp-cheetsheet
# The -R switch means "reverse mirror" which means "put" [upload].
{ echo -e "\n*NOTE, lftp syncs *contents* of local dir (rsync-like syntax doesn't create target dir):"; } 2>/dev/null
lftp -e 'mirror -R -x ".*\.git.*" /tmp/loctest / ; exit' -u user,12345 127.0.0.1
{ echo "show dirs:"; } 2>/dev/null
tree --noreport -a /tmp/srvtest /tmp/loctest
{ echo "cleanup:"; } 2>/dev/null
rm -rf /tmp/srvtest/*
{ echo -e "\n*NOTE, specify lftp target dir explicitly (will be autocreated):"; } 2>/dev/null
lftp -e 'mirror -R -x ".*\.git.*" /tmp/loctest /loctest ; exit' -u user,12345 127.0.0.1
{ echo "show dirs:"; } 2>/dev/null
tree --noreport -a /tmp/srvtest /tmp/loctest
{ echo "cleanup:"; } 2>/dev/null
sudo rm -rf /tmp/srvtest/*
{ echo -e "\n*NOTE, ftpsync syncs *contents* of local dir (rsync-like syntax doesn't create target dir); also info mode -i is buggy (it puts, although it shouldn't):"; } 2>/dev/null
{ echo -e "\n*NOTE, ftpsync --ignoremask is for older unused code; use --exclude instead (but it is buggy; need to change ` 'exclude=s' => \$opt::exclude,` in source)"; } 2>/dev/null
$FTPSYNCPATH/ftpsync -i -d --exclude='.*\.git.*' /tmp/loctest ftp://user:12345@127.0.0.1/
{ echo "show dirs:"; } 2>/dev/null
tree --noreport -a /tmp/srvtest /tmp/loctest
{ echo "cleanup:"; } 2>/dev/null
sudo rm -rf /tmp/srvtest/*
{ echo -e "\n*NOTE, specify ftpsync target dir explicitly (will be autocreated):"; } 2>/dev/null
$FTPSYNCPATH/ftpsync -i -d --exclude='.*\.git.*' /tmp/loctest ftp://user:12345@127.0.0.1/loctest
{ echo "show dirs:"; } 2>/dev/null
tree --noreport -a /tmp/srvtest /tmp/loctest
{ echo "cleanup:"; } 2>/dev/null
sudo rm -rf /tmp/srvtest/*
sudo pkill -f ftpserver-cli.py
{ set +x; } 2>/dev/null
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.