使用python传输文件

Question

I have to transfer files from my remote computer to amazon ec2 and then from there to Amazon s3 using python . I have successfully uploaded one text file but dont know how to upload multiple files. Here are the codes for two programs for one file.

to get file to ec2

import urllib
source = urllib.urlopen('url').read()
fhand = open('file2.txt','w')
fhand.write(source)
fhand.close()

to upload file to s3

import boto
from boto.s3.key import Key
keyId = "acess key"
skeyId = "secret key"

fileName="file2.txt"
bucketName="bname"
file=open(fileName)

conn = boto.connect_s3(keyId,skeyId)
print conn
bucket = conn.get_bucket(bucketName)
print bucket
k = Key(bucket)
print k
k.key=fileName
print k.key
result = k.set_contents_from_file(file)
print result

Answer 1

Looping to get files from ec2

import urllib,os
sources = ['//url1/file1.txt','//url2/file2.txt','//url3/file3.txt'] # List of urls
for myurl in sources:
    source = urllib.urlopen(myurl).read() #Open each url file
    fname = os.path.split(myurl)[1]       #Split the url to get the file name 
    print('Copying file:',fname)
    fhand = open(fname,'w')
    fhand.write(source)
    fhand.close()

Use the same principle to send files from a list mylist=["a","b","c"]
https://wiki.python.org/moin/ForLoop

Notes:
If the same file name is used in different urls, the results will be overwritten, so you will want to add a test or some other coping mechanism, if that is a probability.
I have used os.path.split() to get the file name, you may find that the urlparse function gives you better mileage, with more complicated urls

Answer 2

Here is how you can transfer files with pyformulas , if python is installed on each machine.

First install pyformulas pip install pyformulas==0.2.7

Then run the server (receives the files):

Server:

import pyformulas as pf

class server(pf.net.Stream):
    file = bytes()
    def on_receive(self, conn, buffer):
        self.file += buffer

    def on_disconnect(self, conn):
        with open('file_%i' % self.port, 'wb') as file:
            file.write(self.file)

servers = [server(port=1000+i) for i in range(3)]

Then the client (sends the files):

Client:

import pyformulas as pf

class send_file(pf.net.Stream):
    def __init__(self, port, file, address):
        self.file = file
        super(send_file, self).__init__(port, address)

    def on_connect(self, conn):
        conn.send(self.file)
        conn.disconnect()

filenames = ['0.txt', '1.wav', '2.bin'] # Put the filenames here

[ send_file(port=1000+idx, file=open(fname, 'rb').read(), address='server ip address') for idx, fname in enumerate(filenames) ]