在 web 应用程序上与 javascript 交互的桌面应用程序/脚本

Question

My work has tasked me with determining the feasibility of migrating our existing in-house built change management services(web based) to a Sharepoint solution. I've found everything to be easy except I've run into the issue that for each change management issue (several thousand) there may be any number of attachment files associated with them, called through javascript, that need to be downloaded and put into a document library.

(ex. ... onClick="DownloadAttachment(XXXXX,'ProjectID=YYYY');return false">Attachment... ).

To keep me from manually selecting them all I've been looking over posts of people wanting to do similar, and there seem to be many possible solutions, but they often seem more complicated than they need to be.

So I suppose in a nutshell I'm asking what would be the best way to approach this issue that yields some sort of desktop application or script that can interact with web pages and will let me select and organize all the attachments. (Making a purely web based app (php, javascript, rails, etc.) is not an option for me, so throwing that out there now).

Thanks in advance.

Answer 1

1. Given a document id and project id, XXXXX and YYYY respectively in your example, figure out the URL from which the file contents can be downloaded. You can observe a few URL links in the browser and detect the pattern which your web application uses.

2. Use a tool like Selenium to get a list of XXXXX s and YYYY s of documents you need to download.

3. Write a bash script with wget to download the files locally and put in the correct folders.

Answer 2

This is a "one off" migration, right?

1. Get access to your in-house application's database, and create an SQL query which pulls out rows showing the attachment names ( XXXXX ?) and the issue/project ( YYYY ?), ex:

    |file_id|issue_id|file_name | | 5| 123|Feasibility Test.xls|

2. Analyze the DownloadAttachment method and figure out how it generates the URL that it calls for each download.

3. Start a script (personally I'd go for Python) that will do the migration work.
4. Program the script to connect and run the SQL query, or can read a CSV file you create manually from step #1.
5. Program the script to use the details to determine the target-filename and the URL to download from.
6. Program the script to download the file from the given URL, and place it on the hard drive with the proper name. (In Python, you might use urllib .)

Hopefully that will get you as far as a bunch of files categorized by "issue" like:

issue123/Feasibility Test.xls
issue123/Billing Invoice.doc
issue456/Feasibility Test.xls

Answer 3

Thank you everyone. I was able to get what I needed using htmlunit and java to traverse a report I made of all change items with attachments, go to each one, copy the source code, traverse that to find instances of the download method, and copy the unique IDs of each attachment and build an.xls of all items and their attachments.