ScrapySharp 表单提交导致 System.AggregateException

Question

我花了几个小时苦苦思索为什么这不起作用

我正在尝试使用 ScrapySharp 来抓取网站，现在只是尝试示例网站，然后转移到我的实际网站。

每次我在我的程序中执行form.Submit() ，我都会遇到 System.AggregateException（指定的强制转换无效）

我的代码：

using System;
using System.IO;
using System.Linq;
using System.Net;
using HtmlAgilityPack;
using ScrapySharp.Extensions;
using ScrapySharp.Html;
using ScrapySharp.Html.Forms;
using ScrapySharp.Network;

namespace WebScraper
{
    class MainClass
    {
        public static void Main(string[] args)
        {
            ScrapingBrowser browser = new ScrapingBrowser();

            //set UseDefaultCookiesParser as false if a website returns invalid cookies format
            //browser.UseDefaultCookiesParser = false;
            browser.AllowAutoRedirect = true;
            browser.AllowMetaRedirect = true;
            WebPage homePage = browser.NavigateToPage(new Uri("http://the-internet.herokuapp.com/login"));

            PageWebForm form = homePage.FindForm("login");
            form["username"] = "tomsmith";
            form["password"] = "SuperSecretPassword!";
            form.Method = HttpVerb.Get; //I tried both .Post and .Get
            WebPage resultsPage = form.Submit(); //THIS IS WHERE I GET THE ERROR
            Console.WriteLine(resultsPage);

        }
    }
}

我的错误：

System.AggregateException：发生一个或多个错误。 （指定的强制转换无效。） ---> System.InvalidCastException：指定的强制转换无效。 在 ScrapySharp.Network.ScrapingBrowser.CreateRequest (System.Uri url, ScrapySharp.Network.HttpVerb 动词) [0x0000b] 在 <0a639adc663f45108f057c429262c620>:0 处 (System.Uri url, ScrapySharp.Network.HttpVerb verb) [0x0000b] , System.String data, System.String contentType) [0x00066] in <0a639adc663f45108f057c429262c620>:0 --- 内部异常堆栈跟踪结束 --- 在 System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) [0x00066] ] 在 /Users/builder/jenkins/workspace/build-package-osx-mono/2019-06/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading /Tasks/Task.cs:2027 在 System.Threading.Tasks.Task 1[TResult].GetResultCore (System.Boolean waitCompletionNotification) [0x0002b] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-06/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:496 at System.Threading.Tasks.Task 1[TResult].GetResultCore (System.Boolean waitCompletionNotification) [0x0002b] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-06/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:496 at System.Threading.Tasks.Task 1[TResult].get_Result () [0x00000] 在 /Users/builder/jenkins/workspace/build-package-osx-mono/2019-06/external/bockbuild/builds/mono-x64/ external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:466 at ScrapySharp.Network.ScrapingBrowser.NavigateToPage (System.Uri url, ScrapySharp.Network.HttpVerb 动词, System.String 数据, System.String contentType) [0x0000b] in <0a639adc663f45108f057c429262c620>:0 at ScrapySharp.Html.Forms.PageWebForm.Submit () [0x00023] in <0a639adc663f45108f057c429262c625 [0x00065] 在 /Users/arib/Projects/WebScraper/WebScraper/Program.cs:29

我对这个错误感到厌倦，非常感谢任何和所有帮助..谢谢

Answer 1

问题是当您使用form["username"] ，结果是一个字符串。 您想要获取FormField ，您可以使用以下代码进行操作：

WebPage homePage = browser.NavigateToPage(new Uri("http://the-internet.herokuapp.com/login"));
PageWebForm form = homePage.FindForm("login");
var formFields = form.FormFields;
foreach (var field in formFields)
{
    if (field.Name.Equals("username", StringComparison.OrdinalIgnoreCase))
    {
        field.Value = "tomsmith";

    }
    else if (field.Name.Equals("password", StringComparison.OrdinalIgnoreCase))
    {
        field.Value = "SuperSecretPassword!";

    }
}

WebPage resultsPage = form.Submit();
Console.WriteLine(resultsPage);

或者，您可以使用Find()来获取FormField ：

var usernameField = form.FormFields.Find(x => x.Name == "username");