.net API 尝试接受带有大文件的 model 时出错

Question

I have an API method that receives a null model parameter when a large file is passed to it.
I created a test client to test this endpoint. Both the test client and the API have these same identical models and are using .NET 4.5:

public class FilingPostModel
    {
        public string Id { get; set; }
        public string TypeId { get; set; }
        public string FirstName { get; set; }
        public string MiddleName { get; set; }
        public string LastName { get; set; }
        public string Suffix { get; set; }
        public string Line1 { get; set; }
        public string Line2 { get; set; }
        public string City { get; set; }
        public string State { get; set; }
        public string PostalCode { get; set; }
        public string Country { get; set; }
        public string Email { get; set; }
        public string PhoneNumber { get; set; }
        public string Comment { get; set; }
        public string DateSubmitted { get; set; }
        public string Summary { get; set; }
        public List<FilePostModel> FileData { get; set; }
    }

    public class FilePostModel
    {
        public string FileId { get; set; }
        public string FileName { get; set; }
        public string ContentType { get; set; }
        public string FileContent { get; set; }
        public string DateSubmitted { get; set; }
        public string ClassificationId { get; set; }     
    }

The test client is submitting this model:

City: "j"
Comment: null
Country: "United States"
Email: "test@test.tst"
FileData: Count = 1
TypeId: "f94e264a-c8b1-44aa-862f-e6f0f7565e19"
FirstName: "fname"
Id: null
LastName: "lname"
Line1: "testdrive 1"
Line2: null
MiddleName: null
PhoneNumber: "3748923798"
PostalCode: "12345"
State: "Pennsylvania"
Suffix: null
Summary: null

The FileData component has one item:

FileContent: "xY+v6sC8RHQ19av2LpyFGu6si8isrn8YquwGRAalW/6Q..."
ClassificationId: null
ContentType: "text/plain"
FileName: "large.txt"

This is the test clients method used to create and send the API request

public async Task<ActionResult> PostNewFiling(FilingPostModel model)
{
    Dictionary<string, string> req = new Dictionary<string, string>
        {
            {"grant_type", "password"},
            {"username", "some user name"},
            {"password", "some password"},
        };
    FilingApiPostModel postModel = new FilingApiPostModel(model);
    using (HttpClient client = new HttpClient())
    {
        client.Timeout = TimeSpan.FromMinutes(15);
        client.BaseAddress = new Uri(baseUrl);
        var resp = await client.PostAsync("Token", new FormUrlEncodedContent(req));
        if (resp.IsSuccessStatusCode)
        {
            TokenModel token = JsonConvert.DeserializeObject<TokenModel>(await resp.Content.ReadAsStringAsync());
            if (!String.IsNullOrEmpty(token.access_token))
            {
                foreach (HttpPostedFileBase file in model.Files)
                {
                    if (file != null)
                    {                                    
                        FilePostModel fmodel = new FilePostModel();
                        fmodel.FileName = file.FileName;
                        fmodel.ContentType = file.ContentType;
                        byte[] fileData = new byte[file.ContentLength];
                        await file.InputStream.ReadAsync(fileData, 0, file.ContentLength);
                        fmodel.FileContent = Convert.ToBase64String(fileData);
                        fmodel.ClassificationId = model.Classification1;
                        postModel.FileData.Add(fmodel);
                    }
                }
                
                client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", token.access_token);
                var response = await client.PostAsJsonAsync("api/Filing/PostFiling", postModel);    
                var responseBody = await response.Content.ReadAsStringAsync();

                if (response.IsSuccessStatusCode)
                    return Json(new { responseBody });
                else
                    return Json(new { error = true, message = "Error Uploading", obj = responseBody });
            }
        }
        return Json(new { error = true, message = "Error Uploading" });
    }
}

Here is the API method to receive this client request:

public async Task<StatusModel> PostFiling(FilingPostModel model)

Here is the maxAllowedContentLength setting in web.config:

<system.webServer>
    <security>
      <requestFiltering>
        <requestLimits maxAllowedContentLength="4294967295" />
      </requestFiltering>
    </security>
  </system.webServer>

The API model is always null in this test scenario. I'm receiving two types of errors:

1. Newtonsoft.Json - Array dimensions exceeded supported range
2. Newtonsoft.Json - Unexpected character encountered while parsing value: x. Path 'FileData[0].Bytes', line 1, position 517

The file size of this test file is 560 MB. I used Dummy File Creator to create it. Here's a sample of what the content looks like:

ůêÀ¼Dt5õ«ö.œ…Ȭ®ªìD¥[þ6\hW åz·cʾYP¸‡>•;,–@Ó¶ÿm™­fø@ÃNÇIäÀ¿Y4~ëÆÃc¥EWÀ_÷õ9%«éÀG!WBÍ*G2P×æŸ7ú‚{ÓêRúÅîµMZSªšpt6ä”Òø˜H

I have also tried using "fsutil file createnew" to create a test file but receive similar error.

Everything works properly when testing with a 256 MB file.

Thank you for your help.

Answer 1

you can add two attributes to your action:

[RequestFormLimits(MultipartBodyLengthLimit = 209715200)]
[FromBody]

Answer 2

If you need to post JSON I would change your FileData object to use a Base64 encoded string in place of the byte array.

Change the Bytes property in FileData to a string and then when creating the request change fmodel.Bytes = fileData to fmodel.Bytes = Convert.ToBase64String(fileData) .

After deserialising your JSON you can convert the base64 back to a byte[] using Convert.FromBase64String(String) .

Not only will this prevent you exceeding the array limit in JSON but will also reduce the payload size considerably.

Answer 3

Every implementation has its own limit for example PHP has 2MB by default you can increase it using Post_Max_Size

ASP.NET Core enforces 30MB any file bigger than default limitation can cause errors.

Here's Upload Large Files In ASP.NET Core !

For IIS Express, you need to add web.config to your project and add the following markup into it:

<system.webServer>
  <security>
    <requestFiltering>
      <requestLimits maxAllowedContentLength="209715200" />
    </requestFiltering>
  </security>
</system.webServer>

Answer 4

If the issue is being caused by the total size of the array, rather than the number of elements in the array, the OutOfMemoryException might be avoided by targeting 64-bit compilation and by updating your app.config:

<configuration>
  <runtime>
    <gcAllowVeryLargeObjects enabled="true" />
  </runtime>
</configuration>

gcAllowVeryLargeObjects element