MalformedXML：您提供的 XML 格式不正确或未根据我们发布的架构进行验证

Question

I am having this weird issue while working with AWS S3. I am working on application by which I can store the images to AWS bucket. Using Multer as middleware and S3FS library to connect and upload to AWS.

But the following error pops up when I try uploading the content.

"MalformedXML: The XML you provided was not well-formed or did not validate against our publis hed schema"

Index.js

var express = require('express');
var router = express();
var multer = require('multer');
var fs = require('fs');
var S3FS = require('s3fs');
var upload = multer({
  dest: 'uploads'
})
var S3fsImpl = new S3FS('bucket-name', {
  region: 'us-east-1',
  accessKeyId: 'XXXXXXXXXXXX',
  secretAccessKey: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
});

/* GET home page. */
router.get('/', function (req, res, next) {
  res.render('profile', {
    title: 'Express'
  });
});

router.post('/testupload', upload.single('file'), function (req, res) {
  var file = req.file;
  console.log(file);

  var path = req.file.path;
  var stream = fs.createReadStream(path);
  console.log(stream);

  S3fsImpl.writeFile(file.name, stream).then(function () {
    fs.unlink(file.path, function (err) {
      if (err) {
        console.log(err);
      }
    });
    res.redirect('/profile');
  })
});

module.exports = router;

EDIT Output:

{ fieldname: 'file',
  originalname: '441_1.docx',
  encoding: '7bit',
  mimetype: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
  destination: 'uploads',
  filename: '662dcbe544804e4f50dfef1f52b40d22',
  path: 'uploads\\662dcbe544804e4f50dfef1f52b40d22',
  size: 13938 }
ReadStream {
  _readableState:
   ReadableState {
     objectMode: false,
     highWaterMark: 65536,
     buffer: BufferList { head: null, tail: null, length: 0 },
     length: 0,
     pipes: null,
     pipesCount: 0,
     flowing: null,
     ended: false,
     endEmitted: false,
     reading: false,
     sync: true,
     needReadable: false,
     emittedReadable: false,
     readableListening: false,
     resumeScheduled: false,
     defaultEncoding: 'utf8',
     ranOut: false,
     awaitDrain: 0,
     readingMore: false,
     decoder: null,
     encoding: null },
  readable: true,
  domain: null,
  _events: { end: [Function] },
  _eventsCount: 1,
  _maxListeners: undefined,
  path: 'uploads\\662dcbe544804e4f50dfef1f52b40d22',
  fd: null,
  flags: 'r',
  mode: 438,
  start: undefined,
  end: undefined,
  autoClose: true,
  pos: undefined,
  bytesRead: 0 }

Package.json

{
  "name": "aws-s3-images",
  "version": "1.0.0",
  "private": true,
  "scripts": {
    "start": "node ./bin/www"
  },
  "dependencies": {
    "body-parser": "~1.17.1",
    "connect-multiparty": "^2.0.0",
    "cookie-parser": "~1.4.3",
    "debug": "~2.6.3",
    "express": "~4.15.2",
    "hbs": "~4.0.1",
    "morgan": "~1.8.1",
    "multer": "^1.3.0",
    "s3fs": "^2.5.0",
    "serve-favicon": "~2.4.2"
  },
  "description": "AWS S3 uploading images",
  "main": "app.js",
  "devDependencies": {},
  "keywords": [
    "javascript"
  ],
  "author": "reeversedev",
  "license": "MIT"
}

Answer 1

So if anyone still facing this issue, in my case the issue only happens when you pass an empty array of objects you wanna delete, it causes the server to crash with the following error "MalformedXML".

const data: S3.DeleteObjectsRequest = {
  Bucket: bucketName,
  Delete: {
    Objects: [], <<---here
  },
}

return s3Bucket.deleteObjects(data).promise()

so just check if the array of Objects keys isn't equal to zero before sending that request to aws.

Answer 2

The S3 restricts the file deletion 1000 per DeleteObjectsRequest. Hence after fetching all KeyVersions List, I gave the check if the keys are >1000, then partition the list into sublists and then pass it to the DeleteObjectsRequest with sublists as below-

if (keys.size() > 1000) {
            int count = 0;
            List<List> partition = ListUtils.partition(keys, 1000);
            for (List list : partition) {
                count = count + list.size();
                DeleteObjectsRequest request = new DeleteObjectsRequest(
                        fileSystemConfiguration.getTrackingS3BucketName()).withKeys(list);
                amazonS3Client.deleteObjects(request);
                logger.info("Deleted the completed directory files " + list.size() + " from folder "
                        + eventSpecificS3bucket);
            }
            logger.info("Deleted the total directory files " + count + " from folder " + eventSpecificS3bucket);
        } else {
            DeleteObjectsRequest request = new DeleteObjectsRequest(
                    fileSystemConfiguration.getTrackingS3BucketName()).withKeys(keys);
            amazonS3Client.deleteObjects(request);
            logger.info("Deleted the completed directory files from folder " + eventSpecificS3bucket);
        }

Answer 3

I got this problem when use AmplifyJS library. Follow the document in AWS homepage about Multipart upload overview :

Whenever you upload a part, Amazon S3 returns an ETag header in its response. For each part upload, you must record the part number and the ETag value. You need to include these values in the subsequent request to complete the multipart upload.

But S3 default config does not do it. Just go To Permissions tab -> Add <ExposeHeader>ETag</ExposeHeader> into CORS Configuration. https://github.com/aws-amplify/amplify-js/issues/61

Answer 4

This code should work for you. You need to remember that: 1) use unique bucket name 2) under your file object use 'originalname' instead of 'name' <-- this property does not exist

app.post('/testupload', function(req, res){


    var file = req.files[0];

    console.log(file.path);
    console.log(file.name);

    console.log('FIRST TEST: ' + JSON.stringify(file));

    var stream = fs.createReadStream(file.path);    

    S3fsImpl.writeFile(file.originalname, stream).then(function () 
      {
        console.log('File has been sent - OK');
      },
      function(reason)
      {
          throw reason;
      }
     ); 

     res.redirect('/index');   

});

Answer 5

据我所知，只需交叉检查 BUCKET NAME。

final PutObjectRequest putObjectRequest = new PutObjectRequest(**bucketName**, accessKeyId, is ,meta); 

Answer 6

If you use ActiveStorage with Minio add force_path_style: true to your config

# config/storage.yml
minio:
   service: S3
   access_key_id: name
   secret_access_key: password
   endpoint: http://example.com:9000/
   region: us-east-1
   bucket: myapp-production
   force_path_style: true # add this

Answer 7

input := &s3.DeleteObjectsInput{
        Bucket: bucketName,
        Delete: &s3.Delete{Objects: objs, // <-  up to 1000 keys
       Quiet: aws.Bool(false)},
    }

I am using aws-sdk-go sdk, when the num of the key in objs is overer 1000,I have get the same error like: MalformedXML: The XML you provided was not well-formed or did not validate against our published schema.

The request contains a list of up to 1000 keys. reference: https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html

Answer 8

For those which came here from talend, In my case cross check the tS3Put component's Bucket name and in the key part give any name which you want to see in the s3 as a uploaded file.

As I'm new to StackOverflow, I'm not allowed to attached images here. You can copy the below url to check it out. Thanks

https://i.stack.imgur.com/Q1pW0.png

Answer 9

Can you try this code:

var S3fsImpl = new S3FS('bucket-name', {
  region: 'us-east-1',
  accessKeyId: 'XXXXXXXXXXXX',
  secretAccessKey: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
});

var fsImplStyles = S3fsImpl.getPath(file.name);

// Change us-east-1 for your region
var url = 'https://s3-us-east-1.amazonaws.com/' + fsImplStyles;

Send feedback, if this code works for you.