Postman 无法验证 JSON 架构中的所有对象

Question

It seems like it should be straight-forward enough, but I'm having problems fully validating JSON response data within Postman (Tiny Validator 4) when there is more than one object returned. When doing negative testing, I found it is only checking the first object instead of all the objects to ensure all data is returned. I want to ensure ALL data returned in ALL objects is of the correct type and is present as expected. I realize this should never be an issue, but want to verify for regression in my test environment.

Response Example:

[
    {
        "productID": 1,
        "product": "Desktop",
        "versionID": 123,
        "version": "Win10 x64"
    },
    {
        "productID": 2,
        "product": "Laptop",
        "versionID": 321,
        "version": "Win 10 x64"
    },
    {
        "productID": 3,
        "product": "Monitor",
        "versionID": 456,
        "version": "LCD Panel"
    }
];

Postman Schema and Object Setup:

var schema = {
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "array",
  "items": [
    {
      "type": "object",
      "properties": {
        "productID": {
          "type": "integer"
        },
        "product": {
          "type": "string"
        },
        "versionID": {
          "type": "integer"
        },
        "version": {
          "type": "string"
        }
      },
      "required": [
        "productID",
        "product",
        "versionID",
        "version"
      ]
    }
  ]
}

var jsonData = pm.response.json();

var wrongDataType = [
    {
        "productID": 1,
        "product": "Desktop",
        "versionID": 123,
        "version": "Win10 x64"
    },
    {
        "productID": "2",       //data returned as a string instead of integer
        "product": "Laptop",
        "versionID": 321,
        "version": "Win 10 x64"
    }
];

var dataMissing = [
    {
        "productID": 1,
        "product": "Desktop",
        "versionID": 123,
        "version": "Win10 x64"
    },
    {
        "productID": "2",
        //"product": "Laptop",      //data not present in response
        "versionID": 321,
        "version": "Win 10 x64"
    }
];

Test Validation using Tiny Validator 4:

pm.test('Schema is valid', function() {
  pm.expect(tv4.validate(jsonData, schema)).to.be.true;
});

pm.test('Wrong data type is not valid', function() {
  pm.expect(tv4.validate(wrongDataType, schema)).to.be.false;
});

pm.test('Data missing is not valid', function() {
  pm.expect(tv4.validate(dataMissing, schema)).to.be.false;
});

In the TV4 validation of the returned jsonData against the schema, it returns true because the API response is valid. For the subsequent comparisons, comparing wrongDataType and dataMissing to the schema, the Postman tests show as failures due to the TV4 validation expecting an invalid comparison, but it actually matching.

If I put the wrong data type or missing data in the first object, it catches it, the validation fails, and the test passes. However, if the incorrect data is put in any other returned object (as shown in the example), the test fails because it doesn't catch the errors in the second objects.

How do I get my tests to check all of the objects in a response and fails if any are incorrect or missing?

Answer 1

It took me a while, but you've specified items as an array. This means that each item is positionally specified. Since you only have a single subschema, only the first item is verified.

If you just use the subschema, but not wrapped in an array, it should fix your issue.