对于集成测试，如何验证模式/结构的JSON和XML？ 任何语言

Question

Are there any common tricks, strategies, or tools to natively compare two JSON and/or XML strings for their schema/structure only? I'm hoping to find a technique that fairly primitive and universal, but have not found one so far. I'm also looking for one that will simply compare a stored output versus a live output without having to write field-level validations.

When performing the most basic integration tests against REST API's we don't control, we often want to verify that a response has certain characteristics (certain fields exist, of certain value types, etc). The responses are almost never completely equivalent to a stored sample response, often times because of changed timestamps, or new objects in arrays and such. I have frequently come across the problem in both Java and .NET, but a solution on any platform would be helpful to see.

We have some methods we've come up with, but they all feel very sub-optimal and have downsides. For example: 1. Parsing the response to a generic JSON object and then performing validations that fields exist using "getProperties()", and validating the type of the value using "instanceof", etc. 2. Fully Deserializing to a domain class and essentially doing the same as above. 3. Comparing the raw body as a string to a stored body from a text file, and using string techniques to "strip out" the values and spaces.

Answer 1

We use method 2)

2. Fully Deserializing to a domain class and essentially doing the same as above.

We have a try...catch wrapped around our strongly typed call to the Deserialize method that uses the JavaScriptSerializer .

The exception handled will determine the nature of the error encountered.

We do store/keep record of past & current tests and corresponding API versions and date-test-passed information

It is stored in the DB as a domain class-to-API version mapping catalogue... It stores

• our last tested,versioned domain class library details(dll, type), their corresponding API version, base & resource paths...
• as well as last-passed datetime...

Our testing system picks that information, reflects on the current version of the dll(using its type) containing the domain classess corresponding to the known API and its version.

Using Dependency Injection, it makes it easy for us to even swap out and in previous DLLs to test if they work against a possibly new API.

That way we actually have dates & information to refer to in our cmmunication with the third-party when things break.

Integrations are always hard, so we also make sure we also have 'heartbeat' pings that do simple requests to the API, integrated to our alert system(SCOM) that notifies us of failures.

Answer 2

A less programmatic but simpler approach to ensure a 3rd party provider API responses still comply with what you expect from them is to use tools like Postman .

Postman enables you to create a suite or collection of API calls with assertions on the responses, so the idea would be to collect the basic calls you want to make sure work in the provider and write the assertions for them, focusing on the shape and values you expect in the responses.

With that collection of calls and assertions, you can run the suite on a regular basis either manually or within your builds using Newman (a command-line tool to run Postman collections), detecting bugs, unavailability and API regressions on your providers when they happen, without needing to monitor the provider's status pages or bug tracking tools.

Being Postman a 3rd party tool itself, it would mean that, out of the box, your tests won't be stored in the same repository as your service. Additionally, you now depend on Postman's availability and robustness. On the other hand, you would be using a tool specifically designed for this need, which makes unnecessary to write (and test, and maintain) specific code utils that validate the responses of the external provider.