Despite its seemingly straightforward nature, I am encountering difficulties in fully validating JSON response data within Postman (Tiny Validator 4) when multiple objects are returned. During negative testing, I discovered that it only checks the first object instead of all the objects to ensure that all data is being properly returned. I want to verify that ALL data returned in ALL objects is of the correct type and present as expected. While this should theoretically not be a problem, I want to confirm for regression testing purposes in my test environment.
Response Example:
[
{
"productID": 1,
"product": "Desktop",
"versionID": 123,
"version": "Win10 x64"
},
{
"productID": 2,
"product": "Laptop",
"versionID": 321,
"version": "Win 10 x64"
},
{
"productID": 3,
"product": "Monitor",
"versionID": 456,
"version": "LCD Panel"
}
];
Postman Schema and Object Setup:
var schema = {
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "array",
"items": [
{
"type": "object",
"properties": {
"productID": {
"type": "integer"
},
"product": {
"type": "string"
},
"versionID": {
"type": "integer"
},
"version": {
"type": "string"
}
},
"required": [
"productID",
"product",
"versionID",
"version"
]
}
]
}
var jsonData = pm.response.json();
var wrongDataType = [
{
"productID": 1,
"product": "Desktop",
"versionID": 123,
"version": "Win10 x64"
},
{
"productID": "2", //data returned as a string instead of integer
"product": "Laptop",
"versionID": 321,
"version": "Win 10 x64"
}
];
var dataMissing = [
{
"productID": 1,
"product": "Desktop",
"versionID": 123,
"version": "Win10 x64"
},
{
"productID": "2",
//"product": "Laptop", //data not present in response
"versionID": 321,
"version": "Win 10 x64"
}
];
Test Validation using Tiny Validator 4:
pm.test('Schema is valid', function() {
pm.expect(tv4.validate(jsonData, schema)).to.be.true;
});
pm.test('Wrong data type is not valid', function() {
pm.expect(tv4.validate(wrongDataType, schema)).to.be.false;
});
pm.test('Data missing is not valid', function() {
pm.expect(tv4.validate(dataMissing, schema)).to.be.false;
});
In the TV4 validation of the returned jsonData against the schema, it returns true because the API response is valid. For subsequent comparisons involving wrongDataType and dataMissing against the schema, the Postman tests display failures due to TV4 validation expecting an invalid comparison, despite them actually matching.
If I introduce the wrong data type or missing data in the first object, the validation fails, and the test passes. However, if incorrect data is placed in any other returned object (as shown in the example), the test fails because it does not detect errors in the second objects.
How can I modify my tests to check all objects in a response and fail if any are incorrect or missing?