The biggest problem that I've faced with Postman/Newman is a limited number of libraries available in its sandbox environment where test and pre-request scripts are executed.
Personally, I prefer jest matchers over chai matchers for more detailed assertion error messages and extensibility. When jest matchers show you a coloured diff between expected and actual values, chai matchers simply say the expectation failed. As a result, every time a test fails I need to log actual value in test, and if it is big enough to find the difference manually, I need to paste it to a dummy javascript and then run a jest matcher... or simply use JSON diff tools.
If the test fails in a CI/CD environment, then nothing is logged, and I need to reproduce it to understand the issue.
Another common issue: if an API returns something more sophisticated than JSON/XML or even built on top of them, you have neither libraries to parse the results nor custom matchers. Examples: HTML, YAML, JSON-LD, RDF (TTL) etc.
There are ways to add external libraries (e.g. via CDN), but it looks more like a workaround to me.
The biggest problem that I've faced with Postman/Newman is a limited number of libraries available in its sandbox environment where test and pre-request scripts are executed.
Personally, I prefer jest matchers over chai matchers for more detailed assertion error messages and extensibility. When jest matchers show you a coloured diff between expected and actual values, chai matchers simply say the expectation failed. As a result, every time a test fails I need to log actual value in test, and if it is big enough to find the difference manually, I need to paste it to a dummy javascript and then run a jest matcher... or simply use JSON diff tools.
If the test fails in a CI/CD environment, then nothing is logged, and I need to reproduce it to understand the issue.
Another common issue: if an API returns something more sophisticated than JSON/XML or even built on top of them, you have neither libraries to parse the results nor custom matchers. Examples: HTML, YAML, JSON-LD, RDF (TTL) etc.
There are ways to add external libraries (e.g. via CDN), but it looks more like a workaround to me.
Thanks a lot for your feedback!
For performance/stress API testing I use vegeta, which is written in Go and worked well for different use cases: https://github.com/tsenart/vegeta
Another alternative I have found is Bruno: https://github.com/usebruno/bruno, although it is not as filled with functionality as the others two.
Thanks, I need to update this post. In the following blog post https://blogs.sap.com/2023/08/17/alternatives-to-postman-for-exploratory-and-automated-api-testing/
I have already mentioned Bruno and added some information with regards to Insomnia
I have no updated the substack post also to reflect the current state