An essential part of the DevOps cycle, software test management must be linked to the project management tool. Logically, test management tools are a highly competitive market on the Atlassian Jira marketplace. And it has to be said that each solution tries to go the extra mile. There are several proposals, all of which are very successful.
Here are the main solutions on the market for Testing, taken from our Jira and Marketplace Guide (2023 edition):
QMetry test management for Jira
QMetry’s test management solution is one of the most comprehensive available: planning, creation, execution, automation, detailed reports, etc. To make matters worse, the application manages to maintain an excellent user experience. Why is that? QMetry is best thought of as a separate application connected to Jira, but in fact the links are first and foremost where the publisher has chosen to put them, where they are essential.
It is not then the ‘issue types’ that are used to create the objects of the application (Cycles, plans, and test cases), which can hinder the most technical perspectives, mainly at the level of queries. The solution is also not the most comprehensive in terms of test status reporting. It’s a small price to pay for a comprehensive tool that’s also very well positioned in terms of pricing.
Requirements & Test Management for Jira
“RTM is the latest application in the selection for managing tests. It is a solid proposal for test management tools, but not the only one: RTM is also interested in Requirement Management, which provides a link between tests and requirements. As far as tests are concerned, there’s little to complain about apart from test cycle management, which is not natively offered. Automation isn’t part of the package either, which could prove to be an obstacle for those wishing to connect Jira to automation frameworks.
RTM offers an easy-to-use interface, which ensures that it is used correctly. Finally, the price is also a strong point of the product, especially in the Cloud, compared with the rather expensive competition.
Xray test Management for Jira
XRay is a very popular application on the marketplace for test management, due to its long history and the completeness of its functionality. First and foremost, Xray offers interfaces that are accessible to everyone, without neglecting the advanced functionalities available to administrator profiles. The tool can be finely integrated into software development processes, even Scrum sprints. The integration with Jira is deep, and Xray doesn’t forget to connect with other integration tools on the market: Bamboo, Jenkins, Gitlab, etc. Xray also manages test automation, and can be integrated with your usual framework (Cucumber, JUnit, TestNG, etc.). Xray will remain handy for organising and planning tests. In terms of reporting, the essentials are provided, and Xray data can be exploited even more effectively using eazyBI, which has a native connector.
Xray is the application of choice for Spectrum’s integration projects.
Zephyr Squad – Test Management for Jira
Following its acquisitions, Smartbear is now offering no less than 3 versions of its Zephyr product, one of the marketplace’s flagship test management solutions. The “Squad” edition is the original Zephyr product. On the marketplace, it’s the oldest, the most popular, but also the most expensive, along with its sister ‘Scale’ edition for testing applications. It’s a complete product. It integrates with the majority of automation frameworks and integration tools, and it’s quick to get to grips with. The tool could perhaps do with a little more in the way of test planning and organisation. The reporting section is fine. Zephyr Squad ‘does the job’, but seems a little expensive compared with the sometimes more accomplished competition.
Zephyr Scale – Test Management for Jira
Prior to its acquisition by Smartbear, Zephyr Scale was TM4J. So it’s not a more advanced edition of Zephyr Squad, but a different product. In keeping with its name, Scale is a step above Squad in terms of functionality, particularly in terms of test organisation and planning. Zephyr Scale will automatically be better suited to project management at scale, i.e. on Premium instances, and will work very well with applications such as BigPicture or Structure. However, the application is not reserved for agility at scale, and is not the only one on the market capable of this. It will work just as well on simple V-cycle projects. Above all, Smartbear is segmenting its offering. In fact, for the same price, we would tend to recommend Scale, even though Squad has a better capacity for rapid implementation.
In conclusion
Only Requirements & Tests Management for Jira, the youngest solution, seems to want to differentiate itself with a more essential product (automation is not offered, for example), and above all less expensive (less true for large Data Centre organisations). The others, now all ‘old hands’ in the Jira marketplace, have very complete and mature products. Differentiating them on functional response is a challenge, even if it seems that Zephyr Squad is struggling to keep up with the fierce competition. Testing is still the best way to make a choice after validating the responses to your requirements. Approaches differ, especially when it comes to administration. For the end user, the experience is often a good one.
In our guide to Jira and Marketplace 2023, we analysed this market in terms of functionality and price:
Testing solutions are all expensive compared with other types of application in the marketplace. They benefit from the fact that Jira is an essential part of the software development process. After that, these are highly sophisticated products that meet the challenges of testing.
