Creating Test Plans and Test Suites using Azure Devops

What is Azure Test plan?

Azure Test Plans provides tools to accomplish quality and collaboration throughout the development process. This test management solution provides all the capabilities required for planned manual testing, user acceptance testing, exploratory testing and gathering feedback from stakeholders. It is easy-to-use and browser based .

How does Azure Test Plans work?

Through a combination of browser-based tools—Test Plans, Progress Reports, Parameters Configurations, Runs and Test Tools —and DevOps integration features, Azure Test Plans supports the following test objectives:

Manual and Exploratory Testing : Manual and exploratory testing which includes the following test activities:
- Planned Manual Testing. Manual testing by organizing tests into test plans and test suites by designated testers and test leads.
- User Acceptance Testing. Testing carried out by designated user acceptance testers to verify the value delivered meets customer requirements, while reusing the test artifacts created by engineering teams.
- Exploratory Testing. Testing carried out by development teams, including developers, testers, UX teams, product owners and more, by exploring the software systems without using test plans or test suites.
- Stakeholder Feedback. Testing carried out by stakeholders outside the development team, such as users from marketing and sales divisions.
Automated Testing : Azure Test Plans is fully integrated with Azure Pipelines to support testing within continuous integration/continuous deployment (CI/CD). Test plans and test cases can be associated with build or release pipelines. Pipeline tasks can be added to pipeline definitions to capture and publish test results. Test results can be reviewed via built in progress reports and pipeline test reports.
Traceability : Test cases and test suites linked to user stories, features, or requirements supports end-to-end traceability. Tests and defects are automatically linked to the requirements and builds being tested, which also helps tracking the quality of requirements. Users can add and run tests from the Kanban board, or for larger teams, use the Test plans hub to define test plans and test suites. Pipeline results and the Requirements widget provide a means to track testing of requirements.
Reporting and Analysis : Test result tracking and progress monitoring is supported through configurable tracking charts, test-specific widgets that you can add to dashboards, and built-in reports, such as Progress report, pipeline test result reports, and the Analytics service.

Key Benefits

Azure Test Plans provides software development teams the following benefits.

Test on any platform: With the Test Plans web portal, you can use any supported browser to access all the manual testing capabilities. It enables you to create and run manual tests through an easy-to-use, browser-based interface that users can access from all major browsers on any platform.
Rich diagnostic data collection: Using the web-based Test Runner and Test Runner client you can collect rich diagnostic data during your manual tests. This includes screenshots, an image action log, screen recordings, code coverage, IntelliTrace traces, and test impact data for your apps under test. This data is automatically included in all the bugs you create during test, making it easy for developers to reproduce the issues.
End to End traceability: Azure DevOps provides end-to-end traceability of your requirements, builds, tests and bugs with liking work items to other objects. Users can track their requirement quality from cards on the Kanban board. Bugs created while testing are automatically linked to the requirements and builds being tested, which helps you track the quality of the requirements or builds.
Integrated analytics: The Analytics service provides data that feeds into built-in reports, configurable dashboard widgets, and customizable reports using Power BI. Data tracks test plan progress and trends for both manual and automated tests. Test analytics provides near real-time visibility into test data for builds and releases. Teams can act on this data to improve test collateral to help maintain healthy pipelines.
Extensible platform. You can combine the tools and technologies you already know with the development tools that work best for you to integrate with and extend Azure Devops. Use the REST APIs and contribution model available for the Test platform to create extensions that provide the experience you need for your test management lifecycle.

Share work items across your test experience

For example, you’re building version 1.* of your product and you might create several test cases for that version. Each of these test cases can be updated, and more added, at any time. For each development cycle and release of your product, you create a test plan and import the existing test cases into that plan. You can also, if you wish, divide the test cases into separate test suites within the plan to enable easier management and monitoring of these separate sets of test cases.

After you create your test plan, you assign test configurations and assign testers to cover the required test matrix. These testers run the tests and gauge the quality of the product. Testers continue testing until the product meets exit criteria. For the next development cycle and release, you can create a new test plan and reuse the same test cases. You repeat this development-test-release cycle by importing the same test cases into each new test plan.

The great thing is that, because test plans refer to test cases, updates to a test case automatically reflect in all the test plans and test suites that use it.

In the next version of the product, you can reuse the existing test cases. However, a better option may be to copy or clone the testcases. A copy creates a new baseline. Changes to these new test cases don’t affect your previous test plans.

Prerequisites

You must connect to a project. If you don’t have a project yet, create one
You must be added to a project. To get added, Add users to a project or team.
To view test-related work items, you must have Basic access or higher and permissions to view work items under the corresponding Area Path.
To add test plans and test suites, manual test cases, delete test artifacts, and define test configurations you must have Basic + Test Plans access level or have one of the following Visual Studio subscriptions:
To add or edit test-related artifacts, you must have the following permissions:
- To add or modify test plans, test suites, test cases, or other test-based work item types, you must have Edit work items in this node permission set to Allow under the corresponding Area Path.
- To modify test plan properties such as build and test settings, you must have Manage test plans permission set to Allow under the corresponding Area Path.
- To create and delete test suites, add, and remove test cases from test suites, change test configurations associated with test suites, and modify a test suite hierarchy (move a test suite), you must have Manage test suites permission set to Allow under the corresponding Area Path.

Create a test plan

In general, you create test plans to test requirements. Before creating a test plan, you may want to define your backlog of requirements.

From the web portal, open your project and go to Test Plans. If you already have a test plan, select Test Plans to go to the page that lists all test plans

2. In the Test Plans page, select New Test Plan to create a test plan for your current sprint.

3.In New Test Plan, enter a name for the test plan. Verify that the area path and iteration are set correctly, then select Create.

Rename a Test Plan

To rename a test plan, use this procedure:

Select Test Plans to view test plans.
Next to the test plan name, select More Actions > Edit.

3. Change the name and then select Save & Close.

You can make other changes to the test plan here

Delete a test plan

To delete a test plan, use this procedure:

Select Test Plans to view test plans.
Next to the test plan name, select More Actions > Delete.
The Permanently delete test artifacts dialog box explains exactly what will be deleted. Type the test plan ID to confirm that you want to delete, and then select Permanently delete.

Add a requirement-based test suite

Now add test suites for the backlog items that need manual tests. These tests could be user stories, requirements, or other work items based your project.

To add a suite to a test plan, select More options for the test suite, and then select New Suite > Requirement based suite.

You use requirement-based suites to group your test cases together. That way, you can track the testing status of a backlog item. Each test case that you add to a requirement-based test suite is automatically linked to the backlog item.

In Create requirement-based suites, add one or more clauses to filter your work items by the iteration path for the sprint. Run the query to view the matching backlog items

3. In the list of work items returned by the query, select the backlog items you want to test in this sprint. Select Create suites to create a requirement-based suite for each one

Work with test suites

You can create a static test suite that can contain any type of test suites. Use these test suites like folders. Drag test suites to group them in a static test plan. Drag and drop tests to reorder them.

You can track changes to test plans and test suites. Open the work item for the test plan or test suite, then view the work item history.

For test suites, other actions are tracked in the Test Suite Audit field. For example, adding and removing test cases from a test suite are tracked in this field.

You can export test plans, test suites, and test cases.

Select Export test cases to CSV

Change the test case fields in the report by adding or removing columns from the list view of the test suite.

Find a test case in a test plan

In Test Plans for your test plan, use the filter icon to show the search and filter list. It can help find the tests you want.