Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release #826

Merged
merged 1 commit into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# How to report attributes to ReportPortal

Attributes are essential for categorizing, filtering, and analyzing test data. This article will guide you through the process of reporting attributes to our test results reporting service.

Be sure to check out our dedicated article for [best practices on using attributes](/work-with-reports/LaunchesTestItemsAttributes) effectively.

## Parameters of attributes

* Format – Key : Value
* Key – any string value, optional field
* Value – any string value, mandatory field
* The maximum limit is 512 symbols each for both key and value

Multiple values can be associated with a single key. For example, if we have a `jira_id` key, the value can differ based on the Jira ticket.

<MediaViewer src={require('./img/report-attributes/ReportAttributes1.png')} alt="jira_id key" />

:::important
The only valid use of the colon is to split the key and value.<br />
When a colon is present in the key or in the value, it breaks the string into two parts.
:::

### Colon scenarios

**1.** Key and value reported, colon used in key only.

**Example:**<br />
```{"key": "la:bel", "value": "smoke"}```

**Behavior on UI:**<br />
Key: la<br />
Value: bel

The original value "smoke" is not transmitted at all.

**2.** Key and value reported, colon used in both key and value.

**Example:**<br />
```{"key": "la:bel", "value": "smo:ke"}```

**Behavior on UI:**<br />
Key: la<br />
Value: bel

The original value "smoke" is not transmitted at all.

**3.** Key and value reported, colon used in value only.

**Example:**<br />
```{"key": "label", "value": "smo:ke"}```

**Behavior on UI:**<br />
Key: label<br />
Value: smo

The value is truncated at the colon, and only the part before the colon is transmitted.

**4.** Value reported, colon used in value only.

**Example:**<br />
```{"value": "smo:ke"}```

**Behavior on UI:**<br />
Value: smo

The value is truncated at the colon, and only the part before the colon is transmitted.

## Adding attributes via test framework integration

### JavaScript

Here is an example of adding attributes for a Playwright agent.

You can add attributes for the Launch in ReportPortal [agent config](https://github.com/reportportal/examples-js/blob/main/example-playwright/playwright.config.ts#L8):

<MediaViewer src={require('./img/report-attributes/ReportAttributes2.png')} alt="Add attributes for the Launch in Playwright" />

You can add attributes for the test item in your [test file](https://github.com/reportportal/examples-js/blob/main/example-playwright/tests/rp-features/attributes-and-description.spec.ts#L33):

<MediaViewer src={require('./img/report-attributes/ReportAttributes3.png')} alt="Add attributes for the test item in Playwright" />

### .NET

Attributes in .NET agents can be added declaratively or at runtime. To add launch attributes via standard [ReportPortal.json config file](https://github.com/reportportal/example-net-nunit/blob/68f596712e153fd67e45cbd479a9aa40a7d5423a/src/Example/ReportPortal.config.json#L15):

```json
{
"launch": {
"attributes": ["platform:unix", "arch:x64" ]
}
}
```

And to add test attributes ([sample for NUnit](https://github.com/reportportal/example-net-nunit/blob/68f596712e153fd67e45cbd479a9aa40a7d5423a/src/Example/Tests/Class1.cs#L142)):

```csharp
[Test, Category("browser:chrome")]
public void Test1()
{
ReportPortal.Shared.Context.Current.Metadata.Attributes.Add("browser", "chrome");
}
```

In Gherkin:

```
@browser:chrome
Scenario: Scenario1
```

## Adding attributes via API

You can initiate (launch, suite, test, or step) with one set of attributes and finish with a different set.

Importantly, the new attributes do not overwrite the old ones; instead, they are added.

<MediaViewer src={require('./img/report-attributes/ReportAttributes4.png')} alt="Adding attributes via API" />

## System attributes

In the ReportPortal, there is an opportunity to influence specific processes and data processing rules. To this end, ReportPortal supports system attributes. These attributes can be applied to the selected test items and trigger ReportPortal to take certain actions. System attributes are not visible on the UI (for that, the attribute should be sent with `system: true`).

### immediateAutoAnalysis

If `immediateAutoAnalysis=true`, then each test item will be analyzed right after it is completed, without waiting for the whole launch to finish.

<MediaViewer src={require('./img/report-attributes/ReportAttributes5.png')} alt="immediateAutoAnalysis attribute" />

### immediatePatternAnalysis

When `immediatePatternAnalysis=true`, then each test item will be analyzed right after it is completed, without waiting for the whole launch to finish.

<MediaViewer src={require('./img/report-attributes/ReportAttributes6.png')} alt="immediatePatternAnalysis attribute" />

### skippedIsNotIssue

If `skippedIsNotIssue=true` during launch import, then all test items with the SKIPPED status will be processed without applying a `To Investigate` defect type.

<MediaViewer src={require('./img/report-attributes/ReportAttributes7.png')} alt="skippedIsNotIssue attribute in our test automation reporting dashboard" />
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
115 changes: 32 additions & 83 deletions docs/work-with-reports/LaunchesTestItemsAttributes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ sidebar_label: Launches/test items attributes

For more convenient and effective use of our test report dashboard, we recommend paying attention to attributes.

## What are attributes?

Attributes are specific data points that can be included at all levels in ReportPortal:

**Launch**
Expand All @@ -27,88 +29,13 @@ Attributes are specific data points that can be included at all levels in Report
2. Attributes are used for filtering.
3. You can create a filter based on the attribute and then build widgets using this filter.

## Parameters of attributes

* Format – Key : Value
* Key – any string value, optional field
* Value – any string value, mandatory field
* The maximum limit is 512 symbols each for both key and value.

Multiple values can be associated with a single key. For example, if we have a `jira_id` key, the value can differ based on the Jira ticket.

<MediaViewer src={require('./img/launches-test-items-attributes/Attributes5.png')} alt="jira_id key" />

:::important
The only valid use of the colon is to split the key and value.<br />
When a colon is present in the key or in the value, it breaks the string into two parts.
:::

### Colon scenarios

**1.** Key and value reported, colon used in key only.

**Example:**<br />
```{"key": "la:bel", "value": "smoke"}```

**Behavior on UI:**<br />
Key: la<br />
Value: bel

The original value "smoke" is not transmitted at all.

**2.** Key and value reported, colon used in both key and value.

**Example:**<br />
```{"key": "la:bel", "value": "smo:ke"}```

**Behavior on UI:**<br />
Key: la<br />
Value: bel

The original value "smoke" is not transmitted at all.

**3.** Key and value reported, colon used in value only.

**Example:**<br />
```{"key": "label", "value": "smo:ke"}```

**Behavior on UI:**<br />
Key: label<br />
Value: smo

The value is truncated at the colon, and only the part before the colon is transmitted.

**4.** Value reported, colon used in value only.

**Example:**<br />
```{"value": "smo:ke"}```

**Behavior on UI:**<br />
Value: smo
You can [report attributes to ReportPortal](/log-data-in-reportportal/HowToReportAttributesToReportPortal) via test framework integrations or via API and add or edit them via the UI. For that, click on the pencil icon next to the desired launch/test item. Then click on the existing attribute to edit the key and value.

The value is truncated at the colon, and only the part before the colon is transmitted.
<MediaViewer src={require('./img/launches-test-items-attributes/Attributes5.png')} alt="Pencil icon to edit attribute via UI" />

## Adding attributes
<MediaViewer src={require('./img/launches-test-items-attributes/Attributes6.png')} alt="Clicking on existing attribute" />

Attributes can be added in the following ways:

1. Via test reporting.

You can initiate (launch, suite, test, or step) with one set of attributes and finish with a different set.

Importantly, the new attributes do not overwrite the old ones; instead, they are added.

<MediaViewer src={require('./img/launches-test-items-attributes/Attributes6.png')} alt="Adding attributes via test reporting" />

2. Via UI (manually).

To add or edit attributes, click on the `pencil` icon next to the desired launch/test item. Then click on the `Add new` link or click on the existing attribute to edit the key and value.

<MediaViewer src={require('./img/launches-test-items-attributes/Attributes7.png')} alt="pencil icon next to the desired launch/test item" />

<MediaViewer src={require('./img/launches-test-items-attributes/Attributes8.png')} alt="Add new attribute" />

<MediaViewer src={require('./img/launches-test-items-attributes/Attributes9.png')} alt="Adding attributes via UI" />
<MediaViewer src={require('./img/launches-test-items-attributes/Attributes7.png')} alt="Editing attribute via UI" />

## Best practices for using attributes

Expand Down Expand Up @@ -155,7 +82,9 @@ Below you can find some examples of how attributes can describe launch/test item

<MediaViewer src={require('./img/launches-test-items-attributes/Attributes13.png')} alt="Examples of categorizing tests using attributes" />

## Use Case: using attributes for filtering
## Use Cases

### Using attributes for filtering

Let's consider the following example. In the launch below, there are 31 Failed tests.

Expand All @@ -175,9 +104,9 @@ As a result of the investigation, we found that the least number of failures occ

Thus, we understand that we need to focus on tests with controller type `contr_launch`.

## Use Case: using attributes for creating widgets
### Using attributes for creating widgets

Widgets allow you to visually track metrics for launches. We can define these metrics using filters that can be created based on attributes. For example, to determine the growth trend of summary statistics of launches with the same attribute key, let's construct a [Cumulative trend chart widget](/dashboards-and-widgets/CumulativeTrendChart).
Widgets allow you to visually track [qa and test automation metrics metrics](/dashboards-and-widgets/ReportingAndMetricsInReportPortal/#qa-and-test-automation-metrics-in-reportportal) for launches. We can define these metrics using filters that can be created based on attributes. For example, to determine the growth trend of summary statistics of launches with the same attribute key, let's construct a [Cumulative trend chart widget](/dashboards-and-widgets/CumulativeTrendChart).

There are some launches with the `smoke` attribute.

Expand All @@ -195,6 +124,26 @@ On the screenshot below, we can observe a growth in statistics on the platforms

<MediaViewer src={require('./img/launches-test-items-attributes/Attributes22.png')} alt="Growth in statistics on the platforms from one build to the next" />

Based on attributes specified for test items, you can also build a [Component Health Check widget](/dashboards-and-widgets/ComponentHealthCheck/) to visually represent the passing rate of different application components.

For example, if there are some launches with the `demo` attribute, you can create a filter using this attribute.

<MediaViewer src={require('./img/launches-test-items-attributes/ComponentHealthCheck1.png')} alt="Launches with the demo attribute" />

Next, construct a Component Health Check widget using this filter. Specify the components, functionality, platforms, or other criteria you want to monitor by defining the relevant attributes across different levels.

<MediaViewer src={require('./img/launches-test-items-attributes/ComponentHealthCheck2.png')} alt="Building a Component Health Check widget using filter" />

As a result:

- On the first level, you can view the passing rate for each build.

<MediaViewer src={require('./img/launches-test-items-attributes/ComponentHealthCheck3.png')} alt="First level of Component Health Check widget" />

- On the second level, you can see which platform has the highest number of failed test cases, helping you identify areas that may need more attention.

<MediaViewer src={require('./img/launches-test-items-attributes/ComponentHealthCheck4.png')} alt="Second level of Component Health Check widget" />

Therefore, properly using attributes in ReportPortal can greatly optimize test management and assist in providing clearer visualizations of trends through widgets.

There are also [system attributes](/work-with-reports/SystemAttributes) that allow to extend the functionality of ReportPortal.
There are also [system attributes](/log-data-in-reportportal/HowToReportAttributesToReportPortal#system-attributes) that allow to extend the functionality of ReportPortal.
26 changes: 0 additions & 26 deletions docs/work-with-reports/SystemAttributes.mdx

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading