Changed times and hue steps in TC-CC-3.3 so tolerance does not span wrap #37308

r-recchia · 2025-01-29T18:13:00Z

This PR updates TC-CC-3.3.yaml file. This PR is linked to Fixes #475. The [PR #35525] was used as guidance.

It also includes changes in test specifications that were done in PR 4993.

Updates:

Changed delays and hue steps
Removed manual steps to check values
Updated CC.S.F00 value

Testing:

To not skip test steps CC.S.F00 needs to be 1. Changed this value in src/app/tests/suites/certification/ci-pics-values.

In the first terminal:

Clear commissioning data:
```
rm /tmp/chip_*
```

Build light:

./scripts/build/build_examples.py –target linux-x64-light-no-ble build

Run:

./out/linux-x64-light-no-ble/chip-lighting-app

In the second terminal:

Commission:

./out/linux–x64-chip-tool/chip-tool pairing code 0x12344321 MT:-24J042C00KA0648G00

In the third terminal:

Test:

./scripts/tests/chipyaml/chiptool.py tests Test_TC_CC_3_3 –server_path ./out/linux-x64-chip-tool/chip-tool

…t span wrap

semanticdiff-com · 2025-01-29T18:13:04Z

Review changes with

Changed Files

File	Status
src/app/tests/suites/certification/Test_TC_CC_3_3.yaml	0% smaller
src/app/tests/suites/certification/ci-pics-values	Unsupported file format

CLAassistant · 2025-01-29T18:13:07Z

All committers have signed the CLA.

github-actions · 2025-01-29T19:01:41Z

PR #37308: Size comparison from e80451c to 962fd93

Full report (71 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)

platform	target	config	section	`e80451c`	`962fd93`	change	% change
bl602	lighting-app	bl602+mfd+littlefs+rpc	FLASH	1093546	1093546	0	0.0
			RAM	103298	103298	0	0.0
bl702	lighting-app	bl702+eth	FLASH	650158	650158	0	0.0
			RAM	25265	25265	0	0.0
		bl702+wifi	FLASH	828066	828066	0	0.0
			RAM	13981	13981	0	0.0
		bl706+mfd+rpc+littlefs	FLASH	1056626	1056626	0	0.0
			RAM	23861	23861	0	0.0
bl702l	contact-sensor-app	bl702l+mfd+littlefs	FLASH	888060	888060	0	0.0
			RAM	18504	18504	0	0.0
	lighting-app	bl702l+mfd+littlefs	FLASH	971046	971046	0	0.0
			RAM	16368	16368	0	0.0
cc13x4_26x4	lighting-app	LP_EM_CC1354P10_6	FLASH	838000	838000	0	0.0
			RAM	123464	123464	0	0.0
	lock-ftd	LP_EM_CC1354P10_6	FLASH	823420	823420	0	0.0
			RAM	125344	125344	0	0.0
	pump-app	LP_EM_CC1354P10_6	FLASH	770644	770644	0	0.0
			RAM	113804	113804	0	0.0
	pump-controller-app	LP_EM_CC1354P10_6	FLASH	754888	754888	0	0.0
			RAM	114012	114012	0	0.0
cc32xx	air-purifier	CC3235SF_LAUNCHXL	FLASH	538445	538445	0	0.0
			RAM	205192	205192	0	0.0
	lock	CC3235SF_LAUNCHXL	FLASH	572309	572309	0	0.0
			RAM	205344	205344	0	0.0
cyw30739	light	CYW30739B2-P5-EVK-01	unknown	2040	2040	0	0.0
			FLASH	679409	679409	0	0.0
			RAM	78532	78532	0	0.0
		CYW30739B2-P5-EVK-02	unknown	2040	2040	0	0.0
			FLASH	699261	699261	0	0.0
			RAM	81172	81172	0	0.0
		CYW30739B2-P5-EVK-03	unknown	2040	2040	0	0.0
			FLASH	699261	699261	0	0.0
			RAM	81172	81172	0	0.0
		CYW930739M2EVB-02	unknown	2040	2040	0	0.0
			FLASH	656189	656189	0	0.0
			RAM	73600	73600	0	0.0
	light-switch	CYW30739B2-P5-EVK-01	unknown	2040	2040	0	0.0
			FLASH	615793	615793	0	0.0
			RAM	71516	71516	0	0.0
		CYW30739B2-P5-EVK-02	unknown	2040	2040	0	0.0
			FLASH	635421	635421	0	0.0
			RAM	74060	74060	0	0.0
		CYW30739B2-P5-EVK-03	unknown	2040	2040	0	0.0
			FLASH	635421	635421	0	0.0
			RAM	74060	74060	0	0.0
	lock	CYW30739B2-P5-EVK-01	unknown	2040	2040	0	0.0
			FLASH	635273	635273	0	0.0
			RAM	74524	74524	0	0.0
		CYW30739B2-P5-EVK-02	unknown	2040	2040	0	0.0
			FLASH	654981	654981	0	0.0
			RAM	77068	77068	0	0.0
		CYW30739B2-P5-EVK-03	unknown	2040	2040	0	0.0
			FLASH	654981	654981	0	0.0
			RAM	77068	77068	0	0.0
	thermostat	CYW30739B2-P5-EVK-01	unknown	2040	2040	0	0.0
			FLASH	611709	611709	0	0.0
			RAM	68612	68612	0	0.0
		CYW30739B2-P5-EVK-02	unknown	2040	2040	0	0.0
			FLASH	631569	631569	0	0.0
			RAM	71252	71252	0	0.0
		CYW30739B2-P5-EVK-03	unknown	2040	2040	0	0.0
			FLASH	631569	631569	0	0.0
			RAM	71252	71252	0	0.0
efr32	lock-app	BRD4187C	FLASH	936232	936232	0	0.0
			RAM	159904	159904	0	0.0
		BRD4338a	FLASH	729900	729892	-8	-0.0
			RAM	234764	234764	0	0.0
	window-app	BRD4187C	FLASH	1029256	1029248	-8	-0.0
			RAM	128040	128040	0	0.0
esp32	all-clusters-app	c3devkit	DRAM	97296	97296	0	0.0
			FLASH	`1577036`	`1577036`	0	0.0
			IRAM	83820	83820	0	0.0
		m5stack	DRAM	116092	116092	0	0.0
			FLASH	1544978	1544978	0	0.0
			IRAM	117039	117039	0	0.0
linux	air-purifier-app	debug	unknown	4760	4760	0	0.0
			FLASH	2708701	2708701	0	0.0
			RAM	132800	132800	0	0.0
	all-clusters-app	debug	unknown	5568	5568	0	0.0
			FLASH	`5974756`	`5974756`	0	0.0
			RAM	531616	531616	0	0.0
	all-clusters-minimal-app	debug	unknown	5464	5464	0	0.0
			FLASH	5323442	5323442	0	0.0
			RAM	242728	242728	0	0.0
	bridge-app	debug	unknown	5480	5480	0	0.0
			FLASH	4681630	4681630	0	0.0
			RAM	221464	221464	0	0.0
	chip-tool	debug	unknown	6120	6120	0	0.0
			FLASH	13095836	13095836	0	0.0
			RAM	596746	596746	0	0.0
	chip-tool-ipv6only	arm64	unknown	21848	21848	0	0.0
			FLASH	11161856	11161856	0	0.0
			RAM	648472	648472	0	0.0
	fabric-admin	debug	unknown	5808	5808	0	0.0
			FLASH	11387883	11387883	0	0.0
			RAM	596522	596522	0	0.0
	fabric-bridge-app	debug	unknown	4736	4736	0	0.0
			FLASH	4506228	4506228	0	0.0
			RAM	208648	208648	0	0.0
	fabric-sync	debug	unknown	4976	4976	0	0.0
			FLASH	5612469	5612469	0	0.0
			RAM	483520	483520	0	0.0
	lighting-app	debug+rpc+ui	unknown	6144	6144	0	0.0
			FLASH	5624385	5624385	0	0.0
			RAM	231744	231744	0	0.0
	lock-app	debug	unknown	5416	5416	0	0.0
			FLASH	4730680	4730680	0	0.0
			RAM	207712	207712	0	0.0
	ota-provider-app	debug	unknown	4776	4776	0	0.0
			FLASH	4359256	4359256	0	0.0
			RAM	201352	201352	0	0.0
	ota-requestor-app	debug	unknown	4728	4728	0	0.0
			FLASH	4496760	4496760	0	0.0
			RAM	205936	205936	0	0.0
	shell	debug	unknown	4256	4256	0	0.0
			FLASH	`3004845`	`3004845`	0	0.0
			RAM	160504	160504	0	0.0
	thermostat-no-ble	arm64	unknown	9536	9536	0	0.0
			FLASH	4098640	4098640	0	0.0
			RAM	246136	246136	0	0.0
	tv-app	debug	unknown	5744	5744	0	0.0
			FLASH	5952101	5952101	0	0.0
			RAM	606920	606920	0	0.0
	tv-casting-app	debug	unknown	5320	5320	0	0.0
			FLASH	11269485	11269485	0	0.0
			RAM	710864	710864	0	0.0
nrfconnect	all-clusters-app	nrf52840dk_nrf52840	FLASH	907096	907096	0	0.0
			RAM	142395	142395	0	0.0
		nrf7002dk_nrf5340_cpuapp	FLASH	901624	901624	0	0.0
			RAM	124739	124739	0	0.0
	all-clusters-minimal-app	nrf52840dk_nrf52840	FLASH	845824	845824	0	0.0
			RAM	141323	141323	0	0.0
nxp	contact	k32w0+release	FLASH	584288	584288	0	0.0
			RAM	70860	70860	0	0.0
		mcxw71+release	FLASH	599632	599632	0	0.0
			RAM	63080	63080	0	0.0
	light	k32w0+release	FLASH	610732	610732	0	0.0
			RAM	70252	70252	0	0.0
		k32w1+release	FLASH	685192	685192	0	0.0
			RAM	48664	48664	0	0.0
	lock	mcxw71+release	FLASH	748664	748664	0	0.0
			RAM	67476	67476	0	0.0
psoc6	all-clusters	cy8ckit_062s2_43012	FLASH	1646372	1646372	0	0.0
			RAM	211560	211560	0	0.0
	all-clusters-minimal	cy8ckit_062s2_43012	FLASH	`1553148`	`1553148`	0	0.0
			RAM	208376	208376	0	0.0
	light	cy8ckit_062s2_43012	FLASH	1468828	1468828	0	0.0
			RAM	200352	200352	0	0.0
	lock	cy8ckit_062s2_43012	FLASH	1466860	1466860	0	0.0
			RAM	224688	224688	0	0.0
qpg	lighting-app	qpg6105+debug	FLASH	661984	661984	0	0.0
			RAM	105204	105204	0	0.0
	lock-app	qpg6105+debug	FLASH	619788	619788	0	0.0
			RAM	99648	99648	0	0.0
stm32	light	STM32WB5MM-DK	FLASH	482600	482600	0	0.0
			RAM	144672	144672	0	0.0
telink	bridge-app	tlsr9258a	FLASH	681370	681370	0	0.0
			RAM	91084	91084	0	0.0
	contact-sensor-app	tlsr9528a_retention	FLASH	621400	621400	0	0.0
			RAM	31484	31484	0	0.0
	light-app-ota-compress-lzma-shell-factory-data	tl3218x	FLASH	770240	770240	0	0.0
			RAM	49344	49344	0	0.0
	light-app-ota-shell-factory-data	tl7218x	FLASH	774670	774670	0	0.0
			RAM	99648	99648	0	0.0
	light-switch-app-ota-compress-lzma-shell-factory-data	tlsr9528a	FLASH	708574	708574	0	0.0
			RAM	73376	73376	0	0.0
	lighting-app-ota-factory-data	tlsr9118bdk40d	FLASH	625672	625672	0	0.0
			RAM	142016	142016	0	0.0
	lighting-app-ota-rpc-factory-data-4mb	tlsr9518adk80d	FLASH	813052	813052	0	0.0
			RAM	99560	99560	0	0.0
tizen	all-clusters-app	arm	unknown	5116	5116	0	0.0
			FLASH	1751892	1751892	0	0.0
			RAM	93524	93524	0	0.0
	chip-tool-ubsan	arm	unknown	11408	11408	0	0.0
			FLASH	18695966	18695966	0	0.0
			RAM	8183700	8183700	0	0.0

andy31415

LGTM however we should wait on the dependency in https://github.com/CHIP-Specifications/chip-test-plans/pull/4933 to merge first.

cecille

Can you please go through and double check these numbers against the test plan? Some of them are off.

r-recchia added 2 commits January 29, 2025 14:06

CC.S.F00 updated to 1 to not skip test steps in TC-CC-3.3.yaml

9d8b6a4

Remove manual steps, change delays and step size so tolerance does no…

28b06af

…t span wrap

github-actions bot added app tests labels Jan 29, 2025

r-recchia mentioned this pull request Jan 29, 2025

TC-CC-3.3: Automate project-chip/matter-test-scripts#475

Open

Fixing identation issues

962fd93

pullapprove bot added the review - pending label Jan 29, 2025

dmaranges-gd mentioned this pull request Jan 31, 2025

[TC-CC-8.1] Update test to automate steps #37335

Open

andy31415 approved these changes Feb 4, 2025

View reviewed changes

r-recchia added 2 commits February 11, 2025 20:09

Fix toleFix tolerance calculation}

6e2b61e

Updated comments

c682cff

cecille reviewed Feb 24, 2025

View reviewed changes

r-recchia added 4 commits February 25, 2025 13:40

Fix one comment in the tolerance value

c15803c

Remove json

71b52d1

Updated labels

6162d7b

Added closed ranged in tolerance comments

ea7738d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changed times and hue steps in TC-CC-3.3 so tolerance does not span wrap #37308

Changed times and hue steps in TC-CC-3.3 so tolerance does not span wrap #37308

r-recchia commented Jan 29, 2025 •

edited

Loading

semanticdiff-com bot commented Jan 29, 2025 •

edited

Loading

CLAassistant commented Jan 29, 2025 •

edited

Loading

github-actions bot commented Jan 29, 2025 •

edited

Loading

andy31415 left a comment

cecille left a comment

Changed times and hue steps in TC-CC-3.3 so tolerance does not span wrap #37308

Are you sure you want to change the base?

Changed times and hue steps in TC-CC-3.3 so tolerance does not span wrap #37308

Conversation

r-recchia commented Jan 29, 2025 • edited Loading

Updates:

Testing:

semanticdiff-com bot commented Jan 29, 2025 • edited Loading

CLAassistant commented Jan 29, 2025 • edited Loading

github-actions bot commented Jan 29, 2025 • edited Loading

andy31415 left a comment

Choose a reason for hiding this comment

cecille left a comment

Choose a reason for hiding this comment

r-recchia commented Jan 29, 2025 •

edited

Loading

semanticdiff-com bot commented Jan 29, 2025 •

edited

Loading

CLAassistant commented Jan 29, 2025 •

edited

Loading

github-actions bot commented Jan 29, 2025 •

edited

Loading