[Chore] tools.images_packer: refine previous offline_ota_image_builder to support 2 image packing mode

[Bodong-Yang]

Description

This PR refines the tools.offline_ota_image_builder(renamed to images_packer) to explicitly provides two image packing mode(each requires different command args), which:

1. build-cache-src: build external cache source image and (optionally) prepare specific device as external cache source device,

2. build-offline-ota-imgs-bundle: build offline OTA image bundle for offline OTA.

Note

fully offline OTA helper script is not yet implemented.

For detailed usage, please check tools/images_packer/README.md.

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
otaclient/app
common.py	268	15	94%	55, 185, 200, 275–277, 287, 296–298, 404, 416, 504, 535, 539
configs.py	10	0	100%
copy_tree.py	80	3	96%	42, 93, 122
downloader.py	263	44	83%	68, 81–82, 297, 302, 306, 324–325, 375–379, 398–400, 403–404, 407–408, 429–432, 436–437, 441–442, 446–447, 456, 531–533, 549, 569–571, 575, 577, 580, 585–587
ecu_info.py	59	7	88%	76–77, 90, 95, 118–119, 128
errors.py	113	0	100%
interface.py	5	0	100%
log_setting.py	26	7	73%	27–28, 59, 61–63, 69
main.py	42	4	90%	48–49, 64, 67
ota_client.py	376	130	65%	52–53, 73, 175–176, 189, 192, 196, 199, 238–241, 253–256, 259–260, 264–269, 279–282, 326–329, 334, 338, 341, 354–357, 360–367, 370–377, 383–386, 413, 416–417, 419, 422–425, 427–428, 433–434, 437, 451–458, 465, 468–474, 521–524, 535, 571, 576–579, 584–586, 589–590, 592–593, 595–596, 598, 659–660, 663, 671, 675, 678, 689–690, 693, 701, 705, 708, 719, 738, 749–750, 767, 786, 790–791, 806
ota_client_call.py	40	6	85%	44–46, 82–84
ota_client_service.py	39	5	87%	60–61, 72–74
ota_client_stub.py	394	108	72%	78–80, 82–83, 91, 94–96, 99–101, 105–107, 109–110, 113–116, 118, 121–122, 125–127, 132–137, 141, 144–148, 150–151, 159–161, 164, 206–208, 213, 248, 273, 276, 279, 383, 409, 411, 437, 487, 549, 619–620, 659, 679–681, 687–690, 694–696, 703–705, 708, 712–715, 772, 859–861, 868, 898–899, 902–906, 915–924, 931, 937, 940–941, 945, 948
ota_metadata.py	315	31	90%	145, 150, 186–187, 197–198, 201, 213, 271, 304–306, 323–326, 406, 409, 417–419, 432, 441–442, 445–446, 719–720, 723, 727, 730
ota_status.py	14	2	85%	33, 41
proxy_info.py	47	5	89%	79, 81, 84, 119, 126
update_stats.py	105	2	98%	160, 170
otaclient/app/boot_control
__init__.py	4	0	100%
_cboot.py	293	142	51%	80–84, 86–88, 97–100, 102–104, 108, 110, 116–117, 119, 123, 130, 134, 138, 142–146, 150–154, 161–163, 167–169, 172–173, 176–177, 179, 181–183, 186–190, 193–197, 201, 206–210, 213, 215–216, 219, 225, 228, 231, 234, 237, 240, 243, 246–247, 250–251, 254, 256–258, 261–262, 277, 280, 320, 334–337, 350–353, 363–364, 366, 374, 413, 430–433, 442–445, 450–452, 461, 463–464, 466–467, 470, 477, 507–510, 544–547, 552–554, 559–563, 568–574
_common.py	370	148	60%	64–66, 68–70, 72–73, 82–83, 87–88, 96–97, 108–109, 114–119, 124–129, 138, 147, 151, 155, 159, 173–174, 176, 186–188, 190, 195, 199, 209–211, 213, 221–222, 230–231, 233, 244–248, 252–256, 268–270, 282–284, 295–296, 299–303, 306, 308–309, 319–323, 327–331, 333–340, 359, 363, 366, 375, 392, 395, 401–403, 411–417, 512, 517, 522, 581–582, 636, 640–641, 644, 652, 654–655, 711–712, 715, 729, 734–735, 737, 745, 748, 751–753, 756, 765, 825–828, 849–852, 868–869, 878
_errors.py	47	19	59%	43–46, 50–52, 59–67, 69, 89–90
_grub.py	405	114	71%	216, 264–267, 273–277, 314–315, 322–327, 330, 332, 335–337, 342, 344–346, 355–357, 359–361, 370–372, 374–376, 455–456, 460–461, 513, 519, 545, 564, 568–569, 584–586, 617, 621–623, 625–627, 682, 685–688, 707–710, 735–738, 768–771, 813, 819, 839–840, 842, 854, 857, 860, 863, 867–869, 887–890, 912–915, 920–928, 933–941
_rpi_boot.py	272	125	54%	84–86, 92–93, 95, 98, 101, 104, 109, 117–118, 120, 126, 130–133, 138–140, 144–147, 172–174, 180–182, 197–199, 205–207, 221–228, 230, 234, 238–239, 243–246, 249–250, 255, 259, 263, 267, 301, 331, 334–336, 344, 353–356, 366–369, 373–380, 424–426, 471, 474–477, 494–497, 502, 505, 526–529, 534–542, 547–555, 569–572, 578–580, 583
configs.py	106	5	95%	57–58, 177, 256, 276
firmware.py	31	5	83%	60, 62, 73–75
protocol.py	4	0	100%
selecter.py	38	27	28%	42–44, 46, 49–50, 52–54, 57, 59, 64–66, 74–76, 78–79, 81–83, 85–87, 89, 91
otaclient/app/create_standby
__init__.py	12	5	58%	29–31, 33, 35
common.py	217	92	57%	73, 76–77, 81–83, 85, 87–89, 92, 94, 142, 153, 190–192, 194–196, 198, 201–204, 208, 219, 292–294, 296–301, 303–313, 316–320, 322, 325, 347–348, 352, 356–357, 360–362, 368–369, 373, 375–380, 385–388, 392–393, 397, 405–408, 410, 415–417, 419, 422–423, 425, 431–432
interface.py	6	0	100%
rebuild_mode.py	98	7	92%	82, 84, 95–97, 102, 129
otaclient/app/proto
__init__.py	31	3	90%	37, 44–45
_common.py	407	47	88%	85, 163, 170, 182–184, 203, 208, 219, 256, 262, 267, 298, 302, 306, 403, 464, 471, 474, 494, 501, 503, 529, 535, 538, 540, 565, 571, 574, 576, 605, 609, 611, 625, 642, 670, 673, 677, 680, 708, 714, 761–764, 766, 797
_ota_metafiles_wrapper.py	84	16	80%	36–42, 112–116, 122–125
_otaclient_v2_pb2_wrapper.py	266	37	86%	87, 90–93, 132, 175, 183, 197, 207, 210–211, 213, 258, 261, 264–265, 285, 305, 385, 452, 505, 513–515, 519–520, 522, 525–526, 529–530, 551, 559, 573, 581, 595
streamer.py	43	8	81%	32, 47, 65–66, 71, 80–81, 99
wrapper.py	4	0	100%
otaclient/ota_proxy
__init__.py	31	6	80%	58–59, 61, 70, 79–80
_consts.py	15	0	100%
cache_control.py	68	4	94%	71, 91, 113, 121
config.py	18	0	100%
db.py	146	15	89%	75, 81, 103, 113, 116, 145–147, 166, 199, 208–209, 229, 258, 300
errors.py	10	0	100%
orm.py	113	10	91%	91, 96, 101, 107, 113, 140–141, 154, 231, 235
ota_cache.py	404	88	78%	95–96, 215, 226, 253–255, 275, 291–292, 294, 317–318, 324, 328, 330, 359–363, 379, 440–441, 483–484, 554, 567–570, 620, 639–640, 672–673, 684, 718–722, 726–728, 730, 732–739, 741–743, 746–747, 751–752, 756, 803, 811–813, 892–895, 899, 902–903, 917–918, 920–922, 926–927, 933–934, 965, 971, 998, 1027–1029
server_app.py	138	39	71%	75, 78, 84, 100, 102, 161, 170, 212–213, 215–217, 220, 225–227, 230–231, 234, 237, 240, 243, 256–257, 260–261, 263, 266, 292–295, 298, 312–314, 320–322
utils.py	23	1	95%	31
TOTAL	5900	1332	77%

Tests	Skipped	Failures	Errors	Time
172	0 💤	0 ❌	0 🔥	4m 55s ⏱️

[obi-t4]

I left some comments, please have a check!

[obi-t4]

_ecu_id key must be checked if the key already exists.

[Bodong-Yang]

Updated at a9edc6b. If same ECU has multiple target OTA image being specified, a warning log will be issued, and later setting will override the previous one.

[Bodong-Yang]

ota-client/tools/images_packer/__main__.py

Lines 72 to 79 in a9edc6b

	if _ecu_id in image_files:
	logger.warning(
	(
	f"override previously set OTA target image for ECU@{_ecu_id} "
	f"from {image_files[_ecu_id]} to {_image_fpath}"
	)
	)
	image_files[_ecu_id] = _image_fpath

[obi-t4]

@Bodong-Yang
bash script has been added at:
707857c

[obi-t4]

I think this should not be warning. For the user, this is the unexpected behavior, so it should be error.

[Bodong-Yang]

Yes I agree! This is unexpected behavior. Let me fix it.

[Bodong-Yang]

Updated at 786ef4d, now multiple images specified to the same ECU will cause the images_packer logging an error and then exit.

[obi-t4]

👍

[obi-t4]

LGTM 👍