refactor: integrate new status_monitor framework, otaclient internal now fully decoupled from specific OTA API version, remove otaclient.app package

[Bodong-Yang]

Introduction

Note

This PR is part of #392 and finally completes the #392.
This PR is also an important part of the whole otaclient re-architecture project.

Note

This PR integrates the new status monitor framework introduced at #412 into otaclient.

This PR integrates the new status monitor framework introduced in #412 into otaclient, and finally finishes up decoupling otaclient internal implementation from specific OTA service API version.
With the above integration and cleanup finished, now otaclient.app package is fully disassembled and removed.

Other changes including:

1. _types: fixes OTAClientStatus missing failure-traceback field.
2. errors: fix OTAErrorUnrecoverable has failure_type set to RECOVERABLE.
3. rename status_monitor to _status_monitor, log_setting to _logging as these two packages are fully internal used.
4. main: guard most of the imports, especially grpc related, within the create_otaclient_grpc_server function and main function to avoid import side-effects.
5. main: construct otaclient, ECUStatusStorage and ECUTracker out of OTAClientAPIServicer.
6. ota_core: cleanup and remove unused code.

Tests

• implement tests that cover the changes.
• pytest passed.
• OTA e2e on VM passed.

[Bodong-Yang]

Start to fix up/implement test files for this PR.

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
src/ota_metadata/legacy
__init__.py	1	0	100%
parser.py	326	43	86%	100, 156, 161, 197–198, 208–209, 212, 224, 277–279, 283–286, 320–323, 392, 395, 403–405, 418, 427–428, 431–432, 597–599, 649–650, 653, 681–683, 737, 740–742
types.py	84	13	84%	37, 40–42, 112–116, 122–125
src/ota_metadata/utils
cert_store.py	42	0	100%
src/ota_proxy
__init__.py	36	10	72%	59, 61, 63, 72, 81–82, 102, 104–106
__main__.py	7	7	0%	16–18, 20, 22–23, 25
_consts.py	15	0	100%
cache_control_header.py	68	4	94%	71, 91, 113, 121
cache_streaming.py	144	13	90%	211, 225, 229–230, 265–266, 268, 280, 349, 367–370
config.py	17	0	100%
db.py	73	18	75%	109, 115, 153, 159–160, 163, 169, 171, 192–199, 201–202
errors.py	5	0	100%
lru_cache_helper.py	47	2	95%	84–85
ota_cache.py	215	59	72%	70–71, 140, 151–152, 184–185, 202, 239–243, 247–249, 251, 253–260, 262–264, 267–268, 272–273, 277, 324, 332–334, 413–416, 430, 433–434, 448–449, 451–453, 457–458, 464–465, 496, 502, 529, 581–583
server_app.py	139	39	71%	76, 79, 85, 101, 103, 162, 171, 213–214, 216–218, 221, 226–228, 231–232, 235, 238, 241, 244, 257–258, 261–262, 264, 267, 293–296, 299, 313–315, 321–323
utils.py	14	0	100%
src/otaclient
__init__.py	5	2	60%	17, 19
__main__.py	1	1	0%	16
_logging.py	51	33	35%	43–44, 46–47, 49–54, 56–57, 59–60, 62–65, 67, 77, 80–82, 84–86, 89–90, 92–96
_status_monitor.py	161	11	93%	46, 48–49, 157, 160, 177, 180, 196–197, 204, 207
_types.py	75	0	100%
errors.py	120	1	99%	97
main.py	49	29	40%	30, 32–39, 41–42, 44–45, 47–48, 52, 54, 59, 61, 67–69, 72–73, 77–80, 82
ota_core.py	299	89	70%	97, 124, 126–127, 129, 133, 137–138, 143–144, 150, 152, 191–194, 200, 204, 210, 349, 381–382, 384, 393, 396, 401–402, 405, 411, 413–417, 453–456, 459–466, 502–505, 585–592, 597, 600–606, 639–640, 646, 650–651, 657, 682–684, 686, 761, 789–790, 792–793, 801, 803–809
utils.py	37	2	94%	73–74
src/otaclient/boot_control
__init__.py	4	0	100%
_firmware_package.py	93	22	76%	82, 86, 136, 180, 186, 209–210, 213–218, 220–221, 224–229, 231
_grub.py	413	124	69%	214, 262–265, 271–275, 312–313, 320–325, 328–334, 337, 340–341, 346, 348–350, 359–365, 367–368, 370–372, 381–383, 385–387, 466–467, 471–472, 524, 530, 556, 578, 582–583, 598–600, 624–627, 639, 643–645, 647–649, 708–711, 736–739, 762–765, 777–778, 781–782, 817, 823, 843–844, 846, 871–873, 891–894, 922–925, 930–938, 943–951
_jetson_cboot.py	256	256	0%	20, 22–25, 27–29, 35–40, 42, 58–60, 62, 64–65, 71, 75, 134, 137, 139–140, 143, 150–151, 159–160, 163, 169–170, 178, 187–191, 193, 199, 202–203, 209, 212–213, 218–219, 221, 227–228, 231–232, 235–237, 239, 245, 250–252, 254–256, 261, 263–266, 268–269, 278–279, 282–283, 288–289, 292–296, 299–300, 305–306, 309, 312–316, 321–324, 327, 330–331, 334, 337–338, 341, 345–350, 354–355, 359, 362–363, 366, 369–372, 374, 377–378, 382, 385, 388–391, 393, 400, 404–405, 408–409, 415–416, 422, 424–425, 429, 431, 433–435, 438, 442, 445, 448–449, 451, 454, 462–463, 470, 480, 483, 491–492, 497–500, 502, 509, 511–513, 519–520, 524–525, 528, 532, 535, 537, 544–548, 550, 562–565, 568, 571, 573, 580, 590–592, 594, 596, 599, 602, 605, 607–608, 611–615, 619–621, 623, 631–635, 637, 640, 644, 647, 658–659, 664, 674, 677–685, 689–698, 702–711, 715, 717–719
_jetson_common.py	172	45	73%	132, 140, 288–291, 294, 311, 319, 354, 359–364, 382, 408–409, 411–413, 417–420, 422–423, 425–429, 431, 438–439, 442–443, 453, 456–457, 460, 462, 506–507
_jetson_uefi.py	399	271	32%	124–126, 131–132, 151–153, 158–161, 328, 446, 448–451, 455, 459–460, 462–470, 472, 484–485, 488–489, 492–493, 496–498, 502–503, 508–510, 514, 518–519, 522–523, 526–527, 531, 534–535, 537, 542–543, 547, 550–551, 556, 560–561, 564, 568–570, 572, 576–579, 581–582, 604–605, 609–610, 612, 616, 620–621, 624–625, 632, 635–637, 640, 642–643, 648–649, 652–655, 657–658, 663, 665–666, 674, 677–680, 682–683, 685, 689–690, 694, 702–706, 709–710, 712, 715–719, 722, 725–729, 733–734, 737–742, 745–746, 749–752, 754–755, 762–763, 773–776, 779, 782–785, 788–792, 795–796, 799, 802–805, 808, 810, 815–816, 819, 822–825, 827, 833, 838–839, 858–859, 862, 870–871, 878, 888, 891, 898–899, 904–907, 915–918, 926–927, 939–942, 944, 947, 950, 958, 969–971, 973–975, 977–981, 986–987, 989, 1002, 1006, 1009, 1019, 1024, 1032–1033, 1036, 1040, 1042–1044, 1050–1051, 1056, 1064–1071, 1076–1084, 1089–1097, 1103–1105
_ota_status_control.py	102	11	89%	117, 122, 127, 240, 244–245, 248, 255, 257–258, 273
_rpi_boot.py	282	130	53%	53, 56, 120–121, 125, 133–136, 150–153, 158–159, 161–162, 167–168, 171–172, 181–182, 222, 228–232, 235, 253–255, 259–261, 266–268, 272–274, 284–285, 288, 291, 293–294, 296–297, 299–301, 307, 310–311, 321–324, 332–336, 338, 340–341, 346–347, 354, 357–362, 393, 395–398, 408–411, 415–416, 418–422, 450–453, 472–475, 501–504, 509–517, 522–530, 547–550, 556–558
_slot_mnt_helper.py	12	0	100%
configs.py	51	0	100%
protocol.py	6	0	100%
selecter.py	41	29	29%	44–46, 49–50, 54–55, 58–60, 63, 65, 69, 77–79, 81–82, 84–85, 89, 91, 93–94, 96, 98–99, 101, 103
src/otaclient/configs
__init__.py	17	0	100%
_cfg_configurable.py	49	0	100%
_cfg_consts.py	47	1	97%	97
_common.py	8	0	100%
_ecu_info.py	56	3	94%	59, 64–65
_proxy_info.py	51	4	92%	85, 87–88, 90
cfg.py	19	0	100%
src/otaclient/create_standby
__init__.py	13	1	92%	36
common.py	226	44	80%	59, 62–63, 67–69, 71, 75–76, 78, 126, 174–176, 178–180, 182, 185–188, 192, 203, 279–280, 282–287, 299, 339, 367, 370–372, 388–389, 403, 407, 429–430
interface.py	7	0	100%
rebuild_mode.py	115	10	91%	98–100, 119, 150–155
src/otaclient/grpc
_otaproxy_ctx.py	117	51	56%	68–70, 72–73, 81–84, 87–89, 93, 98–99, 101–102, 105, 107–108, 111–113, 116–117, 120–122, 127–132, 136, 139–143, 145–146, 154–156, 159, 196–198, 203, 239
src/otaclient/grpc/api_v2
ecu_status.py	153	10	93%	75, 78, 81, 147, 171, 173, 300, 370–371, 410
ecu_tracker.py	34	19	44%	40–42, 48, 52–54, 61–63, 66, 70–74, 77–79
servicer.py	127	36	71%	85, 174–176, 183, 194–195, 236–237, 240–244, 253–262, 269, 275, 278–281, 287–288, 295, 298, 304, 307
types.py	46	2	95%	78–79
src/otaclient_api/v2
api_caller.py	39	6	84%	45–47, 83–85
types.py	256	23	91%	86, 89–92, 131, 209–210, 212, 259, 262–263, 506–508, 512–513, 515, 518–519, 522–523, 586
src/otaclient_common
__init__.py	34	15	55%	42–44, 61, 63, 68–77
_io.py	64	1	98%	41
cmdhelper.py	9	0	100%
common.py	106	11	89%	44, 148, 151–153, 168, 175–177, 271, 275
downloader.py	199	10	94%	107–108, 126, 153, 369, 424, 428, 516–517, 526
linux.py	61	15	75%	51–53, 59, 69, 74, 76, 108–109, 133–134, 190, 195–196, 198
logging.py	29	1	96%	55
persist_file_handling.py	118	18	84%	113, 118, 150–152, 163, 192–193, 228–232, 242–244, 246–247
proto_streamer.py	42	8	80%	33, 48, 66–67, 72, 81–82, 100
proto_wrapper.py	398	46	88%	87, 165, 172, 184–186, 205, 210, 221, 257, 263, 268, 299, 303, 307, 402, 462, 469, 472, 492, 499, 501, 526, 532, 535, 537, 562, 568, 571, 573, 605, 609, 611, 625, 642, 669, 672, 676, 707, 713, 762–763, 765, 803–805
retry_task_map.py	105	5	95%	158–159, 161, 181–182
typing.py	31	4	87%	48, 97–98, 100
TOTAL	6438	1608	75%

Tests	Skipped	Failures	Errors	Time
235	0 💤	0 ❌	0 🔥	11m 56s ⏱️

[Bodong-Yang]

Guard most of the imports within the main function and create_otaclient_grpc_server function. Especially grpcio package, this package might have some side-effect during import.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
77.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud