Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: rpi_boot: use chroot to run flash-kernel, ditch double reboot strategy #318

Merged
merged 29 commits into from
Jun 19, 2024

Conversation

Bodong-Yang
Copy link
Member

@Bodong-Yang Bodong-Yang commented Jun 7, 2024

Description

This PR introduces one-step firmware install strategy to rpi_boot to deprecate the previously used double steps firmware update strategy. It also includes some internal refinements in rpi_boot.

New features

  1. support flash-kernel directly in post-update phase with chroot to standby slot.
  2. now if system-boot partition is not mounted, rpi_boot will try to mount it.
  3. rpi_boot now allows extra partitions after partition ID 3.

Check list

  • test file(s) that covers the change(s) is implemented.
  • local test is passed.
  • test on real machine is passed.

Other changes

  1. otaclient_common.linux: implement new subprocess_run_wrapper, which supports chroot.
  2. otaclient.boot_control.common: implement a plain version of mount.

@Bodong-Yang Bodong-Yang added the refactor Rewrite/remove related code instead of patching them label Jun 9, 2024
@Bodong-Yang Bodong-Yang changed the title refactor: make rpi_boot faster by ditching two-stage reboot strategy refactor: faster rpi_boot implementation Jun 10, 2024
commit 2f02ab3
Author: bodong.yang <[email protected]>
Date:   Mon Jun 17 15:34:41 2024 +0000

    rpi_boot: use partition layout to determine the slot

commit 0c89de4
Author: bodong.yang <[email protected]>
Date:   Mon Jun 17 14:18:52 2024 +0000

    boot_control.common: fix prepare_standby_dev set standby fslabel

commit ea2832d
Author: bodong.yang <[email protected]>
Date:   Mon Jun 17 14:42:25 2024 +0000

    boot_control.common: implement get_device_tree

commit 5366b13
Author: Bodong Yang <[email protected]>
Date:   Thu Jun 13 23:26:35 2024 +0900

    deps: bump requests to 2.32 (#319)
@Bodong-Yang Bodong-Yang force-pushed the refactor/rpi_faster branch from 617e6da to f6be2aa Compare June 17, 2024 17:21
@Bodong-Yang Bodong-Yang changed the title refactor: faster rpi_boot implementation refactor: rpi_boot: use chroot to run flash-kernel, ditch double reboot strategy Jun 19, 2024
@Bodong-Yang Bodong-Yang marked this pull request as ready for review June 19, 2024 08:49
Copy link
Contributor

github-actions bot commented Jun 19, 2024

Coverage

Coverage Report
FileStmtsMissCoverMissing
src/ota_metadata/legacy
   __init__.py110100% 
   parser.py3263589%145, 150, 186–187, 197–198, 201, 213, 271, 281–284, 323–326, 406, 409, 417–419, 432, 441–442, 445–446, 720, 723, 735–740
   types.py841384%37, 40–42, 112–116, 122–125
src/ota_proxy
   __init__.py361072%59, 61, 63, 72, 81–82, 102, 104–106
   __main__.py770%16–18, 20, 22–23, 25
   _consts.py150100% 
   cache_control.py68494%71, 91, 113, 121
   config.py180100% 
   db.py1461589%75, 81, 103, 113, 116, 145–147, 166, 199, 208–209, 229, 258, 300
   errors.py50100% 
   orm.py1121091%92, 97, 102, 108, 114, 141–142, 155, 232, 236
   ota_cache.py4018678%98–99, 218, 229, 256–258, 278, 294–295, 297, 320–321, 327, 331, 333, 360–362, 378, 439–440, 482–483, 553, 566–569, 619, 638–639, 671–672, 683, 717–721, 725–727, 729, 731–738, 740–742, 745–746, 750–751, 755, 802, 810–812, 891–894, 898, 901–902, 916–917, 919–921, 925–926, 932–933, 964, 970, 997, 1026–1028
   server_app.py1383971%76, 79, 85, 101, 103, 162, 171, 213–214, 216–218, 221, 226–228, 231–232, 235, 238, 241, 244, 257–258, 261–262, 264, 267, 293–296, 299, 313–315, 321–323
   utils.py23195%33
src/otaclient
   __init__.py5260%17, 19
   __main__.py110%16
   log_setting.py52590%53, 55, 64–66
src/otaclient/app
   __main__.py110%16
   configs.py750100% 
   errors.py1120100% 
   interface.py50100% 
   main.py46589%52–53, 75–77
   ota_client.py39213166%67, 75, 96, 201–203, 214, 260–263, 275–278, 281–284, 294–297, 302–303, 305, 314, 317, 322–323, 326, 332, 334, 337, 379–382, 387, 391, 394, 410–413, 416–423, 426–433, 439–442, 471, 474–475, 477, 480–483, 485–486, 491–492, 495, 509–516, 523, 526–532, 579–582, 590, 626, 631–634, 639–641, 644–645, 647–648, 650–651, 653, 713–714, 717, 725–726, 729, 740–741, 744, 752–753, 756, 767, 786, 813, 832, 850
   ota_client_stub.py39410972%76–78, 80–81, 89–92, 95–97, 101, 106–107, 109–110, 113, 115–116, 119–121, 124–125, 128–130, 135–140, 144, 147–151, 153–154, 162–164, 167, 204–206, 211, 247, 272, 275, 278, 382, 406, 408, 432, 478, 535, 605–606, 645, 664–666, 672–675, 679–681, 688–690, 693, 697–700, 753, 842–844, 851, 881–882, 885–889, 898–907, 914, 920, 923–924, 928, 931
   update_stats.py106298%162, 172
src/otaclient/app/boot_control
   __init__.py40100% 
   _common.py24811254%74–75, 96–98, 114–115, 135–136, 155–156, 175–176, 195–196, 218–220, 235–236, 260–266, 287, 295, 313, 321, 340–341, 344–345, 368, 370–379, 381–390, 392–394, 413, 416, 424, 432, 448–450, 452–457, 550, 555, 560, 673, 677–678, 681, 689, 691–692, 718–719, 721–724, 729, 735–736, 739–740, 742, 749–750, 761–767, 775–777, 781–782, 785–786, 789, 795
   _grub.py41712869%216, 264–267, 273–277, 314–315, 322–327, 330–336, 339, 342–343, 348, 350–352, 361–367, 369–370, 372–374, 383–385, 387–389, 468–469, 473–474, 526, 532, 558, 580, 584–585, 600–602, 626–629, 641, 645–647, 649–651, 710–713, 738–741, 764–767, 779–780, 783–784, 819, 825, 845–846, 848, 860, 863, 866, 869, 873–875, 893–896, 924–927, 932–940, 945–953
   _jetson_cboot.py27021420%69–70, 77–78, 96–105, 117, 124–125, 137, 143–144, 154–156, 168–169, 180–181, 184–185, 188–189, 192–196, 199–200, 204–205, 210–211, 213–217, 219–225, 227–228, 233, 236, 239–240, 243, 247–248, 252–253, 257, 260, 263, 267–273, 275–277, 282, 285, 288, 292, 299, 301–304, 317, 320, 324, 326–328, 332, 339, 341, 344, 350–351, 356, 364, 372–374, 383–384, 386–388, 394, 397–399, 403–404, 406, 409, 418–420, 423, 426, 429–434, 436–438, 441, 444, 448–453, 457–459, 464–465, 469–470, 473, 476, 479–480, 483, 486, 491, 494, 497–498, 500, 502, 505, 508, 510–511, 514–518, 523–524, 526, 534–538, 540, 543, 546, 557–558, 563, 573, 576–584, 589–597, 602–610, 616–618, 621, 624
   _jetson_common.py1416653%50, 74, 129–134, 136, 141–143, 148–151, 159–160, 167–168, 173–174, 190–191, 193–195, 198–200, 203, 207, 211, 215–217, 223–224, 226, 259, 285–286, 288–290, 294–297, 299–300, 302–306, 308, 315–316, 319, 321, 331, 334–335, 338, 340
   _rpi_boot.py28613453%54, 57, 121–122, 126, 134–137, 151–154, 161–162, 164–165, 170–171, 174–175, 184–185, 223, 229–233, 236, 254–256, 260–262, 267–269, 273–275, 285–286, 289, 292, 294–295, 297–298, 300–302, 308, 311–312, 322–325, 333–337, 339, 341–342, 347–348, 355–361, 392, 394–397, 407–410, 414–415, 417–421, 449–452, 471–474, 479, 482, 500–503, 508–516, 521–529, 546–549, 555–557, 560, 563
   configs.py380100% 
   protocol.py40100% 
   selecter.py382631%44–46, 49–50, 54–55, 58–60, 63, 65, 69, 77–79, 81–82, 84–85, 89, 91–93, 95, 97
src/otaclient/app/create_standby
   __init__.py12558%28–30, 32, 34
   common.py2194380%63, 66–67, 71–73, 75, 79–80, 82, 128, 176–178, 180–182, 184, 187–190, 194, 205, 279–280, 282–287, 300, 355, 358–360, 376–377, 391, 395, 417–418
   interface.py50100% 
   rebuild_mode.py99990%94–96, 108–113
src/otaclient/configs
   _common.py80100% 
   ecu_info.py57198%107
   proxy_info.py52296%88, 90
src/otaclient_api/v2
   __init__.py140100% 
   api_caller.py39684%45–47, 83–85
   api_stub.py170100% 
   types.py2562391%86, 89–92, 131, 209–210, 212, 259, 262–263, 506–508, 512–513, 515, 518–519, 522–523, 586
src/otaclient_common
   __init__.py34876%42–44, 59, 61, 67, 74–75
   common.py1541987%41, 45, 200, 203–205, 220, 227–229, 295–297, 307, 316–318, 364, 368
   downloader.py2694384%72, 85–86, 301, 306, 328–329, 379–383, 402–404, 407–408, 411–412, 433–436, 440–441, 445–446, 450–451, 460, 535–537, 553, 573–575, 579, 581, 584, 589–591
   linux.py611575%51–53, 59, 69, 74, 76, 108–109, 133–134, 190, 195–196, 198
   logging.py29196%55
   persist_file_handling.py1131884%112, 114, 146–148, 150, 176–179, 184, 188–192, 218–219
   proto_streamer.py42880%33, 48, 66–67, 72, 81–82, 100
   proto_wrapper.py3984588%87, 165, 172, 184–186, 205, 210, 221, 257, 263, 268, 299, 303, 307, 402, 462, 469, 472, 492, 499, 501, 526, 532, 535, 537, 562, 568, 571, 573, 605, 609, 611, 625, 642, 669, 672, 676, 707, 713, 760–763, 765
   retry_task_map.py84396%141, 143, 158
   typing.py250100% 
TOTAL6012140576% 

Tests Skipped Failures Errors Time
179 0 💤 0 ❌ 0 🔥 5m 7s ⏱️

Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
58.5% Coverage on New Code (required ≥ 80%)

See analysis details on SonarCloud

@Bodong-Yang Bodong-Yang self-assigned this Jun 19, 2024
@Bodong-Yang Bodong-Yang merged commit deeb27d into main Jun 19, 2024
4 of 5 checks passed
@Bodong-Yang Bodong-Yang deleted the refactor/rpi_faster branch June 19, 2024 10:01
Bodong-Yang added a commit that referenced this pull request Jun 21, 2024
…ot strategy (#318)

This PR introduces one-step firmware install strategy to rpi_boot to deprecate the previously used double steps firmware update strategy. It also includes some internal refinements in rpi_boot.

* New features
1. support flash-kernel directly in post-update phase with chroot to standby slot.
2. now if system-boot partition is not mounted, rpi_boot will try to mount it.
3. rpi_boot now allows extra partitions after partition ID 3.

* Other changes
1. otaclient_common.linux: implement new subprocess_run_wrapper, which supports chroot.
2. otaclient.boot_control.common: implement a plain version of mount.
Bodong-Yang added a commit that referenced this pull request Jun 21, 2024
commit b546e8a
Author: bodong.yang <[email protected]>
Date:   Wed Jun 19 15:12:40 2024 +0000

    delta calculation: limit the threads and concurrent tasks

commit ea773ca
Merge: fb41c64 deeb27d
Author: Bodong Yang <[email protected]>
Date:   Thu Jun 20 00:07:19 2024 +0900

    Merge branch 'main' into refactor/delta_calculation_limit_concurrency

commit deeb27d
Author: Bodong Yang <[email protected]>
Date:   Wed Jun 19 19:01:29 2024 +0900

    refactor: rpi_boot: use chroot to run flash-kernel, ditch double reboot strategy (#318)

    This PR introduces one-step firmware install strategy to rpi_boot to deprecate the previously used double steps firmware update strategy. It also includes some internal refinements in rpi_boot.

    * New features
    1. support flash-kernel directly in post-update phase with chroot to standby slot.
    2. now if system-boot partition is not mounted, rpi_boot will try to mount it.
    3. rpi_boot now allows extra partitions after partition ID 3.

    * Other changes
    1. otaclient_common.linux: implement new subprocess_run_wrapper, which supports chroot.
    2. otaclient.boot_control.common: implement a plain version of mount.

commit 0773108
Author: Bodong Yang <[email protected]>
Date:   Wed Jun 19 10:35:26 2024 +0900

    refactor: rpi_boot: detects slot by partition tables, not by checking slot fslabel (#321)

    This PR refines the rpi_boot module to detect slot by examining the partition layout, not relying on reading the fslabel, and explicitly requires the expected partition tables(but support extra partitions after partition ID 3).

    Also this PR implements the feature that rpi_boot will correct the active slot's fslabel with correct slot_id after slot detection.

commit fb41c64
Author: bodong.yang <[email protected]>
Date:   Tue Jun 18 04:52:20 2024 +0000

    create_standby: add semaphore during delta generating
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
refactor Rewrite/remove related code instead of patching them
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant