Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history 
commit b4b9547
Author: bodong.yang <[email protected]>
Date:   Wed Dec 18 07:51:29 2024 +0000

    ota_core: wait on any_child_ecu_in_update not set

commit 1fd8994
Author: bodong.yang <[email protected]>
Date:   Wed Dec 18 07:35:21 2024 +0000

    minor fix

commit 276deff
Author: bodong.yang <[email protected]>
Date:   Wed Dec 18 07:33:38 2024 +0000

    fix up test_ecu_status

commit 2f5d322
Author: bodong.yang <[email protected]>
Date:   Wed Dec 18 07:22:29 2024 +0000

    fix up test_ecu_status

commit 7a57a6f
Author: bodong.yang <[email protected]>
Date:   Wed Dec 18 07:11:40 2024 +0000

    ecu_status: set ecu_status_flags.any_child_ecu_in_update accordingly

commit 1ac7cc9
Author: bodong.yang <[email protected]>
Date:   Wed Dec 18 07:08:32 2024 +0000

    otaclient.main: MultipleECUStatusFlags: rename any_in_update to any_child_ecu_in_update

commit f19560c
Author: bodong.yang <[email protected]>
Date:   Wed Dec 18 07:08:11 2024 +0000

    otaclient._types: MultipleECUStatusFlags: rename any_in_update to any_child_ecu_in_update

commit 3d05444
Author: Keisuke Nakata <[email protected]>
Date:   Tue Dec 17 10:21:54 2024 +0900

    docs: update DEVELOPMENT.md (#455)

    update DEVELOPMENT.md

commit b710bb0
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Tue Dec 17 09:53:24 2024 +0900

    build(deps): Update uvicorn[standard] requirement (#457)

    Updates the requirements on [uvicorn[standard]](https://github.com/encode/uvicorn) to permit the latest version.
    - [Release notes](https://github.com/encode/uvicorn/releases)
    - [Changelog](https://github.com/encode/uvicorn/blob/master/CHANGELOG.md)
    - [Commits](encode/uvicorn@0.30.0...0.34.0)

    ---
    updated-dependencies:
    - dependency-name: uvicorn[standard]
      dependency-type: direct:production
    ...

    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

commit 3ffaf90
Author: Bodong Yang <[email protected]>
Date:   Mon Dec 16 15:30:09 2024 +0900

    feat: deps: bump to use simple-sqlite3-orm v0.5.0, update the otaproxy accordingly (#454)

    This PR bumps to use simple-sqlite3-orm v0.5.0 for otaproxy, and updates the otaproxy accordingly. Also missing indexes for ota_cache metadata db (previously defined before otaclient v3.7.1) are added back to otaproxy.

    Other minor changes:
    1. lru_cache_helper: use burst_suppressed_logger for loggings that might be flooded.
  • Loading branch information
@Bodong-Yang
Bodong-Yang committed Dec 18, 2024
1 parent 15530e3 commit 7a97d2e
Show file tree
Hide file tree
Showing 8 changed files with 77 additions and 72 deletions.
48 changes: 1 addition & 47 deletions DEVELOPMENT.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,52 +14,6 @@ $ python3 --version
Python 3.8.10
```

## How to run OTA client on the target ECU

Appropriate partitioning and configurations are required. See [docs/README.md](docs/README.md).

Installation guide is available at [docs/INSTALLATION.md](docs/INSTALLATION.md).

### Run otaclient directly

```bash
# with virtualenv activated
python3 -m otaclient
# or
python3 -m otaclient.app
```

### Run otaclient installed at custom location

If we install the otaclient to custom directory instead of the default location, we must indicate python the path to the install location.

#### method 1: indicate path by **PYTHONPATH**

```bash
# we have to append the /opt/ota to the PYTHONPATH, to tell the
# python interpreter to search otaclient package under /opt/ota, instead of
# using the one install under <virtualenv>/lib/python3.8/site-packages

# with venv activated:
PYTHONPATH=/opt/ota python3 -m otaclient
# or
PYTHONPATH=/opt/ota python3 -m otaclient.app
```

#### method 2: change working dir to the install location

```bash
# change work dir to /opt/ota
cd /opt/ota

# with venv activated:
# NOTE:python will insert current working dir at index 0 in `sys.path`
# under /opt/ota folder, so that python will first use otaclient under /opt/ota
python3 -m otaclient
# or
python3 -m otaclient.app
```

## How to test OTA client on the development PC

### Build the image for testing
Expand Down Expand Up @@ -158,7 +112,7 @@ Type "help", "copyright", "credits" or "license" for more information.
>>> from otaclient_pb2.v2 import otaclient_pb2_grpc
```
### Updating docs/SERVICES.md
### Creating docs/SERVICES.md
The protobuf document docs/SERVICES.md should be updated by protoc-gen-doc tool.
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ dependencies = [
"simple-sqlite3-orm==0.6.0rc0",
"typing-extensions>=4.6.3",
"urllib3<2.3,>=2.2.2",
"uvicorn[standard]>=0.30,<0.33",
"uvicorn[standard]>=0.30,<0.35",
"zstandard<0.24,>=0.22",
]
optional-dependencies.dev = [
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,5 @@ requests<2.33,>=2.32
simple-sqlite3-orm<0.6,>=0.5
typing-extensions>=4.6.3
urllib3<2.3,>=2.2.2
uvicorn[standard]>=0.30,<0.33
uvicorn[standard]>=0.30,<0.35
zstandard<0.24,>=0.22
2 changes: 1 addition & 1 deletion src/otaclient/_types.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ class OTAClientStatus:

@dataclass
class MultipleECUStatusFlags:
any_in_update: mp_sync.Event
any_child_ecu_in_update: mp_sync.Event
any_requires_network: mp_sync.Event
all_success: mp_sync.Event

Expand Down
22 changes: 11 additions & 11 deletions src/otaclient/grpc/api_v2/ecu_status.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -195,10 +195,11 @@ async def _generate_overall_status_report(self):
"new ECU(s) that acks update request and enters OTA update detected"
f"{_new_in_update_ecu}, current updating ECUs: {in_update_ecus_id}"
)
if in_update_ecus_id:
ecu_status_flags.any_in_update.set()

if self.in_update_child_ecus_id:
ecu_status_flags.any_child_ecu_in_update.set()
else:
ecu_status_flags.any_in_update.clear()
ecu_status_flags.any_child_ecu_in_update.clear()

# check if there is any failed child/self ECU in tracked active ECUs set
_old_failed_ecus_id = self.failed_ecus_id
Expand Down Expand Up @@ -346,7 +347,10 @@ async def on_ecus_accept_update_request(self, ecus_accept_update: set[str]):

ecu_status_flags.all_success.clear()
ecu_status_flags.any_requires_network.set()
ecu_status_flags.any_in_update.set()
if self.in_update_child_ecus_id:
ecu_status_flags.any_child_ecu_in_update.set()
else:
ecu_status_flags.any_child_ecu_in_update.clear()

def get_polling_interval(self) -> int:
"""Return <ACTIVE_POLLING_INTERVAL> if there is active OTA update,
Expand All @@ -355,11 +359,8 @@ def get_polling_interval(self) -> int:
NOTE: use get_polling_waiter if want to wait, only call this method
if one only wants to get the polling interval value.
"""
ecu_status_flags = self.ecu_status_flags
return (
ACTIVE_POLLING_INTERVAL
if ecu_status_flags.any_in_update.is_set()
else IDLE_POLLING_INTERVAL
ACTIVE_POLLING_INTERVAL if self.in_update_ecus_id else IDLE_POLLING_INTERVAL
)

def get_polling_waiter(self):
Expand All @@ -377,13 +378,12 @@ def get_polling_waiter(self):
_inner_wait_interval = 1 # second

async def _waiter():
ecu_status_flags = self.ecu_status_flags
if ecu_status_flags.any_in_update.is_set():
if self.in_update_ecus_id:
await asyncio.sleep(ACTIVE_POLLING_INTERVAL)
return

for _ in range(math.ceil(IDLE_POLLING_INTERVAL / _inner_wait_interval)):
if ecu_status_flags.any_in_update.is_set():
if self.in_update_ecus_id:
return
await asyncio.sleep(_inner_wait_interval)

Expand Down
2 changes: 1 addition & 1 deletion src/otaclient/main.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ def main() -> None: # pragma: no cover
local_otaclient_op_queue = mp_ctx.Queue()
local_otaclient_resp_queue = mp_ctx.Queue()
ecu_status_flags = MultipleECUStatusFlags(
any_in_update=mp_ctx.Event(),
any_child_ecu_in_update=mp_ctx.Event(),
any_requires_network=mp_ctx.Event(),
all_success=mp_ctx.Event(),
)
Expand Down
4 changes: 2 additions & 2 deletions src/otaclient/ota_core.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -656,10 +656,10 @@ def _execute_update(self):
session_id=self.session_id,
)
)

if proxy_info.enable_local_ota_proxy:
wait_and_log(
check_flag=self.ecu_status_flags.any_in_update.is_set,
check_flag=self.ecu_status_flags.any_child_ecu_in_update.is_set,
check_for=False,
message="permit reboot flag",
log_func=logger.info,
Expand Down
67 changes: 59 additions & 8 deletions tests/test_otaclient/test_grpc/test_api_v2/test_ecu_status.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ async def setup_test(self, mocker: MockerFixture, ecu_info_fixture: ECUInfo):
# init and setup the ecu_storage
# NOTE: here we use threading.Event instead
self.ecu_status_flags = ecu_status_flags = MultipleECUStatusFlags(
any_in_update=threading.Event(), # type: ignore[assignment]
any_child_ecu_in_update=threading.Event(), # type: ignore[assignment]
any_requires_network=threading.Event(), # type: ignore[assignment]
all_success=threading.Event(), # type: ignore[assignment]
)
Expand Down Expand Up @@ -380,7 +380,7 @@ async def test_export(
},
# ecu_status_flags
{
"any_in_update": True,
"any_child_ecu_in_update": True,
"any_requires_network": True,
"all_success": False,
},
Expand Down Expand Up @@ -429,7 +429,55 @@ async def test_export(
},
# ecu_status_flags
{
"any_in_update": True,
"any_child_ecu_in_update": True,
"any_requires_network": True,
"all_success": False,
},
),
# case 3:
# only main ECU doing OTA update.
(
# local ECU status: UPDATING
_internal_types.OTAClientStatus(
ota_status=_internal_types.OTAStatus.UPDATING,
update_phase=_internal_types.UpdatePhase.DOWNLOADING_OTA_FILES,
),
# sub ECUs status
[
# p1: SUCCESS
api_types.StatusResponse(
available_ecu_ids=["p1"],
ecu_v2=[
api_types.StatusResponseEcuV2(
ecu_id="p1",
ota_status=api_types.StatusOta.SUCCESS,
),
],
),
# p2: SUCCESS
api_types.StatusResponse(
available_ecu_ids=["p2"],
ecu=[
api_types.StatusResponseEcu(
ecu_id="p2",
status=api_types.Status(
status=api_types.StatusOta.SUCCESS,
),
)
],
),
],
# expected overal ECUs status report set by on_ecus_accept_update_request,
{
"lost_ecus_id": set(),
"in_update_ecus_id": {"autoware"},
"in_update_child_ecus_id": set(),
"failed_ecus_id": set(),
"success_ecus_id": {"p1", "p2"},
},
# ecu_status_flags
{
"any_child_ecu_in_update": False,
"any_requires_network": True,
"all_success": False,
},
Expand Down Expand Up @@ -510,7 +558,7 @@ async def test_overall_ecu_status_report_generation(
},
# ecu_status_flags
{
"any_in_update": True,
"any_child_ecu_in_update": True,
"any_requires_network": True,
"all_success": False,
},
Expand Down Expand Up @@ -562,7 +610,7 @@ async def test_overall_ecu_status_report_generation(
},
# ecu_status_flags
{
"any_in_update": True,
"any_child_ecu_in_update": True,
"any_requires_network": True,
"all_success": False,
},
Expand Down Expand Up @@ -604,15 +652,18 @@ async def test_on_receive_update_request(
for k, v in flags_status.items():
assert getattr(self.ecu_status_flags, k).is_set() == v

async def test_polling_waiter_switching_from_idling_to_active(self):
async def test_polling_waiter_switching_from_idling_to_active(
self, mocker: pytest_mock.MockerFixture
):
"""Waiter should immediately return if active_ota_update_present is set."""
_sleep_time, _mocked_interval = 3, 60

mocker.patch(f"{ECU_STATUS_MODULE}.IDLE_POLLING_INTERVAL", _mocked_interval)

async def _event_setter():
await asyncio.sleep(_sleep_time)
self.ecu_status_flags.any_in_update.set()
await self.ecu_storage.on_ecus_accept_update_request({"autoware"})

self.ecu_status_flags.any_in_update.clear()
_waiter = self.ecu_storage.get_polling_waiter()
asyncio.create_task(_event_setter())
# waiter should return on active_ota_update_present is set, instead of waiting the
Expand Down

0 comments on commit 7a97d2e

Please sign in to comment.