Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add interactivity check for physical devices with display timeout #3635

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

fro0116
Copy link

@fro0116 fro0116 commented Apr 12, 2024

Hi there! I recently started running ALAS on a cheap android tablet with a 1 minute display timeout to save energy. I also have it configured to close the game on tasks completion.

However, after ALAS goes to idle and closes the game, it would always fail with logs that look like this and a pure black screenshot:

═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
                                                      COMMISSION                                                       
═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
2024-04-03 15:54:34.291 | INFO | <<< UI ENSURE >>>                                                                     
2024-04-03 15:54:34.308 | INFO | UI get current page                                                                   
2024-04-03 15:54:34.993 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:35.971 | INFO | [Package_name] com.YoStarJP.AzurLane                                                  
2024-04-03 15:54:36.105 | INFO | [Device Orientation] 1 (HOME key on the right)                                        
2024-04-03 15:54:36.952 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:37.836 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:38.706 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:39.587 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:40.545 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:41.446 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:42.350 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:43.239 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:44.123 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:45.011 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:45.901 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:46.838 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:47.730 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:48.598 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:49.485 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:50.374 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:51.264 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:52.136 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:53.058 | INFO | Unknown ui page                                                                       
2024-04-03 15:54:53.876 | WARNING | Unknown ui page                                                                    
2024-04-03 15:54:53.895 | INFO | [EMULATOR__SCREENSHOT_METHOD] ADB_nc                                                  
2024-04-03 15:54:53.910 | INFO | [EMULATOR__CONTROL_METHOD] ADB                                                        
2024-04-03 15:54:53.921 | INFO | [SERVER] jp                                                                           
2024-04-03 15:54:53.932 | WARNING | Starting from current page is not supported                                        
2024-04-03 15:54:53.939 | WARNING | Supported page: ['page_main', 'page_campaign_menu', 'page_campaign', 'page_fleet', 
'page_unknown', 'page_exercise', 'page_daily', 'page_event', 'page_sp', 'page_coalition', 'page_os', 'page_archives',  
'page_reward', 'page_mission', 'page_guild', 'page_commission', 'page_tactical', 'page_battle_pass', 'page_event_list',
'page_raid', 'page_research', 'page_shipyard', 'page_meta', 'page_storage', 'page_reshmenu', 'page_dormmenu',          
'page_dorm', 'page_meowfficer', 'page_academy', 'page_game_room', 'page_shop', 'page_munitions', 'page_supply_pack',   
'page_build', 'page_rpg_stage', 'page_rpg_story', 'page_rpg_city']                                                     
2024-04-03 15:54:53.972 | WARNING | Supported page: Any page with a "HOME" button on the upper-right                   
2024-04-03 15:54:53.983 | CRITICAL | Please switch to a supported page before starting Alas                            
2024-04-03 15:54:53.991 | INFO | Game server may be under maintenance or network may be broken, check server status now
2024-04-03 15:54:54.508 | INFO | Server "キール" is available.                                                         
2024-04-03 15:54:54.530 | CRITICAL | Game page unknown                                                                 
2024-04-03 15:54:54.560 | WARNING | Saving error: ./log/error/1712127294560                                            

This seems to be because the device went to sleep, so I added a new function to check for device interactivity (which is set to false when display is off), and used it in the ui_get_current_page and handle_app_login loops to wake the device if not interactive.

I've been running with these changes for the past week and it hasn't required any manual interventions yet. I've defaulted the result of the function to true in the case where we can't find the expected adb outputs to keep behavior identical for older devices and emulators, but I haven't actually tested on an emulator yet since I don't have one set up. Would appreciate some help with this from folks that run ALAS on emulators!

I'm a bit of a python noob, so there might be some mistakes in here. Happy to update this as you see fit. Thanks!

@Air111
Copy link
Contributor

Air111 commented Apr 12, 2024

I think this is a niche requirement. Besides, your implementation may encounter the following two issues:

  1. Many devices require additional actions to wake up, such as swiping and entering a password.
  2. If the device remains in a not interactive state, it could enter into a loop.

@LmeSzinc LmeSzinc added the emulator issue / 模拟器问题 Issues caused by emulator, change emulator instead label Apr 18, 2024
Copy link
Owner

@LmeSzinc LmeSzinc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Implement in both ADB and uiautomator2, wrap in Device class
  2. Abstract function to wake device
  3. get_interactive should only be called when a pure black screenshot is received
  4. No interactive check on emulators

Comment on lines +147 to +150
if not self.device.get_interactive():
self.device.wake()
self.device.sleep(1)
continue
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using sleep is forbidden

Comment on lines +722 to +730
@retry
def get_interactive(self):
"""
Makes sure device is interactive before starting tasks
"""
_INTERACTIVE_RE = re.compile(
r'mInteractive=(?P<interactive>.+)'
)
output = self.adb_shell(['dumpsys', 'input_method'])
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
emulator issue / 模拟器问题 Issues caused by emulator, change emulator instead
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants