Skip to content

Platform Acceptance Test for Developer

Jump to bottom
rc-swag edited this page Nov 16, 2023 · 5 revisions

Warning

These acceptance tests are now deprecated as the source for formal testing. Formal testing is done against regression tests stored in a test suite managed by the test team. Developers can update acceptance tests here on the wiki, however, if the developer recommends a test needs to be added to the formal tests contact the test team.

Acceptance testing of the latest Keyman Developer alpha before going into Beta.

Keyman Developer Acceptance Test Procedures

  1. These test procedures are to be run before moving from alpha to beta, or beta to stable, or before PRs are merged into stable branches.
  2. Copy these checklists of tests into a new issue (for tier transitions) or a comment on the PR.

User Testing

Gather Assets for Testing

  • Windows 10 (x64) - VM or physical okay
  • Windows 11 (x64) - VM or physical okay (if prefer for other suites)

Setup Steps

  • Uninstall previous versions of Keyman for Windows and Keyman Developer on the system
  • Restart system, install updates
  • Clean up any broken installation artifacts (ideally, roll back to clean snapshot in VM):
    • %ProgramFiles(x86)%\Keyman or %ProgramFiles%\Keyman
    • %CommonProgramFiles(x86)%\Keyman or %CommonProgramFiles%\Keyman
    • %ProgramData%\Keyman
    • %AppData%\Keyman
    • %LocalAppData%\Keyman
    • HKCU\Software\Keyman
    • HKLM\Software\Wow6432Node\Keyman
    • Manually remove any remaining Keyman input methods from Windows Languages

SUITE_DEVELOPER_BASELINE: Keyman Developer baseline tests

GROUP_WIN10 - to be tested on Windows 10 (x64)

GROUP_WIN11 - to be tested on Windows 11 (x64)

  • TEST_STARTS: Verify that Keyman Developer starts
  • TEST_COMPILES: Open a .kpj project, load keyboards and verify that you can compile them

What's New: test new functionality

SUITE_DEBUGGER: Test new core-based debugger

Setup: open a reasonably complex keyboard project (eg. Amharic and Cameroon keyboards), load the .kmn, and press F7 to compile.

  • TEST_DEBUGGER_STARTS: Start debugging by pressing F5. Verify that the debugger starts and you can type with the keyboard in the debug window. Look out for any unexpected behaviour or visual oddities.
  • TEST_DEBUGGER_SINGLE_STEP: Press the Single Step button (>>) in the Debugger toolbar. Type a key in the debugger window. Use the debugger toolbar (>) button to step through the rules and verify that the matched rules are correct. Examine the Debugger State windows (Elements, Call stack, etc) to verify that the content presented at each point makes sense.
  • TEST_DEBUGGER_PAUSE: Press the Pause button (||) in the Debugger toolbar. Verify that no input is possible in the debugger window.
  • TEST_DEBUGGER_CHANGES: Make a change to the source file and verify that the debugger works correctly after recompiling.
  • TEST_DEBUGGER_PLATFORM: Create some platform-specific rules with platform() statement, and use the Debugger State window 'Platform' to select platform variants, and verify that the debugger selects the appropriate rules in testing.

SUITE_SERVER: Test new Keyman Developer Server

  • TEST_SERVER_STARTS: Start Keyman Developer. Verify that Server starts by examining the tool tray area, and launching http://localhost:8008 in your browser.

  • TEST_SERVER_STOPS: Start Keyman Developer. Check that Tools/Options/Server/Leave Server running after closing IDE is off. Exit Keyman Developer. Verify that Server has shut down by attempting to visit http://localhost:8008 in your browser -- it should fail to load.

  • TEST_SERVER_KEEP_ALIVE: Start Keyman Developer. Check that Tools/Options/Server/Leave Server running after closing IDE is on. Exit Keyman Developer. Verify that Server has is still running by visiting http://localhost:8008 in your browser.

  • TEST_SERVER_KEYBOARD: Make sure that a keyboard debug session works.

    1. In Keyman Developer, load a keyboard project, open a keyboard, and start testing on web.
    2. Open http://localhost:8008 in your browser, and select the keyboard from the Keyboards drop down
    3. Verify that the keyboard is working in the web debug view.

  • TEST_SERVER_KEYBOARD_LIVE_CHANGE: Make sure that changes to the keyboard are dynamically updated.

    1. Follow TEST_SERVER_KEYBOARD steps to start debugging the keyboard.
    2. Make a visual change to the on screen keyboard / touch keyboard view in the keyboard and recompile it (F7</kbd).
    3. Switch back to the browser and verify that the change is visible in the keyboard.
Clone this wiki locally