Added an automated Selenium UI test for a small Zimit2 archive

[THEBOSS0369]

This PR Fixes #1205

Hello Everyone!

Description

In this PR i have added an automated Selenium UI test for a small Zimit2 archive (used tonedear.com_en_2024-09.zim)

The tests i have added are as follows :

1. Load Kiwix JS and verify title
2. Switch to Jquery or SW mode
3. Load Tonedear archive and verify content
4. Navigate from main page to Android & iOS section
5. Verify Android and iOS store images in Both Modes

Test

I have done all the necessary test

1. Checked in both "Restricted" and "ServiceWorker" Everything is working fine check the ss below.
2. Unit tests npm test no issue
3. End-to-end (e2e) tests-e2e-firefox-> All tests Passed Below is the ss
4. extension versions with production code tested
5. Browser Test -> Microsoft Edge, Chrome, Firefox and Brave

Screenshot for tests

[THEBOSS0369]

@Jaifroid I have not added test for android & ios page because i didn't know what test should i do, your suggestion will be appreciated 😅😃 Thanks!
!

[Jaifroid]

Thank you @THEBOSS0369. That's very useful. I would suggest adding a test that switches to the page that shows the Android and iOS store images, and verifying that both images have loaded correctly:

• In Restricted mode (aka JQuery), test that both images' src attribute has changed to a data URI (see screenshot), and assert that both data URIs match the beginning of the expected content (say 30 characters, so you get the beginning of the actual data, not just "data:img/png,base64,". If you want to be thorough, you could hash the string and test the hashes, but that may be overkill.
• In ServiceWorker mode, you can do a basic image verification: check if the image has completed loading using JavaScript properties, and verify that the image has natural dimensions (width/height > 0). Use the computed naturalWidth and naturalHeight properties of the images to verify (see https://developer.mozilla.org/en-US/docs/Web/API/HTMLImageElement/naturalWidth). This will tell us that the image loaded correctly (after appropriate wait).

[THEBOSS0369]

Hey @Jaifroid ! I have added the android and ios test as you asked me but there is a problem occurring when i am running the tests the restricted mode test passes well but in sw mode this error comes

i have tried by making the waititme 120000 , by removin cache but still this won't work. I don't understand what i am doing wrong i checked in the sw mode the image's alt is present and i have added the same in the code. Your help will be appreciated . Thanks!

[Jaifroid]

Thanks @THEBOSS0369. Sorry to hear about the error. It's tricky to get the assertions right with the timing. I'll take a look at the weekend, but in the meantime it's possibly the case that there's some error in selecting the images and checking their natural width. Adding some console logging to see what it's actually finding and what you are expecting it to find might help!

[THEBOSS0369]

Yes sir. I'm doing that by further more progress i have found that there are in total 21 images showing 19 are icon_external_link one is gif and one is kiwix logo this script is still ain't able to find the store images. i will try to find more in the meanitime maybe i find something useful ,right now this is what's showing.

Note: I didn't pushed the code as it has become very messy so if you want to see the code just tell me i will push it. 😃🙂

[Jaifroid]

OK, I ran this myself (in the Edge runner, adding the appropriate commands), and while watching the browser locally it's clear that the test never actually loads the page "Android & iOS App". It remains stuck on "Ear Training Practice" (see screenshot). You're not going to be able to verify the images on the "Android & iOS App" page unless you actually switch to it first!

[Jaifroid]

Please see below for the steps to remedy the issue you're encountering.

[Jaifroid]

As the comment says above this block, the drivers need to be listed in reverse test order here, so that when the user runs the tests locally we see the Ray Charles first (which will be on top of the other windows). So please put the tonedear test above gutenberg in this list, and update the comment to reflect that there are three tests

[THEBOSS0369]

Done Sir

[THEBOSS0369]

@Jaifroid Another problem occurring , when i try to navigate to Android Page it won't, i think this is happening because i didn't know the exact path for it at first i used this android-ios-ear-training-app it didn't worked i tried by adding /android-ios-ear-training-app & ../android-ios-ear-training-app and many more tries but still didn't worked

then i looked into the ray_charles code as it also have a navigation function i tried this which is
const appLink = await driver.wait(until.elementLocated(By.xpath("//ul[contains(@class, 'navbar-nav navbar-right')]/li[1]/a")), 20000);
similar to that present in ray chares
articleLink = await driver.findElement(By.xpath('/html/body/div/div/ul/li[77]/a[2]'));
but still not working.

Then i think i can do that by link but The link for the server remains same
http://localhost:5173/www/index.html?appCache=false
even when i open any other page but when i run the tests it have a new url
http://localhost:8080/dist/www/index.html?title=C/tonedear.com/android-ios-ear-training-app
for different pages i tried by clearing cache and using incognito mode still url remains same. I'm not sure if this is default or not.

Can you help me with this path problem if we get it fixed then every tests will pass correctly.

[Jaifroid]

@THEBOSS0369 Oh no, this app doesn't support navigating to a page by URL fragment. There are two ways to get to a particular page: either click on the link that you highlight in the screenshot (easiest), or search for it using the search bar. Also, remember that the links you're clicking on are in the iframe, not at the top level, so you have to switch to the iframe before finding the link and clicking it. Take a look at the code in the Ray Charles test to see how to switch to the iframe.

This explains why you said you were finding 21 images in the landing page when in fact there are none! You were finding these images in the app itself, as you hadn't switched to the iframe containing the content.

[THEBOSS0369]

@Jaifroid Sirrr ! it worked now its navigating to andorid page although the images are still not found now i will try to fix that

[THEBOSS0369]

@Jaifroid Sir all The tests have passed . Does any thing else needed to be added in the test? If yes then do let me know.

The comments in this image for tonedear tests will not be visible in the main tests i have commented it out.

[Jaifroid]

@THEBOSS0369 That's very good news! I don't think we need more tests. I'll review the new code as soon as I can, but in the meantime, you might want to clean it up, remove any excessive timeouts you previously added, etc. Once the spec is ready, you should add it to all the other runners in the folders under tests/e2e/runners. Then I'll need to run it on BrowserStack to be sure the tests pass on all browsers.

Oh, I see GitHub tests have failed. I'll re-run those to be sure it wasn't a timeout fluke.

[THEBOSS0369]

@Jaifroid I think the tests are failing because when it navigate to android page it asks me to authorize the author and i used to press that maybe that's causing error.

[THEBOSS0369]

@Jaifroid Yes this is happening due to the reason i told above i didn't clicked the trust source this time and tests failed. I don't know how to fix this one

[Jaifroid]

Yes this is happening due to the reason i told above i didn't clicked the trust source this time and tests failed. I don't know how to fix this one

This is because you erased https://github.com/kiwix/kiwix-js/blob/main/tests/e2e/spec/legacy-ray_charles.e2e.spec.js#L125 (the setting "noPrompts=true") from the URL when loading the app. This was in your previous spec, but then you did a hard reset of your branch, and you erased it, so it's no longer in the commit history.

That command prevents the security prompt from showing. All you need to do is copy the appropriate code block from the ray charles version of the test.

Also, in future, try to avoid rewriting history in your PRs. It makes things difficult to review as well, because when you do a force push, I have to reset my copy of your PR instead of simply pulling a new commit! (Don't worry, everyone makes these kinds of mistakes with PRs when learning.)

[THEBOSS0369]

@Jaifroid Done Sir! Now all tests passes well without any issue.

I'm waiting for these tests to pass then i will add the tests in all the files and remove any unnecessary time waits.

Also, in future, try to avoid rewriting history in your PRs. It makes things difficult to review as well, because when you do a force push, I have to reset my copy of your PR instead of simply pulling a new commit! (Don't worry, everyone makes these kinds of mistakes with PRs when learning

My apologies Sir i didn't knew force push can do this much damage. I will be careful from next time 😅

[THEBOSS0369]

I have increased the timewait maybe this can fix it

Edit (11:10): All the tests have been passed so i'm reducing time wait

Edit Again (12:30): Sir i have added the tests in all the files .

[THEBOSS0369]

@Jaifroid Sir, The tests are failing in chrome i am not sure why its happening i tested in firefox it passes well. I just did same steps in chrome's e2e test file as i did in the firefox's e2e test file

[Jaifroid]

Hmm looks like a timeout issue since it works on Windows in the same browsers. Let me re-run and see again.

[Jaifroid]

It might be useful to uncomment this, to check if the link is found.

[THEBOSS0369]

Done Sir

[Jaifroid]

I don't think the element needs to be scrolled into view. You could try without this (comment it out).

[THEBOSS0369]

Done Sir

[Jaifroid]

If you've already found the element (androidLink), it doesn't seem necessary to wait for it to be visible, as you wouldn't have found it if it weren't already visible. You could also try commenting this out.

[THEBOSS0369]

Done Sir

[Jaifroid]

Problem with this is that it's impossible to see screenshots in GitHub Actions AFAIK!

[THEBOSS0369]

@Jaifroid ! This worked sir All the tests have passed.

But sir one doubt if the github tests wasn't able to recognize the images in the linux tests then how it was able to verify for the windows test? Like is it some compatible issue or anything else.

[Jaifroid]

Very good news @THEBOSS0369! I guess it was a timing issue of some kind, but I need to check the verification method to be sure there isn't an issue with the assertions.

The next step is for me to get the Browser Stack tests running against this PR (you can't do that, because you don't have access to the credentials). To do this, I'll make a dummy PR that has the same code as your PR, and this should then run on Browser Stack, and we can get feedback on whether it's passing also on old browsers. The fact that it's already passing on IE11 mode on Windows (Edge) implies that there shouldn't be any problem with other browsers, but it needs to be checked. I'll do this when I can in the next few days.

[THEBOSS0369]

@Jaifroid Got it Sir! If my work's done here then shall i find any other issue to work on or wait till this PR completes all its checks?