[🐛 Bug]: Video Recording Fails After File Integrity Check

[nipun1999]

What happened?

Description: We encountered an issue where, after creating a long video and performing file integrity checks, if a new session is initiated and the request is routed to the same Chrome pod, the video recorder sidecar (which was previously handling the file integrity check) does not immediately start recording. This results in the recording not starting as expected and causes downstream issues.

Attempted Fix: We attempted to resolve this by running the wait_for_file_integrity function in the background (by appending the & symbol to the command). This solution was mainly for logging purposes, as the function doesn't perform any additional actions if the file is corrupted.

Question: Do you foresee any potential issues or unintended side effects with this fix?

Thanks in advance for your insights!

Command used to start Selenium Grid with Docker (or Kubernetes)

Kubernetes

Relevant log output

{ "value": { "ready": true, "message": "Ready", "node": { "availability": "UP", "externalUri": "http:\u002f\u002f172.24.16.215:5555", "heartbeatPeriod": 30000, "maxSessions": 1, "nodeId": "bc01c5f1-4f83-4aaa-b02b-b6ede7163cc8", "osInfo": { "arch": "amd64", "name": "Linux", "version": "5.10.234-225.895.amzn2.x86_64" }, "sessionTimeout": 86400000, "slots": [ { "id": { "hostId": "bc01c5f1-4f83-4aaa-b02b-b6ede7163cc8", "id": "e080f06b-c77d-4ee6-9224-67612e0421aa" }, "lastStarted": "1970-01-01T00:00:00Z", "session": null, "stereotype": { "browserName": "chrome", "browserVersion": "134.0", "container:hostname": "selenium-grid-selenium-node-chrome-6c6f495d8b-6n2j2", "goog:chromeOptions": { "binary": "\u002fusr\u002fbin\u002fgoogle-chrome" }, "platformName": "linux", "se:containerName": "selenium-grid-selenium-node-chrome-6c6f495d8b-6n2j2", "se:downloadsEnabled": true, "se:noVncPort": 7900, "se:vncEnabled": true } } ], "version": "4.30.0 (revision 509c7f1)" } } }
starting stop_recording_function
2025-04-02 12:07:57,991 [video.recorder] - Video recording stopped
2025-04-02 12:07:57,992 [video.recorder] - Add to pipe a signal Uploading video to s3://zamp-prd-uae-selenium-grid-bucket/519ff35a8ab4ba81ddd825830bc3bbd5.mp4
stopping recording, returning back to main loop
after stop recording in main func
starting waiting for file integrity
2025-04-02 12:07:57,994 [video.uploader] - Uploading /videos/519ff35a8ab4ba81ddd825830bc3bbd5.mp4 to s3://zamp-prd-uae-selenium-grid-bucket
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Elapsed time:         0.0sTransferred:   	    1.651 MiB / 1.651 MiB, 100%, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Deleted:                1 (files), 0 (dirs), 1.651 MiB (freed)
Renamed:                1
Transferred:            1 / 1, 100%
Elapsed time:         0.3s
finished wait for file integrity function
finished if condition of max recorded count in main while loop
finished main loop iteration
running main while loop
{ "value": { "ready": false, "message": "No free slots available", "node": { "availability": "UP", "externalUri": "http:\u002f\u002f172.24.16.215:5555", "heartbeatPeriod": 30000, "maxSessions": 1, "nodeId": "bc01c5f1-4f83-4aaa-b02b-b6ede7163cc8", "osInfo": { "arch": "amd64", "name": "Linux", "version": "5.10.234-225.895.amzn2.x86_64" }, "sessionTimeout": 86400000, "slots": [ { "id": { "hostId": "bc01c5f1-4f83-4aaa-b02b-b6ede7163cc8", "id": "e080f06b-c77d-4ee6-9224-67612e0421aa" }, "lastStarted": "2025-04-02T12:07:58.763805305Z", "session": { "capabilities": { "acceptInsecureCerts": false, "browserName": "chrome", "browserVersion": "134.0.6998.165", "chrome": { "chromedriverVersion": "134.0.6998.165 (fd886e2cb29dd984c13deec032832dee68d8abe3-refs\u002fbranch-heads\u002f6998@{#2122})", "userDataDir": "\u002ftmp\u002f.org.chromium.Chromium.oBzW3E" }, "fedcm:accounts": true, "goog:chromeOptions": { "debuggerAddress": "localhost:33663" }, "networkConnectionEnabled": false, "pageLoadStrategy": "normal", "platformName": "linux", "proxy": { }, "se:cdp": "http:\u002f\u002flocalhost:33663", "se:cdpVersion": "134.0.6998.165", "se:containerName": "selenium-grid-selenium-node-chrome-6c6f495d8b-6n2j2", "se:downloadsEnabled": true, "se:noVncPort": 7900, "se:vncEnabled": true, "se:vncLocalAddress": "ws:\u002f\u002f172.24.16.215:7900", "setWindowRect": true, "strictFileInteractability": false, "timeouts": { "implicit": 0, "pageLoad": 300000, "script": 30000 }, "unhandledPromptBehavior": "dismiss and notify", "webauthn:extension:credBlob": true, "webauthn:extension:largeBlob": true, "webauthn:extension:minPinLength": true, "webauthn:extension:prf": true, "webauthn:virtualAuthenticators": true }, "sessionId": "2b48e56b74dfb79d1048fd7f81b93429", "start": "2025-04-02T12:07:58.763805305Z", "stereotype": { "browserName": "chrome", "browserVersion": "134.0", "container:hostname": "selenium-grid-selenium-node-chrome-6c6f495d8b-6n2j2", "goog:chromeOptions": { "binary": "\u002fusr\u002fbin\u002fgoogle-chrome" }, "platformName": "linux", "se:containerName": "selenium-grid-selenium-node-chrome-6c6f495d8b-6n2j2", "se:downloadsEnabled": true, "se:noVncPort": 7900, "se:vncEnabled": true }, "uri": "http:\u002f\u002flocalhost:14426" }, "stereotype": { "browserName": "chrome", "browserVersion": "134.0", "container:hostname": "selenium-grid-selenium-node-chrome-6c6f495d8b-6n2j2", "goog:chromeOptions": { "binary": "\u002fusr\u002fbin\u002fgoogle-chrome" }, "platformName": "linux", "se:containerName": "selenium-grid-selenium-node-chrome-6c6f495d8b-6n2j2", "se:downloadsEnabled": true, "se:noVncPort": 7900, "se:vncEnabled": true } } ], "version": "4.30.0 (revision 509c7f1)" } } }
2025-04-02 12:08:08,761 [video.recorder] - Session: 2b48e56b74dfb79d1048fd7f81b93429 is created
2025-04-02 12:08:08,852 [video.recorder] - Start recording: true, video file name: 2b48e56b74dfb79d1048fd7f81b93429.mp4
{
  "data": {
    "session": {
      "id": "2b48e56b74dfb79d1048fd7f81b93429",
      "capabilities": "{\n  \"acceptInsecureCerts\": false,\n  \"browserName\": \"chrome\",\n  \"browserVersion\": \"134.0.6998.165\",\n  \"chrome\": {\n    \"chromedriverVersion\": \"134.0.6998.165 (fd886e2cb29dd984c13deec032832dee68d8abe3-refs\\u002fbranch-heads\\u002f6998@{#2122})\",\n    \"userDataDir\": \"\\u002ftmp\\u002f.org.chromium.Chromium.oBzW3E\"\n  },\n  \"fedcm:accounts\": true,\n  \"goog:chromeOptions\": {\n    \"debuggerAddress\": \"localhost:33663\"\n  },\n  \"networkConnectionEnabled\": false,\n  \"pageLoadStrategy\": \"normal\",\n  \"platformName\": \"linux\",\n  \"proxy\": {\n  },\n  \"se:bidiEnabled\": false,\n  \"se:cdp\": \"ws:\\u002f\\u002fselenium-grid-selenium-hub.selenium-grid\\u002fsession\\u002f2b48e56b74dfb79d1048fd7f81b93429\\u002fse\\u002fcdp\",\n  \"se:cdpVersion\": \"134.0.6998.165\",\n  \"se:containerName\": \"selenium-grid-selenium-node-chrome-6c6f495d8b-6n2j2\",\n  \"se:downloadsEnabled\": true,\n  \"se:noVncPort\": 7900,\n  \"se:vnc\": \"ws:\\u002f\\u002fselenium-grid-selenium-hub.selenium-grid\\u002fsession\\u002f2b48e56b74dfb79d1048fd7f81b93429\\u002fse\\u002fvnc\",\n  \"se:vncEnabled\": true,\n  \"se:vncLocalAddress\": \"ws:\\u002f\\u002f172.24.16.215:7900\",\n  \"setWindowRect\": true,\n  \"strictFileInteractability\": false,\n  \"timeouts\": {\n    \"implicit\": 0,\n    \"pageLoad\": 300000,\n    \"script\": 30000\n  },\n  \"unhandledPromptBehavior\": \"dismiss and notify\",\n  \"webauthn:extension:credBlob\": true,\n  \"webauthn:extension:largeBlob\": true,\n  \"webauthn:extension:minPinLength\": true,\n  \"webauthn:extension:prf\": true,\n  \"webauthn:virtualAuthenticators\": true\n}",
      "startTime": "02/04/2025 12:07:58",
      "uri": "http://172.24.16.215:5555",
      "nodeId": "bc01c5f1-4f83-4aaa-b02b-b6ede7163cc8",
      "nodeUri": "http://172.24.16.215:5555",
      "sessionDurationMillis": "10009",
      "slot": {
        "id": "e080f06b-c77d-4ee6-9224-67612e0421aa",
        "stereotype": "{\n  \"browserName\": \"chrome\",\n  \"browserVersion\": \"134.0\",\n  \"container:hostname\": \"selenium-grid-selenium-node-chrome-6c6f495d8b-6n2j2\",\n  \"goog:chromeOptions\": {\n    \"binary\": \"\\u002fusr\\u002fbin\\u002fgoogle-chrome\"\n  },\n  \"platformName\": \"linux\",\n  \"se:containerName\": \"selenium-grid-selenium-node-chrome-6c6f495d8b-6n2j2\",\n  \"se:downloadsEnabled\": true,\n  \"se:noVncPort\": 7900,\n  \"se:vncEnabled\": true\n}",
        "lastStarted": "02/04/2025 12:07:58"
      }
    }
  }
}
2025-04-02 12:08:08,856 [video.recorder] - Starting to record video
2025-04-02 12:08:08,865 [video.recorder] - Video recording started
[x11grab @ 0x5647c84abd40] Stream #0: not enough frames to estimate rate; consider increasing probesize
finished main loop iteration
running main while loop

Operating System

EKS

Docker Selenium version (image tag)

4.30.0-20250323

Selenium Grid chart version (chart version)

No response

[github-actions]

@nipun1999, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[VietND96]

I guess the issue is after upload (using rclone command move by default), file didn't exist anymore for func check file integrity to work properly. Let me check on this.

[VietND96]

It will be fixed in #2742