double_tap_action has no effect on iOS

[webjib]

Card diagnostic information:

card_version: 6.3.8
browser: >-
  Mozilla/5.0 (iPhone; CPU iPhone OS 18_3 like Mac OS X) AppleWebKit/605.1.15
  (KHTML, like Gecko) Home Assistant/2025.1 (io.robbie.HomeAssistant;
  build:2025.1073; iOS 18.3.0) Mobile/HomeAssistant, like Safari
date: 2025-01-22T16:27:31.598Z
lang: en
timezone: Europe/Paris
git:
  hash: 805a211
  build_date: Wed, 22 Jan 2025 03:43:55 GMT
  commit_date: Tue, 21 Jan 2025 19:42:47 -0800
ha_version: 2025.1.3
custom_integrations:
  frigate:
    detected: true
    version: 5.7.0
    devices:
      ***: 5.7.0/0.14.1-f4f3cfa
  hass_web_proxy:
    detected: false
config:
  type: custom:frigate-card
  cameras:
    - camera_entity: camera.***

Description of problem:

Configuring a double_tap_action either for the view or just the live, has no effect on iOS. It works on Safari for Mac.
Tap and Hold actions are working fine on iOS.

live:
  actions:
    double_tap_action:
      action: custom:frigate-card-action
      frigate_card_action: ptz_digital
      ptz_action: zoom_in
      absolute:
        zoom: 1.5

[dermotduffy]

I don't have an iPhone to test, but playing around on Browserstack and I see this same behavior. The video element is swallowing the interaction. The same sadly happens on Android:

• https://card.camera/#/troubleshooting?id=double_tap-does-not-work-in-android

Given that touch/mouse interaction with the video element is desirable, I am not sure there is anything that can/should be done here other than updating that troubleshooting description to include iOS in this limitation.

[webjib]

Thanks for your answer! I don't have any clue how Home Assistant is doing that, but they support the double tap action on the Picture Glance card. This card is using an HTML video element, but maybe there's some hidden overlay over the video that capture the event? If you configure a double tap, it takes precedence over the tap action.

Maybe Home Assistant source code can help?

[dermotduffy]

Thanks for your answer! I don't have any clue how Home Assistant is doing that, but they support the double tap action on the Picture Glance card. This card is using an HTML video element, but maybe there's some hidden overlay over the video that capture the event? If you configure a double tap, it takes precedence over the tap action.

This is likely because (i) by default Picture Glance shows an image not a video, (ii) when you configure it to show a video, it won't show media controls.

If you're prepared to sacrifice media controls, maybe this will work for you:

live:
  controls:
    builtin: false                <------------------------ Turns off builtin media controls
  actions:
    double_tap_action:
      action: custom:frigate-card-action
      frigate_card_action: ptz_digital
      ptz_action: zoom_in
      absolute:
        zoom: 1.5

[webjib]

Are you sure? I've tried to add a Picture Glance card, with Live mode for the camera (instead of default Auto mode), and the HTML source code shows a video element:

About the builtin controls, I had already disabled them when I reported the issue.

[dermotduffy]

Yes, it's a video element when you configure it that way (think we're agreeing!). I just tested on Android, double tap works fine on this card, when built-in controls are off (just like it does on picture glance).

It doesn't work on iOS, but picture glance does?

[webjib]

Yes, with built-in controls disabled on the Frigate card, the double tap does nothing on iOS, but it's working on macOS (safari).
With the Picture Glance card, double tap is working on both iOS and macOS (safari).

I've tried adding an overlay to the Frigate card (a semi transparent PNG), and double tap is working fine on this overlay. However this is not the solution, because I lose ability to pinch/pan the video.

[dermotduffy]

It's not related to the card zooming either, turning that off doesn't change anything. Sadly it's extremely painful for me to debug problems like this, as I don't actually have any iOS device, so I'm left pawing around on Browserstack ...

[webjib]

Could you tell me where to precisely find the source code responsible of the double tap, I could try to help in debugging.

[webjib]

EDIT:
I've done some other test. I've implemented the exact same logic than HA:
https://github.com/home-assistant/frontend/blob/dev/src/panels/lovelace/common/directives/action-handler-directive.ts
...and it's now working!
The only change is to implement this.cancelled from HA source code.
If you'd like, we can discuss this a little bit more (here or elsewhere)

ORIGINAL COMMENT:
I've done some tests.

Your 'end' function in 'ActionHandler' is generating 3 events when double taping (iOS):

• TouchEvent of type 'touchend', detail 0
• TouchEvent of type 'touchend', detail 0
• PointerEvent of type 'click', detail 1

FYI, I've tried to add an event listener on 'dblclick' event, and it generates this event:
MouseEvent, type 'dblclick', detail 2

dblclick is maybe the solution?

[dermotduffy]

@webjib Amazing! Yes, this is (obviously) exactly the right spot. Do you want to prepare a PR? I'd very much welcome it. There's a few special cases we'll need to test here that HA doesn't support, e.g.:

• Other devices/browsers (but this is probably fine if this is in the HA source).
• Embedded WebRTC-card (it needs to receive sufficient passthrough of events) for the webrtc-card provider.
• Zooming / Panning.
• Does it work with video controls on? (I assume not, but we need to update the troubleshooting docs).

[webjib]

Sure, I can prepare a PR.

• I don't have WebRTC card anymore to test.
• Zooming / panning seems to work as usual
• no, it's working only when video controls are off