Button press calls actions multiple times (UE 4.25)

[rproepp]

I'm using a complex action binding with many different actions mappings and multiple keys for each to support various controllers. All controls are also duplicated for both hands to allow switching between left-handed and right-handed within the game.

This setup worked fine with the plugin in 4.21, but since migrating to 4.25 I've noticed strange behavior for some of the actions. I've tried to narrow down the issue and produced the attached example project by modifying the default VR template. It defines five action mappings, the left and right hand B button on the Index is configured for each of the mappings. The MotionControllerPawn has a print node for each action.

When starting the VR preview and pressing one of the B buttons, each action is called several times, once for each time the button is used in any action mapping. I don't think that is the desired behavior, and each action should only be called once per button press?

With a more complex mapping, there's more strange behavior, including actions getting called that don't even have the pressed button configured. The behavior changes depending on the existence of other unrelated action mappings, and depending on configured buttons for other motion controller types that aren't connected.

The generated .json files seem fine and look the same as they did with 4.21. I've also added keyboard keys to the input actions to determine whether this is a general Unreal issue, but each action is only called once when using the keyboard keys.

VRTest.zip

[1runeberg]

Hi @rproepp - thanks for the report.

For multiple actions on a single key, with the integrated version we don't have the luxury of unique temporary keys which we autogenerate in the Marketplace version. So you will need to keep track of handedness state in your project unfortunately to gate the input events.

In saying that, we've added a feature in the plugin to look for a unique key in tke Key Mappings and will use the unique key in favour of any other key that's already in use by another action. E.g. if you put in a Magic Leap or Oculus Go key in the action (assuming you are not using this key or supporting those platforms), the plugin will then use this key to trigger the action (as the engine under the hood only recognizes keys and not actions).

The feature seems broken though in 4.25.4 from your test project. Will produce a fix for this and also see if we can have transient generated temporary keys approved for a hotfix. This is unlikely to be accepted as a fix for 4.25 though, but we may be able to get it into 4.26 - so will create a separate 4.25 PR which we expect to be rejected, but at least developers can merge it to their engine builds.

[rproepp]

Hi, thanks for the quick response! Gating based on handedness is not an issue, I'm already doing that. But I don't quite understand what will change with the fix: will actions for keys that are configured in multiple actions only be triggered once, and does that require to map an additional unique key from an unused SteamVR device?

[1runeberg]

Hi @rproepp - no prob. Ideally we'll have Action Sets to handle this, but we don't have a means at the moment to plumb that in UE.

So yes, gating via project code or by adding a unique key to actions are the options (Magic Leap keys for instance). I don't think we can pass the Temporary Key solution we had similar to the Marketplace version in the Engine Integrated version.

[1runeberg]

Hi @rproepp - sorry it seems I got this confused with another issue. I've traced the code and the feature to use a unique key from the list is indeed working (just not for Magic Leap as a pseudo temporary/unique key).

we could add those in to be recognized, but for your use case however, even if you place unique keys on each of your actions, it will still be triggered multiple times. The runtime (SteamVR) will indicate all three actions have been triggered. There is no current way to filter this on the engine side without an action set so you need to keep track of your game's state (either which actions should run when or which hand is active) and gate them appropriately.

unfortunately you cant use a lot of the logic that's provided by the SteamVR runtime for this use case.

[rproepp]

Ok, I've added a second gate on all actions and everything seems to work again like in 4.21. To be clear on what the setup looks like now: For every pressed and released on every action, first there's a gate that depends on handedness, and then another gate that ensures that the action is not called more than once between ticks. Is it safe to assume that all duplicate action triggers will happen in the same tick?

[1runeberg]

yes, they'll all happen on the same tick... the plugin goes through all actions in a tick and checks with the steamvr runtime if it has been triggered or not.

[rproepp]

After testing with more headsets, I found that on the Vive actions are not only called multiple times, but some buttons also call actions that are not configured for them. In the cases I've seen, the correct action was triggered first so it was possible to do another workaround and stop the additional actions from getting executed in the same tick.

I don't have a minimal example this time, the behavior only seems to happen once the button configuration becomes complex. I haven't found a pattern on when it starts and it can affect different buttons/actions depending on how other buttons are mapped.

Do you have an idea why this could be happening, and is a real fix possible? I'm also worried if this additional workaround is stable across user configurations: it could just be luck that the correct action is called first?

[1runeberg]

Hi @rproepp - there shouldnt be any buttons that gets triggered if the action that was triggered isnt bound to it. My guess is that the manifest hasnt been updated or steamvr is picking up an older version some reason. Yeah it would be great if you can send a minimal example as it'll be easier to test or repro steps.

[rproepp]

Ok, here's a somewhat minimal example: it works in the same way as the one from my original post, but the input mapping is a lot more complex. When you press the menu button on the right Vive wand, you should see three prints, indicating that three actions have been triggered (see the MotionControllerPawn Blueprint for the event and print nodes). But only two actions are configured on that button, the third action is configured for a different button both in DefaultInput.ini and vive_controller.json
VRTest.zip

[emretanirgan]

Hey @rproepp, I know this was a while ago but did you ever find a solution to this? We have a relatively complex input mapping in our app, and we're running into the exact same issue. We have the same key bound to multiple actions in some instances, or multiple keys bound to the same action, and we're running into issues where one key triggers an action for a completely different key. So we might pull on the trigger, and end up triggering an unrelated action bound to the grip. This is extremely frustrating and quite hard to track down, so I was just wondering if you ever got to the bottom of it.

Or @1runeberg do you happen to know any more details about this, or have you looked into the minimal example at all? Any help would be greatly appreciated - thank you!

[rproepp]

I haven't found a proper solution: for my particular mapping, the button combinations where this happens can be safely worked around, so I didn't look further into the underlying problem.

[1runeberg]

Hi @emretanirgan - have you seen this response in this thread:

#172 (comment)

I had another look at your test project however, and yeah, the reason is as stated here is that since Action Sets isnt feasible in UE at the moment and since there's no way in UE's core input to trigger an action without a specific input key, sans a core engine input change - you will need to manually keep track of multiple actions that gets triggered by the same key, especially with your test project since you have quite a number of them.

Since with the current way the engine Core Input is architectured, UE can't trigger an action without a key, internally, the plugin selects one key that is bound from an action to use (the last key in the list):

For example in your demo project, the key "Mixed Reality Trackpad Up" is setup to both "Switch Cursor" and "Chat". When "Chat" is triggered, switch cursor is then triggered as well since the plugin sends "Mixed Reality Trackpad Up" to UE's core Input which then triggers all the other actions bound to it in the UI.

So in this case, pressing the "Vive Menu" to trigger "Chat" will also trigger "Switch cursor" since the key "Trackpad Up" is last in the list for Chat, and is also last in the list for "Switch cursor"

As such, you have to "gate" your inputs via game logic so only the correct actions are triggered depending on the state of your game. Or if possible, ensure that the last key in the list in an action is unique and/or the actions they are bound to won't clash with a specific game level/state.

While there is logic in the plugin to look for a unique key bound to an action (not duplicated or used on any other action), and prefer that over the last key in the binding list - it's still best to use game logic to group your actions with duplicate keys based on the current state of your game and only trigger the correct action if your game is in the correct state (In SteamVR we call this "Action Sets", but its not currently supported with UE core input, this is why this Issue is tagged as "Engine Enhancement" as we don't have the ability to add this in the plugin level).

We were able to work around this in the Marketplace version of the plugin by creating "Temporary Keys", however there was no way to make them invisible in the UI so it wasn't a comfortable solution for integration. We have requested action sets however with Epic. Should they implement action sets, it will very likely be implemented via/with OpenXR though since Action Sets are native to OpenXR.

[emretanirgan]

Thanks for the in-depth explanation @1runeberg! This is very helpful, and I'll go with the workarounds you've suggested until action sets are hopefully implemented within the engine.