Multitouch simulation and/or touchscreen support in Editor

[hybridherbst]

What workflow / functionality is missing?
Multitouch simulation and/or touchscreen support in Editor. Currently only single-mouse-click-to-touch is supported.

Describe the solution you'd like

• a way to emulate/simulate multitouch (e.g. 2-finger-rotate) in Editor with a mouse as input
• proper support for Windows touchscreens in AR Simulation / Device Simulator

Currently blocked by Unity's touch injection code randomly breaking when more than one touch ID is used.

[Scrantch]

Hey again guys,

Sent a mail on this but didn't hear back so maybe need to put it in this thread:

I’m having issues again with this. You seemed to fix the mouse drag dropping out but now it seems to be back, or wasn’t really fixed after all. It doesn’t lock up Game view but it does just drop out if I’m dragging objects, etc. Objects will just be left floating in space for example. Also, and likely related, I’ve been killing myself trying to find a bug for two days that turns out is ONLY in Game view again. I am implementing OnDragStart, OnDrag, OnDragEnd and these are being called TWICE sometimes randomly. But not on actual iPhone builds. Finally found that in the PonterEventData sent to the routines that it is processing both pointID 0 AND pointerID 1. For a press and drag it should only be triggering 0. Are you guys somehow trying to simulate multitouch and sending two pointerIDs?

I’m on v1.0.5 Preview.2.

Let me know as this is causing all kinds of problems in my code interactively in Game view.

Thanks!

[Scrantch]

Bump bump. In addition to the above post on problems in ARF 3.x, I can't get any input into the AR scene to work now in Game view after upgrading to ARF 4.1.1. AR Simulation does not appear to be working at all in AR Foundation 4.1.1. I have also upgraded to the 1.0.5 release of ARS. Getting this warning every time it's run:

"No active UnityEngine.XR.XRInputSubsystem is available. Please ensure that a valid loader configuration exists in the XR project settings."

@hybridherbst I saw you post a few months back on a Unity forum about the same error message when you were developing this. What is the fix? Is AR Simulation broke now? What can we do here? I'm dead in the water. Please advise.

Thank you

[hybridherbst]

Sorry for not following up earlier!

ARSimulation is definitely not broken, we've resolved that "XRInputSubsystem" issue long ago, iirc it was just a temporary one on some specific version combinations (and was quickly resolved by Unity). So let's figure out what's going on here.

Would you be able to post a simple repro project here?

If not, can you please list exact

• your platform
• Unity version
• AR Foundation version
• Device Sim version

Regarding multitouch, this certainly sounds strange; it's great that you can verify this works on device though.
Would you be able to post a simple repro project?

[Scrantch]

Thanks for getting back! Can’t disentangle my project easily to post a repo but here’s the specs:

• Mac OS Catalina
• Unity 2020.1
• ARF 4.1.1 (same ARKit, etc.)
• Have been using the Device Sim just Game view

I can click on UI elements in the scene and they function but not any of the AR simulated planes or even move the view with right mouse button of WASD keys. Seems completely frozen in the regard which is why I was wondering if the error message about XRInputSubsystem meant anything. And by that being resolved should we still be seeing it? Because it pops up every time.

Let's first get this working again and then I'll test latest version with all the multi-touch stuff in my first post.

Thanks!

[Scrantch]

Also, your demo scenes are working in the same project. So something happened with upgrading my scene from ARF 3.x to ARF 4.1.1. All project settings are the same. Any ideas what it could be? Do I need to create all new ARSession, etc components on my Game Objects?

[Scrantch]

More info... it appears that this is do to scene initialization. I have a Launcher scene where users can select options and then launch the main scene. If we somehow start from the main scene (as I usually do from the editor) it loads the Launcher first. Going this route I get:

"No AR Session in scene, no automatic setup"

Which makes sense because there is no AR in the Launcher scene, only in the main scene. Once they select options and enter the main scene ARS outputs:

"Added SimulatedARPoseProvider because ARPoseDriver was found."

However, I'm wondering if the setup isn't being done correctly when switching from main scene to launcher and back to main scene as in this case. When it runs this way, no input is allowed in the AR scene in Game view. If I just open and launch directly from the Launcher scene, it works.

Again, all this worked in ARF 3.x and ARS 1.0.5-preview. Did anything along these lines change in ARF 4.1.1 and/or ARS 1.0.5?

[marwie]

Hello @Scrantch thanks for all the info and sorry for the inconvinience.

There should be no need to upgrade any components due to upgrading AR Foundation.

So if I understand correctly it breaks when:

1. Loading scene with AR Session + triggering Automatic Setup
2. Loading another scene without AR
3. Loading scene with AR Session again (+ triggering Automatic Setup?)

And you still get simulated AR planes but only input stops working?

[marwie]

Hello @Scrantch, I tried reproducing your bug (tried both on osx and windows using Unity versions 2020.1 and 2019.4) with ARSim 1.0.5 and ARF 4.1.1

When trying to reproduce I saved a scene using "Tools/Setup Basic AR" and created another empty scene and a scene with "Extended AR". I switched between the scenes using a keyboard shortcut. I got the same messages in the console as you but that at least seems not to be enough to break input.

1. When starting from the main scene you mention the launcher being loaded: at what moment is the launcher scene being loaded (via some attribute/OnInitializeOnLoad/RuntimeInitialize/Awake?)
2. Do you save any state in the scenes before/after loading?
   3a) When you say "Have been using Device Sim just Game View" do you mean you're NOT using the device simulator or are you using the device simulator window but set to "game view mode"?
   3b) Is the device simulator installed and is the window open or docked somewhere in your layout?

Thanks in advance for your help :) I hope we can find the root 🌱 for this issue soon

[Scrantch]

Hi, sorry for the delay in getting back. Had a big sprint we needed to get through. To answer your questions:

1. Basically we are using Photon Networking. The launcher screen sets up username, room they're looking for etc. So if I'm in the main scene (which I'm usually working from) and press Play in the editor the controller script looks and sees if we're not connected to Photon in the Start() method. If not, then it loads the Launcher scene which allows you to input network parameters. Then when you click Connect, after connecting to the network, it loads the main scene where all the AR stuff is.

2. Sorry, yes that was a typo. I am NOT using Device sim. Just Game View.

Many thanks.

[Scrantch]

@marwie Hi, any update on this?

[marwie]

Not yet @Scrantch, sorry for the delay :(