ROI with Arduplane - Get the GPS coordinate where the Pan-tilt Camera is looking at.

[Elios007]

HI,

We would like to have the ROI feature in arduplane with Advanced options.

We would like to know GPS coordinates for the location of the crosshairs of a PTZ camera, otherwise known as ROI or Geo-pointing. This will essentially give us the LAT/LONG of what we are looking at.

The Brushless Gimbal will be connected to the Pixhawk so the Pixhawk knows where it is looking, and because there is a terrain following inside the Pixhawk, so we will combine both feature to get us where the camera is looking at regardless the direction and position of the plane.

Some Options how to use this new ROI inside the Arduplane:

Scenario 1: If someone is searching for a missing person in a mountain or any place, the plane does not need to fly over the found person to get his GPS coordinate because when he saw him in the camera the OSD and Ground station will show his GPS coordinate because the Camera is looking at him. It can save time by sending his coordinates to the ground team to go and get him and the plane will save time to go and search for other missing people.

Scenario 2: If we know the GPS Coordinate of an accident, we can add them into the mission planner and the plane will go there but it's not the same existing "fly-to-here" feature, because we will define the altitude and the ground distance for the plane to be away from the accident, which means the Gimbal Camera will look at the accident from a certain defined distance and altitude instead of being above the accident and the plane will continue doing loiter or flying manually...

I hope you will be able to help us in this feature as it can save lives.

[magicrub]

That's a pretty major feature. Would love to see it in ArduPilot.
Auto-search mode!

On Thu, Apr 2, 2015 at 4:22 AM, Elios007 [email protected] wrote:

HI,

We would like to have the ROI feature in arduplane with Advanced options.

We would like to know GPS coordinates for the location of the crosshairs
of a PTZ camera, otherwise known as ROI or Geo-pointing. This will
essentially give us the LAT/LONG of what we are looking at.

[image: image]
https://cloud.githubusercontent.com/assets/8313659/6962969/641414bc-d942-11e4-9dd5-22410c180fa0.png

[image: image]
https://cloud.githubusercontent.com/assets/8313659/6958113/81bd0830-d90f-11e4-9c8d-d5fd3170f104.png

[image: image]
https://cloud.githubusercontent.com/assets/8313659/6958116/a2de2864-d90f-11e4-8999-b164b87e0940.png

The Brushless Gimbal will be connected to the Pixhawk so the Pixhawk knows
where it is looking, and because there is a terrain following inside the
Pixhawk, so we will combine both feature to get us where the camera is
looking at regardless the direction and position of the plane.

Some Options how to use this new ROI inside the Arduplane:

Scenario 1: If someone is searching for a missing person in a mountain or
any place, the plane does not need to fly over the found person to get his
GPS coordinate because when he saw him in the camera the OSD and Ground
station will show his GPS coordinate because the Camera is looking at him.
It can save time by sending his coordinates to the ground team to go and
get him and the plane will save time to go and search for other missing
people.

Scenario 2: If we know the GPS Coordinate of an accident, we can add them
into the mission planner and the plane will go there but it's not the same
existing "fly-to-here" feature, because we will define the altitude and the
ground distance for the plane to be away from the accident, which means the
Gimbal Camera will look at the accident from a certain defined distance and
altitude instead of being above the accident and the plane will continue
doing loiter or flying manually...

I hope you will be able to help us in this feature as it can save lives.

—
Reply to this email directly or view it on GitHub
#2041.

[tridge]

To do this properly with arbitrary terrain is a ray casting problem, which is quite tricky to do on the STM32, especially as we may not have all the terrain data paged in from microSD
It may be better to do this on the GCS. The GCS already has all the information it needs, but it will also have 30m grid terrain data, and lots of CPU and memory to do the ray casting

[magicrub]

What if we don't use terrain and assume a flat earth?

[magicrub]

Right, which makes it a harder, and less probable to get implemented,
problem. However, if we start with the framework and get the simple case
working then adding elevation info is just an enhancement and less
daunting. Baby steps!

On Sun, Apr 5, 2015 at 8:23 AM, Elios [email protected] wrote:

Because the GPS will be wrong if you assume it is flat while there is a
mountain.

Regards
Elie Waked
skype: DonElie
00441212868687
On Apr 5, 2015 6:19 PM, "Tom Pittenger" [email protected] wrote:

What if we don't use terrain and assume a flat earth?

—
Reply to this email directly or view it on GitHub
<
https://github.com/diydrones/ardupilot/issues/2041#issuecomment-89789852>

.

—
Reply to this email directly or view it on GitHub
#2041 (comment)
.

[lvale]

@tridge The original question made at RCGroups also had a 10x Zoom Camera involved in the equation....

[ionoy]

Are you using Mavlink to send gimbal orientation down to GCS? I'm actually implementing similar solution on my own, but I've decided to use separate link for gimbal communication at the moment, because ArduCopter doesn't have a good solution for sending this kind of data from UserCode.

I would love to see some extension points accessible from UserCode to receive and send packets of data.

[squilter]

related to #510

[ionoy]

@Elios007 At the moment our software is closed, sorry.

[raptortrias]

This function is of interest, has any functionality been implemented

thanks

[TunaLobster]

The first part of this was implemented with 6d65b35. Adding in the terrain data to account for a large change in altitude from the home position has not been implemented.

[amilcarlucas]

I think Scenario 2 is already implemented in the current set ROI command

[rmackay9]

Knowing the lat, lon, alt of what the gimbal is pointing at is available using the mount-poi Lua script. We have a c++ solution coming in 4.5 once this PR is merged #24657

[peterbarker]

We have a merged solution for this now, so closing.