jetson TX2 camera #12

imxboards · 2018-03-30T07:21:53Z

hi ,
I install tx2 with
Ubuntu 16.04
Python 2.7
Tensorflow 1.7
OpenCV 3.3.1
But when I ruan object_detection.py issue is coming

vidia@tegra-ubuntu:~/tf/realtime_object_detection$ python object_detection.py

Model found. Proceed.
Loading frozen model into memory
2018-03-30 07:11:08.517989: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:865] ARM64 does not support NUMA - returning NUMA node zero
2018-03-30 07:11:08.518123: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties:
name: NVIDIA Tegra X2 major: 6 minor: 2 memoryClockRate(GHz): 1.3005
pciBusID: 0000:00:00.0
totalMemory: 7.66GiB freeMemory: 2.29GiB
2018-03-30 07:11:08.518172: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0
2018-03-30 07:11:10.014011: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-03-30 07:11:10.014083: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0
2018-03-30 07:11:10.014109: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N
2018-03-30 07:11:10.014269: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1694 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
Loading label map
Building Graph
2018-03-30 07:11:33.197609: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0
2018-03-30 07:11:33.197707: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-03-30 07:11:33.197737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0
2018-03-30 07:11:33.197758: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N
2018-03-30 07:11:33.197847: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1694 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
start!!!
2018-03-30 07:11:33.199546: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0
2018-03-30 07:11:33.199636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-03-30 07:11:33.199671: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0
2018-03-30 07:11:33.199713: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N
2018-03-30 07:11:33.199817: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1694 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
2018-03-30 07:11:33.200175: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0
2018-03-30 07:11:33.200259: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-03-30 07:11:33.200292: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0
2018-03-30 07:11:33.200315: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N
2018-03-30 07:11:33.200394: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1694 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
VIDEOIO ERROR: V4L2: Pixel format of incoming image is unsupported by OpenCV
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in cvCaptureFromCAM_GStreamer, file /home/nvidia/opencv/modules/videoio/src/cap_gstreamer.cpp, line 887
VIDEOIO(cvCreateCapture_GStreamer(CV_CAP_GSTREAMER_V4L2, reinterpret_cast<char *>(index))): raised OpenCV exception:

/home/nvidia/opencv/modules/videoio/src/cap_gstreamer.cpp:887: error: (-2) GStreamer: unable to start pipeline
in function cvCaptureFromCAM_GStreamer

Start video stream with shape: 0,0
Press 'q' to Exit
Starting Detection
OpenCV Error: Assertion failed (scn == 3 || scn == 4) in cvtColor, file /home/nvidia/opencv/modules/imgproc/src/color.cpp, line 11016
Traceback (most recent call last):
File "object_detection.py", line 302, in
main()
File "object_detection.py", line 298, in main
detection(graph, category, score, expand)
File "object_detection.py", line 213, in detection
image_expanded = np.expand_dims(cv2.cvtColor(image, cv2.COLOR_BGR2RGB), axis=0)
cv2.error: /home/nvidia/opencv/modules/imgproc/src/color.cpp:11016: error: (-215) scn == 3 || scn == 4 in function cvtColor

can you help me ? should i change config.yml parm?

gustavz · 2018-04-02T11:26:43Z

are you using the onboard camera of the jetson?
do you know how to use it in opencv, have a look here:
https://devtalk.nvidia.com/default/topic/1017860/using-on-board-tx2-camera-with-opencv-3-2-/?offset=7 and others

Alro10 · 2018-04-14T15:11:13Z

Hi! for my experience:
In config.sample.yml

video_input : nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)600, height=(int)600,format=(string)I420, framerate=(fraction)5/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink

warisz · 2018-04-27T06:08:51Z

@AlexanderRobles21 @gustavz
Hi, I tried setting video_input as you suggested for Jetson TX2 camera, however, I get the following error.

nvidia@tegra-ubuntu:~/Desktop/realtime_object_detection$ python object_detection.py 
> Model found. Proceed.
> Loading frozen model into memory
2018-04-26 22:56:07.466932: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:865] ARM64 does not support NUMA - returning NUMA node zero
2018-04-26 22:56:07.467125: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 0 with properties: 
name: NVIDIA Tegra X2 major: 6 minor: 2 memoryClockRate(GHz): 1.3005
pciBusID: 0000:00:00.0
totalMemory: 7.66GiB freeMemory: 2.29GiB
2018-04-26 22:56:07.467194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-04-26 22:56:10.025344: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1687 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
> Loading label map
> Building Graph
2018-04-26 22:56:32.894441: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-04-26 22:56:32.894590: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 577 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
VIDEOIO ERROR: V4L: device nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)600, height=(int)600,format=(string)I420, framerate=(fraction)5/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink: Unable to query number of channels
2018-04-26 22:56:32.897282: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-04-26 22:56:32.897409: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 577 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
2018-04-26 22:56:32.902302: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-04-26 22:56:32.902475: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 577 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)

Available Sensor modes : 
2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 1 WxH = 2592x1458 FrameRate = 30.000000 ...

Socket read error. Camera Daemon stopped functioning.....
gst_nvcamera_open() failed ret=0
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in icvStartPipeline, file /home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp, line 450
Traceback (most recent call last):
  File "object_detection.py", line 301, in <module>
    main()
  File "object_detection.py", line 297, in main
    detection(graph, category, score, expand)
  File "object_detection.py", line 201, in detection
    video_stream = WebcamVideoStream(video_input,width,height).start()
  File "/home/nvidia/Desktop/realtime_object_detection/stuff/helper.py", line 108, in __init__
    self.stream.set(cv2.CAP_PROP_FRAME_WIDTH, self.width)
cv2.error: /home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp:450: error: (-2) GStreamer: unable to start pipeline
 in function icvStartPipeline

I have installed applicable gstreamer dev packages.
I am able to invoke the camera through terminal via gst-launch, for example gst-launch-1.0 -ev nvcamerasrc ! nvvidconv ! ximagesink and can also access camera via other scripts for example

  # Use Jetson onboard camera
    gst_str = ("nvcamerasrc ! "
               "video/x-raw(memory:NVMM), width=(int)2592, height=(int)1458, format=(string)I420, framerate=(fraction)30/1 ! "
               "nvvidconv ! video/x-raw, width=(int){}, height=(int){}, format=(string)BGRx ! "
               "videoconvert ! appsink").format(width, height)
    return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)

Any help would be appreciated. Thanks.

gustavz · 2018-04-27T06:37:21Z

I make a guess:
you are using OpenCV that comes preinstalled with flashing JetPack right?

The problem is that Nvidia/JetPack Build OpenCV without Gstreamer support.
That’s why it is not possible to use the onboard camera with the standard OpenCV (ask NVIDIA why they install OpenCV that way that it can’t use its own hardware. They also disabled CUDA support for OpenCV, so i guess they dont want you to use it but instead Visionworks...)

Long story short:
you need to completely deinstall OpenCV and reinstall it from source with Gstreamer Support Enabled.

warisz · 2018-04-27T06:53:16Z

Hi Gustav,

Thanks for your quick reply. I actually reinstalled OpenCV from source following this tutorial: https://jkjung-avt.github.io/opencv3-on-tx2/
I am able to launch the onboard camera, like for example, via the last code snippet I posted in my previous message.

I am using TF1.6 with JetPack 3.1 (and have also tried TF1.5 and gotten same error). From prying around the other issues in this repo, TF1.4 is recommended for this repo; however the error I have posted seems to be Gstreamer related.

I will try reinstalling OpenCV from source and switch to JetPack3.1 to use TF1.4 and try that and post an update. Any input from your side is also appreciated.

Thanks a lot.

gustavz · 2018-04-27T08:31:43Z

I use and always have used JetPack 3.2, maybe you should switch to that?

Alro10 · 2018-04-27T14:38:57Z

Hi @imxboards,

I recently tested object_detection.py for Jetpack 3.1 and TF 1.3 without any problems using onboard cam and webcam ( video_input = /dev/video1) achieved 20fps. But when I used TF 1.6 rc1 there is a problem:

2018-04-27 11:06:05.483867: E tensorflow/stream_executor/event.cc:33] error destroying CUDA event in context 0x337fbf0: CUDA_ERROR_LAUNCH_FAILED

I recommend to use TF 1.3, I think this version works for CUDA 8. For Jetpack 3.2 (CUDA 9) I didn't tested yet.

gustavz · 2018-04-27T14:44:08Z

I successfully tested it on 1.4, 1.5, 1.6 but with varying performance results.

If you want to achieve highest results use the versions that I recommend in the readme.

Also I strongly recommend you all to use JetPack 3.2 because of CUDA 9 / cuDNN 7 among other improvements

lfelipesv · 2018-05-14T07:38:15Z

@warisz could you solve the problem? I am experiencing the same problem trying to install with tensorflow 1.6 and opencv 3.4.0 that i built by myself... cheers!

scbean · 2018-09-17T05:52:44Z

I am experiencing the same problem, I have Jetpack 3.3 and TF 1.9

I installed open CV using this tutorial :
https://jkjung-avt.github.io/opencv3-on-tx2/

VIDEOIO ERROR: V4L: device nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink: Unable to query number of channels

Available Sensor modes :
2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 1 WxH = 2592x1458 FrameRate = 30.000000 ...

Available Sensor modes :
2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 1 WxH = 2592x1458 FrameRate = 30.000000 ...

Socket read error. Camera Daemon stopped functioning.....
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in icvStartPipeline, file /home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp, line 450
Traceback (most recent call last):
File "run_stream.py", line 123, in main
detection.start(cfg)
File "/home/nvidia/Desktop/realtime_object_detection-master/lib/detection_nms_v2.py", line 192, in start
video_reader.start(VIDEO_INPUT, WIDTH, HEIGHT, save_to_file=SAVE_TO_FILE)
File "/home/nvidia/Desktop/realtime_object_detection-master/lib/webcam.py", line 45, in start
self.vid.set(cv2.CAP_PROP_FRAME_HEIGHT, height)
cv2.error: /home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp:450: error: (-2) GStreamer: unable to start pipeline
in function icvStartPipeline

Merwanski · 2019-02-13T16:05:06Z

Hi @scbean have similar problem to the one you are describing
did you find a solution for it ???

Merwan

Alro10 · 2019-02-14T15:55:02Z

Hi @Merwanski and @scbean

You can check my repository: https://github.com/Alro10/realtime_object_detection

Use this file: object_detectionjetson.py

Please write back if that solves your problem!

Merwanski · 2019-02-15T21:58:25Z

Thank you @Alro10 ... the camera is working but I am still getting this error message
""""""
VIDEOIO ERROR: V4L: device nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink: Unable to query number of channels
"""""""

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

jetson TX2 camera #12

jetson TX2 camera #12

imxboards commented Mar 30, 2018

gustavz commented Apr 2, 2018

Alro10 commented Apr 14, 2018

warisz commented Apr 27, 2018 •

edited

Loading

gustavz commented Apr 27, 2018 •

edited

Loading

warisz commented Apr 27, 2018

gustavz commented Apr 27, 2018

Alro10 commented Apr 27, 2018

gustavz commented Apr 27, 2018 •

edited

Loading

lfelipesv commented May 14, 2018

scbean commented Sep 17, 2018

Merwanski commented Feb 13, 2019

Alro10 commented Feb 14, 2019

Merwanski commented Feb 15, 2019

jetson TX2 camera #12

jetson TX2 camera #12

Comments

imxboards commented Mar 30, 2018

gustavz commented Apr 2, 2018

Alro10 commented Apr 14, 2018

warisz commented Apr 27, 2018 • edited Loading

gustavz commented Apr 27, 2018 • edited Loading

warisz commented Apr 27, 2018

gustavz commented Apr 27, 2018

Alro10 commented Apr 27, 2018

gustavz commented Apr 27, 2018 • edited Loading

lfelipesv commented May 14, 2018

scbean commented Sep 17, 2018

Merwanski commented Feb 13, 2019

Alro10 commented Feb 14, 2019

Merwanski commented Feb 15, 2019

warisz commented Apr 27, 2018 •

edited

Loading

gustavz commented Apr 27, 2018 •

edited

Loading

gustavz commented Apr 27, 2018 •

edited

Loading