OpenCV4 instead of opencv3

[YuTingLiu]

I'm using latest vision of opencv, opencv 4.0.1, is there any plan to update repo to this version?
If you guys don't have a plan, is there any way to modify code to using this version of opencv?
I have already modify src/vision_opencv/cv_bridge/CMakeLists.txt, change:

find_package(OpenCV 3 REQUIRED
  COMPONENTS
    opencv_core
    opencv_imgproc
    opencv_imgcodecs
 CONFIG
)

to

set(OpenCV_DIR /usr/local/share/opencv4)
find_package(OpenCV 4.0.1 EXACT REQUIRED)`

and output shows ROS find OPENCV:

system
-- +++ processing catkin package: 'cv_bridge'
-- ==> add_subdirectory(vision_opencv/cv_bridge)
-- Boost version: 1.62.0
-- Found the following Boost libraries:
--   python
-- Found OpenCV: /usr/local (found suitable exact version "4.0.1")
-- Found PythonLibs: /usr/lib/arm-linux-gnueabihf/libpython2.7.so (found suitable version "2.7.13", minimum required is "2.7")

But report error when 👍
rospack depends cv_bridge

[rospack] Error: package 'cv_bridge' depends on non-existent package 'opencv3' and rosdep claims that it is not a system dependency. Check the ROS_PACKAGE_PATH or try calling 'rosdep update'

[sreevan]

I too am facing this, is there any specific workaround or solution for this please? I am on Jetson Tx2 and compiling opencv again would not be possible as i can do that only after flashing to free up space. Please help.

[sreevan]

I was able to successfully compile cv_bridge with opencv4 below are the rough notes of what i did:

1. Add set (CMAKE_CXX_STANDARD 11) to your top level cmake
2. In cv_bridge/src CMakeLists.txt line 35 change to if (OpenCV_VERSION_MAJOR VERSION_EQUAL 4)
3. In cv_bridge/src/module_opencv3.cpp change signature of two functions
   3.1) UMatData* allocate(int dims0, const int* sizes, int type, void* data, size_t* step, int flags, UMatUsageFlags usageFlags) const
   to
   UMatData* allocate(int dims0, const int* sizes, int type, void* data, size_t* step, AccessFlag flags, UMatUsageFlags usageFlags) const
   3.2) bool allocate(UMatData* u, int accessFlags, UMatUsageFlags usageFlags) const
   to
   bool allocate(UMatData* u, AccessFlag accessFlags, UMatUsageFlags usageFlags) const

[acxz]

It seems like there is work for adding opencv4 support on #259 .

[hgaiser]

Yes but it's been open for half a year, it seems there isn't much maintenance on these repositories anymore. My advice: use that PR and compile all the necessary packages yourself. It's what we did.

[acxz]

@vrabaud and/or @mjcarroll,
Can you guys comment on this? Will this package be updated for opencv4 or is this package development stale?

[mjcarroll]

@acxz I'm going to at least get it added for ROS2, since I'm maintaining that at the moment. I can't speak to the plans for ROS1 at the moment, but the change looks reasonable.

[acxz]

Thank you for your comment! I hope it also gets added into ROS1: melodic.

[seanyen]

@mjcarroll Curious any updates on this, at least for ROS2?

[YuTingLiu]

Since this issue has openned for so long time, if there any progress on this, please close it.

[tkircher]

There hasn't been any progress, and ROS2 is basically a dead project. Plenty of systems have OpenCV 4 installed as the only version and yet ROS is still built against OpenCV 3.2. That means no modules depending on OpenCV can be built from source without building the entirety of ROS from source. Hilariously, on Ubuntu if you try to install OpenCV 3.2 dev libraries, it forces ROS to be removed.

[mjcarroll]

Sorry all, here is the plan moving forward:

On the ROS1 side, @ahcorde is going to help out getting vision_opencv ready for the noetic release, which includes targeting Ubuntu focal, and by extension, OpenCV4 as well as Python 3. That work is ongoing here: #323

On the ROS2 side, I'm going to be getting vision_opencv ready for the foxy release, which includes targeting Ubuntu focal, and by extension, OpenCV4 as well.

Both of these releases are currently planned for May 23.

[mjcarroll]

Additionally, because @vrabaud has been missing for so long, it may be time for someone to claim maintainership on the ROS1 side by following the process here: http://wiki.ros.org/MaintenanceGuide#Claiming_Maintainership

@seanyen I don't know if this is something that you and your team would be interested in.

[jorgemiar]

So I just started with the Jetson Nano and ros melodic and it comes with OpenCV 4.1.1. preinstalled. Should I remove it and install OpenCV 3.2. to ensure no compatibility issues with cv_bridge or other packages I have (currently having some compatibility issues when running catkin make because I tried to install the image_view package)? Will ROS be removed if I do that? Don't want to have to install a new OpenCV if possible @tkircher

[tkircher]

@jorgemia I recommend leaving the system OpenCV in place and building ROS from scratch. Skip the repositories completely and start from a clean install of Jetpack. One big problem though is that there are dozens and dozens of ROS modules that are included in the Ubuntu repo that are not present in the ROS source - rtabmap, for example. It isn't built in ROS Melodic by default, so you have to pull it down from github into your catkin workspace and build it manually.

[tkircher]

I decided to summarize what I did to get it working, here, for lack of a better place.

We'll just start where the usual instructions start.
See: http://wiki.ros.org/Installation/Source

We have to install the base dependencies, which work fine from repo:

$ sudo apt-get install python-rosdep python-rosinstall-generator python-wstool python-rosinstall build-essential

Then we initialize rosdep:

$ sudo rosdep init
$ rosdep update

Then create our catkin workspace and pull down what ROS thinks are the main packages:

$ mkdir ~/ros_catkin_ws
$ cd ~/ros_catkin_ws
$ rosinstall_generator desktop_full --rosdistro melodic --deps --tar > melodic-desktop-full.rosinstall
$ wstool init -j8 src melodic-desktop-full.rosinstall

Now we have to fix it to actually include the main packages, along with a few others:

$ git clone https://github.com/pal-robotics/ddynamic_reconfigure src/ddynamic_reconfigure

$ rm -r src/navigation_msgs
$ git clone https://github.com/ros-planning/navigation_msgs.git src/navigation_msgs

$ rm -r src/geometry2/
$ git clone https://github.com/ros/geometry2.git src/geometry2

$ rm -r src/image_transport_plugins
$ git clone https://github.com/ros-perception/image_transport_plugins.git src/image_transport_plugins

$ git clone https://github.com/ros-planning/navigation.git src/navigation
$ git clone https://github.com/AprilRobotics/apriltag_ros.git src/apriltag_ros
$ git clone https://github.com/OctoMap/octomap_msgs.git src/octomap_msgs
$ git clone https://github.com/introlab/find-object.git src/find_object_2d
$ git clone https://github.com/introlab/rtabmap_ros.git src/rtabmap_ros

Then we have to modify several files for building against OpenCV 4. First make the modifications specified in sreevan's comment above (or one of the various PRs). Then,

In src/image_pipeline/image_publisher/CMakeLists.txt

Change the OpenCV section from:

set(opencv_2_components core highgui)
set(opencv_3_components core imgcodecs videoio)
find_package(OpenCV REQUIRED COMPONENTS core)
message(STATUS "opencv version ${OpenCV_VERSION}")
if(OpenCV_VERSION VERSION_LESS "3.0.0")
  find_package(OpenCV 2 REQUIRED COMPONENTS ${opencv_2_components})  
else()
  find_package(OpenCV 3 REQUIRED COMPONENTS ${opencv_3_components})  
endif()

to:

message(STATUS "opencv version ${OpenCV_VERSION}")
find_package(OpenCV REQUIRED COMPONENTS core imgcodecs videoio)

In src/image_pipeline/image_publisher/src/nodelet/image_publisher_nodelet.cpp

change CV_LOAD_IMAGE_COLOR to cv::IMREAD_COLOR
change CV_CAP_PROP_POS_FRAMES to cv::CAP_PROP_POS_FRAMES, etc.

In src/gazebo_ros_pkgs/gazebo_plugins/src/gazebo_ros_prosilica.cpp

Change

    #include <opencv/cv.h>
    #include <opencv/highgui.h>
    #include <opencv/cvwimage.h>

to

    #include <opencv2/core.hpp>
    #include <opencv2/highgui/highgui.hpp>

Then we start the build:

./src/catkin/bin/catkin_make_isolated --install -DCMAKE_BUILD_TYPE=Release

This works for me, as they say. ;)

[Stigmaru]

Sorry all, here is the plan moving forward:

On the ROS1 side, @ahcorde is going to help out getting vision_opencv ready for the noetic release, which includes targeting Ubuntu focal, and by extension, OpenCV4 as well as Python 3. That work is ongoing here: #323

On the ROS2 side, I'm going to be getting vision_opencv ready for the foxy release, which includes targeting Ubuntu focal, and by extension, OpenCV4 as well.

Both of these releases are currently planned for May 23.

Is OpenCV4 or vision_opencv preinstalled into ROS Noetic or do I need to install OpenCV separately?

[ahcorde]

@AeroWRX Please don't ask the same question two times. Ask this kind of questions here https://answers.ros.org/questions/

[jveitchmichaelis]

@AeroWRX To answer the question, it depends. When you install vision_opencv, it pulls in opencv from your package manager. This has some advantages, since it means we can guarantee a working install on a particular release of Ubuntu (or whatever), provided you use the matching supported ROS version, but also it's a pain because the version of OpenCV in the package repository has been lagging for a long time (at 3.2).

Ubuntu focal now packages OpenCV 4.2: https://packages.ubuntu.com/focal/libopencv-core-dev so things ought to improve.

So provided you're also installing ROS on the latest version of Ubuntu, you should be OK. If you try on an older release, like 18/19, you will get OpenCV 3.2 unless you install both OpenCV and vision_opencv manually.

[Algabri]

@sreevan
which one you meant 1. Add set (CMAKE_CXX_STANDARD 11) to your top level cmake

Which one ?
:~/catkin_ws/src/vision_opencv/cv_bridge/cmake$ cv_bridge-extras.cmake.in or the top level ~/catkin_ws/src/vision_opencv/cv_bridge$ CMakeLists.txt file.

I'd like to build opencv4.2 with Ubuntu 16.04 and python 2.7

[sreevan]

@Algabri it is the second one (~/catkin_ws/src/vision_opencv/cv_bridge$ CMakeLists.txt). You might also have troubles in image_transport_plugins, image_pipeline and gazebo_plugins packages of ros (assuming you need ros as you are compiling CV_bridge) and so you would need to recompile them too after the below changes:

• change in ~/ros_catkin_ws/src/image_transport_plugins/compressed_image_transport/src/compressed_publisher.cpp
  all CV_IMWRITE_ flags to cv::IMWRITE_

• change in ~/ros_catkin_ws/src/image_transport_plugins/compressed_depth_image_transport/src/codec.cpp
  CV_LOAD_IMAGE_UNCHANGED to cv::IMREAD_UNCHANGED

• change in ~/ros_catkin_ws/src/image_pipeline/image_publisher/CMakeLists.txt
  OPENCV 3 REQUIRED to OPENCV 4 REQUIRED

• change in ~/ros_catkin_ws/src/image_pipeline/image_publisher/src/nodelet/image_publisher_nodelet.cpp
  all CV_CAP_POS_FRAMES to 1
  CV_LOAD_IMAGE_COLOR to cv::IMREAD_COLOR

• change in gazebo_plugins/src/gazebo_ros_prosilica.cpp
  #include <opencv/cv.h>
  #include <opencv/highgui.h>
  #include <opencv/cvwimage.h>

            TO
  
            #include <opencv2/core/core_c.h>
            #include <opencv2/highgui/highgui_c.h>
  

[Algabri]

First of all, thank you so much for your help @sreevan

Additional to this modification, I did as following:

1. I added set (CMAKE_CXX_STANDARD 11) to~/catkin_ws/src/vision_opencv/cv_bridge$ CMakeLists.txtfile.

2. for this point : ~/ros_catkin_ws/src/image_transport_plugins/compressed_image_transport/src/compressed_publisher.cpp
   all CV_IMWRITE_ flags to cv::IMWRITE_

I downloaded this and put it in ~/catkin_ws/src/, then I went to~/catkin_ws/src/image_transport_plugins-indigo-devel/compressed_image_transport/src$ compressed_publisher.cpp .

I didn't find CV_IMWRITE_ . I found:

params.resize(9, 0);
      params[0] = IMWRITE_JPEG_QUALITY;
      params[1] = config_.jpeg_quality;
      params[2] = IMWRITE_JPEG_PROGRESSIVE;
      params[3] = config_.jpeg_progressive ? 1 : 0;
      params[4] = IMWRITE_JPEG_OPTIMIZE;
      params[5] = config_.jpeg_optimize ? 1 : 0;
      params[6] = IMWRITE_JPEG_RST_INTERVAL;
      params[7] = config_.jpeg_restart_interval;

and

 params.resize(3, 0);
      params[0] = IMWRITE_PNG_COMPRESSION;
      params[1] = config_.png_level;

3. I found it and changed in ~/ros_catkin_ws/src/image_transport_plugins/compressed_depth_image_transport/src/codec.cpp
   CV_LOAD_IMAGE_UNCHANGED to cv::IMREAD_UNCHANGED

4. for this point: change in ~/ros_catkin_ws/src/image_pipeline/image_publisher/CMakeLists.txt
   OPENCV 3 REQUIRED to OPENCV 4 REQUIRED

I changed twice, first time:

set(opencv_3_components core highgui)
set(opencv_4_components core imgcodecs videoio)
find_package(OpenCV REQUIRED COMPONENTS core)
message(STATUS "opencv version ${OpenCV_VERSION}")
if( NOT (OpenCV_VERSION VERSION_LESS "4.0.0"))
  find_package(OpenCV 4 REQUIRED COMPONENTS ${opencv_4_components})  
else()
  find_package(OpenCV 3 REQUIRED COMPONENTS ${opencv_3_components})  
endif()

Also second time after made comment out for above lines:

message(STATUS "opencv version ${OpenCV_VERSION}")
find_package(OpenCV REQUIRED COMPONENTS core imgcodecs videoio)

5. for this point: change in ~/ros_catkin_ws/src/image_pipeline/image_publisher/src/nodelet/image_publisher_nodelet.cpp
   all CV_CAP_POS_FRAMES to 1
   CV_LOAD_IMAGE_COLOR to cv::IMREAD_COLOR

change CV_LOAD_IMAGE_COLOR to cv::IMREAD_COLOR
change CV_CAP_PROP_POS_FRAMES to cv::CAP_PROP_POS_FRAMES,

Also,

first time : cap_.set(cv::CAP_PROP_POS_FRAMES,0);
second time: cap_.set(cv::CAP_PROP_POS_FRAMES,1);
last time: cap_.set(1,0);
If it is : cap_.set(1); .................> it gave an error.

6. for this point:

change in gazebo_plugins/src/gazebo_ros_prosilica.cpp
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <opencv/cvwimage.h>

I didn't implement it because I am running the code with real robot not gazebo, is it necessary??

As you know last step:
~/catkin_ws$ catkin_make

I see this line six times:

-- Found OpenCV: /usr/local (found version "3.3.1") not 4.2

Note: If lines order in the code as below, it runs with OpenCV 4.2

import rospy
import sys
ros_path2 = '/usr/local/lib/python2.7/site-packages'
ros_path3 = '/usr/lib/python2.7/dist-packages'
if  ros_path2 and ros_path3 in sys.path:
    sys.path.remove(ros_path2)
    sys.path.remove(ros_path3)
import cv2

But if they as:

import sys
ros_path2 = '/usr/local/lib/python2.7/site-packages'
ros_path3 = '/usr/lib/python2.7/dist-packages'
if  ros_path2 and ros_path3 in sys.path:
    sys.path.remove(ros_path2)
    sys.path.remove(ros_path3)
import cv2
import rospy

The output:

import rospkg
ImportError: No module named rospkg

Without import sys, it runs with OpenCV 3.3.1

Thanks

[dmc146]

Sorry all, here is the plan moving forward:

On the ROS1 side, @ahcorde is going to help out getting vision_opencv ready for the noetic release, which includes targeting Ubuntu focal, and by extension, OpenCV4 as well as Python 3. That work is ongoing here: #323

On the ROS2 side, I'm going to be getting vision_opencv ready for the foxy release, which includes targeting Ubuntu focal, and by extension, OpenCV4 as well.

Both of these releases are currently planned for May 23.

ROS Noetic ships with the latest versions of OpenCV (4.2.0), vision-opencv, cv_bridge. Yet cv_bridge.h (line 43) still #includes "opencv2/core/core.hpp" which throws up an error and apparently prevents building a package. Am I missing something (probably) or is this part of an awaited fix?

[mjcarroll]

Yet cv_bridge.h (line 43) still #includes "opencv2/core/core.hpp" which throws up an error and apparently prevents building a package.

#include "opencv2/..." is still valid, as that's what OpenCV ships. For example, this is the include directory on master at the moment: https://github.com/opencv/opencv/tree/master/modules/core/include

[dmc146]

Yet cv_bridge.h (line 43) still #includes "opencv2/core/core.hpp" which throws up an error and apparently prevents building a package.

#include "opencv2/..." is still valid, as that's what OpenCV ships. For example, this is the include directory on master at the moment: https://github.com/opencv/opencv/tree/master/modules/core/include

Thanks for your tolerant response to an uninformed question... The misplaced question came up because I am getting "No such file or directory" for opencv2/core/core.hpp when trying to build a package via catkin_make. I assume this is something to do with some missing dependencies or some such on my part and will continue searching

[mjcarroll]

The misplaced question came up because I am getting "No such file or directory" for opencv2/core/core.hpp when trying to build a package via catkin_make

It may be that the dependency isn't fulfilled. Have you verified that the corresponding opencv packages have been installed? I believe that libopencv-dev should be sufficient on Debian/Ubuntu.

[dmc146]

It may be that the dependency isn't fulfilled. Have you verified that the corresponding opencv packages have been installed? I believe that libopencv-dev should be sufficient on Debian/Ubuntu.

Yes, have just verified again - "libopencv-dev is already the newest version (4.2.0+dfsg-5)"