SVET Linux sample appliction 2020 R3 release

Dependency: MediaSDK 20.3 and OpenVINO 2021.1

New features on R3:

Support CPU inference by option “-infer::device GPU”.
Support shared inference network instance between sessions
Support setting decoding output to RGBP by option “-dc::rgbp”
Support 2 video decoder outputs from SFC and VPP in different sizes and color format
Support more detection networks by specify XML files

Signed-off-by: Elaine Wang <[email protected]>

README.md

@@ -27,33 +27,33 @@ See [FAQ](./doc/FAQ.md)
   * [Known limitations](#know-limitations)
 
 # License
-The sample application is licensed under MIT license. See [LICENSE](./LICENSE) for details.
+The sample application is licensed under MIT license. See [LICENSE](./video_e2e_sample/LICENSE) for details.
 
 # How to contribute
 See [CONTRIBUTING](./doc/CONTRIBUTING.md) for details. Thank you!
 
 # Documentation
-See [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.2.0.pdf)
+See [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.3.0.pdf)
 
 # System requirements
 
 **Operating System:**
 * Ubuntu 18.04.02
 
 **Software:**
-* [MediaSDK 20.1.1](https://github.com/Intel-Media-SDK/MediaSDK/releases/tag/intel-mediasdk-20.1.1)
-* [OpenVINO™ 2020.3](https://software.intel.com/en-us/openvino-toolkit)
+* [MediaSDK 20.3.0](https://github.com/Intel-Media-SDK/MediaSDK/releases/tag/intel-mediasdk-20.3.0)
+* [OpenVINO™ 2021.1](https://software.intel.com/en-us/openvino-toolkit)
 
 **Hardware:** 
-* Intel® platforms supported by the MediaSDK 20.1.1 and OpenVINO 2020.3. 
+* Intel® platforms supported by the MediaSDK 20.3.0 and OpenVINO 2021.1. 
 * For Media SDK, the major platform dependency comes from the back-end media driver. https://github.com/intel/media-driver
 * For OpenVINO™, see details from here: https://software.intel.com/en-us/openvino-toolkit/documentation/system-requirements
 
 # How to build
 
 Run build_and_install.sh to install dependent software packages and build sample application video_e2e_sample. 
 
-Please refer to ”Installation Guide“ in [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.2.0.pdf) for details.
+Please refer to ”Installation Guide“ in [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.3.0.pdf) for details.
 
 ## Build steps
 
@@ -68,10 +68,10 @@ cd cva_sample
 ```
 This script will install the dependent software packages by running command "apt install". So it will ask for sudo password. Then it will download libva, libva-util, media-driver and MediaSDK source code and install these libraries. It might take 10 to 20 minutes depending on the network bandwidth.
 
-After the script finishing, the sample application video_e2e_sample can be found under ./bin. Please refer to "Run sample application" in [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.2.0.pdf) for details.
+After the script finishing, the sample application video_e2e_sample can be found under ./bin. Please refer to "Run sample application" in [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.3.0.pdf) for details.
 
 # Known limitations
 
-The sample application has been validated on Intel® platforms Skylake(i7-6770HQ), Coffee Lake(i7-8559U i7-8700) and Whiskey Lake(i7-8665UE).
+The sample application has been validated on Intel® platforms Skylake(i7-6770HQ), Coffee Lake(i7-8559U i7-8700), Whiskey Lake(i7-8665UE) and Tiger Lake U(i7-1185G7E, i5-1135G7E).
 
 

build_and_install.sh

@@ -1,14 +1,23 @@
 #!/bin/bash
 
 root_path=$PWD
+#download_method="git"
+download_method="wget"
 
 if [[ ! -d "${INTEL_OPENVINO_DIR}" ]];
 then echo "Please make sure openvino has been installed and enviroment variables has been set by "
     echo "source intel/openvino/bin/setupvars.sh"
     exit -1;
 fi
 
-./msdk_pre_install.py
+sudo apt install python
+
+if [[ $download_method == "git" ]]
+then
+python msdk_pre_install_internal.py | tee svet_download.log
+else
+python msdk_pre_install.py | tee svet_download.log
+fi
 
 git_projects="MediaSDK media-driver libva"
 for i in $git_projects;
@@ -21,6 +30,33 @@ do
     fi
 done
 
+#check md5sum if wget is used to download source code packages
+
+if [[ $download_method == "wget" ]]
+then
+    echo "Checking the md5sum of downloaded packages"
+    declare -A  media_packages_md5sum
+    media_packages_md5sum[intel-gmmlib-20.3.2.tar.gz]=583d9fbef52d880238629f2ced50a9be
+    media_packages_md5sum[intel-media-20.3.0.tar.gz]=9c9219c09447567254aec00ade5dd3b8
+    media_packages_md5sum[intel-mediasdk-20.3.0.tar.gz]=1fd2cb03d15b3c308e6a9bd1735d2845
+    media_packages_md5sum[libva.tar.gz]=fdc00aedad4a51e058403f5f24d9abac
+    media_packages_md5sum[libva-utils.tar.gz]=a22479f85f7693536c5756ad46192b32 
+
+    for c in ${!media_packages_md5sum[@]}; do
+        md5_res=`md5sum $c`
+        if [[ ${media_packages_md5sum[$c]} ==  ${md5_res%% *} ]]
+        then
+            echo "$c" "${media_packages_md5sum[$c]}" correct
+        else
+            echo "$c" "md5sum is wrong " ${md5_res%% *}  vs ${media_packages_md5sum[$c]}
+            echo "Pleaes remove libva/media-driver/MediaSDK re-run this script"
+            echo "If you still see md5sum not match, please refer to msdk_pre_install.py and download, uncomparess the package manually"
+            echo "Or you can modify line 5 in this scirpt and change the download_method to \"git\". Then re-run this script."
+            exit
+        fi
+    done
+fi
+
 cd "MediaSDK"
 patch -N --no-backup-if-mismatch -p1 < ../patches/0001-MSDK-Enable-AVC-decode-SFC-RGB4.patch
 patch -N --no-backup-if-mismatch -p1 < ../patches/0002-sample_common-Add-support-to-multiple-displays.patch
@@ -30,15 +66,21 @@ if [ $? != 0 ]; then
 fi
 
 cd ../
-cp -rf video_e2e_sample  MediaSDK/samples/
 
-./msdk_pre_install.py -b cfl
+if [[ $download_method == "git" ]]
+then
+python msdk_pre_install_internal.py -b cfl| tee svet_build.log
+else
+python msdk_pre_install.py -b cfl | tee svet_build.log
+fi
 
-if [[ -f MediaSDK/build/__bin/release/video_e2e_sample ]];
+#video_e2e_sample depends on libsample_common.a
+if [[ -f MediaSDK/build/__lib/release/libsample_common.a ]];
 then
     rm -f bin
     ln -s MediaSDK/build/__bin/release/ bin
 
+
     va_env_set=`grep "LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri" ~/.bashrc -c`
 
     if (( va_env_set == 0 )) 
@@ -56,10 +98,17 @@ then
 
     fi
 
-    if [[ $LD_LIBRARY_PATH != *"/opt/intel/mediasdk/lib"* ]]
+    if [[ -d "/opt/intel/mediasdk/lib64" ]]
+    then
+        echo "A old MediaSDK version is found under folder /opt/intel/mediasdk/lib64"
+        echo "Rename /opt/intel/mediasdk/lib64 as /opt/intel/mediasdk/oldlib64"
+        mv  /opt/intel/mediasdk/lib64 /opt/intel/mediasdk/oldlib64
+    fi
+
+    if [[ $LD_LIBRARY_PATH != *"/opt/intel/mediasdk/lib/"* ]]
     then
-        export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/mediasdk/lib"
-        echo 'export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/mediasdk/lib"' >> ~/.bashrc
+        export LD_LIBRARY_PATH="/opt/intel/mediasdk/lib/:$LD_LIBRARY_PATH"
+        echo 'export LD_LIBRARY_PATH="/opt/intel/mediasdk/lib/:$LD_LIBRARY_PATH"' >> ~/.bashrc
     fi
 
     if [[ -d $cl_cache_dir ]];
@@ -73,8 +122,34 @@ then
         echo "mkdir -p ~/cl_cache" >> ~/.bashrc
         echo "export cl_cache_dir=~/cl_cache" >> ~/.bashrc
     fi
-    echo "Sample application building has completed!"
-    echo "Please use ./bin/video_e2e_sample for testing"
+
+    #Start to build SVET sample application
+    #Copy MediaSDK sample headers and libraries that weren't installed by default
+    sudo mkdir -p /opt/intel/mediasdk/include/sample_common
+    sudo mkdir -p /opt/intel/mediasdk/include/vpp_plugin
+    sudo mkdir -p /opt/intel/mediasdk/include/rotate_cpu
+    sudo cp -rf MediaSDK/samples/sample_common/include/ /opt/intel/mediasdk/include/sample_common/
+    sudo cp -rf MediaSDK/samples/sample_plugins/vpp_plugin/include/ /opt/intel/mediasdk/include/vpp_plugin/
+    sudo cp -rf MediaSDK/samples/sample_plugins/rotate_cpu/include/ /opt/intel/mediasdk/include/rotate_cpu/
+    sudo cp -f MediaSDK/build/__lib/release/libsample_common.a  /opt/intel/mediasdk/lib/
+    sudo cp -f MediaSDK/build/__lib/release/libvpp_plugin.a /opt/intel/mediasdk/lib/
+
+    mkdir -p video_e2e_sample/build
+    cd video_e2e_sample/build
+    cmake ../ | tee /tmp/svet_build.log
+    cat /tmp/svet_build.log >> svet_build.log
+    make -j4 | tee /tmp/svet_build.log
+    cat /tmp/svet_build.log >> svet_build.log
+
+    cd $root_path
+    if [[ -f bin/video_e2e_sample ]];
+    then
+        echo "SVET sample application building has completed!"
+        echo "Please use ./bin/video_e2e_sample for testing"
+    else
+        echo "SVET sample application building failed!"
+        echo "Please create an IPS with SVET in title on premiersupport.intel.com and upload ./svet_build.log ./svet_download.log to IPS."
+    fi
 fi
 
 #Download openvino models

doc/CONTRIBUTING.md

@@ -14,8 +14,8 @@ Steps:
 
 ## Requirements
 
-Hardware Requirements: Coffee Lake or Whiskey Lake
-Software Requirements: Ubuntu 18.04, MediaSDK 19.4.0 and OpenVINO 2019 R3 
+Hardware Requirements: Coffee Lake, Sky Lake, Kaby Lake, Whiskey Lake or Tiger Lake
+Software Requirements: Ubuntu 18.04, MediaSDK 20.3.0 and OpenVINO 2021.1 
 
 ## How to test your changes
 
@@ -24,11 +24,13 @@ Software Requirements: Ubuntu 18.04, MediaSDK 19.4.0 and OpenVINO 2019 R3
 ```sh
 ./build_and_install.sh
 ```
-Apply the changes to MediaSDK/samples/video_e2e_sample/. Then build video_e2e_sample:
+Apply the changes to video_e2e_sample. Then build video_e2e_sample:
 ```sh
-cp msdk_build.sh MediaSDK
-cd MediaSDK
-./msdk_build.sh
+cd video_e2e_sample
+mkdir -p build
+cd build
+cmake ../
+make -j4
 ```
 
 ### 2. Run tests to make sure below tests can run without error 

doc/FAQ.md

@@ -46,3 +46,7 @@ face-detection-retail-0004, human-pose-estimation-0001, vehicle-attributes-recog
 
 ## Can I use other OpenVINO version rather than 2020.3 ?
 Yes, but you have to modify some code due to interfaces changing. And also you need to download the IR files and copy them to ./model manually. Please refer to script/download_and_copy_models.sh for how to download the IR files.
+
+## When run 4 channel decode plus inference and display on APL, the CPU occupy ratio is very high and fps is low
+Please refer to par file par_file/inference/n4_face_detection_rgbp.par. It uses option "-dc::rgbp" that make the SFC outputs RGB4 for display and VPP outputs RGBP for inference input. Then there is no need to use OpenCV for resizing and color conversion which consume much CPU time on APL.
+Note, "-dc::rgbp" only works with "-infer::fd". Will support more inference types in the future.

msdk_pre_install.py

@@ -127,7 +127,7 @@ def fnParseCommandline():
 
         print "Please input the sudo password to proceed\n"
         cmd ="sudo apt-get -y install git libssl-dev dh-autoreconf cmake libgl1-mesa-dev libpciaccess-dev build-essential curl imagemagick unzip yasm libjpeg-dev libavcodec-dev libavutil-dev libavformat-dev;"
-        cmd+="sudo apt-get -y install checkinstall pkg-config"
+        cmd+="sudo apt-get -y install coreutils checkinstall pkg-config opencl-clhpp-headers opencl-c-headers ocl-icd-opencl-dev"
         os.system(cmd)
 
         print ""
@@ -138,38 +138,38 @@ def fnParseCommandline():
         # Pull all the source code
         print "libva"
         if not os.path.exists("%s/libva"%(WORKING_DIR)):
-            cmd = "cd %s; rm -f libva.tar.gz; wget https://github.com/intel/libva/archive/2.7.1.tar.gz -O libva.tar.gz;"%(WORKING_DIR) 
-            cmd+= "tar zxf libva.tar.gz; mv libva-2.7.1 libva"
+            cmd = "cd %s; rm -f libva.tar.gz; wget https://github.com/intel/libva/archive/2.9.0.tar.gz -O libva.tar.gz;"%(WORKING_DIR) 
+            cmd+= "tar zxf libva.tar.gz --one-top-level=libva --strip-components 1"
             print cmd
             os.system(cmd);
 
         print "libva-utils"
         if not os.path.exists("%s/libva-utils"%(WORKING_DIR)):
             cmd = "cd %s;rm -f libva-utils.tar.gz;"%(WORKING_DIR)
-            cmd += "wget https://github.com/intel/libva-utils/archive/2.7.1.tar.gz -O libva-utils.tar.gz;"
-            cmd += "tar zxf libva-utils.tar.gz; mv libva-utils-2.7.1 libva-utils;"
+            cmd += "wget https://github.com/intel/libva-utils/archive/2.9.1.tar.gz -O libva-utils.tar.gz;"
+            cmd += "tar zxf libva-utils.tar.gz --one-top-level=libva-utils --strip-components 1;"
             print cmd
             os.system(cmd);
 
         print "media-driver"
         if not os.path.exists("%s/media-driver"%(WORKING_DIR)): 
-            cmd = "cd %s; rm -f intel-media-20.1.1.tar.gz;"%(WORKING_DIR)
-            cmd += "wget https://github.com/intel/media-driver/archive/intel-media-20.1.1.tar.gz; "
-            cmd += "tar zxf intel-media-20.1.1.tar.gz; mv media-driver-intel-media-20.1.1 media-driver"
+            cmd = "cd %s; rm -f intel-media-20.3.0.tar.gz;"%(WORKING_DIR)
+            cmd += "wget https://github.com/intel/media-driver/archive/intel-media-20.3.0.tar.gz; "
+            cmd += "tar zxf intel-media-20.3.0.tar.gz --one-top-level=media-driver --strip-components 1"
             print cmd
             os.system(cmd);
 
         print "gmmlib"
         if not os.path.exists("%s/gmmlib"%(WORKING_DIR)): 
-            cmd = "cd %s;rm -f intel-gmmlib-20.1.1.tar.gz; wget https://github.com/intel/gmmlib/archive/intel-gmmlib-20.1.1.tar.gz; "%(WORKING_DIR)
-            cmd += "tar zxf intel-gmmlib-20.1.1.tar.gz; mv gmmlib-intel-gmmlib-20.1.1 gmmlib"
+            cmd = "cd %s;rm -f intel-gmmlib-20.3.2.tar.gz; wget https://github.com/intel/gmmlib/archive/intel-gmmlib-20.3.2.tar.gz; "%(WORKING_DIR)
+            cmd += "tar zxf intel-gmmlib-20.3.2.tar.gz --one-top-level=gmmlib --strip-components 1"
             print cmd
             os.system(cmd);
 
         print "MediaSDK"
         if not os.path.exists("%s/MediaSDK"%(WORKING_DIR)): 
-            cmd = "cd %s; rm -f intel-mediasdk-20.1.1.tar.gz;  wget https://github.com/Intel-Media-SDK/MediaSDK/archive/intel-mediasdk-20.1.1.tar.gz; "%(WORKING_DIR)
-            cmd+= "tar zxf intel-mediasdk-20.1.1.tar.gz; mv MediaSDK-intel-mediasdk-20.1.1 MediaSDK"
+            cmd = "cd %s; rm -f intel-mediasdk-20.3.0.tar.gz;  wget https://github.com/Intel-Media-SDK/MediaSDK/archive/intel-mediasdk-20.3.0.tar.gz;"%(WORKING_DIR)
+            cmd+= "tar zxf intel-mediasdk-20.3.0.tar.gz --one-top-level=MediaSDK --strip-components 1"
             print cmd
             os.system(cmd);
 
@@ -181,7 +181,6 @@ def fnParseCommandline():
         print "************************************************************************"
 
         # Build and install libVA including the libVA utils for vainfo.
-        # libVA origin:fbf7138389f7d6adb6ca743d0ddf2dbc232895f6 (011118), libVA utils origin: 7b85ff442d99c233fb901a6fe3407d5308971645 (011118)
         cmd ="cd %s/libva; "%(WORKING_DIR)
         cmd+="./autogen.sh --prefix=/usr --libdir=/usr/lib/x86_64-linux-gnu; make -j4; sudo make install"
         print cmd