Install OpenCV in Ubuntu

What is OpenCV?

OpenCV (stands for open-source computer vision library) is the programming function's library primarily focused on real-time computer vision. It was originally integrated by Intel and later offered by Willow Garage then Itseez (later inherited by Intel). OpenCV is free and cross-platform for use upon the open-source Apache 2 License. It features acceleration of GPU for real-time implementations beginning with 2011.

What is Computer Vision?

Computer vision can be described as a process from which we can know the concept of the videos and images how they are stored and how we can retrieve and manipulate information from them. Computer vision is a base or mostly applied for Artificial Intelligence. It is playing a big role in robotics, self-driving cars, and photo correcting applications as well.

Computer vision is a discipline that describes how to understand, interrupt, and reconstruct a 3D view using its 2D images, regarding the structure properties available in the view. It deals with replicating and modeling human vision with computer hardware and software.

Computer vision can significantly overlap with the below-mentioned fields:

• Pattern Recognition: Pattern recognition describes various methods to classify several patterns.
• Image Processing: Image processing emphasis on image manipulation.
• Photogrammetry: Photogrammetry is concerned with getting correct measurements from images.

Functionalities of OpenCV

Some important functionalities of OpenCV are mentioned below:

• CUDA acceleration
• Clustering and machine learning (flann, ml)
• Computational photography (video, photo, superres)
• Stereo computer vision or geometry-based monocular (videostab, stitching, calib3D)
• Feature/object detection (nonfree, features2d, objdetect)
• Video/image I/O, display, processing (highgui, core, imgproc)

What is Image processing?

It is a method for performing a few operations on any image, to receive an enhanced image, or to extract a few useful information using it. The basic image processing definition can be, "Image processing is the manipulation and analysis of digitized image for improving its quality".

Digital image

The images may be categorized as a two-dimensional function f(x,y), in which x and y are plane (spatial) coordinates, and the fat amplitude for any co-ordinate pair is known as the grey or intensity level of an image at that particular point.

The images are nothing but the two-dimensional matrix, which is specified by a mathematical function f(x,y) at any particular point, is providing the value of a pixel of the image at that point, the value of that pixel specifies how bright that particular pixel is, and which color it will be.

Basically, image processing is signal processing where the input is known as an image and the outcome is known as characteristics or image according to need related to that image.

Image processing has the below three steps:

• Manipulating and analysis of the image
• Importing an image
• Output where the result could be altered report or image that is based on the image analysis

History of OpenCV

In 1999, OpenCV was officially launched and it was an initiative of Intel Research to advance applications of CPU-intensive, a series part of projects with 3D display walls and real-time ray tracing. The primary contributors to this project contained several optimization experts in the Performance Library Team and Intel Russia. In the starting time of OpenCV, the project goals were specified as:

1. The knowledge of dissemination vision by offering a basic infrastructure that all developers can create on, so that code will be more readily transferable and readable.
2. The research of advanced vision by offering not just open but optimized code as well for common vision infrastructure.
3. Advance vision-based technical applications by enabling performance-optimized and portable code available at no cost along with a license that didn't need code to be free for open itself.

In 2000, the first OpenCV alpha version was published to the public at the IEEE Conference on Computer Vision and Pattern Recognition, and 5 betas were published during 2001-2005. In 2006, the initial 1.0 version was published. The 1.1 version "pre-release" was published in October 2008. In October 2009, the second major OpenCV version was released.

OpenCV 2 contains major modifications to the interface of the C++, focusing on better implementations, new functions, more type-safe patterns, and easier for existing ones in contrast to performance (mainly on multi-core systems). Now, official versions appear every 6 months, and improvement is now implemented by the Russian team which is supported by technical corporations.

For OpenCV, the support was taken over via a non-profit foundation which is known as OpenCV.org in August 2012. This Foundation manages a user and developer site.

Intel sealed an agreement for acquiring Itseez which is a leading OpenCV developer in May 2016.

OpenCV disclosed and started a Kickstarter Campaign for OpenCV AI Kit which is a hardware module series and inclusions to OpenCV supporting AI.

Applications of OpenCV

The application areas of OpenCV include:

• Augmented reality
• Motion tracking
• Structure from motion (short for SFM)
• Stereopsis stereo vision
• Recognition and segmentation
• Object detection
• Motion understanding
• Mobile robotics
• Human-computer interaction (short for HCI)
• Gesture recognition
• Facial recognition system
• Egomotion estimation
• 3D and 2D feature toolkits

For supporting a few of the above-mentioned areas, Open CV has a statical library for machine learning that includes:

• Deep neural networks (short for DNN)
• Support vector machine (short for SVM)
• Random forest
• Artificial neural networks
• Naive Bayes classifier
• k-nearest neighbor algorithm
• Expectation-maximization algorithm
• Gradient boosting trees
• Decision tree learning
• Boosting

Programming language of OpenCV

OpenCV is specified in the C++ language and its main interface is inside this language, but it still has a less broad though the extensive older interface of C. Every new algorithm and development occurs in the C++ interface. There are several bindings in MATLAB/OCTAVE, Java, and Python. The API for the interfaces could be detected in the online documentation.

Many wrappers in various programming languages have been improved for encouraging adoption from a wider audience. JavaScript bindings for a specific OpenCV function's subset were published as OpenCV.js to be applied for web platforms in the 3.4 version.

OS support in OpenCV

OpenCV executes on many desktop OSes. Some of these are mentioned below:

• Windows
• OpenBSD
• NetBSD
• FreeBSD
• macOS
• Linux

OpenCV also runs on several mobile OSes which are mentioned below:

• BlackBerry 10
• Maemo
• iOS
• Android

The users can receive official versions from SourceForge or get the latest sources using GitHub. Also, OpenCV applies the concept of CMake.

Hardware acceleration

When the library detects the Integrated Performance Primitives of Intel on the computer, it will apply this proprietary routine for accelerating itself:

• An OpenCL-based interface of GPU has been in development since 2012 October, documentation for the 2.4.13.3 version could be found at the docs.opencv.orgwebsite.
• A CUDA-based interface of GPU has been in development since 2010 September.

Platforms of OpenCV

OpenCV was developed to be cross-platform. Hence, the library was specified in the C language, and it makes OpenCV compact to almost any economic system, through PowerPC Macs to several robotic dogs. Since the 2.0 version, OpenCV contains its transitional C interface and the new C++ one. For almost all parts, new algorithms of OpenCV are now written in C++.

For languages, including Java and Python, wrappers have also been designed to motivate adoption through a wider audience. OpenCV executes on both mobile (iOS, Maemo, Android) and desktop (OpenBSD, FreeBSD, MacOS, Android, Linux, Windows).

• Android: OpenCV was moved to the Android environment since 2010. It permits the use of the complete library power in the development of mobile applications.
• ARM: Almost every embedded device currently uses ARM architecture-based CPUs, such as Cortex-M and Cortex-A series. Usually, deep learning algorithms are trained in x86/x64-based servers along with robust Nvidia GPUs. Then, the interface is required to be implemented on low-power ARM chips.
• CUDA: A new module that offers GPU acceleration was included in OpenCV in 2010. The module, i.e., 'gpu', covers an important part of the functionality of the library and is still in development. It's implemented with CUDA and hence has advantages from the ecosystem of CUDA, such as libraries like NVIDIA Performance Primitives (or NPP). With the inclusion of CUDA OpenCV acceleration, developers can execute more sophisticated and accurate OpenCV algorithms on high-resolution in real-time images during the consumption of less power.
• iOS: The development team of OpenCV actively operated on adding continued iOS support in 2012. Full integration has existed since the 2.4.2 (2012) version.
• OpenCL: A new module offering the OpenCV algorithm's OpenCL accelerations was included in the library in 2011. It enabled the code based on OpenCL to take benefits of heterogeneous hardware, using the potential of integrated and discrete GPUs. The official OpenCV winMegaPack contains the ocl module since the 2.4.6 (2013) version.

License of OpenCV

OpenCV is licensed upon two distinct licenses depending on the library version.

The 4.5.0 and higher versions of OpenCV: The 4.5.0 and higher versions of OpenCV are licensed upon the Apache 2 license.

The 4.4.0 and lower versions of OpenCV: The 4.4.0 and lower versions of OpenCV, such as OpenCV 1.x, OpenCV 2.x, and OpenCV 3.x, are licensed upon the 3-clause BSD license.

Releases of OpenCV

• Version 4.8.0
  • Supports TFLite model, such as int8 quantized models.
  • Enabled build of DNN module with no Protobuf dependency.
  • FaceDetectorYN for better facial landmarks, accuracy, and performance support.
  • New ArUco code-based detection algorithm for QR code.
  • Bar code decoder and detector moved to the primary repository from Contrib.
• Version 4.7.0
  • Added backend support for CANN.
  • Accelerating convolution, for ARM CPU especially.
  • Convolution optimization of Winograd.
  • Neutral network-based Nanotrack v2 tracker.
  • Supports FFmpeg 5.x.
  • Supports video codecs (hardware accelerated) on NVIDIA platforms along with current Video Codec SDK (NVENCODEAPI and NVCUVID).
• Version 4.6.0
  • Supports Clang 15 and GCC 12.
  • Supports FFmpeg 5.0.
  • Deactivated processing of floating-point denormals.
  • In ONNX importer, modified layer names to properly support "output" entities.
  • Added backend support for TIM-VX NPU.
  • Added support for the sample of the audio speech recognition (C++).
• Version 4.5.5
  • Updated handling rules of SOVERSION.
  • Added tests to manage several test suites of ONNX conformance.
  • RISC-V platform's more optimization.
  • Upgraded protobuf (built-in) from 3.5.2-3.19.1.
  • Fixed several analysis problems for the 2021.4 version of OpenVINO.
  • Fixed many build warnings addressed after the OpenVINO update.
• Version 4.5.4
  • 8-bit quantization is available in the DNN Module.
  • Upgraded OpenCV bindings.
  • Supports speech recognition.
  • Optimization of OpenCV DNN.
  • Added support for SoftNMS implementation.
• Version 4.5.3
  • Supports MatMul in CUDA backend.
  • Batch reinstallation has been fixed.
  • Supports the 2021.4 LTS version of OpenVINO.
  • Active OpenCL kernels cache.
  • Improved Python bindings along with more options for G-API configuration.
• Version 4.5.2
  • Imported TensorFlow parsing error debugging.
  • Upgraded DetectionOutput and NMS processing.
  • Fixed Reshape, MatMul, and Div with constant.
  • Supports Exp, SAM, TanH, LeakyReLU, NormalizeL2, and Mish ONNX subgraph.
  • Supports the 2021.3 version of OpenVINO.
• Version 4.5.1
  • Optimization of the 1D pool and 1D convolution.
  • ReduceMean, Resize, Gather with two or more results, and Faster RCNN ONNX models have been fixed.
  • Supports INT32 ONNX tensors.
  • Supports the 2021.4 version of OpenVINO.
  • Preview supports HDDL.
• Version 4.5.0
  • Supports the 2021.1 version of OpenVINO.
  • Optimizations and files in DNN CUDA backend.
  • Supports Python bindings for basic G-API, but streaming and inference supports are in progress currently.
  • Addressed serialization API.
  • The backend support for OpenVINO NN has been shipped to the OpenVINO "Core" APIs.
  • It is now convenient to represent a graph using a dynamic number of I/O.
  • Supports multiple contexts of OpenCL.
• Version 4.4.0
  • The Scale-Invariant Feature Transfer (or SOFT) algorithm has been shipped to the primary repository.
  • Supports Resnet_backbone.
  • Supports EfficientDet models.
  • Add a sample for text recognition.
  • Supports the 2020.3 LTS/2020.4 versions of OpenVINO.
  • Supports NN Builder API, which is planned for deletion in the upcoming release.
• Version 4.3.0
  • Supports ONNX, including Slice with two or more inputs, Algebra over constants, Broadcasting, and LSTM.
  • Supports DarkNet, including scale_channels, swish, sigmoid, and grouped convolution.
  • Supports MobileNet-SSD version 3.
  • Supports custom laters via nGraph OpenVINO API.
  • Supports more operations within the default library, such as NV12toGray, WarpPerspective, and WarpAffine.
• Version 4.2.0
  • Improved GSoC project using CUDA backend.
  • Supports nGraph OpenVINO API (experimental/preview).
  • Added in-graph inference.
  • Increased execution model along with the support for streaming.
  • Supports Multi-threading
  • Supports SIMD intrinsics
• Version 4.1.2
  • Supports 2019R3.
  • Supports the latest IE Core API.
  • New mechanism for the management of custom laters.
  • Supports high-level, which addresses dnn:model class or group of task-specific classes, including dnn:Segmentation model, dnn:DetectionModel, and dnn: ClassificationModel. It also supports automatic post- and pre-processing for deep learning networks.
  • MSA SIMD implementation has been shared for MIPS platforms.
  • Optimized OpenCV.js.
• Version 4.1.1
  • Initial support for 3D convolution networks. There are Eltwise, Pooling, BatchNorm, and Convolution layers implemented.
  • Now, the 2018R5 release is a minimal supported release.
  • Supports asynchronous inference.
  • Visualization of deep learning network diagrams with dot files.
  • Several improvements for TensorFlow and ONNX importers.
• Version 4.1.0
  • Reduced the consumption of peak memory for a few models by up to 30%.
  • Improved and updated integration using Intel Inference Engine.
  • Now, Inference Engine 2018R3 is the earliest supported version.
  • Now, Myriad X is supported and certified.
  • Added automatic network reshaping addressed in IE IR format.
  • New networks are supported, such as SSDs, Faster-RCNNs, and Mask-RCNN, along with FPN SSD and dilated convolution from TensorFlow Object Detection API.
• Version 4.0.0
  • Several APIs have been deleted from OpenCV 1.x. Some affected modules are calib3d, imgcodecs, videoio, video, photo, and objdetect.
  • Persistence (loading and storing structured data from/to JSON, YAML, or XML) has been reimplemented completely in the C++ language and also missed the C API in the core module.
  • Now, OpenCV is a C++11 library and needs a C++11-complaint compiler. Also, remember that OpenCV 3.x can be created as a C++11 library.
  • Supports Mask-RCNN model.
  • Implemented ONNX parser.
  • Further upgraded DNN module performance when it is created with the support for Intel DLDT by using more layers through DLDT.
  • Supports shortcuts for almost all famous deep learning networks.
• Version 3.4.2
  • Added the DNN_TARGET_OPENCL_FP16 computational target for deep learning network's half-precision floating point arithmetic with OpenCL.
  • Added support for the Inference Engine backend of Intel to activate models on VPU and GPU devices.
  • Enabled import of the OpenVINO pre-trained networks of Intel from IR (intermediate representation).
  • Introduced support for custom layers, which let us specify unimplemented layers or revoke existing ones.
  • Introduced a new deep learning sample influenced by An Efficient and Accurate Scene Text Detector (or EAST).
• Version 3.4.1
  • Supports quantized TensorFlow networks.
  • Now, OpenCV can utilize the Intel DL inference engine as a backend of DNN acceleration.
  • Added acceleration of AVX-512 to the performance-critical kernels, including fully connected and convolution layers.
  • SSD-based models retrained and trained in TensorFlow Object Detection API can be conveniently imported by one python script invocation, creating a text graph representation.
  • pthreads backend performance of cv::parallel_for_() has been improved greatly on several core machines.
• Version 3.4
  • Added support for faster R-CNN and the associated example.
  • JavaScript bindings have been continued to facilitate the DNN module.
  • DNN has also been further increased for iGPU with OpenCL.
  • On-disk pre-compiled OpenCL kernel caching has been implemented finally.
  • It is now also possible to download and execute pre-compiled OpenCL kernels through T-API.
• Version 3.3.1
  • Added the lightweight Resnet-10, face detection sample, and SSD-based network.
  • The limited Darknet parser, sufficient to parse several models of YOLO, as well as various layers to add some YOLO object detection network variants, has been implemented.
  • Supports FP16 networks.
  • Many layers have newly been added to aid image colorization, text detection, and other networks.
  • OpenCV has been upgraded for PPC64 architecture.
  • The bioinspired module's OpenCL acceleration path has been improved.
  • Hardware-accelerated video decoding/encoding through MediaSDK is now also present on Windows.

Installing OpenCV in Ubuntu

As we explained above, OpenCV is a computer vision library that stands for Open-Source Computer Vision Library. In this abbreviation, open defines an open-source library along with bindings for Java, C++, and python and supports distinct types of operating systems such as Linux and Windows.

It can use GPU acceleration and multi-core processing for real-time operating operations. The library of OpenCV can be used for a number of applications including medical image analysis, 3D model extraction, face detection and recognition, surveillance video, motion tracking, and much more.

We will explain how we can install OpenCV in the 20.04 version of Ubuntu in this article.

Installing OpenCV with Ubuntu Repository

In the 20.04 version of Ubuntu, the library of OpenCV is available inside the official Ubuntu repository. First of all, we need to update our apt cache by entering the following command in the terminal window:

Then, we will install OpenCV by executing the following command in the terminal window:

This command will help us to install every necessary package on our system.

After finishing the installation process of OpenCV, we can also verify the installation process by importing the module, i.e., cv2, and print the installed OpenCV version by running the following command in the terminal window: