Using OpenCV APIs to capture video from a camera is convenient. However, OpenCV does not provide an API for listing all available devices. If you have multiple cameras connected to your PC, you have no idea how to choose the right one. To get device information on Windows, you need to invoke DirectShow APIs. In this post, I will share how to create a Python extension that lists camera devices for OpenCV-Python on Windows.

Bridging DirectShow APIs to OpenCV-Python

How to run DirectShow sample

To use DirectShow APIs, read the Microsoft’s tutorial – Selecting a Capture Device, which shares how to list video and audio devices in C++.

To run the sample, create an empty Win32 Console Application in Visual Studio: File > New > Project > Templates > Visual C++ > Win32.

Create main.cpp, and copy all codes snippets from the tutorial page to the C++ file.

Build and run the project.

How to wrap up DirectShow C++ code

Define Python module and change main() function to a Python method:

static PyObject *
getDeviceList(PyObject *self, PyObject *args)
{
	PyObject* pyList = NULL; 
	
	HRESULT hr = CoInitializeEx(NULL, COINIT_MULTITHREADED);
	if (SUCCEEDED(hr))
	{
		IEnumMoniker *pEnum;

		hr = EnumerateDevices(CLSID_VideoInputDeviceCategory, &pEnum);
		if (SUCCEEDED(hr))
		{
			pyList = DisplayDeviceInformation(pEnum);
			pEnum->Release();
		}
		CoUninitialize();
	}

    return pyList;
}

static PyMethodDef Methods[] =
{
    {"getDeviceList", getDeviceList, METH_VARARGS, NULL},
    {NULL, NULL, 0, NULL}
};

PyMODINIT_FUNC
initdevice(void)
{
     (void) Py_InitModule("device", Methods);
}

Create Python list to save device information:

PyObject* pyList = PyList_New(0);
Char *pValue = _com_util::ConvertBSTRToString(var.bstrVal);
HRESULT hr = PyList_Append(pyList, Py_BuildValue("s", pValue));

Note: you have to convert BSTR to string. To use ConvertBSTRToString, we need a header file #include <comutil.h>, and the corresponding library #pragma comment(lib, “comsuppw.lib”).

How to build and use Python extension

Create setup.py:

from distutils.core import setup, Extension

module_device = Extension('device',
                        sources = ['device.cpp'], 
                        library_dirs=['G:\Program Files\Microsoft SDKs\Windows\v6.1\Lib']
                      )

setup (name = 'WindowsDevices',
        version = '1.0',
        description = 'Get device list with DirectShow',
        ext_modules = [module_device])

Build and install the extension:

python setup.py build install

If you see the error ‘Unable to find vcvarsall.bat’, set Visual Studio environment:

• Visual Studio 2010 (VS10): SET VS90COMNTOOLS=%VS100COMNTOOLS%
• Visual Studio 2012 (VS11): SET VS90COMNTOOLS=%VS110COMNTOOLS%
• Visual Studio 2013 (VS12): SET VS90COMNTOOLS=%VS120COMNTOOLS%
• Visual Studio 2015 (VS14): SET VS90COMNTOOLS=%VS140COMNTOOLS%

Create test.py to use the extension for OpenCV-Python:

import device
import cv2

def select_camera(last_index):
    number = 0
    hint = "Select a camera (0 to " + str(last_index) + "): "
    try:
        number = int(input(hint))
        # select = int(select)
    except Exception ,e:
        print("It's not a number!")
        return select_camera(last_index)

    if number > last_index:
        print("Invalid number! Retry!")
        return select_camera(last_index)

    return number


def open_camera(index):
    cap = cv2.VideoCapture(index)
    return cap

def main():
    # print OpenCV version
    print("OpenCV version: " + cv2.__version__)

    # Get camera list
    device_list = device.getDeviceList()
    index = 0

    for name in device_list:
        print(str(index) + ': ' + name)
        index += 1

    last_index = index - 1

    if last_index < 0:
        print("No device is connected")
        return

    # Select a camera
    camera_number = select_camera(last_index)
    
    # Open camera
    cap = open_camera(camera_number)

    if cap.isOpened():
        width = cap.get(3) # Frame Width
        height = cap.get(4) # Frame Height
        print('Default width: ' + str(width) + ', height: ' + str(height))

        while True:
            
            ret, frame = cap.read();
            cv2.imshow("frame", frame)

            # key: 'ESC'
            key = cv2.waitKey(20)
            if key == 27:
                break

        cap.release() 
        cv2.destroyAllWindows() 

if __name__ == "__main__":
    main()

Source Code

https://github.com/yushulx/python-capture-device-list

The post Listing Multiple Cameras for OpenCV-Python on Windows appeared first on Code Pool.

Recently, I was inspired by a blog post “Python Live Video Streaming Example” and thinking whether it is possible to save the camera streaming to a video file. Based on the example code, I managed to figure out a solution. In this post, I want to share the process of building the web camera recorder using OpenCV and Flask.

How to Use OpenCV to Record a Video

Let’s start with the code snippet posted on OpenCV website:

import numpy as np
import cv2

cap = cv2.VideoCapture(0)

# Define the codec and create VideoWriter object
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('output.avi',fourcc, 20.0, (640,480))

while(cap.isOpened()):

    ret, frame = cap.read()
    if ret==True:
        frame = cv2.flip(frame,0)
        # write the flipped frame
        out.write(frame)
        cv2.imshow('frame',frame)

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    else:
        break

# Release everything if job is finished
cap.release()
out.release()
cv2.destroyAllWindows()

After running the code on Windows, I got a 0KB file. The reason is the codec does not exist in my Windows 10. Replace fourcc with -1 to check the available codec list:

out = cv2.VideoWriter('output.avi', -1, 20.0, (640,480))

Instead of XVID, using MJPG will work.

fourcc = cv2.VideoWriter_fourcc(*'MJPG')

MJPG codec results in high size video. To get a smaller size, we need to install X264, which is not in the codec list by default.

Change codec to X264:

fourcc = cv2.VideoWriter_fourcc(*'X264')

Once you run the app, an annoying log window will pop up:

I have found the solution here. Open Windows registry and set log_level value 0.

How to Build Camera Recorder in Web Browser

The source code of video_streaming_with_flask_example is handy.

def get_frame(self):
        success, image = self.video.read()
        # We are using Motion JPEG, but OpenCV defaults to capture raw images,
        # so we must encode it into JPEG in order to correctly display the
        # video stream.
        ret, jpeg = cv2.imencode('.jpg', image)
        return jpeg.tobytes()

In my case, I have to save camera instance globally for video recording.

def video_stream():
    global video_camera 
    global global_frame

    if video_camera == None:
        video_camera = VideoCamera()
        
    while True:
        frame = video_camera.get_frame()

        if frame != None:
            global_frame = frame
            yield (b'--frame\r\n'
                    b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')
        else:
            yield (b'--frame\r\n'
                            b'Content-Type: image/jpeg\r\n\r\n' + global_frame + b'\r\n\r\n')

Use XMLHttpRequest to start and stop video recording event.

Client:

    var xhr = new XMLHttpRequest();
    xhr.onreadystatechange = function() {
        if (xhr.readyState == 4 && xhr.status == 200) {
            // alert(xhr.responseText);
        }
    }
    xhr.open("POST", "/record_status");
    xhr.setRequestHeader("Content-Type", "application/json;charset=UTF-8");
    xhr.send(JSON.stringify({ status: "true" }));

Server:

@app.route('/record_status', methods=['POST'])
def record_status():
    global video_camera 
    if video_camera == None:
        video_camera = VideoCamera()

    json = request.get_json()

    status = json['status']

    if status == "true":
        video_camera.start_record()
        return jsonify(result="started")
    else:
        video_camera.stop_record()
        return jsonify(result="stopped")

Every time the event is triggered, create a new thread to save camera stream to a video file.

class RecordingThread (threading.Thread):
    def __init__(self, name, camera):
        threading.Thread.__init__(self)
        self.name = name
        self.isRunning = True

        self.cap = camera
        fourcc = cv2.VideoWriter_fourcc(*'MJPG')
        self.out = cv2.VideoWriter('./static/video.avi',fourcc, 20.0, (640,480))

    def run(self):
        while self.isRunning:
            ret, frame = self.cap.read()
            if ret:
                self.out.write(frame)

        self.out.release()

    def stop(self):
        self.isRunning = False

    def __del__(self):
        self.out.release()

Run the app:

python server.py

Note: if you are using Python 2, you will see the socket connection issue:

To get rid of the exception, enable threaded mode:

if __name__ == '__main__':
    app.run(host='0.0.0.0', threaded=True)

Or use Python 3 to run the app.

Source Code

https://github.com/yushulx/web-camera-recorder

The post How to Build Web Camera Recorder Using OpenCV and Flask appeared first on Code Pool.

Many excellent document mobile apps support not only image capture, but also edge detection and perspective transformation. If you are interested in these computer vision technologies, you can use OpenCV to create a free document scanner app yourself. In this post, I want to share how to use OpenCV-Python to create a web document scanner step by step.

Setting Up Environment

Download Python 3.5.

Install Flask:

pip3 install flask

Install OpenCV 3.3.0 for Python:

pip3 install opencv-python

Download the latest NumPy 1.11.2. Unzip the package and build it:

python3 setup.py build install

To compile NumPy source code on Windows 10, install Microsoft Visual C++ Compiler for Python 2.7.

Web Document Scanner

Article and Code References

• https://github.com/vipul-sharma20/document-scanner
• http://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/
• http://docs.opencv.org/3.1.0/dd/d49/tutorial_py_contour_features.html
• https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm
• http://www.pyimagesearch.com/2015/04/06/zero-parameter-automatic-canny-edge-detection-with-python-and-opencv/

Steps of Building the App

Create document.py to do edge detection and perspective transformation:

import cv2
import rect
import numpy as np

class Scanner(object):
    # http://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/
    def four_point_transform(self, image, rect):
    	# obtain a consistent order of the points and unpack them
        # individually
        (tl, tr, br, bl) = rect
    
        # compute the width of the new image, which will be the
        # maximum distance between bottom-right and bottom-left
        # x-coordiates or the top-right and top-left x-coordinates
        widthA = np.sqrt(((br[0] - bl[0]) ** 2) + ((br[1] - bl[1]) ** 2))
        widthB = np.sqrt(((tr[0] - tl[0]) ** 2) + ((tr[1] - tl[1]) ** 2))
        maxWidth = max(int(widthA), int(widthB))
    
        # compute the height of the new image, which will be the
        # maximum distance between the top-right and bottom-right
        # y-coordinates or the top-left and bottom-left y-coordinates
        heightA = np.sqrt(((tr[0] - br[0]) ** 2) + ((tr[1] - br[1]) ** 2))
        heightB = np.sqrt(((tl[0] - bl[0]) ** 2) + ((tl[1] - bl[1]) ** 2))
        maxHeight = max(int(heightA), int(heightB))
    
        # now that we have the dimensions of the new image, construct
        # the set of destination points to obtain a "birds eye view",
        # (i.e. top-down view) of the image, again specifying points
        # in the top-left, top-right, bottom-right, and bottom-left
        # order
        dst = np.array([
            [0, 0],
            [maxWidth - 1, 0],
            [maxWidth - 1, maxHeight - 1],
            [0, maxHeight - 1]], dtype = "float32")
    
        # compute the perspective transform matrix and then apply it
        M = cv2.getPerspectiveTransform(rect, dst)
        warped = cv2.warpPerspective(image, M, (maxWidth, maxHeight))
    
        # return the warped image
        return warped

    # https://github.com/vipul-sharma20/document-scanner
    def detect_edge(self, image, enabled_transform = False):
        dst = None
        orig = image.copy()

        gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
        blurred = cv2.GaussianBlur(gray, (5, 5), 0)
        edged = cv2.Canny(blurred, 0, 20)
        _, contours, _ = cv2.findContours(edged, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)

        contours = sorted(contours, key=cv2.contourArea, reverse=True)

        for cnt in contours:
            epsilon = 0.051 * cv2.arcLength(cnt, True)
            approx = cv2.approxPolyDP(cnt, epsilon, True)

            if len(approx) == 4:
                target = approx
                cv2.drawContours(image, [target], -1, (0, 255, 0), 2)

                if enabled_transform:
                    approx = rect.rectify(target)
                    # pts2 = np.float32([[0,0],[800,0],[800,800],[0,800]])
                    # M = cv2.getPerspectiveTransform(approx,pts2)
                    # dst = cv2.warpPerspective(orig,M,(800,800))
                    dst = self.four_point_transform(orig, approx)
                break

        return image, dst

Create camera.py to capture frames from a camera:

import cv2
from document import Scanner

class VideoCamera(object):
    def __init__(self):
        # Open a camera
        self.cap = cv2.VideoCapture(2)
      
        # Initialize video recording environment
        self.is_record = False
        self.out = None
        self.transformed_frame = None

        self.scanner = Scanner()
        self.cached_frame = None
    
    def __del__(self):
        self.cap.release()

    def get_video_frame(self):
        ret, frame = self.cap.read()
        if ret:
            frame, _ = self.scanner.detect_edge(frame)
            self.cached_frame = frame
            ret, jpeg = cv2.imencode('.jpg', frame)
            return jpeg.tobytes()
        else:
            return None

    def capture_frame(self):
        ret, frame = self.cap.read()
        if ret:
            _, frame = self.scanner.detect_edge(frame, True)
            ret, jpeg = cv2.imencode('.jpg', frame)
            self.transformed_frame = jpeg.tobytes()
        else:
            return None

    def get_cached_frame(self):
        return self.cached_frame

    def get_image_frame(self):
        return self.transformed_frame

Note: if you have only one device connected, the parameter in cv2.VideoCapture() should be 0.

Create server.py to stream camera frames to your web client:

from flask import Flask, render_template, Response, jsonify, request
from camera import VideoCamera

app = Flask(__name__)

video_camera = None

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/capture_status', methods=['POST'])
def capture_status():
    global video_camera 

    if video_camera == None:
        video_camera = VideoCamera()

    json = request.get_json()

    status = json['status']

    if status == "true":
        video_camera.capture_frame()
        return jsonify(result="done")

def video_frame():
    global video_camera 

    if video_camera == None:
        video_camera = VideoCamera()
        
    while True:
        frame = video_camera.get_video_frame()

        if frame is not None:
            yield (b'--frame\r\n'
                    b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')
        else:
            yield (b'--frame\r\n'
                    b'Content-Type: image/jpeg\r\n\r\n' + video_camera.get_cached_frame() + b'\r\n\r\n')

def image_frame():
    global video_camera 

    if video_camera == None:
        video_camera = VideoCamera()
        
    frame = video_camera.get_image_frame()

    if frame is not None:
        yield (b'--frame\r\n'
                b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')

@app.route('/video_viewer')
def video_viewer():
    return Response(video_frame(),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

@app.route('/image_viewer')
def image_viewer():
    return Response(image_frame(),
                        mimetype='multipart/x-mixed-replace; boundary=frame')

if __name__ == '__main__':
    app.run(host='0.0.0.0', threaded=True)

Run the app:

python server.py

Source Code

https://github.com/yushulx/web-document-scanner

The post How to Build Web Document Scanner Using OpenCV-Python appeared first on Code Pool.

Document Scanning is nothing new for mobile apps. I remember the first time that I installed a document scanning app was in 2010. However, Apple, Microsoft, and Google the giant tech companies got started to recognize the importance of document scanning technology recently. It is good to see the function appears in iOS Notes, Office Lens and Google Drive. Why is the document scanning being hot now? I think the reason is the quality of images captured by mobile cameras is getting better and better. More and more users tend to scan documents with cameras instead of traditional scanners. If you are not familiar with Android and iOS development, you can use HTML5. In this post, I will share how to create a simple web document management app using OpenCV.js.

How to Use FileSystem APIs to Write, Read and Remove Images

While creating a web client-side document management app, if you keep all image data in memory, you will see the “out of memory” warning soon. To avoid the issue, we can write memory data to cache. There are several storage options available in HTML5, such as Local Storage, Session Storage, IndexedDB, Web SQL, and Cookies. Due to the size limitation, they are not suitable for my scenario. My use case is to load and edit images that are large binary blobs in the web browser. It is why I need to use FileSystem APIs which aims to deal with data outside of the context of the browser.

Load an image to memory

Create an input element and set “file” as the type. The following code can change the button text:

<input type="button" id="load" value="Load image" onclick="document.getElementById('upload').click();" />
<input type="file" id="upload" onchange="loadImage();" style="display:none;" />

Comparing to desktop platforms, the “file” type running on mobile platforms trigger not only a file selection window but also the default camera app.

Use FileReader to load an image into an array buffer and wrap it as a blob.

fileReader.onload = function (event) {
    if (!cacheManager.hasStorage(event.target.result.byteLength)) {
      alert('Image storage is full. Please clear some images to get storage quota.');
      return;
    }
    let size = event.target.result.byteLength / 1024;
    log('size = ' + size + 'K');

    var arrayBufferView = new Uint8Array(this.result);
    var blob = new Blob([arrayBufferView], {
      type: "image/png"
    });

    renderBlobImage(blob, 'canvas');

  };
  fileReader.readAsArrayBuffer(file);

Draw the image on a canvas:

function renderBlobImage(blob, canvasID) {
  var urlCreator = window.URL || window.webkitURL;

  let canvas = document.getElementById(canvasID);
  let ctx = canvas.getContext('2d');
  var image = new Image();
  image.onload = function () {
    ctx.clearRect(0, 0, canvas.width, canvas.height);
    var canvasWidth = 600;
    var canvasHeight = 400;

    var scaleFactor = Math.min((canvasWidth / image.width), (canvasHeight / image.height));
    canvas.width = image.width * scaleFactor;
    canvas.height = image.height * scaleFactor;
    ctx.drawImage(image, 0, 0, image.width * scaleFactor, image.height * scaleFactor);
  }
  image.src = urlCreator.createObjectURL(blob);
}

Write a file

Here is the code for writing a file:

CacheManager.prototype.writeFile = function (
    fileName, blob, successCallback, errorCallback) {

    function onInitFs(fs) {
      fs.root.getFile(fileName, {
        create: true
      }, function (fileEntry) {
        fileEntry.createWriter(function (fileWriter) {

          fileWriter.truncate(0);

        }, errorHandler);

        fileEntry.createWriter(function (fileWriter) {

          fileWriter.onwriteend = function (e) {
          
            console.log('Write completed.');
            successCallback();
          };

          fileWriter.onerror = function (e) {
            console.log('Write failed: ' + e.toString());
            errorCallback();
          };

          fileWriter.write(blob);

        }, errorHandler);

      }, errorHandler);
    }

    window.requestFileSystem(
      window.TEMPORARY, this.storageSize, onInitFs, errorHandler);
  };

Why do we need to use createWriter() twice? The first createWriter() is used to clear the existing data.

We can compare the storage usage before and after:

Before

After

Read a file

CacheManager.prototype.readFile = function (fileName, callback) {
    function onInitFs(fs) {
      fs.root.getFile(
        fileName, {},
        function (fileEntry) {

          // Get a File object representing the file,
          // then use FileReader to read its contents.
          fileEntry.file(callback, errorHandler);

        },
        errorHandler);
    }

    window.requestFileSystem(
      window.TEMPORARY, this.storageSize, onInitFs, errorHandler);
  };

Remove a file

CacheManager.prototype.removeFile = function (fileName) {
    function onInitFs(fs) {
        fs.root.getFile(fileName, {
          create: false
        }, function (fileEntry) {

          fileEntry.remove(function () {
            console.log('File removed.');
          }, errorHandler);

        }, errorHandler);
      }

      window.requestFileSystem(
        window.TEMPORARY, this.storageSize, onInitFs, errorHandler);

    
  };

How to Use OpenCV.js

OpenCV.js is a JavaScript binding that exposes OpenCV library to the web.

To use OpenCV.js, you just need to include cv.js:

<script async src="cv.js"></script>

When opening your web page, OpenCV.js will load the cv.data file via XMLHttpRequest. Here is the code snippet that I extracted from the complex sample code for monitoring the download status:

var Module = {
  setStatus: function (text) {
    if (!Module.setStatus.last) Module.setStatus.last = {
      time: Date.now(),
      text: ''
    };
    if (text === Module.setStatus.text) return;
    var m = text.match(/([^(]+)\((\d+(\.\d+)?)\/(\d+)\)/);
    var now = Date.now();
    if (m && now - Date.now() < 30) return; // if this is a progress update, skip it if too soon
    if (m) {
      text = m[1];

    }
    if (text === '') {
      isOpenCVReady = true;
      console.log('OpenCV is ready');
      // onPreprocess();
    }

  },
  totalDependencies: 0,
  monitorRunDependencies: function (left) {
    this.totalDependencies = Math.max(this.totalDependencies, left);
    Module.setStatus(left ? 'Preparing... (' + (this.totalDependencies - left) + '/' + this.totalDependencies + ')' : 'All downloads complete.');
  }
};
Module.setStatus('Downloading...');

Once OpenCV context is ready, we can call any OpenCV methods in JavaScript. However, there seems to be a bug:

I debugged line by line and finally focused on the following code:

cv.convexHull(contours.get(0), item, false, true);

As long as I run the snippet before calling other APIs, I can get the expected result.

The full code for skipping the bug:

function onPreprocess() {

  var canvas = document.createElement('canvas');
  var ctx = canvas.getContext('2d');
  var imgData = ctx.getImageData(0, 0, canvas.width, canvas.height);
  var src = cv.matFromArray(imgData, cv.CV_8UC4);
  var canny_output = new cv.Mat();
  var blurred = new cv.Mat();
  var cthresh = 75;
  cv.blur(src, blurred, [5, 5], [-1, -1], cv.BORDER_DEFAULT);
  cv.Canny(blurred, canny_output, cthresh, cthresh * 2, 3, 0);

  var contours = new cv.MatVector();
  var hierarchy = new cv.Mat();
  cv.findContours(canny_output, contours, hierarchy, 3, 2, [0, 0]);

  var item = new cv.Mat();
  // For preprocessing. Bug?
  cv.convexHull(contours.get(0), item, false, true);
  item.delete();

  src.delete();
  blurred.delete();
  contours.delete();
  hierarchy.delete();
  canny_output.delete();

}

Now, let’s play the web document management app:

Source Code

https://github.com/yushulx/javascript-image-storage

The post A Simple Web Document Management App with OpenCV.js appeared first on Code Pool.

OpenCV is written in C++.  If you install OpenCV library on Windows, you will see OpenCV officially provides wrappers for Python and Java but not for C#. Fortunately, there are many .NET open source projects for wrapping OpenCV C++ APIs, and thus we don’t need to write a wrapper from scratch. In this post, I will share how to use OpenCV library and Dynamsoft Barcode Reader SDK to create a .NET barcode reader app on Windows.

Prerequisites

• Visual Studio 2015
• OpenCvSharp
• Dynamsoft Barcode Reader

.NET Barcode Reader with OpenCV

I tried difference open source projects and finally decided to use OpenCvSharp which is continuously maintained by open source community.

Install OpenCvSharp and Dynamsoft Barcode Reader via Package Manager in Visual Studio. Tools >  NuGet Package Manager > Package Manager Console:

PM > Install-Package OpenCvSharp3-AnyCPU
PM> Install-Package Dynamsoft.DotNet.Barcode

Create a video capture object:

VideoCapture capture = new VideoCapture(0);

Get a video frame:

Mat image = capture.RetrieveMat();

Render the frame in a Window:

Cv2.ImShow("video", image);

Break the infinite loop when pressing ‘ESC’ key:

int key = Cv2.WaitKey(20);
// 'ESC'
if (key == 27)
{
    break;
}

Create a barcode reader object:

BarcodeReader reader = new BarcodeReader("t0068MgAAALLyUZ5pborJ8XVc3efbf4XdSvDAVUonA4Z3/FiYqz1MOHaUJD3d/uBmEtXVCn9fw9WIlNw6sRT/DepkdvVW4fs=");

To recognize barcodes, you can use DecodeBuffer() function. However, the type of the first parameter is byte[]. Now the biggest problem is how we can make the function work with Mat type.

Get the data pointer, width, height and element size as follows:

IntPtr data = image.Data;
int width = image.Width;
int height = image.Height;
int elemSize = image.ElemSize();
int buffer_size = width * height * elemSize;

Copy the data to a byte array:

using System.Runtime.InteropServices;
byte[] buffer = new byte[buffer_size];
Marshal.Copy(data, buffer, 0, buffer_size);

Decode the buffer and return barcode results:

BarcodeResult[] results = reader.DecodeBuffer(buffer, width, height, width * elemSize, ImagePixelFormat.IPF_RGB_888);
if (results != null)
{
    Console.WriteLine("Total result count: " + results.Length);
    foreach (BarcodeResult result in results)
    {
        Console.WriteLine(result.BarcodeText);
    }
}

Build and run the program:

API References

• https://shimat.github.io/opencvsharp_docs/index.html
• https://www.dynamsoft.com/help/Barcode-Reader/index.html

Source Code

https://github.com/dynamsoft-dbr/opencv-dotnet

The post Building .NET Barcode Reader with OpenCV and DBR 5.2 appeared first on Code Pool.

A dot code is a 2D barcode symbology composed of disconnected dots. It is widely used in the tobacco industry. Recently, Dynamsoft rolled out barcode reader SDK v7.4, which added DotCode support. In this post, I will share a command-line app and a GUI app, demonstrating how to build Java DotCode reader on Windows 10.

Decoding DotCode in Java Command-Line App

The command-line app is simple. What I am going to do is to invoke an API to decode DotCode from an image file.

Configure Dynamsoft Barcode Reader in pom.xml:

  <repositories>
    <repository>
      <id>dbr</id>
      <url>https://download2.dynamsoft.com/maven/dbr/jar</url>
    </repository>
  </repositories>
  <dependencies>
    <dependency>
      <groupId>com.dynamsoft</groupId>
      <artifactId>dbr</artifactId>
      <version>7.4.0</version>
    </dependency>
  </dependencies>

To simply the running command, we can add a plugin to assemble all dependencies into one jar file:

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <configuration>
          <descriptorRefs>
            <descriptorRef>jar-with-dependencies</descriptorRef>
          </descriptorRefs>
        </configuration>
      </plugin>
    </plugins>
  </build>

Instantiate Dynamsoft Barcode Reader and set DotCode as the target barcode:

        BarcodeReader br = null;
        try {
            br = new BarcodeReader("LICENSE-KEY");
            br.initRuntimeSettingsWithString("{\"ImageParameter\":{\"Name\":\"BestCoverage\",\"DeblurLevel\":9,\"ExpectedBarcodesCount\":512,\"ScaleDownThreshold\":100000,\"LocalizationModes\":[{\"Mode\":\"LM_CONNECTED_BLOCKS\"},{\"Mode\":\"LM_SCAN_DIRECTLY\"},{\"Mode\":\"LM_STATISTICS\"},{\"Mode\":\"LM_LINES\"},{\"Mode\":\"LM_STATISTICS_MARKS\"}],\"GrayscaleTransformationModes\":[{\"Mode\":\"GTM_ORIGINAL\"},{\"Mode\":\"GTM_INVERTED\"}]}}", EnumConflictMode.CM_OVERWRITE);
            PublicRuntimeSettings runtimeSettings = br.getRuntimeSettings();
            runtimeSettings.barcodeFormatIds_2 = EnumBarcodeFormat_2.BF2_DOTCODE;
            br.updateRuntimeSettings(runtimeSettings);
        } catch (Exception e) {
            System.out.println(e);
            return;
        }

By default, the SDK will decode all supported barcode formats. If you want to focus on DotCode and speed up the decoding process, disable other 1D and 2D barcode formats:

runtimeSettings.barcodeFormatIds = EnumBarcodeFormat.BF_NULL;

Pass the file path to the decodeFile() function which will soon return the barcode results:

        TextResult[] results = null;
        try {
            results = br.decodeFile(filename, "");
        } catch (Exception e) {
            System.out.println("decode buffered image: " + e);
        }

Build and run the project:

mvn clean install assembly:assembly -Dmaven.test.skip=true

java -cp target/command-line-dotcode-1.0-SNAPSHOT-jar-with-dependencies.jar com.java.barcode.App test.png

A Webcam DotCode Reader Built with OpenCV Java and Java Swing

A GUI app needs more effort.

How to install OpenCV Java?

The OpenCV Java SDK contains a jar package and a shared library. For Windows users, you just need to install the pre-built package and find the OpenCV Java SDK at opencv-4.3\opencv\build\java. Since I’m using a Maven project, I have to install the jar file to the Maven local repository beforehand:

mvn install:install-file -Dfile=opencv-430.jar -DgroupId=org -DartifactId=opencv -Dversion=4.3.0 -Dpackaging=jar

Thereafter, add the configuration to pom.xml:

    <dependency>
      <groupId>org</groupId>
      <artifactId>opencv</artifactId>
      <version>4.3.0</version>
    </dependency>

What about the dll file? If you don’t put the dll in the right place, you will get the error log as loading the library:

Exception in thread "main" java.lang.UnsatisfiedLinkError: no opencv_java430 in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
        at java.lang.Runtime.loadLibrary0(Runtime.java:870)
        at java.lang.System.loadLibrary(System.java:1122)
        at com.java.barcode.App.main(App.java:65)

Here are the workarounds for the issue.

Check the available java library path. Copy the dll file to current working directory or add the dll path to system environment PATH:

System.out.println(System.getProperty("java.library.path"));

Use the full path to load the library:

System.load("D:/opencv-4.3/opencv/build/java/x64/opencv_java430.dll");

Define the Java library path when running your Java app:

java -Djava.library.path=<dll path> -cp target/opencv-dotcode-1.0-SNAPSHOT-jar-with-dependencies.jar com.java.barcode.App

A “hello world” program using OpenCV Java

Once OpenCV is ready, you can test the library as follows:

System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
Mat mat = Mat.eye(3, 3, CvType.CV_8UC1);
System.out.println("mat = " + mat.dump());

Display Webcam video stream in Java Swing component

Inspired by the OpenCV Java docs that guide JavaFX programming, I studied the OpenCV capture logic and rendered the video frames in a JLable:

public void updateViewer(final BufferedImage image) {
        if (!SwingUtilities.isEventDispatchThread()) {
            SwingUtilities.invokeLater(new Runnable() {
                @Override
                public void run() {
                    mImage.setIcon(new ImageIcon(image));
                }
            });
            return;
        }
    }

Runnable frameGrabber = new Runnable() {

                    @Override
                    public void run() {
                        Mat frame = grabFrame();
                        byte[] data = Utils.matToByteArray(frame);

                        if (!status.get()) {
                            status.set(true);
                            barcodeTimer.schedule(new BarcodeRunnable(frame, mBarcodeReader, callback, status), 0, TimeUnit.MILLISECONDS);
                        }
                    
                        BufferedImage bufferedImage = Utils.byteToBufferedImage(data, frame.width(), frame.height(), frame.channels());
                        if (isRunning) updateViewer(bufferedImage);
                    }
                };
this.timer = Executors.newSingleThreadScheduledExecutor();
this.timer.scheduleAtFixedRate(frameGrabber, 0, 33, TimeUnit.MILLISECONDS);

Read DotCode and show results

In the code above, you can see I created a barcode timer instance for reading DotCode:

barcodeTimer = Executors.newSingleThreadScheduledExecutor();

We have to run the barcode decoding API in a worker thread in order to avoid decreasing the frame rate.

To draw the DotCode position, create a class CustomJLable that extends JLable:

    private ArrayList<Point[]> data = new ArrayList<>();

    @Override
    protected void paintComponent(Graphics g) {
        super.paintComponent(g);
        Graphics2D g2d = (Graphics2D) g.create();
        if (data.size() > 0) {
            g2d.setColor(Color.RED);
            for (Point[] points : data) {
                for (int i = 0; i < points.length; ++i) {
                    if (i == 3) {
                        g2d.drawLine(points[i].x, points[i].y, points[0].x, points[0].y);
                    } else {
                        g2d.drawLine(points[i].x, points[i].y, points[i + 1].x, points[i + 1].y);
                    }
                }
            }

        }
        g2d.dispose();
    }

    public void appendPoints(Point[] points) {
        data.add(points);
    }

    public void clearPoints() {
        data.clear();
    }

Build and run the GUI DotCode reader:

mvn clean install assembly:assembly -Dmaven.test.skip=true
java -cp target/opencv-dotcode-1.0-SNAPSHOT-jar-with-dependencies.jar com.java.barcode.App

Q&A

How to convert OpenCV Mat to Java byte array?

    public static byte[] matToByteArray(Mat original)
    {
        int width = original.width(), height = original.height(), channels = original.channels();
        byte[] sourcePixels = new byte[width * height * channels];
        original.get(0, 0, sourcePixels);
        return sourcePixels;
    }

How to convert Java byte array to Java BufferedImage?

    public static BufferedImage byteToBufferedImage(byte[] sourcePixels, int width, int height, int channels)
    {
        BufferedImage image = null;
        
        if (channels > 1)
        {
            image = new BufferedImage(width, height, BufferedImage.TYPE_3BYTE_BGR);
        }
        else
        {
            image = new BufferedImage(width, height, BufferedImage.TYPE_BYTE_GRAY);
        }
        final byte[] targetPixels = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
        System.arraycopy(sourcePixels, 0, targetPixels, 0, sourcePixels.length);
        
        return image;
    }

Source Code

https://github.com/yushulx/java-dotcode-reader

The post How to Build DotCode Reader in Java on Windows 10 appeared first on Code Pool.