What would be the best detector to classify objects with simple geometry and color such as a blue cube ?

Hi everyone,

for a project I want to use a web cam to classify objects with simple geometry and color such as a blue cube (see picture attached).

The model will have to be executed on a STM32 (nucleo64). From what I've read I think it is possible.

The objects are presented one by one to the camera. Depending on the object presented to the camera an action will be taken.

If it's a cube then it will be pushed towards a basket else nothing is done.

It's not clear to me what would the right approach to build a customized object detection model : "semantic segmentation algorithm" or an "objectDetector algorithm".

For "objectDetector algorithm" there are different algorithms (RCNN, YOLO, ACF ...). Here again I don't know what algorithm to choose.

Is ACF suited for this specific detection task ?

With ACF for instance, what would be as a rule of thumb the minimum number of labeled images required to train the model ? 50,100, 200, 500 ?

Thanks by advance,

Alain.

Answer 1

[Image Analyst]

That's a pretty simple problem. You don't have to use deep learning if you don't want to. Traditional methods would work fine. In fact if the blocks are always in that position you can just use the Color Thresholder App to find blue, then use regionprops to look at the area. The area of the cylinder will be in one range and the area of the cube in another range.

If you want to use deep learning, since the objects are so simple and there are only two classes, you might be able to get by with just 50 training images.

Try this:

% Demo by Image Analyst
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 16;
markerSize = 20;
%--------------------------------------------------------------------------------------------------------
%    READ IN TEST IMAGE
folder = pwd;
baseFileName = 'blue shapes.jpeg';
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
    % The file doesn't exist -- didn't find it there in that folder.
    % Check the entire search path (other folders) for the file by stripping off the folder.
    fullFileNameOnSearchPath = baseFileName; % No path this time.
    if ~exist(fullFileNameOnSearchPath, 'file')
        % Still didn't find it.  Alert user.
        errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
        uiwait(warndlg(errorMessage));
        return;
    end
end
rgbImage = imread(fullFileName);
%--------------------------------------------------------------------------------------------------------
% Display the image.
subplot(2, 1, 1);
imshow(rgbImage, []);
impixelinfo;
axis('on', 'image');
title('Original RGB Image', 'FontSize', fontSize, 'Interpreter', 'None');
% Update the dimensions of the image.
% numberOfColorChannels should be = 1 for a gray scale image, and 3 for an RGB color image.
[rows, columns, numberOfColorChannels] = size(rgbImage)

rows =    313

columns =    640

numberOfColorChannels =      3

% Maximize window.
g = gcf;
g.WindowState = 'maximized';
g.Name = 'Demo by Image Analyst';
g.NumberTitle = 'off';
drawnow;
%--------------------------------------------------------------------------------------------
% Threshold image.
[mask,maskedRGBImage] = createMask(rgbImage);
% Fill holes.
mask = imfill(mask, 'holes');
% Remove blobs smaller than 2000 pixels.
mask = bwareaopen(mask, 2000);
subplot(2, 1, 2);
imshow(mask)
impixelinfo;
axis('on', 'image');
title('Mask', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
%--------------------------------------------------------------------------------------------
% Now measure blobs.
props = regionprops(mask, 'BoundingBox', 'Area');
allAreas = [props.Area]

allAreas = 1×4

9334 6479 6310 8834

%--------------------------------------------------------------------------------------------
% Now put up bounding box and title
subplot(2, 1, 1);
hold on;
for k = 1 : numel(props)
    rectangle('Position', props(k).BoundingBox)
    x = props(k).BoundingBox(1);
    y = props(k).BoundingBox(2) - 10;
    if allAreas(k) > 7000
        caption = sprintf('Cube.  Area = %d', allAreas(k));
        captionColor = 'b';
    else
        caption = sprintf('Cylinder.  Area = %d', allAreas(k));
        captionColor = 'r';
    end
    text(x, y, caption, 'Color', captionColor, 'FontSize', 11, 'FontWeight','bold');
end
hold off;

%================================================================================================
function [BW,maskedRGBImage] = createMask(RGB)
%createMask  Threshold RGB image using auto-generated code from colorThresholder app.
%  [BW,MASKEDRGBIMAGE] = createMask(RGB) thresholds image RGB using
%  auto-generated code from the colorThresholder app. The colorspace and
%  range for each channel of the colorspace were set within the app. The
%  segmentation mask is returned in BW, and a composite of the mask and
%  original RGB images is returned in maskedRGBImage.
% Auto-generated by colorThresholder app on 29-Dec-2023
%------------------------------------------------------
% Convert RGB image to chosen color space
I = rgb2hsv(RGB);
% Define thresholds for channel 1 based on histogram settings
channel1Min = 0.474;
channel1Max = 0.717;
% Define thresholds for channel 2 based on histogram settings
channel2Min = 0.153;
channel2Max = 1.000;
% Define thresholds for channel 3 based on histogram settings
channel3Min = 0.000;
channel3Max = 0.799;
% Create mask based on chosen histogram thresholds
sliderBW = (I(:,:,1) >= channel1Min ) & (I(:,:,1) <= channel1Max) & ...
    (I(:,:,2) >= channel2Min ) & (I(:,:,2) <= channel2Max) & ...
    (I(:,:,3) >= channel3Min ) & (I(:,:,3) <= channel3Max);
BW = sliderBW;
% Initialize output masked image based on input image.
maskedRGBImage = RGB;
% Set background pixels where BW is false to zero.
maskedRGBImage(repmat(~BW,[1 1 3])) = 0;
end

[Alain]

Hi, thank you for this code. I'll try it.

I ve been able to implement ACF algorithm with 70 images and got good results on my validation set (12 images : 100% true positive detected, no false pos nor false neg)

The strongest box is a bit too small and not perfectly centered compare to your results. I coded :

% gTruthImageCube2 was exported from the imageLabeller
trainingData2 = objectDetectorTrainingData(gTruthImageCube2, ...
    'samplingFactor', 10, ...
    'writeLocation', 'TrainingData2');

summary(trainingData2)
acfDetector = trainACFObjectDetector(trainingData2, 'NumStages',10,NegativeSamplesFactor=2);

I = imread('./ValidationData_2/cube_12.jpeg');
imshow(I)
[bbox,score] = detect(acfDetector,I,'SelectStrongest', true); 
[selectedBbox,selectedScore] = selectStrongestBbox(bbox,score,'OverlapThreshold',0.5, 'NumStrongest', 1);

annotation = acfDetector.ModelName;
I_annot = insertObjectAnnotation(I,'rectangle',selectedBbox,selectedScore, TextBoxOpacity=0.9,FontSize=40);

figure 
imshow(I_annot)

If you have some advices to enhance my code I would appreciate a lot.

Anyway,

Thank you very much,

Alain.