Skip to content
CrocoDocs
Starter GuideProgrammingEngineeringDesignPortfolio
Documentation

Introduction

IntroductionBIOBUZZ game overview

Getting started

Getting startedBlocksOnBot JavaAndroid StudioYour first OpModeJava basicsMotors and servosNonblocking codeDebugging robot code

Control systems

Control systemsJoystick mappingField-centric mecanum drivePID controlFeedforwardFlywheel velocity controlSystem identificationMotion profilingKalman filterLow-pass filter

Autonomous

AutonomousTimed and encoder movementOdometryLocalization troubleshootingMotion planningReliable autonomous sequencesPure PursuitSensor fusionAuto to TeleOp handoff

Season topics

Season topicsProjectile mathematicsTurret aimingShooting on the move

Code practices

Code practicesNaming conventionsCode organizationLoop timing and bulk readsFormattingComments and documentationTeam collaboration

Libraries

LibrariesNextFTCFTCLibSolversLibPedro PathingRoad RunnerFTC DashboardMercurial v2Mercurial v1PanelsSloth

Community software

Community softwareBlaze FTCOpenLoopHive Vision

Sensors and vision

Vision overviewVision basicsLimelight 3A
EasyOpenCV basics
OpenCV pipelines
VisionBench
Starter Guide
Engineering
Design
Portfolio
Appearance
EasyOpenCV

OpenCV pipelines

Build a small color mask step by step with EasyOpenCV

A pipeline is a method EasyOpenCV calls once for each camera frame. The example turns each frame into a black-and-white mask, finds connected matching pixels, draws a box around the largest match, and shares its center with the .

Use the real BIOBUZZ Pollen photo below to follow a yellow object through the pipeline. The photo is an isolated product shot from AndyMark, so its white background is much easier than a real field. We ran OpenCV on that image to make the mask and blob overlay shown here; the example bounds still need retuning for your 's camera and lighting.

The unprocessed BIOBUZZ Pollen product photo used as the pipeline's input.

This controlled product photo is easier to detect than a real field image; its mask thresholds are teaching values, not camera calibration.

Prerequisites

This page assumes you have confirmed that the camera opens. If it does not, work through EasyOpenCV Basics first. EasyOpenCV passes RGBA frames to the pipeline, so the color conversion below starts with COLOR_RGB2HSV (which accepts RGB or RGBA input and ignores alpha).

Working images

OpenCV represents an image as a Mat. The input image is supplied by EasyOpenCV; make Mat objects for the converted HSV image and the mask. Reuse them for each frame instead of constructing them every time.

private final Mat hsv = new Mat();
private final Mat mask = new Mat();
private final Mat blurred = new Mat();
private final Mat kernel = Imgproc.getStructuringElement(
    Imgproc.MORPH_ELLIPSE, new Size(5, 5));
private volatile double centerX = Double.NaN;
private val hsv = Mat()
private val mask = Mat()
private val blurred = Mat()
private val kernel = Imgproc.getStructuringElement(
    Imgproc.MORPH_ELLIPSE, Size(5.0, 5.0))
@Volatile private var centerX = Double.NaN

centerX is shared between the camera thread and the OpMode loop. NaN means “no valid detection yet.” Never treat it as a real pixel coordinate.

Color mask

The conversion separates hue from brightness. inRange paints matching pixels white and other pixels black:

Imgproc.GaussianBlur(input, blurred, new Size(5, 5), 0);
Imgproc.cvtColor(blurred, hsv, Imgproc.COLOR_RGB2HSV);
Core.inRange(
    hsv,
    new Scalar(15, 60, 70),
    new Scalar(40, 255, 255),
    mask
);
Imgproc.morphologyEx(mask, mask, Imgproc.MORPH_OPEN, kernel);
Imgproc.morphologyEx(mask, mask, Imgproc.MORPH_CLOSE, kernel);
Imgproc.GaussianBlur(input, blurred, Size(5.0, 5.0), 0.0)
Imgproc.cvtColor(blurred, hsv, Imgproc.COLOR_RGB2HSV)
Core.inRange(
    hsv,
    Scalar(15.0, 60.0, 70.0),
    Scalar(40.0, 255.0, 255.0),
    mask
)
Imgproc.morphologyEx(mask, mask, Imgproc.MORPH_OPEN, kernel)
Imgproc.morphologyEx(mask, mask, Imgproc.MORPH_CLOSE, kernel)

These OpenCV 8-bit HSV bounds (hue 15–40, saturation 60–255, brightness 70–255) were tuned against the pictured product photo. Blur reduces pixel noise; the opening removes tiny specks; the closing joins small gaps. Tune these bounds using your camera under the lighting you expect on the field. A black mask means no pixels passed the range. If nearly everything is white, check the range and colour conversion.

The example photo has one isolated Pollen, so it produces one blob. Touching balls can merge into a single contour.

For one test, temporarily return mask from processFrame to view the black-and-white result. Return the annotated color input after the mask is working.

The actual threshold output after the yellow HSV range and cleanup operations: white pixels passed the range, and black pixels did not.

This is the intermediate result to inspect before moving on. If the Pollen is not mostly white, adjust the range or lighting first; contour code cannot recover pixels that the mask removed.

Contours

A connected group of white pixels is a contour. Find contours in the mask, then keep the largest contour above a minimum pixel area:

List<MatOfPoint> contours = new ArrayList<>();
Mat hierarchy = new Mat();
Imgproc.findContours(mask, contours, hierarchy,
    Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
hierarchy.release();

MatOfPoint largest = null;
double largestArea = 250.0; // Example minimum, measured in pixels
for (MatOfPoint contour : contours) {
    double area = Imgproc.contourArea(contour);
    if (area > largestArea) {
        largestArea = area;
        largest = contour;
    }
}
val contours = mutableListOf<MatOfPoint>()
val hierarchy = Mat()
Imgproc.findContours(mask, contours, hierarchy,
    Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE)
hierarchy.release()

var largest: MatOfPoint? = null
var largestArea = 250.0 // Example minimum, measured in pixels
for (contour in contours) {
    val area = Imgproc.contourArea(contour)
    if (area > largestArea) {
        largestArea = area
        largest = contour
    }
}

A mask often contains tiny white specks caused by reflections or image noise. The area test removes those small candidates. This is only a simple selection rule; if multiple real objects may be visible, the largest one may not be the object you want.

The largest accepted contour filled in white after the minimum-area check.

This view shows the candidate the example will measure. A mask with two touching balls can produce one connected contour, so this step does not promise one contour per game element.

Blob measurement

A bounding rectangle gives a practical center in pixels. Draw that rectangle so you can compare the program's result with the camera image:

if (largest != null) {
    Rect box = Imgproc.boundingRect(largest);
    centerX = box.x + box.width / 2.0;
    Imgproc.rectangle(input, box, new Scalar(0, 255, 0, 255), 3);
    Imgproc.circle(input, new Point(centerX, box.y + box.height / 2.0),
        5, new Scalar(255, 0, 0, 255), -1);
} else {
    centerX = Double.NaN;
}
if (largest != null) {
    val box = Imgproc.boundingRect(largest)
    centerX = box.x + box.width / 2.0
    Imgproc.rectangle(input, box, Scalar(0.0, 255.0, 0.0, 255.0), 3)
    Imgproc.circle(input, Point(centerX, box.y + box.height / 2.0),
        5, Scalar(255.0, 0.0, 0.0, 255.0), -1)
} else {
    centerX = Double.NaN
}

EasyOpenCV's frame is RGBA, so the color scalars above use four channels. The center is relative to the image, with the in its top-left corner; it does not directly give distance or field coordinates.

The processed Pollen with its detected contour, bounding box, and center marker drawn over the input image.

Compare the center marker with the object before trusting centerX. It is a pixel location in this camera frame, not a robot-relative angle or field position.

Release contours after use so native OpenCV memory does not accumulate:

for (MatOfPoint contour : contours) contour.release();
for (contour in contours) contour.release()

Complete pipeline

The class below assembles the steps. Read it by matching each block to the earlier stages. The camera setup and telemetry belong in a separate OpMode, as shown in EasyOpenCV Basics.

import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.MatOfPoint;
import org.opencv.core.Point;
import org.opencv.core.Rect;
import org.opencv.core.Scalar;
import org.opencv.core.Size;
import org.opencv.imgproc.Imgproc;
import org.openftc.easyopencv.OpenCvPipeline;
import java.util.ArrayList;
import java.util.List;

public class PollenTargetPipeline extends OpenCvPipeline {
    private final Mat hsv = new Mat();
    private final Mat mask = new Mat();
    private final Mat blurred = new Mat();
    private final Mat kernel = Imgproc.getStructuringElement(
        Imgproc.MORPH_ELLIPSE, new Size(5, 5));
    private volatile double centerX = Double.NaN;

    @Override
    public Mat processFrame(Mat input) {
        Imgproc.GaussianBlur(input, blurred, new Size(5, 5), 0);
        Imgproc.cvtColor(blurred, hsv, Imgproc.COLOR_RGB2HSV);
        Core.inRange(hsv, new Scalar(15, 60, 70),
            new Scalar(40, 255, 255), mask);
        Imgproc.morphologyEx(mask, mask, Imgproc.MORPH_OPEN, kernel);
        Imgproc.morphologyEx(mask, mask, Imgproc.MORPH_CLOSE, kernel);

        List<MatOfPoint> contours = new ArrayList<>();
        Mat hierarchy = new Mat();
        Imgproc.findContours(mask, contours, hierarchy,
            Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
        hierarchy.release();

        MatOfPoint largest = null;
        double largestArea = 250.0;
        for (MatOfPoint contour : contours) {
            double area = Imgproc.contourArea(contour);
            if (area > largestArea) {
                largestArea = area;
                largest = contour;
            }
        }

        if (largest == null) {
            centerX = Double.NaN;
        } else {
            Rect box = Imgproc.boundingRect(largest);
            centerX = box.x + box.width / 2.0;
            Imgproc.rectangle(input, box, new Scalar(0, 255, 0, 255), 3);
            Imgproc.circle(input,
                new Point(centerX, box.y + box.height / 2.0),
                5, new Scalar(255, 0, 0, 255), -1);
        }

        for (MatOfPoint contour : contours) contour.release();
        return input;
    }

    public double getCenterX() {
        return centerX;
    }

    // Owner calls this only after closeCameraDevice() has stopped callbacks.
    public void release() {
        hsv.release();
        mask.release();
        blurred.release();
        kernel.release();
    }
}
import org.opencv.core.Core
import org.opencv.core.Mat
import org.opencv.core.MatOfPoint
import org.opencv.core.Point
import org.opencv.core.Rect
import org.opencv.core.Scalar
import org.opencv.core.Size
import org.opencv.imgproc.Imgproc
import org.openftc.easyopencv.OpenCvPipeline

class PollenTargetPipeline : OpenCvPipeline() {
    private val hsv = Mat()
    private val mask = Mat()
    private val blurred = Mat()
    private val kernel = Imgproc.getStructuringElement(
        Imgproc.MORPH_ELLIPSE, Size(5.0, 5.0))
    @Volatile private var centerX = Double.NaN

    override fun processFrame(input: Mat): Mat {
        Imgproc.GaussianBlur(input, blurred, Size(5.0, 5.0), 0.0)
        Imgproc.cvtColor(blurred, hsv, Imgproc.COLOR_RGB2HSV)
        Core.inRange(hsv, Scalar(15.0, 60.0, 70.0),
            Scalar(40.0, 255.0, 255.0), mask)
        Imgproc.morphologyEx(mask, mask, Imgproc.MORPH_OPEN, kernel)
        Imgproc.morphologyEx(mask, mask, Imgproc.MORPH_CLOSE, kernel)

        val contours = mutableListOf<MatOfPoint>()
        val hierarchy = Mat()
        Imgproc.findContours(mask, contours, hierarchy,
            Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE)
        hierarchy.release()

        var largest: MatOfPoint? = null
        var largestArea = 250.0
        for (contour in contours) {
            val area = Imgproc.contourArea(contour)
            if (area > largestArea) {
                largestArea = area
                largest = contour
            }
        }

        if (largest == null) {
            centerX = Double.NaN
        } else {
            val box: Rect = Imgproc.boundingRect(largest)
            centerX = box.x + box.width / 2.0
            Imgproc.rectangle(input, box, Scalar(0.0, 255.0, 0.0, 255.0), 3)
            Imgproc.circle(input,
                Point(centerX, box.y + box.height / 2.0),
                5, Scalar(255.0, 0.0, 0.0, 255.0), -1)
        }

        for (contour in contours) contour.release()
        return input
    }

    fun getCenterX() = centerX

    // Owner calls this only after closeCameraDevice() has stopped callbacks.
    fun release() {
        hsv.release()
        mask.release()
        blurred.release()
        kernel.release()
    }
}

In the OpMode loop, read pipeline.getCenterX(). Check Double.isNaN(centerX) before using it. Do not block the camera callback waiting for the OpMode thread; pass simple measurements such as numbers or an enum between them.

Pipeline testing

  1. Run the camera with no detection. Confirm a live, correctly oriented image.
  2. Display the mask. Confirm the practice object is white and most of its background is black.
  3. Return the annotated image. Confirm the box and center dot stay on the object.
  4. Move the object left and right. Confirm centerX changes in the matching direction.
  5. Remove the object and introduce a similar-colored object. Confirm “no detection” and false positives are handled.
  6. Only after these checks, use the result to choose a robot action.

If detections flicker, improve the lighting or narrow the region of interest before adding complex logic. If your frame size changes, re-check area thresholds; a 250-pixel contour at one resolution is not the same fraction of the image at another.

Pipeline overview

Summary of the source, mask, cleaned candidate, and measured overlay shown at their corresponding steps above.

Sources

  • OpenFTC EasyOpenCV repository — installation status, RGBA note, examples, and SDK compatibility.
  • EasyOpenCV pipeline overview
  • FTC Color Blob Concepts — how the FTC processor defines a blob, contour, and fitted box.
  • Color Locator filtering — official FTC built-in processor filtering.
  • AndyMark: BIOBUZZ scoring elements — source photo

EasyOpenCV basics

Open a camera and understand how a custom OpenCV pipeline receives frames

VisionBench

Test an EasyOpenCV pipeline on your computer before using the robot

On this page

PrerequisitesWorking imagesColor maskContoursBlob measurementComplete pipelinePipeline testingPipeline overviewSources