Vision Tools

helios.vision provides prepared, synchronous tools for common image-analysis tasks:

• find colored shapes inside a region;
• compare a fixed area with a same-sized reference; and
• find the best location for a smaller template.

Create these objects once in CVWorker.__init__ and reuse them. During process(frame), their per-frame methods automatically use the current Helios frame: call find() or match() without passing frame. A per-frame call outside process() fails because no current frame is available.

Vision and Meter are active-development APIs. Use the SDK and examples packaged with the installed Helios release and review their reference pages when updating a script.

Color Contours

Define one or more BGR ranges, a search region, and acceptable result sizes:

from helios import vision


class CVWorker:
    def __init__(self, width, height):
        indicator = vision.BgrRange(
            low=(20, 120, 20),
            high=(140, 255, 140),
        )

        self.finder = vision.ContourFinder(
            colors=[indicator],
            roi=(0, 0, 1920, 1080),
            width=(12, 400),
            height=(12, 400),
            max_results=16,
        )

    def process(self, frame):
        results = self.finder.find()

        for result in results:
            x, y, result_width, result_height = result.bounds
            center_x, center_y = result.centroid
            confidence = result.confidence

The configured ROI and size ranges use Helios's fixed 1920x1080 design space and are scaled to the current capture size. A BgrRange accepts only low and high BGR triples. Connected grouping includes diagonal neighbors.

Constructor

vision.ContourFinder(
    colors,
    roi,
    width,
    height,
    grouping=vision.GROUP_CONNECTED,
    max_results=64,
    focus=False,
    focus_padding=16,
    search_buttons=None,
    search_stick=None,
    search_stick_radius=None,
    search_stick_less_than=False,
    shape=None,
    shape_tolerance=1,
    shape_score=0.72,
    color_growth=0,
    adaptive_color=False,
    keep_focus_on_release=False,
)

GROUP_CONNECTED returns separate connected shapes. GROUP_ALL_IN_ROI combines all matching pixels in the ROI. An optional two-dimensional uint8 shape mask can reject regions that do not resemble the prepared shape.

With focus=True, the search region follows the largest retained result using focus_padding; after a miss it returns to the configured ROI. keep_focus_on_release=True keeps an already-focused search active after its input trigger is released until that focused result is missed.

Use color_growth to admit nearby BGR values with a fixed tolerance. Use adaptive_color=True when a stable indicator changes gradually with capture conditions. Keep the ROI and size limits specific so adaptation starts from the intended element.

Optional Input Gating

search_buttons can be a list of button indices, a list of (button_index, state) conditions, or up to eight OR clauses containing those conditions. State 1 means pressed and 0 means released.

# Search when either button 0 is pressed, or buttons 1 and 2 are pressed.
self.finder.set_search_buttons([
    [(0, 1)],
    [(1, 1), (2, 1)],
])

# Also search while stick 1 magnitude is greater than 25.
self.finder.set_search_stick(1, radius=25)

The stick is 1 or 2, and radius is 0..100. Set less_than=True for activation below the radius, or call set_search_stick(None) to disable the stick trigger. Button and stick triggers are alternatives: either one activates the search. With neither configured, every frame is searched.

Results and Lifetime

find() returns a reusable live ContourResults sequence. Each Contour exposes:

• bounds, centroid, area, and fill_ratio;
• color_score, shape_score, and confidence;
• core_pixel_count, halo_pixel_count, and color_mask;
• observed_low, observed_high, result_index, and generation.

results.truncated is true when more matches existed than max_results allowed. results.generation identifies the current result generation. The sequence and contour objects are refreshed in place by later searches or detection-setting changes, so copy individual values when a historical snapshot is required.

Detailed per-color diagnostics are produced only when requested:

stats = self.finder.color_stats(result_index=0, color_index=0)

The returned dictionary includes count, BGR minimum/maximum, percentiles, mean, and core/halo counts for that result and configured color.

Runtime Settings and Visuals

The finder supports:

set_colors(colors)
set_roi(roi)
set_visual_roi(roi)
set_vertical_gap_close(maximum_gap)
set_minimum_vertical_support(minimum_support)
set_size_range(width, height)
set_grouping(grouping)
set_focus(enabled, padding=None)
set_keep_focus_on_release(enabled)
set_search_buttons(search_buttons)
set_search_stick(stick, radius=None, less_than=False)
set_color_growth(tolerance)
set_adaptive_color(enabled)
color_stats(result_index, color_index=0)

set_visuals(...) controls the ROI, result outline, optional result metrics, colors, thickness, text scale/gap, and Overlay target. Visuals are enabled by default. Use show_metrics=True for setup and diagnostics, then disable it when those details are no longer needed. The active values are available through visual_settings; finder.roi and finder.results expose the current ROI and live results.

set_visual_roi((x, y, width, height)) keeps the displayed search rectangle fixed without changing the detection region. It uses the same 1920x1080 design coordinates as set_roi(). Pass None to show the active detection region again.

set_vertical_gap_close(maximum_gap) joins matching vertical fragments separated by a small gap. set_minimum_vertical_support(minimum_support) filters out columns with too few matching pixels. Both accept integers from 0 through 1080 in design pixels, scaled to the capture height. Zero disables the respective filter. Start with small values and inspect the results before widening the search.

Load Reference Images

Load PNG references once during worker setup:

from pathlib import Path
from helios import vision

reference_image = vision.load_png(Path(__file__).with_name("indicator.png"))
matcher = vision.TemplateMatcher(reference_image, threshold=0.95)

vision.load_png(path) accepts a filesystem path. vision.decode_png_base64(data) accepts an ASCII string or bytes containing standard Base64 PNG data, including its required padding. Supply the encoded data without a data: prefix or line breaks. Both return an HxWx3 uint8 BGR NumPy array; PNG alpha is not retained. Use those arrays with NormMatcher or TemplateMatcher.

Same-Size Image Comparison

NormMatcher compares a prepared reference with a same-sized area of the current frame:

matcher = vision.NormMatcher(reference_image, threshold=0.98)

def process(frame):
    matcher.match(roi=(100, 80, 320, 180))
    if matcher.matched:
        similarity = matcher.similarity

The ROI must have the same width and height as the prepared reference. The default threshold is 0.95. Results are held on the matcher through raw_value, sample_count, mean_error, similarity, matched, and threshold.

Use set_threshold(value) to adjust the match floor. set_visuals(...) can enable or disable the status border and set matched/unmatched colors, thickness, and target. Its current settings are available through visual_settings.

Template Matching

TemplateMatcher finds the single best location for a prepared image inside the current frame's search ROI:

matcher = vision.TemplateMatcher(template_image, threshold=0.95)

def process(frame):
    matcher.find(roi=(0, 0, 1920, 1080))
    if matcher.found:
        x, y, width, height = matcher.bounds
        score = matcher.score

The search ROI must be at least as large as the template. The default threshold is 0.90. Results are held on the matcher through found, bounds, raw_value, score, and threshold.

Use set_threshold(value) to adjust the match floor. set_visuals(...) controls the search-region and match borders, colors, thicknesses, and target. Its current settings are available through visual_settings.

C++

CV C++ provides the same prepared tools under Helios::Vision:

• ContourFinderConfig and ContourFinder::find();
• NormMatcher::match(roi); and
• TemplateMatcher::find(roi).

Configuration uses BgrRange, RectI32, and PixelRange. Retain prepared objects as worker members and copy values from borrowed results only when they must outlive the next operation.

Use Helios::Vision::loadPng(pathUtf8) or decodePngBase64(encodedPng) to create an Image, then pass image.bgr() to a matcher. Keep the Image alive while using its BGR view. The contour equivalents are setVisualRoi(roi), clearVisualRoi(), setVerticalGapClose(maximumGap), and setMinimumVerticalSupport(minimumSupport).

Use HeliosVisionSDK.hpp from the current packaged SDK. The Vision and Meter contracts are still under active development and may change before they are finalized. Use only the public functions declared by the packaged SDK.

Choosing a Tool

Troubleshooting

• Call find() and matcher operations only from process(frame) so a current frame exists.
• Use BGR order for threshold ranges; visual colors accept RGB or RGBA.
• Tighten ROI and size ranges before increasing color tolerance.
• Confirm reference and template arrays are HxWx3 uint8 BGR images.
• Copy result values before the next finder or matcher operation when you need history.