C++ Inference

CV C++ scripts use Helios::Inference::Session from the SDK included with the current Helios release. Helios uses the Compute GPU selected in Preferences.

Basic Flow

#include <helios/HeliosCVSDK.h>

class CVWorker {
    Helios::Inference::Session engine_;

public:
    CVWorker(std::uint32_t, std::uint32_t)
        : engine_(Helios::Inference::Session::create("MODEL_UUID")) {
        if (!engine_.valid()) {
            throw std::runtime_error("Could not create inference session");
        }

        engine_.setConfidenceThreshold(0.45f);
        engine_.setNmsThreshold(0.50f);

        if (!engine_.start()) {
            throw std::runtime_error("Could not start inference");
        }
    }

    void process(Helios::Frame& frame) {
        if (const HeliosInferenceResultsBlock* result =
                engine_.waitForResultsRaw()) {
            for (std::uint32_t i = 0; i < result->num_detections; ++i) {
                const HeliosInferenceDetection& detection =
                    result->detections[i];

                const float confidence = detection.confidence;
                const float centerX = detection.center_x;
                const float centerY = detection.center_y;
                (void)confidence;
                (void)centerX;
                (void)centerY;
            }
        }

        (void)frame;
    }
};

HELIOS_CV_SCRIPT(CVWorker)

Create and start the session once in the worker constructor. waitForResultsRaw() waits indefinitely by default so the result stays paired with the current frame. Pass a finite timeout only when the script intentionally accepts dropping a result after that deadline.

Session::create(uuid) creates a detection, pose, or segmentation session. Session::create() creates a session that can be configured for OCR without an object model.

Results

HeliosInferenceResultsBlock includes:

• frame_sequence, timestamp_ns, and inference_time_ms;
• model_type, frame_width, and frame_height;
• num_detections;
• up to 64 entries in detections across all classes.

Only read the first num_detections entries. Detection, pose, and segmentation share this global 64-result capacity. Each HeliosInferenceDetection provides:

• class id and confidence;
• box coordinates, size, center, and area;
• configured anchor plus polar distance and angle;
• predicted anchor, prediction offsets/input/motion, continuous detection age, prediction quality, and prediction flags;
• apparent_distance, a dimensionless size-derived relative distance; and
• 17 pose keypoints when the model supplies them.

Larger apparent_distance values represent smaller, apparently farther detections; they are not physical distance units. Configure the expected target height-to-width ratio with setApparentDistanceAspectRatio(...). The default ratio is 2.5.

The returned block is a borrowed view. Consume it during the current callback and do not retain its pointer.

Segmentation Results

After waitForResultsRaw() succeeds for a segmentation model, read the companion result:

const HeliosInferenceSegmentationResultsBlock* masks =
    engine_.segmentationResultsRaw();

if (masks) {
    for (std::uint32_t i = 0; i < masks->mask_count; ++i) {
        const std::uint8_t* plane =
            masks->masks + i * masks->mask_plane_stride;
        (void)plane;
    }
}

Mask i corresponds to detection i. Rows use mask_row_bytes; planes use mask_plane_stride. Pixels are packed least-significant-bit first, and the origin/scale fields map mask coordinates to the source frame. This block is also borrowed and must not be retained across frames.

Common Settings

Colors passed to inference drawing calls use BGR order. Pose sessions also provide keypoint and skeleton drawing settings. Drawing is independent of returned results. Apparent-distance text is disabled by default.

The session supports class priorities, anchors, ignore regions, search/zoom controls, pausing, and model changes. setSearchStick(...) accepts stick 1 or 2, a radius from 0 through 100, and an optional less-than comparison. Search-button and stick triggers are alternatives; either activates inference. With neither configured, inference runs on every frame. Only the highest-priority tier represented by retained candidates is published; lower tiers do not backfill unused slots.

One session holds one object model. Use multiple sessions for multiple models and account for the added CPU/GPU load. Helios supports up to 30 simultaneously started inference sessions in total, and each session supports up to 16 OCR regions. A pause or resume requested during a frame takes effect with the next frame, and a wait during that transition returns immediately.

SDK and ABI Compatibility

Use the complete SDK under versions/<version>/sdk/ from the installed Helios release. Keep all headers from the same release and include HeliosCVSDK.h for normal CV C++ scripts. Python scripts do not compile against the native ABI.

Build against the public SDK packaged with Helios. Existing finalized public calls remain compatible as new functions are added. A script that uses a newer function requires a Helios release that provides it; update the runtime and SDK together when adopting new features.

HELIOS_CV_SCRIPT(CVWorker) generates the three lifecycle exports helios_cv_script_create, helios_cv_script_process, and helios_cv_script_destroy. Low-level consumers should use InferenceCore.h, InferenceCoreApi.h, and the matching result headers under sdk/include/helios instead of copying declarations. The complete callable reference is included under sdk/docs.

Vision and Meter are separate active-development surfaces and may change before they are declared finalized.

Anchor Prediction

For workflows that compensate an anchor using measured camera motion, configure an InfcoreAnchorPredictionConfig and pass it to:

InfcoreAnchorPredictionConfig config{};
config.struct_size = sizeof(config);
config.x_enabled = 1;
config.y_enabled = 1;
config.response_delay_ms = 8.0f;
config.lead_x = 1.0f;
config.lead_y = 1.0f;

engine_.setAnchorPrediction(config);

Provide motion samples with:

engine_.recordAnchorPredictionMotion(timestampNs, velocityX, velocityY);

Use the same monotonic nanosecond time domain as the frame timestamp. X and Y velocities are measured in screen-heights per second. Enable only the axes the script can measure reliably.

For text recognition, use the OCR methods on the same session.

Practical Guidance

• Use the HeliosCVSDK.h and ABI headers included with the running Helios version.
• Create and start a session outside process().
• Consume raw result views promptly and only through their documented fields.
• If a wait returns no result unexpectedly, check engine_.runtimeError() and the Output Panel; a hard runtime failure is not an ordinary zero-detection frame.
• INFCORE_ERROR_INSTALLATION_INVALID (18) means the installation could not be verified. Install the latest complete official release and check whether security software quarantined a packaged file.
• Begin with the model's recommended ROI and class-priority settings.
• Report persistent setup failures once rather than once per frame.
• Check the Output Panel when model startup fails.