Try CVAT Online
PRODUCT
CVAT CommunityCVAT OnlineCVAT Enterprise
SERVICES
Labeling ServicesAudio Annotation Services
COMPANY
AboutCareersContact usLinkedinYoutubeGitHub
PRICING
CVAT OnlineCVAT Enterprise
SOLUTIONS
AutomotiveGeospatialInfrastructureLogistics
RESOURCES
All ResourcesBlogDocsCase StudiesChangelogAcademyFeature HighlightsPlaybooksTutorials
COMMUNITY
DiscordGitHub
Beyond Vision: CVAT Goes Multimodal and Introduces Audio Annotation. Learn more ➔

Captic brings AI vision to industrial depalletizing robots with CVAT Online

A Conversation With
Tim De Smet
Co-founder & CTO
About

Captic is a Belgian Physical AI company founded in Ghent and spun out of ML6, a Belgian AI consultancy. Captic builds AI vision kits for industrial robot automation, giving machine builders and OEMs a vision intelligence layer that enables robots to reliably locate and pick deformable bags, stacked boxes, and FMCG goods off pallets. Their foundation models generalize across packaging types and customer environments, and are deployed at customer manufacturing sites.

Customer
Captic
Headquarters
Ghent, Belgium
Industry
Industrial AI
Robotics
Computer vision
Use case
Instance segmentation annotation for AI foundation models powering industrial robot depalletizing
in the stack
CVAT Online (Team plan)
Captic Edge
Captic Cloud
Captic Train
Segment Anything Model 2
Segment Anything Model 3
NVIDIA Jetson
Orbbec RGB-D Cameras
Percipio RGB-D Cameras
Before choosing CVAT, we tried Label Studio, Roboflow, and Azure ML’s labeling tool. CVAT turned out to be the best fit for our instance segmentation workflow, both in terms of features and pricing. And since it’s such a popular tool, the annotators we work with already know it well and can get started without extra onboarding.
Tim De Smet
Co-founder & CTO
CHALLENGE

Point-cloud-only robot vision fails when items are stacked flush or packaging is deformable. No visible 3D edge means no reliable pick location.

SOLUTION

Instance segmentation models trained on RGB images annotated through CVAT Online. Masks identify object boundaries precisely, then guide the depth data query to the right region.

IMPACT

Foundation models covering bags, boxes, and FMCG goods, generalizing across materials and designs, retrained continuously from customer deployments, with datasets reaching 200,000+ labeled images.

When the edge disappears

To understand what Captic's vision system does differently, it helps to understand what conventional industrial vision systems do, and where they break down.

Standard robot vision for depalletizing relies primarily on depth cameras and point cloud analysis. The system builds a 3D map of the pallet surface, looks for edges in the depth data where one object ends and another begins, and uses those edges to plan pick locations. For well-separated, uniformly designed boxes, this produces reliable results. The geometry is predictable and the edges are clear.

Two scenarios break that logic. The first is tight stacking: when boxes are pressed flush against each other on a pallet, there is no visible 3D edge between them. The depth camera sees a flat surface with no meaningful discontinuity, and the system cannot determine where one box ends and the next begins. The second is deformable packaging: a bag has no consistent geometry. It deforms, creases, and shifts. Its surface offers no stable edges to detect.

Captic's approach starts one step earlier. Rather than relying on depth data to locate objects, Captic first runs an instance segmentation model on the RGB image, producing a precise mask for each object in the scene. Those masks are then used to query the relevant region of the point cloud, extracting the 3D information needed for pick planning. The system is no longer searching for edges in depth data. It is using masks derived from RGB to tell the depth sensor exactly where to look.

Beyond reliability, this approach also changes the hardware requirements. Making point-cloud-only systems work reliably at robot arm distances requires high-precision 3D hardware. Using instance segmentation from RGB as the primary localization layer allows Captic to work with more accessible RGBD cameras from vendors such as Orbbec and Percipio, which makes the technology available to a broader range of machine builders.

Foundation models for every packaging type

Captic's customers are OEM machine builders and system integrators who need vision intelligence built into their equipment. Building and maintaining AI vision from scratch is not simply a matter of willingness: the sustained engineering required to do it well, and to keep models improving as new products and environments are encountered, is routinely underestimated. With Captic, machine builders get a more powerful option from day one that they do not need to maintain themselves.

The core of that offering is a set of foundation models, one per primary packaging category: bags, cardboard boxes, and FMCG goods. Each foundation model generalizes across the full range of designs and materials that fall within that category. Captic retrains these models continuously and can fine-tune or adjust them for specific customers or partners.

For larger machine builder partners, Captic builds dedicated models. Everyone else depalletizing bags uses, and benefits from, one generalized model. The annotation work that makes this possible is continuous: every customer deployment is a source of new training images, and that data flows back through the annotation pipeline regularly to keep models improving.

How Captic uses CVAT Online

Annotation at Captic is not a one-time setup. It is an ongoing production process, with datasets for some models growing past 200,000 labeled images, all annotated through CVAT Online. Three workflows sit at the center of that process.

SAM 2 and SAM 3 for bag masking. Bags are deformable, with irregular boundaries and surfaces that fold, crease, or overlap. CVAT Online's single-object interactive segmentation mode, powered by SAM 2 and SAM 3, is the primary tool for handling them. Annotators click to include or exclude specific regions, add manual keypoints where SAM would otherwise pick the wrong object, and convert the result to a polygon. The interaction loop gives annotators the control needed to get the boundary right on harder cases.

Mask brush for edge precision on difficult cases. Two conditions make bag segmentation reliably hard: shadows and creases. When a bag sits partially in shadow, the boundary between the bag surface and the floor becomes ambiguous, and SAM without guidance will often track the shadow edge rather than the object edge. Creased bags, brown paper bags in particular, present a similar problem: SAM can read a crease as an object border instead of part of the bag. CVAT Online's mask brush tool lets annotators add or remove regions from a mask directly, correcting those misinterpretations without redoing the full annotation from scratch.

Before:

During:

After:

API integration for direct export from Captic Cloud. Images collected from customer deployments via Captic Edge flow into Captic Cloud. From there, batches are exported directly to CVAT Online via the API using an access token, arriving as structured tasks with labeling context and instructions attached. External labelers, sourced through Upwork, work through those tasks without needing any manual file transfer step between systems. The connection between Captic Cloud and CVAT Online is part of the standard data pipeline, not a manual step.

Beyond bags, the same CVAT Online environment handles annotation for boxes and FMCG goods. Mask joining and splitting are also central to Captic's workflow: for some use cases they are not a nice-to-have but crucial.

Why CVAT Online

Captic evaluated annotation platforms systematically at the start, running Label Studio, Roboflow's labeling tools, and Azure ML Labeling Studio in parallel before committing to any of them.

CVAT Online won on two dimensions. The first was instance segmentation tooling. The combination of SAM-powered interactive segmentation, mask joining, mask splitting, and the mask brush is what Tim De Smet describes as "overpowered" for the task. The second was pricing.

Roboflow was a poor fit for a further reason: it ties labeling and model training together, and Captic wants full control over where and how it trains its models. Tim De Smet also notes that annotation is more of a side gig for Roboflow, and that its advanced features are lacking.

From factory floor to updated model

The full data loop at Captic runs from customer installation to production model update, with CVAT Online at the annotation stage.

Captic Edge runs on an industrial PC installed at the customer's manufacturing site, where inference runs locally. Every request the system processes is logged, and the images are securely synced to Captic Cloud. From Captic Cloud, batches are exported to CVAT Online via the API. Upwork-sourced labelers, coordinated through a single point of contact, work through those tasks in CVAT Online. Annotated data is then pulled back and fed into Captic Train for model training.

Not every image goes to labeling. Captic gathers feedback from human operators and robots and runs statistics between picks to surface outlier combinations of masks and images, and combines this with a substantial amount of manual review. Tim De Smet stresses that dataset quality is key to a good model. Captic has built tooling to automate most of the effort required and uses combinations of model training+inference as well as purely statistical methods to ensure quality of labelling and distribution.

With the full pipeline connected, the constraint that shapes how fast Captic can improve their models has shifted. As Tim De Smet puts it: "The bottleneck is now our human labelers rather than the labeling tool." Each image is different enough to require individual attention.

Building for the bulk industry

Captic's current focus is Physical AI systems for the bulk industry: smarter check-weighers and intelligent bag handling robots. Focusing on one industry lets Captic build new systems faster, because of the large amount of data it already has there. FMCG and box depalletizing are more geared toward logistics, which Captic views as markets to grow into. 

The annotation challenge is not simply a matter of data volume: it is about keeping labeler context narrow enough that annotation quality stays high. Captic has found that the more focused a dataset is, the easier it is for labelers to understand how to label and what is expected of them. When they have to switch context between boxes, crates, and bags, mistakes become more likely.

CVAT Online is the annotation layer that keeps Captic's retraining loop running.

Building a vision AI product that needs custom training data?

Explore CVAT Online for cloud-based annotation, CVAT Enterprise for self-hosted deployments, or our Labeling Services for fully managed annotation.

Get Started Today

Build, scale, and deliver high-quality training data for your AI models with CVAT.
Free plan available • No credit card required • GDPR & CCPA compliant