The University of Illinois Chicago Department of Urology is a nationally recognized academic urology program and one of the earliest adopters of the single-port (SP) robotic system. The department runs a dedicated fellowship in minimally invasive urology and conducts research in AI-enabled surgical guidance and robotic technique.
Annotating surgical video requires surgeons who have been at the console and understand the anatomy first-hand. Those surgeons have full clinical schedules and little time for labeling. Manual contouring of anatomical structures frame by frame is too slow to build a usable training dataset at research scale.
CVAT Online's SAM integration generates automatic segmentations for each frame, letting expert annotators focus on review and correction rather than manual contouring. Multi-user project access enables faculty, residents, and collaborators across institutions to contribute to the same dataset simultaneously.
Over 1,500 annotated surgical frames, a working anatomy-recognition prototype deployed via the SP console's TilePro feature, and a published manuscript — with a planned study to validate real-time AI guidance for medical student anatomy training.
Robotic surgery trades haptic feedback for precision. A surgeon at the console sees anatomy in extraordinary detail but cannot feel tissue consistency the way they would in open surgery. For trainees and early-career urologists, that absence of tactile information makes spatial orientation a genuine challenge: knowing exactly what structure lies behind the tissue you are cutting, and how far to go, demands experience that takes years to build.
The Department of Urology at the University of Illinois Chicago (UIC) is building AI to give some of that orientation back. Led by Dr. Simone Crivellaro, one of the earliest adopters of the single-port (SP) robotic system worldwide, and Dr. Luca Morgantini, whose research focuses on AI-enabled decision support in robotic surgery, the team is training computer vision models to recognize anatomy in real time during robotic urological procedures. CVAT Online is the annotation infrastructure behind the dataset.
What really separates a good robotic surgeon is never losing their orientation inside the patient and always knowing exactly what anatomy they're looking at. Real-time anatomy recognition is meant to give some of that orientation back.
Luca Morgantini
MD Resident, Department of Urology
The team's AI pipeline overlays anatomical labels directly onto the surgeon's console view during a robotic procedure. In transvesical enucleation of the prostate, the model recognizes three anatomical structures: the bladder neck, the prostatic adenoma, and the peripheral zone, tracking them frame by frame as the surgery progresses. The output appears through the console's TilePro feature, a second video channel built into the SP system, so the surgeon sees the AI's real-time assessment alongside the endoscopic view without interrupting the surgical workflow.
The underlying model is trained on instance segmentation labels rather than bounding boxes, because the clinical value lies in knowing precisely where a structure begins and ends, not just its approximate location. YOLO is the primary architecture the team trains on, though U-Net has also been used depending on the task.

UIC was among the first programs worldwide to adopt the single-port SP robot when Intuitive, maker of the Da Vinci robotic system, introduced it in 2018, and it runs one of the few dedicated fellowships in minimally invasive urology. That combination of early clinical adoption and structured training pipeline is what makes the department both motivated and positioned to build this kind of surgical AI.
Surgical video is difficult to label. Every frame that goes into training must be extracted from a recorded procedure, de-identified, and annotated by someone who understands what they are looking at from a surgeon's perspective. Off-the-shelf computer vision annotators cannot interpret the structures inside a robotic procedure. The annotators have to have been at the console.
“It matters a great deal that the people doing the annotation have actually been at the console and understand the anatomy first-hand,” says Dr. Morgantini. “The flip side is that we’re busy surgeons without much spare time for labeling, so that expertise is a real bottleneck.”
The same expertise constraint shapes quality control. Surgical anatomy can be ambiguous in video: different annotators may segment the same structure differently depending on surgical phase, camera angle, or tissue state. The team handles this through a two-expert review protocol. Every annotation is confirmed by a second expert, and any genuine uncertainty is escalated to Dr. Crivellaro, who is internationally recognized in robotic urological surgery and trusted to resolve those conflicts definitively.
The regulatory groundwork also cannot be skipped. IRB and institutional approval are required before any surgical video enters the annotation pipeline. Because every image is fully de-identified before upload, that approval is generally more straightforward to obtain. But it must come first.
The annotation pipeline runs in three stages:
Frame extraction and de-identification. Key frames are extracted from recorded SP robotic procedures and fully de-identified before any annotation work begins. The de-identified frames are uploaded directly into a CVAT Online project, where the whole team can begin working immediately.
AI-assisted segmentation with SAM. CVAT Online's Segment Anything Model (SAM) integration runs a first-pass segmentation on each uploaded frame, generating anatomical outlines automatically. The expert annotator then reviews, adjusts, and confirms the AI-generated masks. Manual contouring of every frame by hand is replaced by review and correction, which is substantially faster given the complexity of surgical anatomy.
Expert review and export. Each annotation is confirmed by a second expert. Verified labels are exported from CVAT Online in any required format. CVAT's multi-format export is a practical asset: the same labeled dataset can feed YOLO, U-Net, or other architectures without re-labeling, which matters as the team experiments across model types.
The workflow turns the expert-time bottleneck into a manageable review task. Rather than asking a surgeon to draw every segmentation mask by hand, CVAT Online compresses their contribution to confirming what the model got right and correcting what it did not.
The standout feature is the AI-assisted annotation, in particular the Segment Anything Model (SAM) integration. It automatically segments structures and objects in our frames, which streamlines the entire workflow and makes labeling dramatically faster than manually contouring every image by hand.
Luca Morgantini
MD Resident, Department of Urology
UIC arrived at CVAT Online through peer recommendation. When the team asked other medical AI researchers what annotation tool to use, CVAT came up consistently. "We got there the smart way," Dr. Morgantini notes, "as we asked other researchers what the most state-of-the-art option was, and they consistently pointed us to CVAT."
The standout capability for surgical video annotation has been the SAM integration. Instance segmentation of complex anatomical structures requires tracing precise contours around irregular shapes in every frame. SAM reduces that to a review-and-confirm workflow, which is what makes the dataset feasible to build with a small team of expert annotators.
Multi-user access has been equally important. The team spans faculty, residents, and collaborators across different locations, including international partners. CVAT Online allows everyone to work within the same project simultaneously. As Dr. Morgantini puts it: "Multiple teams across multiple sites collaborating on a single effort" is how research actually works now, and the platform reflects that.
Cost was also a factor. CVAT Online came in less expensive than the commercial alternatives the team evaluated, which matters for a research program operating on grant funding.
CVAT is a central part of our annotation pipeline going forward. One thing that’s especially important is being able to have multiple accounts working within the same team, with people contributing from different locations, even across continents. Everyone can review and contribute to the same project, which reflects how scientific research actually works now: multiple teams across multiple sites collaborating on a single effort.
Luca Morgantini
MD Resident, Department of Urology
The dataset has grown well beyond the initial pilot described in the team's publication, now exceeding 1,500 annotated surgical frames with annotation ongoing. A working anatomy-recognition model has been implemented through the TilePro feature on the SP console, overlaying real-time anatomical labels on the surgeon's view during procedures in research mode.

The team has published a manuscript describing the pipeline and the results. The next planned study will test whether real-time AI anatomy guidance helps medical students recognize surgical structures faster than studying from textbooks alone, a step toward validating the tool in a structured educational setting before broader clinical adoption.
The longer-term roadmap includes expansion across additional urological procedures, multi-institutional collaboration, and eventually intraoperative use in a clinical setting. The multi-user, cloud-based architecture of CVAT Online is already suited to that scale: annotators can contribute from anywhere, across time zones and institutions, to a single shared project.
UIC's work sits at the intersection of surgical training, clinical AI, and real-time decision support. The annotation pipeline built on CVAT Online is what allows a small team of expert surgeons to build training data at research scale, without sacrificing the domain-expertise standard that surgical AI demands.
CVAT Online provides the multi-user access that makes distributed collaboration possible, the SAM integration that compresses manual labeling into focused review, and the flexible export that keeps the labeled data useful across architectures. As the program grows across procedures and institutions, the same infrastructure scales with it.
Explore CVAT Online for cloud-based annotation, CVAT Enterprise for self-hosted deployments, or our Labeling Services for fully managed annotation.