Blog

The success of every modern computer vision system relies on one thing: data. Specifically, it relies on computer vision datasets that are well-annotated, diverse, and representative of the real world. These datasets are the fuel that drives object detection, semantic segmentation, visual recognition, and other tasks in AI.But in 2026, computer vision is entering a new stage. The rise of generative adversarial networks, synthetic data pipelines, and 3D object detection has changed how teams think about data altogether. Systems are no longer trained on simple labeled images, they now rely on dynamic, multimodal datasets that capture texture, movement, and depth.That's why choosing the right dataset has become less about quantity and more about context, structure, and how well it mirrors the world your model is meant to understand.In this guide, we’ll break down the most influential and widely used computer vision datasets in 2026. Our goal is to compare them based on format, task coverage, relevance, and how well they support emerging use cases like autonomous driving, image captioning, scene recognition, and multimodal AI so that you can make an informed decision.Criteria for Choosing a Computer Vision DatasetChoosing the right computer vision dataset isn’t just about finding the largest collection of images. It’s about aligning the dataset with your task, architecture, and domain constraints.In our opinion, there are four core factors that determine how useful a dataset will be. Let’s walk through each one so you can make confident, well-informed decisions.Scale and StructureLarge datasets are essential for training deep learning models, but volume alone isn’t enough. A high-quality dataset should include:Well-balanced class distributionClearly defined training, validation, and test setsDetailed annotations like bounding boxes, image-level labels, or segmentation masksDatasets like COCO and Open Images V7 offer strong structure and multi-label annotations, making them effective for object detection and visual recognition tasks.Diversity and RealismDiversity improves generalization, and a model trained on narrow or biased data won’t perform well in production. That’s why we suggest you look for datasets with:Variation in environments, weather, lighting, and anglesRepresentation across different demographics, geographies, and object typesRealistic examples that match your deployment settingFor example, Cityscapes is known for capturing a wide range of urban driving scenarios, making it ideal for autonomous vehicles and pedestrian detection.Use Case FitThe dataset must support your specific application. A project focused on face verification requires different annotations than one focused on optical flow or handwriting recognition.Before committing to a dataset, check:Are the right annotations included? (e.g., segmentation masks, temporal data, point clouds)Does the format align with your tooling? (COCO JSON, Pascal VOC XML, TensorFlow TFRecords, etc.)Is the level of detail sufficient for your model type?The more aligned the dataset is with your use case, the less time you’ll spend converting formats or creating custom labels.Adoption and EcosystemA well-adopted dataset benefits from mature documentation, tooling support, and community contributions. When a dataset is widely used, it’s easier to integrate with frameworks like YOLO.Highly adopted datasets often come with:Active GitHub communitiesPrebuilt loaders and evaluation scriptsLong-term maintenance and version trackingHigh adoption also signals trust. If other teams are using the dataset for training ML models or benchmarking Vision AI systems, it’s more likely to fit into your pipeline without friction.Computer Vision Datasets ComparedEvery dataset plays a different role in how teams build, test, and refine machine learning models. Some focus on broad image classification, while others capture depth, motion, or real-world context for 3D object detection and scene understanding.The table below gives a quick overview of each dataset’s strengths and best uses.DatasetKey StrengthsBest Used ForImageNetOver 14 million labeled images across 21,000 categories. Strong benchmark for classification and transfer learning.Image classification, object recognition, pretraining ML models, face recognition.COCO (Common Objects in Context)330K+ images with detailed bounding boxes, segmentation masks, and captions. Context-rich, multi-object scenes.Object detection, instance segmentation, pedestrian detection, scene recognition, optical flow validation.Open Images Dataset (by Google)9M+ images with 600+ categories, 15M bounding boxes, and 2.8M segmentation masks. Cloud-scale and diverse.Large-scale model training, 3D object detection, object recognition, handwriting recognition, transfer learning.Pascal VOC20-class dataset with bounding boxes and segmentation masks in VOC XML format. Simple and lightweight.Model prototyping, educational projects, small-scale image segmentation and detection tests.LVISOver 1,000 fine-grained categories with long-tail coverage and 2M+ masks. COCO-compatible JSON format.Instance segmentation, fine-grained classification, long-tail recognition, rare-object detection.ADE20K25K+ images with pixel-level annotations for 150 categories, covering both “stuff” and “object” classes.Semantic segmentation, scene parsing, AR/VR model training, 3D face recognition, synthetic data validation (Unreal Engine).ImageNetImageNet is the cornerstone of modern computer vision. Introduced in 2009 by researchers at Princeton and Stanford, it provided the foundation for nearly every major breakthrough in deep learning and visual recognition over the last decade. Containing over 14 million labeled images across more than 21,000 categories, it became the standard benchmark for training and evaluating image classification models.Data FormatEach image in ImageNet is annotated with an image-level label corresponding to a WordNet hierarchy concept. The dataset also includes bounding boxes for over one million images, allowing it to support object detection and localization tasks. The files are typically organized by category folders, making them easily exportable for formats like COCO JSON, TensorFlow TFRecords, or Pascal VOC XML.Key FeaturesLarge-scale dataset covering diverse object categoriesHierarchical labeling system aligned with WordNetAvailability of both classification and detection subsetsSupported by nearly all modern frameworks (PyTorch, TensorFlow, MXNet)Used as a pretraining source for transfer learning in downstream tasksBest Use CasesPretraining for deep learning models in classification and object recognitionTransfer learning for custom datasets and domain adaptationBenchmarking model performance against established standardsFine-tuning tasks like scene recognition, face verification, or image captioningProsExtremely large and diverse datasetUniversally supported across frameworksStrong benchmark for visual recognition modelsEnables faster convergence during model trainingConsLacks domain-specific or multimodal annotationsSome images are outdated or low-resolutionLimited segmentation or 3D data supportLicensing restrictions for certain research usesCurrent RelevanceWhile newer datasets have emerged, ImageNet continues to hold immense value. Its influence is evident in how most Vision AI and generative model pipelines still begin with ImageNet pretraining. Even synthetic datasets are often validated against ImageNet accuracy benchmarks.ImageNet also continues to be cited in thousands of academic papers annually and has appeared in over 40,000 research papers and 250 patents, reflecting its ongoing importance across academia and industry.Even as models evolve toward multimodal and generative architectures, it continues to serve as the baseline reference for training, validation, and performance benchmarking in the field.COCO (Common Objects in Context)The COCO dataset, or Common Objects in Context, is one of the most widely used computer vision datasets for object detection, instance segmentation, and keypoint tracking. Released by Microsoft in 2014, it set a new benchmark for real-world image understanding by emphasizing the importance of context. Rather than focusing on isolated objects, COCO captures how multiple objects interact within complex scenes, making it far more representative of real-world environments.Data FormatCOCO contains over 330,000 images, with more than 1.5 million object instances labeled across 80 core categories. Each image is annotated using COCO JSON format, which supports detailed metadata including segmentation masks, keypoints, and bounding boxes. It also includes captions and labels for image captioning and visual relationship tasks, expanding its utility beyond detection.Key FeaturesRich annotations for object detection, keypoint estimation, and segmentationContext-driven images showing multiple overlapping objectsBuilt-in captions for image captioning and visual recognition tasksFine-grained instance segmentation masksBest Use CasesObject detection and instance segmentationImage captioning and visual question answeringKeypoint estimation and human pose detectionScene recognition and relationship modelingBenchmarking performance for Vision AI and autonomous driving modelsProsHigh-quality, richly annotated datasetComprehensive support for multiple vision tasksStrong compatibility with open-source pipelines and frameworksRemains a universal benchmark across research and industryConsLimited category set compared to datasets like LVIS or Open ImagesFocuses primarily on everyday objects, lacking domain-specific scenesComputationally demanding for model training due to annotation densityCurrent RelevanceAs of 2025, COCO remains one of the most cited and actively used computer vision datasets worldwide, appearing in over 60,000 academic papers in a single year. Its structured format, visual diversity, and consistent annotation standards make it an indispensable resource for anyone developing deep learning models in vision-related tasks.From YOLO and Faster R-CNN to newer architectures like SAM and Ultralytics YOLO11, nearly every major object detection and segmentation benchmark is measured on COCO.Open Images Dataset (by Google)The Open Images Dataset, developed by Google, is one of the largest and most comprehensive computer vision datasets available today. First released in 2016 and continually expanded through multiple versions, it was designed to bridge the gap between image-level classification and fine-grained object detection, segmentation, and visual relationship understanding. Its goal was to create a dataset that could support every stage of modern computer vision development from pretraining and model validation to object recognition and scene analysis.Data FormatThe dataset contains over 9 million images, each annotated with image-level labels and, for a subset, bounding boxes and segmentation masks. It supports a wide range of file formats, including COCO JSON and TensorFlow TFRecords, making it compatible with most ML frameworks. The Open Images V7 release added detailed object relationships, human pose annotations, and localized narratives for image captioning.Key FeaturesOver 600 object categories with bounding boxes for 15 million objects2.8 million instance segmentation masksImage-level labels for over 19,000 visual conceptsAnnotations for object relationships and human posesPublicly hosted through Google Cloud for large-scale accessBest Use CasesLarge-scale model training for image classification and object detectionInstance segmentation and visual relationship modelingBenchmarking Vision AI or multimodal model performanceData augmentation and transfer learning across multiple visual domainsProsMassive dataset with rich, multi-level annotationsCovers a wide range of visual categories and contextsExcellent interoperability with standard formats and frameworksSupported by cloud-hosted infrastructure and community toolsConsHigh storage and computational requirementsAnnotation inconsistencies in certain object categoriesLess suitable for domain-specific or specialized use casesSome subsets require Google authentication for accessCurrent RelevanceOpen Images continues to play a crucial role for teams developing large-scale AI and Vision AI pipelines. Its scale and variety make it ideal for training deep learning models that require high visual diversity and balanced label distribution. Because it integrates both instance segmentation and image-level labeling, it remains useful for general-purpose computer vision tasks and multimodal pretraining.As of 2025, it has appeared in over three thousands research papers and remains a reference point for deep learning and computer vision research across both academia and enterprise.Pascal VOCThe Pascal Visual Object Classes (VOC) dataset is one of the earliest and most influential benchmarks in computer vision. Released between 2005 and 2012 as part of the PASCAL Visual Object Challenge, it helped standardize how researchers evaluate tasks like object detection, classification, and segmentation. Although smaller in scale compared to modern datasets like COCO or Open Images, Pascal VOC remains a cornerstone for model benchmarking and algorithm development.Data FormatPascal VOC includes roughly 20 object categories across thousands of labeled images. Each file comes with annotations in the Pascal VOC XML format, which defines bounding boxes, segmentation masks, and image-level labels. It’s widely supported by frameworks such as TensorFlow, PyTorch, and Keras, and it remains a go-to dataset for educational and prototype-level projects due to its simplicity and accessibility.Key Features20 well-defined object classes for detection and segmentationClear annotation standards in XML formatIncludes both image classification and pixel-level segmentation tasksConsistent train, validation, and test splits for fair benchmarkingLightweight dataset size for fast experimentationBest Use CasesTraining and benchmarking small to medium-sized modelsEducational and academic computer vision researchModel prototyping and pretraining before large-scale deploymentObject detection and semantic segmentation experimentsProsEasy to download, interpret, and integrateLightweight, making it ideal for rapid testingCompatible with a wide range of frameworks and export formatsHistorically important for evaluating visual recognition systemsConsLimited scale and class diversityLacks the contextual depth of modern datasetsNo support for complex relationships or 3D objectsOutdated for large-scale model training and evaluationCurrent RelevanceDespite its age, Pascal VOC remains one of the most recognized names in the field. Its influence extends to nearly every major dataset released since, and its simple, structured annotations continue to teach new generations of data scientists the fundamentals of computer vision dataset design.It’s widely used in academic settings for introducing new architectures or validating lightweight models before scaling up to larger datasets like COCO. The Pascal VOC format also remains foundational, with many modern datasets, such as Cityscapes and Open Images that borrow its structure and export compatibility.While it may no longer set state-of-the-art benchmarks, its influence persists in transfer learning, model validation, and open-source frameworks. Many real-world projects still use Pascal VOC as a quick and reliable dataset for initial model training or small-scale proof-of-concept experiments.LVISThe Large Vocabulary Instance Segmentation (LVIS) dataset was introduced to address a critical limitation in earlier benchmarks like COCO: the lack of diversity and long-tail representation. Developed by researchers from Facebook AI Research, LVIS builds on the COCO dataset but dramatically expands the number of object categories and annotations, making it ideal for fine-grained object detection and instance segmentation.Data FormatLVIS includes over 1,000 object categories across approximately 160,000 images. Each image contains detailed instance segmentation masks, bounding boxes, and object-level annotations stored in JSON format. The dataset structure is COCO-compatible, allowing seamless use with the same APIs, frameworks, and annotation tools. It is also organized to capture both frequent and rare object classes, enabling balanced model training for long-tail distributions.Key FeaturesOver 1,200 object categories, from common to rare classesMore than 2 million segmentation masks with precise boundariesCompatibility with COCO APIs and annotationsInclusion of long-tail and fine-grained object categoriesDesigned to improve generalization in real-world visual recognition tasksBest Use CasesInstance segmentation and object detectionLong-tail recognition and fine-grained classificationTransfer learning and domain adaptation researchBenchmarking generalization and model robustnessProsLarge number of detailed object categoriesExcellent representation of rare and fine-grained classesCompatible with existing COCO tools and pipelinesIdeal for testing generalization and open-vocabulary modelsConsMore complex and computationally intensive than COCOImbalanced category distribution can complicate trainingLimited support for non-visual or 3D dataAnnotation errors may appear in rare classesCurrent RelevanceBy 2026, LVIS will be one of the most important datasets for training deep learning models that need to handle a wide variety of object types. It’s widely used for research in instance segmentation, open-vocabulary detection, and fine-tuning models for edge cases in autonomous vehicles, robotics, and scene understanding.LVIS’s structure also makes it particularly useful for transfer learning, as models trained on LVIS tend to perform better on datasets with rare or domain-specific objects. With the rise of open-vocabulary and multimodal AI systems, LVIS continues to be a standard dataset for evaluating how well models generalize beyond high-frequency object classes.ADE20KThe ADE20K dataset, short for the ADE (Annotated Dataset) for Scene Parsing, is one of the most comprehensive resources for semantic segmentation and scene understanding. Developed by MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), it focuses on parsing complex scenes with pixel-level precision. Unlike datasets centered on individual objects, ADE20K provides a holistic understanding of both foreground and background elements within an image.Data FormatADE20K contains over 25,000 images with detailed pixel-level annotations across 150 object and stuff categories. Every image is manually annotated to include all visible objects and regions, ensuring accurate scene segmentation. The dataset is distributed in a format compatible with COCO JSON and Pascal VOC XML, making it easy to integrate with popular frameworks for training deep learning models. It also includes pre-defined train, validation, and test splits for reproducibility and benchmarking.Key FeaturesPixel-level semantic segmentation for 150 object categoriesCovers both “object” and “stuff” classes for complete scene parsingManually annotated by trained professionals for precisionCompatible with major frameworks like TensorFlow, PyTorch, and Detectron2Frequently used for training segmentation models such as DeepLab and PSPNetBest Use CasesSemantic segmentation and scene parsingTraining and evaluation of segmentation and panoptic modelsBenchmarking transformer-based architectures for visual understandingValidation for synthetic or domain-adapted datasetsProsHigh-quality, dense annotations with strong accuracyBalanced coverage of object and environmental categoriesMaintained by a reputable academic research groupIdeal for developing and benchmarking segmentation modelsConsLimited dataset size compared to Open Images or COCOFocuses mainly on segmentation, with no instance or 3D dataComputationally heavy due to detailed pixel-level labelingCurrent RelevanceAs of 2025, ADE20K remains a top benchmark for semantic segmentation and scene recognition research. Its fine-grained annotations make it essential for developing models that must interpret complex, multi-object environments, particularly in fields like robotics, autonomous driving, and aerial image segmentation. Even as new datasets emerge, it continues to define what “high-quality segmentation data” looks like in the era of large-scale Vision AI and multimodal learning.Examples of Use Cases for Computer Vision DatasetsEvery team uses computer vision datasets differently. For some, it’s about training models that recognize products on a shelf. For others, it’s about helping a car see the road or a robot understand its surroundings. So to shed a bit more light on how they’re used, let’s look at some of the most common and emerging applications shaping the future of computer vision today.1. Image Classification and Object RecognitionThis remains the entry point for most computer vision systems. Datasets like ImageNet, COCO, and Open Images have become industry benchmarks for training models to recognize objects, people, and scenes in real-world contexts.These datasets are ideal for applications such as:Product recognition in retail and e-commerceVisual search and image taggingQuality inspection in manufacturingFace or gesture recognition in security systemsImageNet provides broad visual diversity for general classification, while COCO adds contextual depth through overlapping objects and captions. LVIS and Pascal VOC are excellent choices for refining recognition models on more detailed or long-tail object categories.2. Object Detection and Instance SegmentationFor models that need to locate and classify multiple objects in a single frame, COCO, LVIS, and Open Images remain the gold standard. These datasets feature dense annotations, segmentation masks, and bounding boxes that teach AI to interpret complex, multi-object environments.Key applications include:Autonomous retail checkout and shelf monitoringCrowd and pedestrian detectionIndustrial defect and anomaly detectionWildlife tracking and environmental monitoringLVIS extends COCO’s capabilities with over 1,000 categories, supporting fine-grained detection and rare-object recognition, while Open Images helps scale detection to millions of diverse scenes.3. Scene Understanding and Semantic SegmentationWhen the goal is to help AI understand the entire scene, datasets like ADE20K and Cityscapes are indispensable. They include pixel-level labels for every region of an image, allowing models to learn spatial relationships and contextual awareness.Use cases include:Smart city infrastructure and traffic analyticsAR/VR environment mappingInterior design and robotics navigationAerial and satellite imagery analysisADE20K covers both “stuff” (background) and “object” classes for scene parsing, while Cityscapes provides high-resolution street-level data ideal for autonomous vehicles.4. 3D Object Detection and Spatial MappingDatasets like KITTI, nuScenes, and Matterport3D power AI systems that must understand depth, motion, and geometry. These are critical for self-driving cars, drones, and robots operating in 3D space.They support tasks such as:LiDAR and sensor fusion for autonomous drivingDrone-based warehouse mappingRobotics path planning and obstacle detectionDepth estimation and 3D reconstructionKITTI and nuScenes combine LiDAR, radar, and camera data to train robust perception models, while Matterport3D provides high-quality 3D scans for indoor spatial analysis.5. Medical Imaging and Healthcare AIIn medical data annotation, computer vision datasets help accelerate diagnosis and automate complex visual analysis. Datasets such as LIDC-IDRI, BraTS, and CheXpert provide expertly labeled scans across radiology and pathology disciplines.Common applications include:Tumor segmentation and lesion detection3D organ modeling and reconstructionDisease classification and triage systemsAutomated medical image review and workflow optimizationThese datasets mirror the structure of segmentation datasets like ADE20K, but focus on medical-specific modalities such as CT and MRI.What You Need to Know to Choose the Right Dataset for Your ProjectEvery great computer vision model starts with a decision: what data should it learn from? And that choice matters more than most people realize, as the dataset shapes how your model sees the world, what it pays attention to, and how well it performs once it faces the real thing.If you’re building something practical, start with the classics. ImageNet and COCO are perfect for object recognition, pedestrian detection, or face recognition projects where variety and accuracy matter. But as models grow more specialized, many teams are moving beyond general-purpose datasets to ones built for specific challenges, like Open Images V7 for large-scale training, KITTI for 3D object detection, or ADE20K for scene understanding. And for projects where no ready-made dataset quite fits, the next step is to collect and label their own data. That’s where CVAT can really make a difference.With CVAT, teams can turn raw data into structured, ready-to-train datasets tailored to their exact use case. You can upload images or videos, organize them into datasets, and apply consistent, high-quality annotations using tools like bounding boxes, polygons, segmentation masks, and keypoints. Once complete, datasets can be exported in formats compatible with TensorFlow, PyTorch, and other ML frameworks, making it easy to move from data preparation to model training without friction.If you’re ready to start building, CVAT gives you everything you need to manage, label, and refine your datasets in one place. Use CVAT Online if you prefer a managed cloud solution with no setup required, offering access to advanced labeling and automation features.Set up CVAT Community if you want a self-hosted, open-source version that provides full control and customization.Or, choose CVAT Enterprise if your organization needs a secure, scalable, feature-rich, self-hosted solution with professional support and tailored integrations.

Announcing the new Ultralytics YOLO support for automatic annotation via CVAT agents. Powerful computer vision libraries such as Ultralytics YOLO, Detectron2, and MMDetection have made it easier to train high-performing models for a wide variety of tasks. However, using these models for automated annotation often requires custom code, format conversions, and one-off integrations, especially when labeling workflows span multiple tasks. As a result, many teams fall back on manual labeling because they find automation too complex to adopt at scale. Ultralytics YOLO is one of the most widely used model families in the computer vision community. Until now, CVAT included a single built-in YOLO model for auto-annotation, but expanding beyond that required manual setup. That's why we're excited to announce our new integration with Ultralytics YOLO via the CVAT AI annotation agent. Introducing the new Ultralytics YOLO and CVAT integration With this new integration, you can use native Ultralytics models (YOLOv5, YOLOv8, YOLO11) and third-party YOLO models with Ultralytics compatibility (YOLOv7, YOLOv10, etc.) for automatic image or video annotation for a wide range of computer vision tasks, including: Classification Object detection Instance segmentation Oriented object detection Pose estimation Just pick a YOLO model you want to label your dataset with, connect it to CVAT via the agent, run the agent, and get fully labeled frames or even entire datasets, complete with the right shapes and attributes, and all, in a fraction of the time. Annotation possibilities unlocked This integration opens up multiple workflow optimization and automation opportunities for ML and AI teams. Here are just a few. Pre-label data using the right model for the task Connect the YOLO models that match your annotation goals and run them sequentially to pre-label your data. Each model can be triggered individually through the CVAT interface, allowing you to generate different types of labels for the same dataset without custom scripts or external tools. This works for any YOLO model, out-of-the-box or fine-tuned. Label entire tasks in bulk Working with a large dataset? You don’t have to annotate each frame manually. Apply a YOLO model to the entire task in one step. Just open the Actions menu in your task and select Automatic annotation. CVAT will send the job to the agent and automatically annotate all frames across all jobs in a task, saving you time and reducing repetitive work. Share models across teams and projects Register a model once via a native function and agent, and make it instantly available across your organization in CVAT. Team members can use it in their own tasks without any local setup. Validate model performance on real data Test your fine-tuned YOLO model directly on annotated datasets and compare its predictions side-by-side with human labels in CVAT. Spot mismatches, edge cases, or underperforming classes, all without leaving your annotation environment. How it works Here’s what a typical YOLO auto-annotation setup via agents looks like: Step 1. Write and register the function Start by implementing a native function–a Python script that loads your YOLO model (e.g., yolov8n, yolo11m-seg) and defines how predictions will be generated and returned to CVAT. Then register this function in CVAT using the CLI. Note: You can reuse the same native function both in CLI-based annotation and agent-based mode. Step 2. Start the agent Once the function is registered, launch an agent using the CLI command. This starts a local service that automatically connects to your account in CVAT Online or Enterprise, and listens for annotation requests from CVAT. The agent then runs the model (inside your function), generates predictions, and sends them back to CVAT. For more in-depth information about how to set up automated data annotation with a YOLO or any custom model using a CVAT AI agent, read this article. Step 3. Create or select a task in CVAT Log into your CVAT instance and create a new task (or select an existing one). Upload your images or video, and define the labels you want to annotate (e.g., "person", "car", "helmet"). Depending on your use case, you can define different types of labels such as bounding boxes, polygons, or skeletons to match the expected output from your model. Step 4. Choose the model in the UI Once your task and the job are created and the agent is running, go to the AI Tools panel inside your job. Select the Detector tab and the YOLO model you registered earlier. Step 5. Run AI annotation on selected frames After selecting the model, CVAT sends a request to the running agent. The agent runs the model and returns predictions in the form of shapes (e.g., boxes, polygons, or keypoints), each associated with a label ID. Get started now Ready to speed up your annotation workflow with YOLO? Sign in to your CVAT Online account and try it out yourself. For more information about Ultralytics YOLO models and the tasks they support, check the Ultralytics documentation page. For more information about CVAT AI annotation agents, visit Announcing CVAT AI Agents: A New (and Better) Way to Automate Data Annotation using Your Own Models

The 10 Biggest AI & Computer Vision Conferences in 2026For business leaders, data engineers, and researchers, attending an AI or computer vision conference in 2026 should be a top priority. These gatherings showcase breakthroughs in generative AI models, deep learning, and annotation, but the biggest value of attending a computer vision or AI conference lies in their ability to connect you with industry leaders. Through expert sessions, workshops, and hands-on demonstrations, attendees get the chance to work and learn AI strategies from the top thought leaders in the space.In the next section, we break down the 10 biggest computer vision and AI conferences in 2026, giving you a roadmap to maximize your learning, networking, and innovation opportunities.‍The Top AI & Computer Vision Conferences at a GlanceWe will go over each conference in more detail later in the article. But if you want a quick overview of the 10 biggest conferences, the table below will help.‍‍The Top Computer Vision ConferencesComputer vision sits at the heart of today’s most exciting advances in artificial intelligence. From powering autonomous driving to enabling smarter recommendation engines and behavior prediction, this field is shaping how we see and interact with the world.Below, we highlight the most influential gatherings that set the direction for research and business applications alike.CVPR (Conference on Computer Vision and Pattern Recognition)CVPR is the world’s premier computer vision and artificial intelligence conference, attracting thousands of researchers, engineers, and business leaders each year.Organized by the IEEE and CVF, it serves as a launchpad for groundbreaking work in deep learning, behavior prediction, motion planning, remote sensing, and autonomous driving. The conference balances academic rigor with applied innovation, making it one of the most influential AI conferences globally.Date: June 3-7, 2026.Location: Denver, Colorado.Virtual Access: Live streamed keynotes and access to recorded tutorials, workshops, and proceedings, though not all sessions are streamed.Focus / Who Should Attend:CVPR is ideal for researchers, data engineers, and companies investing in computer vision, generative AI, and AI strategies.Attendees gain exposure to expert sessions, tutorials, and workshops covering high-performance computing, edge computing, and data analytics, with direct applications in fields like autonomous driving, network optimization, and customer experience.International Conference on Computer Vision (ICCV)ICCV is one of the most prestigious global gatherings in computer vision and machine learning, held every two years. It brings together academics, researchers, and industry innovators to present cutting-edge work on topics like deep learning models, motion planning, and autonomous driving.Date: October 2026.Location: To be announced (previous editions have rotated worldwide, including Paris, Seoul, and Venice).Virtual Access: access to live streams and replays, but paper presenters are generally required to attend in person.Focus / Who Should Attend:ICCV is best suited for researchers, PhD students, and organizations seeking to explore theoretical advancements alongside applied breakthroughs.Expect intensive paper presentations, tutorials, and workshops designed for those shaping the next generation of AI strategies.European Conference on Computer Vision (ECCV)ECCV is Europe’s flagship AI conference for computer vision, alternating years with ICCV. It emphasizes both foundational theory and practical applications in natural language processing, remote sensing, and data analytics.Date: September 8-13, 2026.Location: Malmö, Sweden.Virtual Access: Includes virtual passes with online session access and recordings, while accepted authors must usually register in person.Focus / Who Should Attend:ECCV is ideal for European researchers, startups, and business leaders looking to collaborate on projects that link vision with AI-driven business value. ECCV offers a strong mix of academic sessions, poster presentations, and networking opportunities.Embedded Vision SummitThe Embedded Vision Summit focuses on practical applications of computer vision and edge computing in real-world products. It highlights how vision systems are transforming industries like robotics, healthcare, and autonomous driving, with a strong emphasis on deployment at scale.Date: May 11-13, 2026.Location: Santa Clara, California (Silicon Valley).Virtual Access: Offers a virtual pass that streams keynotes, selected technical sessions, and on-demand recordings.Focus / Who Should Attend:The Embedded Vision Summit is perfect for engineers, product managers, and business leaders building vision-enabled solutions. Attendees gain insights into system design, AI strategies, and deep learning innovations that deliver measurable business value.The event also features an expo showcasing the latest hardware and software tools for accelerating development.‍The Top AI ConferencesAI conferences provide a front-row seat to breakthroughs in machine learning, generative AI, and high-performance computing, while also showcasing the latest tools and AI strategies for real-world impact.The following conferences connect industry leaders, researchers, and business leaders, offering both technical depth and strategic insight.NVIDIA GTCNVIDIA GTC is one of the most influential global events for artificial intelligence and high-performance computing. It covers everything from generative AI and deep learning to quantum computing and data engineering.With a mix of expert sessions, hands-on labs, and keynote addresses from leaders at NVIDIA and partners like Google DeepMind, the conference highlights both research breakthroughs and real-world business applications.Date: March 16-19, 2026.Location: San Jose, California.Virtual Access: Fully supports virtual attendance with live keynotes, technical talks, training, and on-demand replay access.Focus / Who Should Attend:NVIDIA GTC was designed for data engineers, developers, researchers, and business leaders seeking to translate AI into measurable business value.It’s particularly valuable for those working in cloud conferences, AI strategies, and industries pushing the limits of autonomous driving, network optimization, and scientific insights.AAAI Conference on Artificial Intelligence (AAAI)AAAI is one of the longest-running and most respected gatherings in artificial intelligence. It serves as a bridge between academic research and applied innovation, featuring sessions on machine learning, natural language processing, generative AI, and emerging areas like agentic AI.The conference emphasizes both technical rigor and societal impact, including ethics and public trust.Date: January 20-27, 2026.Location: Singapore.Virtual Access: Runs as a hybrid event where virtual attendees can join livestreams of talks and panels and access recordings.Focus / Who Should Attend:AAAI offers deep dives into theory, workshops on applied systems, and panels connecting industry leaders with academia to shape the future of intelligent systems. Making it best suited for researchers, graduate students, and business leaders looking to understand the future of AI strategies.Data + AI SummitThe Data + AI Summit, hosted by Databricks, is the leading event at the intersection of data engineering, machine learning, and artificial intelligence. It showcases advances in data analytics, generative AI models, and real-world use cases that demonstrate the direct business value of unified data and AI strategies.Date: June 15-18, 2026.Location: San Francisco, California.Virtual Access: Provides both in-person and virtual registration, with streaming of sessions, training, and on-demand access.Focus / Who Should Attend:The Data + AI Summit is ideal for data engineers, analysts, scientists, and business leaders who want to harness the power of AI in enterprise settings.The event combines technical workshops, expert sessions, and keynote talks, making it a must-attend for organizations aiming to scale AI strategies and drive innovation in cloud and enterprise systems.Rise of AIRise of AI is Europe’s flagship artificial intelligence conference, centered on the societal, business, and ethical dimensions of AI adoption.While it includes technical tracks on machine learning and deep learning, its main strength lies in connecting industry leaders, policymakers, and entrepreneurs to discuss the long-term impact of AI strategies on business and society.Date: May 5-6, 2026.Location: Berlin, Germany.Virtual Access: Offers hybrid participation with livestreamed talks and recordings available to virtual ticket holders.Focus / Who Should Attend:Rise of AI is a strong fit for business leaders, policymakers, and executives exploring the strategic and regulatory aspects of AI.Attendees benefit from thought-leadership keynotes, panels on public trust and governance, and networking sessions designed to link startups, enterprises, and government stakeholders.HumanXHumanX is a high-impact AI conference where enterprise leaders, innovators, and capital converge to turn AI into real outcomes. It emphasizes bridging the gap between research, product strategy, and business deployment—anchoring programming by job function rather than just technology.Date: April 6‑9, 2026.Location: San Francisco, California.Virtual Access: No.Focus / Who Should Attend:HumanX is well suited for senior executives, product leaders, AI strategists, and technical decision‑makers who aim to translate AI from concept to large-scale business impact. Sessions and experiences focus on function‑driven tracks (e.g. builders, command desk, customer engine, ecosystem) and facilitate buyer‑seller matchmaking, peer exchange, and 1:1 networkingIBM ThinkThink is IBM’s flagship technology conference, with artificial intelligence, cloud computing, and high-performance computing at its core. The event blends thought leadership with technical depth, highlighting use cases in generative AI, network optimization, and quantum computing.Date: May 2026.Location: Las Vegas, Nevada.Virtual Access: Provides a strong virtual option with live streams of keynotes, breakout sessions, and replay libraries.Focus / Who Should Attend:Think is ideal for enterprise executives, IT decision-makers, and technical teams. Attendees gain insights from IBM and partner experts on AI strategies, data analytics, and the Power of Networks.The event also offers hands-on labs and solution showcases for those seeking enterprise-grade applications of AI.‍What Conference Do We Suggest You Attend?Each of these conferences brings something unique, whether it’s the research depth of CVPR, the enterprise focus of Data + AI Summit and Think, or the strategic dialogue at Rise of AI. Together, they highlight how fast artificial intelligence and machine learning are moving from theory into practice, creating real business value across industries.The key takeaway? Pick conferences that match your goals. If you’re a researcher, events like ICCV or AAAI will sharpen your technical expertise. If you’re a business leader, forums such as NVIDIA GTC or Embedded Vision Summit will show you how to turn innovation into strategy. And if you’re looking for cross-disciplinary insights, gatherings like AI Con and ECCV bridge the gap between academia and applied impact.In short, the question isn’t which AI conference should you attend, it’s which one will best accelerate your growth, network, and vision for the future.

Artificial Intelligence is only as good as the data it learns from. In the world of machine learning, “garbage in, garbage out” isn’t just a cliché, it’s a business risk. That's because poorly labeled data doesn’t just slow down your models; it can quietly sabotage entire AI initiatives. This is where data annotation quality metrics like Precision, Recall, and Accuracy come in. Far more than academic formulas, these metrics help you avoid costly annotation errors, and produce the kind of ground truth data that keeps AI models reliable at scale. For beginners, mastering these three concepts is a gateway into understanding both the performance of your models and the integrity of your training datasets. What Are Data Annotation Quality Metrics? Data annotation quality metrics provide a structured way to evaluate how well your data labeling efforts are performing. They highlight whether labels are consistent, comprehensive, and aligned with the ground truth needed for model training. For annotation teams, these metrics act as both a diagnostic tool and a roadmap for improvement, ensuring that data quality issues are identified early. Think of it like this. Whenever a job boils down to making repeatable decisions with verifiable outcomes, a human effectively functions like a classifier or detection model. That means we can evaluate the quality of those decisions with model metrics: precision (how often actions are right), recall (how many true cases we catch), and accuracy (overall hit rate). Precision Precision measures the proportion of correctly labeled items among all items labeled as positive, where "positive" refers to the items that the model has predicted as belonging to the target class (e.g., spam emails, diseased patients, or images containing a specific object). It serves as a measure of label trustworthiness, making it central to quality assurance in any annotation project. It is measured through the following formula: Precision = True Positives (TP) ÷ [True Positives (TP) + False Positives (FP)] Example: Imagine you labeled 10 objects as cats, but only 8 truly are cats. Your precision is 80%, meaning 2 of your labels were false positives. Another thing to note, if you label 1 cat as a cat, the precision will be 100% even if you were unable to find 9 other cats. Why It Matters: In the annotation process, high precision shows that annotators are applying labels accurately and avoiding unnecessary noise. For tasks like autonomous vehicle perception, this translates into fewer mistakes (false positives) in detecting pedestrians, vehicles, or obstacles and greater trust in the models that guide navigation and safety decisions. Recall Recall measures the proportion of actual positive items that were correctly labeled. It reflects how complete your annotations are, ensuring important objects or signals are not overlooked during the annotation process. It is measured through the following formula: Recall = True Positives (TP) ÷ [True Positives (TP) + False Negatives (FN)] Example: Suppose there are 10 cats in an image, butand you labeled only 8 of them. Your recall is 80%, meaning 2 cats were missed. Why It Matters: High recall indicates that annotators are capturing the majority of relevant items, reducing the risk of missing critical data. In fields like autonomous vehicles or sentiment analysis, missing positive cases can severely undermine model performance and introduce hidden data quality issues. Accuracy Accuracy measures the overall proportion of correctly labeled items, including both true positives and true negatives, across the entire dataset. A true negative refers to an item that is not the target class and is correctly left unlabeled. For example, if you are labeling cats, a dog that remains unlabeled counts as a true negative. Overall, accuracy provides a broad view of data annotation quality but can sometimes mask problems when working with imbalanced classes. It is measured through the following formula: Accuracy = (True Positives (TP) + True Negatives (TN)) ÷ (TP + TN + False Positives (FP) + False Negatives (FN)) For clarity, a true negative refers to an item correctly left unlabeled—e.g., when labeling “cats,” a dog that is not labeled counts as a true negative. Example: If you label 100 items and 90 of them are correct, whether labeled or unlabeled, your accuracy is 90%. However, this might hide issues. For instance, in object detection , a model might only detect 90 out of 100 objects (10 false negatives) and mislabel another 10 (10 false positives), resulting in the same 90% accuracy. But that doesn’t reflect the underlying errors. Why It Matters: Accuracy gives a quick, high-level snapshot of annotation performance, but it can be misleading if one class dominates the dataset. In highly imbalanced scenarios, like when one class intentionally makes up 99% of examples, skipping annotations for rare classes won’t drastically hurt overall accuracy, yet the resulting dataset can be highly unreliable. To get a clearer picture, accuracy should be paired with class-level metrics like precision and recall for each category. Connecting the Metrics Precision, Recall, and Accuracy each highlight different aspects of data annotation quality. Precision emphasizes trustworthiness, showing whether labels avoid unnecessary false positives. Recall emphasizes completeness, ensuring important objects or signals are not missed. Accuracy offers a big-picture view, showing the overall correctness of annotations. Used together, these metrics provide a balanced perspective on how well your annotation process is performing and create the foundation for annotation QA, linking day-to-day labeling decisions to long-term model performance. Why These Metrics Matter for Annotation QA Formulas alone don’t capture the real value of Precision, Recall, and Accuracy. Their importance lies in how they connect day-to-day labeling decisions with long-term model reliability. Direct Impact on Model Performance Annotation quality metrics directly determine whether an AI system performs reliably in production. Precision, Recall, and Accuracy quantify how annotation decisions ripple through the entire ML pipeline, exposing weaknesses before they undermine results. When these metrics are ignored, the consequences quickly surface in real-world applications: Low Precision → Too many false positives. In retail and ecommerce, this could mean mistakes in product classification, shelf monitoring, logo detection, and attribute tagging for catalogs. Low Recall → Missed signals. In autonomous vehicles, failing to detect pedestrians or traffic signs poses serious safety risks. Misleading Accuracy → Inflated performance. In sentiment analysis, accuracy may appear high simply because neutral reviews dominate, while misclassified positives and negatives go unnoticed. By monitoring these metrics, data scientists and annotation teams can identify weaknesses in the annotation workflow early. This ensures training data does not silently introduce bias, inflate the error rate, or degrade model performance. Consistency and Objectivity High-quality data annotation is not just about labeling correctly; it’s about labeling consistently across annotation teams, projects, and time. Without consistency, even well-labeled data can become unreliable, creating hidden biases that degrade model performance. Precision, Recall, and Accuracy provide the objective lens needed to measure and maintain this consistency. These metrics reduce reliance on subjective reviewer opinions and bring structure to quality assurance. Key ways these metrics improve consistency include: Standardizing Quality Checks → Metrics provide a common benchmark for all annotators, ensuring alignment with the intended annotation scheme. Reducing Subjectivity → Instead of relying on gut feeling, teams can use numbers to decide whether an annotation meets the gold standard. Comparing Performance → Metrics reveal differences in accuracy between annotators or teams, highlighting where additional QA steps or manual inspections are needed. Enabling Reproducibility → With consistent measurement, results can be validated across projects, supporting reproducibility checklists and reducing the risk of data drift. By embedding these metrics into the annotation workflow, organizations gain both transparency and control, making data annotation quality measurable, repeatable, and scalable. Ground Truth Validation Even with strong annotation processes, you need a reliable way to measure new data against a gold standard dataset. This is where Precision, Recall, and Accuracy become essential. They provide the framework for comparing fresh annotations to trusted ground truth labels, ensuring that new data is not drifting away from established quality benchmarks. Ground truth validation acts as a safeguard, keeping the annotation process aligned with project goals. Key benefits of applying metrics to ground truth validation include: Error Detection → Quickly identifies false positives and false negatives in new annotations. Confidence Intervals → Provides reliable quality estimates, ensuring that data annotations meet required sample sizes for validation. Bias Monitoring → Highlights systematic issues, such as ambiguous annotation guidelines or skill gaps among annotators. Long-Term Quality Control → Detects data drift by checking if new labels remain consistent with established ground truth data. Specification Validation → Reveals gaps or ambiguities in the data annotation specification itself, helping teams improve guidelines and reduce repeated errors. By embedding ground truth validation into the annotation workflow, teams transform QA from a one-time checkpoint into an ongoing quality management process that scales with industry-level AI applications. How CVAT Online & Enterprise Helps You Measure These Metrics Both CVAT’s SaaS and On-prem editions elevate annotation QA from manual guesswork to a streamlined, metrics-driven workflow. Built with high-scale, mission-critical applications in mind, it empowers teams with automated tools, modular validation options, and analytics that map directly to precision, recall, and accuracy. Consensus-Based Annotation CVAT Enterprise enables the creation of Consensus Replica jobs, where the same data segment is annotated by multiple people independently. These replica jobs function just like standard annotation tasks: they can be assigned, annotated, imported, and exported separately. Why it Matters: Reduces Bias & Noise: By merging multiple perspectives, consensus helps eliminate outlier annotations and noise. Improves Ground Truth Reliability: Ideal for validating especially important samples within your dataset. Cost-Efficient Quality Control: Achieve robust ground truth with minimal additional annotation workload. Consensus matters because high-quality machine learning models require trusted data to benchmark against. When ground truth annotations are unavailable or too costly to produce manually, consensus provides a practical path forward. By merging multiple annotations with majority voting, CVAT creates reliable ground truth that strengthens precision, recall, and accuracy scores in automated QA pipelines. Learn more about consensus-based annotation here. Automated QA Using Ground Truth & Honeypots CVAT Enterprise enables automated quality assurance through two complementary validation modes: Ground Truth jobs and Honeypots. Ground Truth jobs are carefully curated, manually validated annotation sets that serve as reliable reference standards for measuring accuracy. Honeypots build on this by embedding Ground Truth frames into regular annotation workflows without the annotator’s awareness, enabling ongoing quality checks across the pipeline. The process of using this in CVAT is simple and scalable: A Ground Truth job is created within a task, annotated carefully, and marked as “accepted” and “completed,” after which CVAT uses it as the quality benchmark. In Honeypot mode, CVAT automatically mixes validation frames into annotation jobs at task creation, allowing continuous and unobtrusive QA sampling. By comparing annotator labels against the trusted Ground Truth, CVAT calculates precision, recall, and accuracy scores automatically. This ensures quality is monitored in real time while reducing the need for manual spot checks, giving teams confidence in both speed and consistency at scale. Learn more about automated QA with ground truth and honeypots here. Manual QA and Review Mode CVAT also includes a specialized Review mode designed to streamline annotation validation by allowing reviewers to pinpoint and document errors directly within the annotation interface. Why It Matters: Focused Error Identification: The streamlined interface ensures that annotation flaws aren’t overlooked amidst complex tools. Structured Feedback Loop: QA findings are clearly documented and addressed systematically through issue assignment and resolution. Improved Annotation Reliability: By resolving human-reported mistakes, your dataset’s precision, recall, and overall trustworthiness are actively enhanced. This mode elevates quality assurance beyond metrics, adding a human-in-the-loop layer for nuanced judgment. Learn more about Manual QA and Review Mode in CVAT Enterprise here. Summary & Takeaways Precision, Recall, and Accuracy are more than academic metrics, they are the backbone of annotation quality assurance. Together, they provide a balanced view of trustworthiness, completeness, and overall correctness in labeled data. High annotation quality directly translates into more reliable, fair, and high-performing AI systems, reducing costly errors and mitigating risks like data drift. If you want to track and measure annotation quality at scale, CVAT provides the consensus workflows, automated QA, and analytics dashboards to make it happen. Try it for free today and see how we make training data more trustworthy, annotation teams more efficient, and AI models more reliable.

Far too often, AI and ML projects begin with a model-first mindset. Teams dedicate talent and compute into tuning architectures and experimenting with deep learning techniques, while treating the dataset as fixed. But this quickly leads to a painful realization. All that time you spent and all those GPUs you utilized to build the model is often worthless, because model performance is impacted less by design and more by the quality of the labels. This is where data-centric AI changes the conversation. Instead of assuming data is static, it treats annotation quality (consistent & accurate labels), dataset creation, and ongoing curation as the real drivers of reliable outcomes. Data-Centric AI vs. Model-Centric AI: Understanding the Difference AI has long been built on a model-centric foundation in which researchers optimized architectures and fine-tuned parameters while treating datasets as static. That mindset produced progress in controlled benchmarks, but it often fails in production. What is Model-Centric AI? Model-centric AI is the traditional way many teams have tried to improve artificial intelligence systems. Progress in this model-centric era was possible largely because massive datasets like ImageNet became available, enabling deep learning models to achieve breakthroughs despite imperfections in the data. Some common traits of a model-centric approach include: Focusing on algorithm tweaks, not annotation quality Relying on compute power rather than improving training data Producing models that look strong in testing but fail under distribution shift One key thing to note in model-centric AI is that data is treated as fixed, so whatever annotations exist are simply taken as-is. The downside of this approach is that in real-world computer vision tasks, datasets are rarely clean. In computer vision applications, for example, label errors, class imbalance, and noise in the dataset often cause failures. A good illustration of these failures comes from medical image classification. In datasets like OrganAMNIST and PathMNIST, researchers found that mislabeled images and class imbalance significantly lowered accuracy. This resulted in the model failing to distinguish between known conditions, which is especially harmful in healthcare settings. Because of that, a shift is underway to a data-centric approach, which prioritizes annotation quality and dataset curation to provide more sustainable results. What is Data-Centric AI? Data-centric AI flips the priority, and instead of pushing models harder, it focuses on improving the dataset itself. In data-centric AI, datasets are never fixed. Labels are audited, refined, and expanded to capture real-world variation. Subtle issues, like a scratch on a phone screen or a flaw in a medical device, are annotated precisely so they are not missed in deployment. The advantage is that models trained on curated, high-quality data perform more reliably. They continue to work when real-world data looks different from the training data, reduce false rejections in inspection tasks, and achieve higher accuracy even in smaller data regimes. Some common traits of a data-centric approach include: Prioritizing annotation accuracy and consistency across annotators Using active learning to surface uncertain samples for review Applying confident learning to detect and correct mislabeled data Treating data curation and dataset creation as ongoing, iterative processes Comparing datasets over time to measure progress in quality In short, data-centric AI does not replace models but strengthens them. By prioritizing annotation quality instead of model performance, organizations create a stronger foundation for their AI models. We recommend watching this video from Andrej Karpathy for added information. Why Label Quality Often Beats Model Complexity The contrast between model-centric and data-centric AI makes one thing clear: model performance ultimately depends on the quality of the labels it learns from. This is because every AI system is only as strong as the data it learns from. While deep learning and advanced model architecture get attention, the reality is simple: if the training data is flawed, performance will suffer. High-quality AI models depend on accurate labels and the quality of the images themselves. Poor image resolution, inconsistent samples, or unrepresentative datasets can weaken even the strongest models. At CVAT, we focus on delivering precise annotations, while our partners like FiftyOne help teams select and curate the right data, ensuring that both images and labels contribute to stronger, more reliable AI systems. Garbage Labels = Garbage Predictions When annotations are incorrect, the model has no way to understand it is incorrect, so mislabeled or inconsistent data becomes baked into the predictions. This means a model trained on poor labels doesn’t just perform badly, it scales those mistakes across every deployment. In computer vision tasks, these consequences are costly. For example, a mislabeled defect in a medical device inspection dataset can lead to false rejections, while confusing scratches with cracks in consumer electronics creates unreliable quality checks. This is why no amount of deep learning complexity or hyperparameter tuning can undo bad data. Poor labels also increase the risk of overfitting, where a model learns noise instead of useful patterns. High-Quality Labels Enable Reliable Feedback Loops If bad labels lock errors into every prediction, high-quality annotations do the opposite. They create a foundation for models to improve. When labels are consistent and precise, they allow models to signal where they are uncertain, enabling teams to act on that feedback. This is where feedback loops become powerful, as accurate datasets make it possible to: Run active learning to identify ambiguous samples for re-labeling Apply confident learning to detect hidden annotation mistakes Each cycle of annotation, model training, and review builds stronger data assets, which means that instead of scaling mistakes, teams scale improvements. The result is an AI system that adapts better to distribution shifts, reduces false rejections in inspection tasks, and delivers more trustworthy outcomes in production. Practical Steps to Improve Labeled Data Quality If poor labels create unreliable models and high-quality labels enable feedback loops, the obvious question is, “How can you improve your data label quality?” Well, it often comes down to an ongoing process of review, refinement, and curation. To get started, here are three practical steps any team can take. 1 - Use Active Learning to Target Low-Confidence Labels If annotation errors are the root of unreliable AI, the solution lies in focused review. The key here is targeting the data labels most likely to contain errors. This may sound like a challenging manual process, but with CVAT, teams can integrate active learning to automatically surface low-confidence samples for re-labeling. CVAT also takes this one step further by: Flagging low-confidence samples generated by model outputs or heuristic checks Providing “Immediate job feedback” via validation‑set (ground truth) jobs to catch errors early. These practices help teams surface and correct mistakes efficiently, making human effort count where it matters most. 2 - Balance and Expand The Dataset Even perfectly labeled data loses value if one category dominates. This is because class imbalance creates blind spots that models can’t recover from. CVAT helps by making it straightforward to ingest new data into active projects. You can import full datasets or annotation files, including images, videos, and project assets, directly in the UI, then continue labeling within the same workspace. If your imbalance involves spatial understanding, CVAT also supports 3D point cloud annotation with cuboids, so you can increase coverage for classes that are underrepresented in 3D scenarios like robotics or logistics. 3 - Track Progress Through Accurate Analytics Improving annotation quality only matters if you can measure it. Without reliable metrics, teams can’t see whether label consistency is improving, class balance is shifting, or error rates are dropping. This lack of visibility makes it difficult to justify investments in annotation or to understand why models still fail in production. Thankfully, CVAT provides built-in analytics and automated QA to make progress transparent and actionable. These include: Ground truth jobs and auto-qa tools to compare annotations against reference subsets and catch errors early Analytics dashboards to monitor annotation speed, class distribution, and annotator agreement over time By embedding measurement into everyday workflows, CVAT turns datasets into evolving assets. Data-Centric AI Helps You Get the Most from Your AI Investments The reality of artificial intelligence is clear: most failures don’t come from model design, they come from inaccurate data. And while chasing improvements through a model-centric approach of grid search, hyper parameter tuning, and ever-larger architectures may boost results on benchmarks, it rarely solves real-world problems caused by poor label quality. This is why a data-centric approach is more sustainable. By investing in annotation quality, ongoing data management, and techniques like data augmentation or active learning, teams strengthen their foundation and avoid scaling mistakes into production. With CVAT, you can put a data-centric approach into practice. From annotation to data management and quality measurement, CVAT gives your team the tools to fix label errors, improve datasets, and build AI systems that actually perform in the real world. Want to see how CVAT Works? Try CVAT Online, CVAT Enterprise, and the CVAT Community version now.

TL;DR One-step user off-boarding: remove a user with all their data via a new Django CLI command. Smarter label schema checks: the Raw Labels Editor now blocks invalid configs before they hit the server. Smoother reviews: Issue dialogs auto-reposition to stay fully visible near frame edges. Cleaner APIs: deleted_frames in job meta now only reports frames from the current job segment. Lower backend load: preview requests for Projects/Tasks/Jobs are sent sequentially to avoid spikes. Added Delete a user together with all their resources (admin CLI) Context: Adds a safe server-side flow to remove a user and all associated data in one go. Impact: Admins can fully deprovision accounts without manual cleanup. Built-in validation prevents unsafe deletions (e.g., community users still in orgs with other members; SaaS users with active subscriptions). Example: python manage.py deleteuser <user_id> Changed Raw Labels Editor: stronger validation Context: UI now catches malformed/unsupported label configurations earlier. Impact: Fewer bad payloads reaching the backend; clearer inline feedback for editors. No action needed unless your current config is invalid. Preview requests sent sequentially to reduce server load Context: Fetching previews for Projects/Tasks/Jobs could spike concurrency. Impact: More stable performance under load; slightly lower peak concurrency for preview calls. Fixed Issue dialog never opens off-screen Context: In review mode, dialogs near frame corners could render outside the viewport. Impact: Dialogs now auto-reposition and remain fully visible. No more zooming out to find them. deleted_frames in job meta constrained to the job segment Context: API could return deleted_frames outside the job’s segment in /jobs/{id}/data/meta. Impact: Client code relying on this field now receives only relevant frames. If you worked around the previous behavior, you can simplify your client logic. Notes: Includes a test for video tasks with frame step > 1.

In the world of machine learning and neural networks, annotations are more than labels. They are the ground truth that shapes how models learn, adapt, and perform. For example, in an autonomous driving system, correctly labeling trees and other road hazards helps the neural network distinguish between safe and unsafe obstacles, allowing it to make accurate, split-second decisions when navigating complex traffic scenarios. At its core high-quality annotations guide the learning algorithm, reduce biases, and improve inference. And with the right annotation strategies and smart tools like CVAT to streamline and refine the process, teams can create training datasets that lead to better model performance. Why High-Quality Data is Critical for Model Training Annotation High-quality annotations are essential because they directly shape how a machine learning model understands its training data. Without precise and consistent labels, even the most advanced architecture, whether it is a convolutional neural network for image recognition or an object detection model for computer vision, will misinterpret patterns. This leads to lower accuracy, poor inference, and unreliable results when the model is deployed. Impact on Model Training When annotations are precise, consistent, and aligned with the project’s objectives, the model can learn the correct patterns and relationships within the dataset. This results in stronger generalization, faster convergence, and higher performance in production. However, when annotations are flawed, the model’s ability to interpret data is compromised. This is where the Garbage In, Garbage Out (GIGO) principle comes into play. No matter how sophisticated the architecture or training algorithm, if the labeled data fed into the model is inaccurate or inconsistent, the output will be equally unreliable. Poor annotations can lead to overfitting, underfitting, or skewed model bias, all of which degrade predictive performance. Key effects of annotation quality on model training include: The accuracy of learned patterns and feature recognition Model confidence scores during inference Overall training time and convergence speed Increasing or reducing the risk of overfitting to mislabeled data Enhancing or limiting the model’s ability to generalize to unseen scenarios Ultimately, investing in high-quality annotations is the foundation for a model’s ability to deliver reliable, actionable insights. Without this, even large datasets and advanced algorithms will fail to produce dependable results. (e.g., models trained on the refined COCO‑ReM dataset converge faster and score higher than those using original COCO annotations) Examples of Errors from Poor Annotation In machine learning, even small annotation mistakes in the training dataset can have significant downstream effects on an ML model’s behavior. During data preparation, these errors often remain hidden, only to surface during the inference stage when the machine learning algorithm must evaluate and predict in real-world conditions. For instance, in medical classification tasks, an incorrectly annotated MRI scan, such as tagging a benign tumor as malignant, could trigger unnecessary treatment, impacting both the patient and the healthcare production system. Other impacts of annotation errors include: Model confusion on edge cases – Neural network architectures misinterpret visually similar features or overlapping classes. Reduced confidence scores – The training model produces low model confidence scores during evaluation, weakening performance metrics. Propagation of bias – Systematic labeling errors introduce biases into the learning algorithm and affect correlation patterns. Misclassification in critical applications – Incorrect predictions in safety-critical domains such as predictive maintenance, financial fraud detection, or medical diagnostics. Poor generalization – The ML model fails to adapt to new datasets, validation sets, or unseen scenarios in production. Addressing these risks requires rigorous validation systems, manual review of annotations, and re-trainings supported by quality-controlled data pipelines. This can be done through platforms like CVAT, which have built-in audit trails and annotation consistency checks and which can be integrated with TensorFlow or PyTorch workflows to ensure the training set remains accurate, helping the learning algorithm reach optimal performance in both classification and regression tasks. The Role of High-Quality Data in Neural Network Training Neural networks are only as strong as the data that teaches them. In machine learning, annotations transform raw data into meaningful training signals, giving the learning algorithm the context it needs to recognize patterns, classify inputs, and make accurate predictions. Annotation as a Source of Truth In supervised learning, high-quality annotations serve as the ground truth that a machine learning algorithm depends on to learn from its training set. Whether the task involves image recognition, binary classification, or regression, accurate labels guide the training process, allowing the model to identify features, optimize hyperparameters, and improve predictive performance. When your annotated data is accurate, the outputs are more reliable, the loss function converges faster, and performance metrics improve in both training and validation sets. Key benefits of treating annotations as a source of truth include: Providing a reliable foundation for ML model training and re-trainings Reducing confusion in decision trees, random forest, and neural network architectures Improving generalization in deep learning models, from CNNs to large language models Supporting effective cross-validation for more robust predictions Enabling reproducible workflows for distributed training across GPUs and cloud platforms By incorporating manual review, annotation guidelines, and consistent quality checks into the workflow, teams can ensure annotations remain accurate over time. Bias and Noise Reduction Even the most advanced deep learning architectures can fail if the training dataset is filled with bias or noise. Poorly annotated data skews the learning algorithm’s understanding of correlations, causing systematic errors that can harm both model accuracy and customer trust. High-quality annotation reduces these risks by ensuring consistency across the training set and minimizing human error in the labeling process. Whether using supervised learning for classification or unsupervised learning methods like k-means clustering, accurate labels help the model predict with greater confidence and adapt to varied test data in production systems. Here are some ways that precise annotations help reduce bias and noise: Maintaining consistent class definitions across the entire training dataset Eliminating mislabeling that introduces false correlations into the regression model or clustering algorithms Preventing performance degradation in models used for critical tasks like predictive maintenance or spam detection Improving gradient descent convergence by reducing variability in the training process Supporting re-trainings and model evaluation cycles that catch emerging biases early By combining automated machine learning pipelines with manual review, federated data strategies, and scalable annotation tools, teams can deliver ground truth data that is free from systemic bias. How High-Quality Annotation Workflows Are Built — and How CVAT Helps High-quality annotation workflows don’t happen by accident. They are the result of structured processes, clear guidelines, and reliable tools. CVAT supports these workflows by providing a collaboration platform that enables accuracy, scalability, and quality control for every stage of the machine learning data labeling process. Redundancy and Consensus for Reliability Using multiple annotators per task helps achieve consensus on labels, reducing the likelihood of errors in the training dataset. CVAT allows for configurable redundancy so ML model training benefits from diverse perspectives while maintaining accuracy through agreement-based labeling. The benefits of this include: Fewer mislabeled examples in the training set Stronger ground truth accuracy for supervised learning Reduced bias in machine learning model outputs Annotation Consistency Over Time Consistency ensures that features are labeled the same way across the training set. CVAT supports predefined label sets (such as object categories like car, pedestrian, traffic light) and annotation guidelines (which define how these labels should be applied, including attributes or tagging rules). These guidelines make it easier for distributed teams to maintain uniformity in supervised learning workflows. Built-In Quality Control CVAT provides several automated QA and control mechanisms that help catch annotation errors early and streamline the training process. These include: Ground Truth (GT) jobs: A curated validation set is used as a benchmark, allowing statistical evaluation of annotation quality across the dataset. Honeypots: Hidden validation frames are randomly inserted into jobs, helping monitor annotation accuracy in real time without alerting annotators. Immediate Job Feedback: Once a job is completed, CVAT automatically evaluates it against GT or honeypots and shows the annotator a quality score with the option to correct mistakes immediately. Traceability and Audit Trails Traceability is critical for large-scale datasets, which is why CVAT has an analytics page for all your project data, ensuring teams can track events, annotations and more. This transparency is essential for model evaluation, regulatory compliance, and maintaining customer trust. Flexible Workflows for Diverse Data From image recognition to 3D sensor data, CVAT adapts to different data types and annotation styles. Flexible task management, support for multiple annotation formats, and integration with ML pipelines make it suitable for varied AI and deep learning applications. Best Practices for Ensuring High-Quality Data The structured workflows above provide the foundation for producing reliable training data at scale. But building a strong annotation pipeline is only part of the equation. Maintaining quality over time requires discipline, clear standards, and continuous evaluation. By following these proven best practices, teams can ensure that every dataset, whether used for image recognition or predictive modeling, remains accurate, consistent, and free from bias. Follow Annotation Guidelines Clear, documented annotation guidelines are essential for ensuring that every label in a training set is applied consistently, regardless of who is doing the work. Without them, inconsistencies can creep in, creating noise in the training data and reducing the accuracy of the machine learning model. A good place for a CTA block How to implement clear annotation guidelines: Define each label class with clear descriptions and examples. Specify rules for handling edge cases and overlapping classes. Document attributes that need to be captured (for example, color, orientation, or object state). Ensure guidelines are regularly updated and accessible to all annotators. For a deeper breakdown of what makes strong labeling specifications, see CVAT’s guide on creating data labeling specifications. Conduct Annotation Review Cycles Even experienced annotators make mistakes, which is why conducting annotation review cycles is so critical. Review cycles help catch errors early, before flawed data is used in ML model training. How to conduct annotation review cycles: Schedule periodic reviews of completed annotations. Use CVAT’s Immediate Job Feedback to automatically evaluate annotations against ground truth or honeypots and provide annotators with instant quality scores. Assign multiple reviewers for critical or complex datasets. Use feedback loops to train annotators on corrections. CVAT’s built-in review mode allows reviewers to approve, reject, or edit annotations in real time. Task assignment tools ensure the right people review the right data, while commenting features make feedback easy to share. Perform Continuous Model Evaluation A dataset is never truly “finished.” Models improve when their training data is updated and re-evaluated. Continuous model evaluation measures whether annotation improvements are actually boosting accuracy, reducing loss, and improving performance metrics. How to perform continuous model evaluation: Benchmark model performance before and after annotation updates. Track changes in accuracy, precision, and recall over time. Re-train models when new patterns or edge cases are identified. CVAT makes model evaluation easy. We integrate with ML pipelines so teams can quickly export labeled datasets, test them in TensorFlow or PyTorch, and compare results. Plus, our version control ensures teams can roll back to previous annotations and measure improvement with confidence. The Data-Driven Path to Smarter Annotations Adopting best practices like clear guidelines, structured review cycles, and continuous model evaluation helps you build a feedback-driven workflow where data constantly informs better decisions. CVAT makes this easier by giving organizations the tools to embed their standards into the workflow, collaborate effectively across teams, and integrate seamlessly with existing ML pipelines. Because in a world where machine learning models are judged by their ability to perform in real-world scenarios, data labeling quality is non-negotiable. The more accurate your ground truth, the more reliable your predictions, and the faster your AI projects deliver real value. Don’t let poor annotations hold your models back. Join the thousands of teams using CVAT to build better datasets, streamline labeling, reduce errors, and deliver higher-performing AI faster. Get started now.

In early June, Business Insider and other outlets reported that Scale AI had left at least 85 Google Docs publicly accessible, which exposed thousands of pages of confidential AI (Artificial Intelligence) and ML (Machine Learning) project materials tied to clients like Meta, Google, and xAI. These documents included internal training guidelines, proprietary prompts, audio examples labeled "confidential," and even contractor performance data and private email addresses. The leak wasn’t just an operational lapse, it highlighted a growing risk that AI and ML teams can no longer ignore. When confidential data is exposed, it can compromise AI model integrity, violate data agreements, and erode your competitive advantage. Scale AI’s Leak Is a Cautionary Tale for the Industry The Scale AI breach serves as a stark warning for enterprises across the AI and ML landscape. When third-party vendors entrusted with high-value training data leaves sensitive documents publicly accessible, it reveals systemic lapses in access control and data security While no malicious breach occurred, the scope of exposure reveals deep systemic risks that extend far beyond simple oversight. Four Key Risks Surfaced From the Leak: Confidential Client Data Exposure: Documents related to AI model training for companies like xAI and Bard (Google) were accessible to anyone with the link. These files detailed labeling instructions and dataset structures, exposing sensitive technical workflows and proprietary methodologies. Private Contractor Data Leaked: Spreadsheets with names, performance metrics, and work history of global annotation contractors were included in the leak. This not only violates privacy laws like GDPR but also risks long-term reputational harm and trust breakdown with the human workforce underpinning AI development. Editable Files Created Tampering Risks: Several documents were not only viewable but editable. This opened the door to potential sabotage from altering instructions, inserting malicious data, or deleting critical content, all without any authentication barrier. IP Leakage from Client Datasets: When proprietary data structures, labeling schemas, and annotation logic become public, they reveal how a client frames its machine learning problems. This can offer competitors rare insight into AI training projects, domain assumptions, and even model behavior. In short, this incident highlights a glaring truth: as AI systems scale, so do the stakes of even minor security lapses. Meta–Scale AI: When Your Labeling Vendor Becomes Your Competitor The story doesn’t stop there though. On June 12th, 2025 Meta announced a $14.3 billion investment in Scale AI that sent shockwaves through the AI industry, not just for its size, but for what it implies. By taking a 49% stake in one of the most widely used data labeling vendors, Meta didn’t just acquire a service provider. It embedded itself deep within the data supply chains of competing AI labs and enterprise teams. This immediately raised uncomfortable questions. If you previously entrusted Scale with sensitive internal data, how confident are you that none of that institutional knowledge, annotation strategy, or model-adjacent metadata could now inform Meta’s roadmap? Even with supposed conflict-of-interest firewalls in place, the optics are difficult to ignore. What happens when your annotation pipeline is owned, in part, by the company you're trying to out-innovate? Clients like Google, OpenAI, and xAI reportedly began distancing themselves from Scale AI within days of the deal’s announcement. That reaction speaks volumes. And for many other enterprise AI leaders, this is a moment to re-evaluate who they trust at the most sensitive layers of their model development process. Why Most Annotation Pipelines Are a Breach Waiting to Happen The Scale AI leak revealed just how easily annotation workflows can become a liability when basic security principles are overlooked. Despite handling sensitive datasets and proprietary model inputs, most annotation workflows remain poorly secured. Here are some of the most common and critical vulnerabilities. Orphaned Credentials from Ex-Contractors The use of rotating contractor pools is standard in annotation, but many teams fail to offboard users properly. Scale AI’s leak reportedly included internal documentation still accessible long after project completion, a sign of credential sprawl and poor deactivation hygiene. Common failure points include: No automated credential expiration or deactivation Shared logins reused across projects Lack of audit trails to flag old or unused accounts Inadequate Identity Verification Across Roles and Regions In many global labeling operations, the pressure to scale quickly and cut costs often comes at the expense of trust and traceability. Identity verification is frequently treated as an afterthought, with teams relying on manual invites or generic user accounts to onboard annotators. For example, Scale AI’s documents were accessible through public links, some editable by anyone, which clearly highlights the lack of verified, accountable user access. This leads to several security gaps, including: Weak or absent identity checks before access is granted No multi-factor authentication Inability to map user actions to verified individuals No Fine-Grained Project-Level Access Without project-level controls, users may see more than they should. The Scale AI leak showed internal materials were accessible by too many annotators, contractors, and managers. This lack of compartmentalization creates several common security gaps, including: Broad access across unrelated client projects Inability to isolate users to specific datasets No role-based constraints on actions like exporting or editing Why Treating Annotation Like “Just Labeling” Is a Mistake Annotation is often seen as a routine step in the ML pipeline, but the Scale AI leak shattered that assumption. It showed that the labeling layer is not just vulnerable, but deeply entangled with a company’s intellectual property, strategic intent, and model logic. And when organizations treat annotation as a disposable service, they often neglect essential safeguards around identity verification, access control, and data governance. This mindset is risky. In Scale AI’s case, loosely controlled labeling environments reportedly left project-specific instructions, confidential prompts, and internal guidelines open to the public. That level of exposure can’t be undone, and the consequences extend far beyond a single document. For enterprise AI teams, the message is clear. Labeling workflows must be treated like any production-critical environment: monitored, locked down, and governed by principle, not convenience. Anything less invites unnecessary risk. What Enterprise ML Leaders Should Do to Protect Themselves The Scale AI leak made one thing painfully clear: the weakest part of your AI pipeline can compromise everything else. For enterprise ML teams who want to prioritize data security, that means treating the annotation environment as a high-risk, high-value component of the stack. So how can you prevent your own data from becoming the next headline? It starts with a security-first approach grounded in the following enterprise principles: Centralized Identity Management One of the clearest failures in the Scale AI leak was access control. Documents were left open, sometimes editable, with no clear identity attribution. In an ideal environment, every annotator, reviewer, and admin should authenticate through a centralized identity provider. This reduces credential sprawl, enables automatic deactivation, and ensures access can be instantly revoked when roles change or contracts end. Role-Based Access Controls (RBAC) The Scale AI leak exposed files including sensitive labeling guidelines and client-specific project data, material that should never be visible across teams or contractors. RBAC enforces boundaries by ensuring users only access the specific projects, tools, and data required for their role. This limits unnecessary exposure and contains potential damage if a breach occurs. Auditable Activity Every action within the annotation environment should be logged and traceable. Without logging audit trails, you can't know when a breach occurred or who was responsible. In the Scale AI case, the trail of accountability was murky at best. Enterprise annotation environments must track all user activity, from file access to export actions, to support compliance, detect threats, and respond quickly to any incident. These security pillars are not optional for enterprise-grade AI projects. They’re the baseline for responsible, resilient machine learning pipelines, especially as the value and sensitivity of training data continues to rise. How CVAT Enterprise Supports Secure Annotation at Scale If you want to project yourself, CVAT Enterprise is the perfect option. The first critical advantage of CVAT Enterprise is that it removes the risk of vendor lock-in. If a supplier disappears or changes their terms, customers do not have to search for a new data annotation platform. Reason being? Around 90% of CVAT’s features are available in the open-source version under the permissive MIT license, ensuring long-term access and control. Plus, for CVAT Enterprise customers, even private modules are fully inspectable. This transparency allows organizations to verify code security, meet internal compliance requirements, and maintain complete control over their data workflows. Getting started with CVAT is equally simple. There’s no need to contact procurement teams or wait for a sales process to run its course. Companies can download and test the open-source version immediately, evaluate its capabilities, and determine if it meets their needs. Beyond the advantages listed above CVAT Enterprise offers numerous security features to meet the demanding operational needs of modern AI and ML teams. Scalable Identity Management CVAT integrates with enterprise-grade identity systems including SSO, LDAP, and SAML. This allows teams to manage access through their existing identity infrastructure, eliminating siloed logins and reducing the risk of outdated or orphaned accounts. It also enables consistent enforcement of security policies such as password strength, session limits, and multi-factor authentication across the entire organization. Granular Role-Based Access Control (RBAC) With CVAT’s fine-grained RBAC and group-level permissions, organizations can tailor access by role, team, and project. Internal staff, external contractors, and QA reviewers can each be granted exactly the level of access they require. This limits the spread of sensitive information and protects high-value datasets from accidental exposure or unauthorized use. Flexible, Secure Deployment Options CVAT supports both on-premise and air-gapped deployments, giving security-conscious teams complete control over their infrastructure. For organizations operating in regulated industries or with strict data residency requirements, this means training data can remain fully contained within internal networks and compliance zones. These features make CVAT not just a powerful annotation tool, but a secure foundation for enterprise-scale AI development. Recommended Actions to Protect Your Annotation Pipeline The Scale AI leak is a warning about the fragility of unsecured data privacy in ML workflows. Prudent enterprise AI teams must take these proactive steps to secure their annotation environments before exposure turns into damage. Audit Your Annotation Pipeline The first step to securing your pipeline is performing a comprehensive review of your current workflows, tools, and access points. Understand where your vulnerabilities lie and address them systematically. Key steps include: Take inventory of all annotation platforms, datasets, and projects in use List all active and inactive users, including third-party contractors Identify accounts with excessive or outdated access rights Map how data flows between teams, tools, and storage systems Mandate SSO and IAM Integration Next, you need to tightly control who can access your annotation systems by enforcing centralized identity management. This ensures consistent access policies and faster response to personnel changes. Recommended actions: Require SSO, LDAP, or SAML for all annotation tools Disable platforms that do not support enterprise IAM Integrate access provisioning with your IT team’s existing workflows Automatically revoke credentials upon contract termination or offboarding Treat Annotation Like a Production System As you move forward, it’s a good idea to treat annotation environments similar to production environments, as they both contain sensitive data that powers production models. This data must be secured, monitored, and governed accordingly. The best practices for this include: Enable full activity logging and keep detailed audit trails Monitor for anomalies such as unusual login times or data exports Enforce role-based access and restrict permissions by project Conduct periodic access reviews and compliance checks These steps aren’t just for damage control, they are core to building resilient, secure, and trustworthy AI systems. Training Smarter Means Securing Sooner For AI and ML leaders, the lesson is clear: waiting until a breach occurs is too late. If your labeling workflows are open, unmonitored, or loosely governed, then your entire AI system is vulnerable. So don’t wait until another data breach occurs, now is the time to act. That means rethinking how you manage, secure, and monitor every part of the annotation process. This is where CVAT Enterprise comes in. As an enterprise-ready platform, CVAT gives teams the tools they need to protect high-value training data without slowing down the annotation process. With support for centralized identity, role-based permissions, and secure deployment options, CVAT Enterprise helps organizations label smarter and safer. Don’t wait for your training data to become tomorrow’s headline. Secure your annotation workflow today with CVAT Enterprise.

Summer is in full bloom, and we haven’t slowed down. Here’s a quick roundup of our team’s July deliveries across CVAT Online, Enterprise, and Community platforms, and more. CVAT Academy After months of preparation, we’re excited to launch CVAT Academy! CVAT Academy is a hands-on online course to help you and your team master key annotation tools and techniques in CVAT, from your first bounding box to advanced workflows. We’ve published the first module on core CVAT annotation tools like bounding boxes, polygons, and AI tools. You can watch them on the CVAT Academy page or YouTube and leave your likes and comments. Your feedback is valued, and we’ve already made some tweaks based on user input. The best part: after talking to our customers and soon-to-be annotators, we decided to make this course completely free! So, whether you want to improve your annotation skills or onboard new members faster, you can do it at no cost with CVAT Academy. New features (Releases 2.41 + 2.42) SAM2 Object Tracking via AI Agents (CVAT Online) CVAT Online users can now automatically track shapes across video frames with SAM2 or any other tracking model, custom or pre-trained, using AI agents. Read more Improved Navigation on Listing Pages (All Platforms) On listing pages like Tasks, Jobs, Projects, and Models, users can choose how many items to display per page: 10, 20, 50, or 100, and quickly jump to a specific page by entering its number. These updates enhance browsing large datasets and make navigation more efficient. Quick Edit from List Views (All Platforms) We’ve added a dynamic Edit option for Task, Job, and Project list pages. Users can now update key fields directly from the list view without opening each item: Assignee can be changed for tasks, jobs, and projects. State and Stage can be updated for jobs. This simplifies routine updates and reduces extra clicks. Other Changes & Fixes (All Platforms) Enabled multi-threaded image downloading from cloud storage when preparing chunks, enhancing performance. Resolved COCO keypoints export issues with absent keypoints. Updated the organization Actions menu to match the style of other menus. Enabled shortcuts configuration in tag annotation mode. Ensured “Automatically go to the next frame” setting applies correctly when adding the first tag in the tag annotation workspace. Improved performance of GET /api/jobs/<id>/annotations and GET /api/tasks/<id>/annotations with many tracks containing mutable attributes. Enforced email verification when using Basic HTTP authentication, if ACCOUNT_EMAIL_VERIFICATION is set to mandatory, preventing unverified access. Updated Python runtime for the Segment Anything Nuclio function from 3.8 to 3.10 to ensure compatibility and support. API: Deprecated legacy token authentication. Updated API token and session/CSRF token auth schemas for simplified and more secure access. The PATCH and PUT endpoints at /api/tasks/<id>/annotations and /api/jobs/<id>/annotations now enforce that annotation IDs are present when updating and absent when creating. This improves consistency and prevents mismatched operations. Updated API schema for session authentication. To get into all the nitty-gritty details, visit our GitHub changelog.

SAM2 Object Tracking Comes to CVAT Online Through AI Agent Integration Previously on this blog, we described the use of the Segment Anything Model 2 (SAM2) for quickly annotating videos by tracking shapes from an initial frame. However, this feature was limited to self-hosted CVAT Enterprise deployments. We have also covered using arbitrary AI models via agents and auto-annotation functions to annotate a CVAT task from scratch. Today we’ll talk about a new CVAT feature that combines the benefits of the two approaches: tracking support in auto-annotation (AA) functions. This enables each user of CVAT Online to make use of an arbitrary tracking AI model by writing a small wrapper (AA function) around it, and running a worker process (AI agent) on their hardware to handle requests. In addition, we have implemented a ready-to-use AA function based on SAM2, so that users who want to make use of that particular model can skip the first step and just run an agent. In this article we will explain how to use the SAM2-based AA function, as well as walk through some of the implementation details. Quick start Let’s get started. You will need: Installed Python (3.10 or a later version) and Git. An account at either CVAT Online or an instance of CVAT Enterprise version 2.42.0 or later. First, clone the CVAT source repository into some directory on your machine. We’ll call this directory <CVAT_DIR>: git clone https://github.com/cvat-ai/cvat.git <CVAT_DIR> Next, install the Python packages for CVAT CLI, SAM2 and Hugging Face Hub: pip install cvat-cli -r <CVAT_DIR>/ai-models/tracker/sam2/requirements.txt If you have issues with installing SAM2, note that the SAM2 install instructions contain solutions to some common problems. Next, register the SAM2 function with CVAT and run an agent for it: cvat-cli --server-host <CVAT_BASE_URL> --auth <USERNAME>:<PASSWORD> \ function create-native "SAM2" \ --function-file=<CVAT_DIR>/ai-models/tracker/sam2/func.py -p model_id=str:<MODEL_ID> cvat-cli --server-host <CVAT_BASE_URL> --auth <USERNAME>:<PASSWORD> \ function run-agent <FUNCTION_ID> \ --function-file=<CVAT_DIR>/ai-models/tracker/sam2/func.py -p model_id=str:<MODEL_ID> where: <CVAT_BASE_URL> is the URL of the CVAT instance you want to use (such as https://app.cvat.ai). <USERNAME> and <PASSWORD> are your CVAT credentials. <FUNCTION_ID> is the number output by the function create-native command. <MODEL_ID> is one of the SAM2 model IDs from Hugging Face Hub, such as facebook/sam2.1-hiera-tiny. Optionally: Add -p device=str:cuda to the second command to run the model on your NVIDIA GPU. By default, the model will run on the CPU. Add --org <ORG_SLUG> to both commands to share the function with your organization. <ORG_SLUG> must be the short name of the organization; it is the name displayed under your username when you switch to the organization in the CVAT UI. The last command should stay running, indicating that the agent is listening to annotation requests from the server. This completes the setup steps. Now you can try the function in action: Open the CVAT UI. Create a new CVAT task or open an existing one. The task must be created either from a video file or from a video-like sequence of images (all images having the same dimensions). Open one of the jobs from the task. Draw a mask or polygon shape around an object. Right-click the shape, open the action menu and choose “Run annotation action”. Choose “AI Tracker: SAM2” in the window that appears. Enter the number of the last frame that you want to track the object to and press Run. Wait for the annotation process to complete. Examine the subsequent frames. You should now see a mask/polygon drawn around the same object on every frame up to the one you selected in the previous step. Instead of selecting an individual shape, you can also track every mask & polygon on the current frame by opening the menu in the top left corner and selecting “Run actions”. Implementation Now let’s take a peek behind the curtains and see how the SAM2 tracking function works. This will be useful if you need to troubleshoot, or if you want to implement a tracking function of your own. Unfortunately, the source of the module is too long to explain in its entirety in this article, but we’ll cover the overall structure and key implementation features. First, let’s look at the top-level structure of func.py: @dataclasses.dataclass(frozen=True, kw_only=True) class _PreprocessedImage: ... @dataclasses.dataclass(kw_only=True) class _TrackingState: ... class _Sam2Tracker: ... create = _Sam2Tracker Since we wanted to support multiple model variants, as well as multiple devices, with a single implementation, we did not place the function’s required attributes directly in the module. Instead, we put them inside a class, _Sam2Tracker, which we want to be instantiated by the CLI with the parameters passed via the -p option. To tell the CLI which class to instantiate, we alias the name create to our class. There are also two auxiliary dataclasses, _PreprocessedImage and _TrackingState. These are not part of the tracking function interface, but an implementation detail. We will see their purpose later. Let’s now zoom in on _Sam2Tracker. __init__ and spec Similar to detection functions that we’ve covered before, in the constructor we load the underlying model (SAM2VideoPredictor). We also create the PyTorch device object and create an input transform. def __init__(self, model_id: str, device: str = "cpu") -> None: self._device = torch.device(device) self._predictor = SAM2VideoPredictor.from_pretrained(model_id, device=self._device) self._transform = torchvision.transforms.Compose([...]) Also similar to detection functions, our tracker must define a spec, although it has to be of type TrackingFunctionSpec: spec = cvataa.TrackingFunctionSpec(supported_shape_types=["mask", "polygon"]) In a tracking function, the spec describes which shape types the function is able to track. However, the other attributes of _Sam2Tracker are entirely unlike those of detection functions. On a high level, a tracking function must analyze an image with a shape on it, then predict the location of that shape on other images. However, to allow more efficient tracking of multiple shapes per image, as well as to enable interactive usage, this functionality is split across three methods. preprocess_image def preprocess_image( self, context: cvataa.TrackingFunctionContext, image: PIL.Image.Image ) -> _PreprocessedImage: image = image.convert("RGB") image_tensor = self._transform(image).unsqueeze(0).to(device=self._device) backbone_out = self._predictor.forward_image(image_tensor) ... return _PreprocessedImage( original_width=image.width, original_height=image.height, vision_feats=...(... backbone_out...), ..., ) This method is supposed to perform any processing that the function can do without knowing the details of the shape it’s tracking. In this way, the results can be reused for multiple shapes. In our case, the underlying model has a dedicated method for doing this, so we transform our input image, and pass it to this method. We then return all information we’ll need later as a new instance of our class _PreprocessedImage. The agent does not care what type of object is returned by preprocess_image - it just saves that object so it can pass it to the other methods. Speaking of which… init_tracking_state def init_tracking_state( self, context: cvataa.TrackingFunctionShapeContext, pp_image: _PreprocessedImage, shape: cvataa.TrackableShape, ) -> _TrackingState: mask = torch.from_numpy(self._shape_to_mask(pp_image, shape)) resized_mask = ...(... mask ...) current_out = self._call_predictor(pp_image=pp_image, mask_inputs=resized_mask, ...) return _TrackingState( frame_idx=0, predictor_outputs={"cond_frame_outputs": {0: current_out}, ...}, ) def _call_predictor(self, *, pp_image: _PreprocessedImage, frame_idx: int, **kwargs) -> dict: out = self._predictor.track_step( current_vision_feats=pp_image.vision_feats, frame_idx=frame_idx, ... **kwargs, ) return ...(... out ...) This method is supposed to analyze the shape on the initial frame. Here we convert the input shape to a mask tensor (for brevity we’ll omit the definition of _shape_to_mask here), and then pass it, alongside the preprocessed image, to the underlying model (via a small wrapping function). The method then encapsulates all information that will be needed to track the shape on subsequent frames in a new _TrackingState object and returns it. Much like preprocess_image, the agent doesn’t care what type of object the method returns, so the tracking function can choose the type in order to best suit its own needs. The agent will simply pass this object into our final method… track def track( self, context: cvataa.TrackingFunctionShapeContext, pp_image: _PreprocessedImage, state: _TrackingState ) -> cvataa.TrackableShape: state.frame_idx += 1 current_out = self._call_predictor( pp_image=pp_image, frame_idx=state.frame_idx, output_dict=state.predictor_outputs, ... ) non_cond_frame_outputs = state.predictor_outputs["non_cond_frame_outputs"] non_cond_frame_outputs[state.frame_idx] = current_out ... output_mask = ...(... current_out["pred_masks"] ...) if output_mask.any(): return self._mask_to_shape(context, output_mask.cpu()) else: return None This method is supposed to locate the shape being tracked on another frame. Here we pass data from the state object and the preprocessed image to the model and get a mask back. If the mask has any pixels set, we return it as a TrackableShape object. The _mask_to_shape method (whose definition we’ll omit) will convert the mask to a shape of the same type as the original shape passed to init_tracking_state. If the mask is all zeros, we presume that we lost track of the shape, and return None. The model also returns additional data that can be used to better track the shape on subsequent frames. track adds it to the tracking state, as can be seen with the non_cond_frame_outputs update. This way, future calls to track are able to make use of this data. Agent behavior Now that we’ve examined the purpose of each method, we can see how they all fit together by looking at the tracking process from the agent’s perspective. Let’s say an agent has loaded tracking function F, and a user makes a request for shape S0 from image I0 to be tracked to images I1, I2, and I3. In this case, the agent will make the following calls to the tracking function: STATE = F.init_tracking_state(SC, F.preprocess_image(C, I0), S0) S1 = F.track(SC, F.preprocess_image(C, I1), STATE) S2 = F.track(SC, F.preprocess_image(C, I2), STATE) S3 = F.track(SC, F.preprocess_image(C, I3), STATE) It will then return resulting shapes S1, S2, and S3 to CVAT. Here C and SC are context objects, created by the agent. For more information on these, please refer to the reference documentation. Limitations There are a few things to keep in mind when using tracking functions (SAM2 included). First, agents currently keep the tracking states in their memory. This means that: Only one agent can be run at a time for any given tracking function. If you run more than one agent for a function, users may see random failures as agents try to complete requests referencing some other agent’s tracking states. If the agent crashes or is shut down, all tracking states are destroyed. If this happens while a user is tracking a shape, the process will fail. Second, tracking functions can only be used via agents. There is no equivalent of the cvat-cli task auto-annotate command. Third, tracking functions may be used either via an annotation action (as was shown in the quick start), or via the AI Tools dialog (accessible via the sidebar). However, the latter method only works with tracking functions that support rectangles - other functions will not be selectable. Fourth, skeletons cannot currently be tracked. Conclusion Tracking with SAM2 saves significant time compared to manually annotating each frame. If you are a user of CVAT Online, this feature is now available to you - sign in and try it out! If there is another model you’d like to use for tracking, you can likely do that as well, as long as you implement the corresponding auto-annotation function. For more details on that, refer to the reference documentation: https://docs.cvat.ai/docs/api_sdk/sdk/auto-annotation/ https://docs.cvat.ai/docs/api_sdk/cli/#examples---functions For more information on other capabilities of AA functions and AI agents, see our previous articles on the topic: https://www.cvat.ai/resources/blog/an-introduction-to-automated-data-annotation-with-cvat-ai-cli https://www.cvat.ai/resources/blog/announcing-cvat-ai-agents https://www.cvat.ai/resources/blog/cvat-ai-agents-update