- Notifications
You must be signed in to change notification settings - Fork26
A package to read and convert object detection datasets (COCO, YOLO, PascalVOC, LabelMe, CVAT, OpenImage, ...) and evaluate them with COCO and PascalVOC metrics.
License
laclouis5/globox
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
This framework can:
- parse all kinds of object detection datasets (ImageNet, COCO, YOLO, PascalVOC, OpenImage, CVAT, LabelMe, etc.) and show statistics,
- convert them to other formats (ImageNet, COCO, YOLO, PascalVOC, OpenImage, CVAT, LabelMe, etc.),
- and evaluate predictions using standard object detection metrics such as
$AP_{[.5:.05:.95]}$ ,$AP_{50}$ ,$mAP$ ,$AR_{1}$ ,$AR_{10}$ ,$AR_{100}$ .
This framework can be used both as a library in your own code and as a command line tool. This tool is designed to be simple to use, fast and correct.
You can install the package using pip:
pip install globox
The library has three main components:
BoundingBox: represents a bounding box with a label and an optional confidence scoreAnnotation: represent the bounding boxes annotations for one imageAnnotationSet: represents annotations for a set of images (a database)
TheAnnotationSet class contains static methods to read different dataset formats:
# COCOcoco=AnnotationSet.from_coco(file_path="path/to/file.json")# YOLOv5yolo=AnnotationSet.from_yolo_v5(folder="path/to/files/",image_folder="path/to/images/")# Pascal VOCpascal=AnnotationSet.from_pascal_voc(folder="path/to/files/")
Annotation offers file-level granularity for compatible datasets:
annotation=Annotation.from_labelme(file_path="path/to/file.xml")
For more specific implementations theBoundingBox class contains lots of utilities to parse bounding boxes in different formats, like thecreate() method.
AnnotationsSets are set-like objects. They can be combined and annotations can be added:
gts=coco|yologts.add(annotation)
Iterators and efficient lookup byimage_id's are easy to use:
ifannotationingts:print("This annotation is present.")if"image_123.jpg"ingts.image_ids:print("Annotation of image 'image_123.jpg' is present.")forboxingts.all_boxes:print(box.label,box.area,box.is_ground_truth)forannotationingts:nb_boxes=len(annotation.boxes)print(f"{annotation.image_id}:{nb_boxes} boxes")
Datasets stats can printed to the console:
coco_gts.show_stats()
Database Stats ┏━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓┃ Label ┃ Images ┃ Boxes ┃┡━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩│ aeroplane │ 10 │ 15 ││ bicycle │ 7 │ 14 ││ bird │ 4 │ 6 ││ boat │ 7 │ 11 ││ bottle │ 9 │ 13 ││ bus │ 5 │ 6 ││ car │ 6 │ 14 ││ cat │ 4 │ 5 ││ chair │ 9 │ 15 ││ cow │ 6 │ 14 ││ diningtable │ 7 │ 7 ││ dog │ 6 │ 8 ││ horse │ 7 │ 7 ││ motorbike │ 3 │ 5 ││ person │ 41 │ 91 ││ pottedplant │ 6 │ 7 ││ sheep │ 4 │ 10 ││ sofa │ 10 │ 10 ││ train │ 5 │ 6 ││ tvmonitor │ 8 │ 9 │├─────────────┼────────┼───────┤│ Total │ 100 │ 273 │└─────────────┴────────┴───────┘Datasets can be converted to and saved in other formats:
# ImageNetgts.save_imagenet(save_dir="pascalVOC_db/")# YOLO Darknetgts.save_yolo_darknet(save_dir="yolo_train/",label_to_id={"cat":0,"dog":1,"racoon":2})# YOLOv5gts.save_yolo_v5(save_dir="yolo_train/",label_to_id={"cat":0,"dog":1,"racoon":2},)# CVATgts.save_cvat(path="train.xml")
COCO Evaluation is also supported:
evaluator=COCOEvaluator(ground_truths=gts,predictions=dets)ap=evaluator.ap()ar_100=evaluator.ar_100()ap_75=evaluator.ap_75()ap_small=evaluator.ap_small()...
All COCO standard metrics can be displayed in a pretty printed table with:
evaluator.show_summary()
which outputs:
COCO Evaluation┏━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━┳...┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┓┃ Label ┃ AP 50:95 ┃ AP 50 ┃ ┃ AR S ┃ AR M ┃ AR L ┃┡━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━╇...╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━┩│ airplane │ 22.7% │ 25.2% │ │ nan% │ 90.0% │ 0.0% ││ apple │ 46.4% │ 57.4% │ │ 48.5% │ nan% │ nan% ││ backpack │ 54.8% │ 85.1% │ │ 100.0% │ 72.0% │ 0.0% ││ banana │ 73.6% │ 96.4% │ │ nan% │ 100.0% │ 70.0% │. . . . . . . .. . . . . . . .. . . . . . . .├───────────┼──────────┼────────┼...┼────────┼────────┼────────┤│ Total │ 50.3% │ 69.7% │ │ 65.4% │ 60.3% │ 55.3% │└───────────┴──────────┴────────┴...┴────────┴────────┴────────┘The array of results can be saved in CSV format:
evaluator.save_csv("where/to/save/results.csv")
Custom evaluations can be achieved with:
evaluation=evaluator.evaluate(iou_threshold=0.33,max_detections=1_000,size_range=(0.0,10_000))ap=evaluation.ap()cat_ar=evaluation["cat"].ar
Evaluations are cached by(iou_threshold, max_detections, size_range) keys. This means that repetead queries to the evaluator are fast!
If you only need to use Globox from the command line like an application, you can install the package throughpipx:
pipx install globox
Globox will then be in your shell path and usable from anywhere.
Get a summary of annotations for one dataset:
globox summary /yolo/folder/ --format yolo
Convert annotations from one format to another one:
globox convert input/yolo/folder/ output_coco_file_path.json --format yolo --save_fmt coco
Evaluate a set of detections with COCO metrics, display them and save them in a CSV file:
globox evaluate groundtruths/ predictions.json --format yolo --format_dets coco -s results.csv
Show the help message for an exhaustive list of options:
globox summary -hglobox convert -hglobox evaluate -h
Clone the repo with its test data:
git clone https://github.com/laclouis5/globox --recurse-submodules=tests/globox_test_datacd globoxInstall dependencies withuv:
uv sync --dev
Run the tests:
uv run pytest tests
Speed benchmark can be executed with:
uv run python tests/benchmark.py -n 5
The following speed test is performed using Python 3.11 andtimeit with 5 iterations on a 2021 MacBook Pro 14" (M1 Pro 8 Cores and 16 GB of RAM). The dataset is COCO 2017 Validation which comprises 5k images and 36 781 bounding boxes.
| Task | COCO | CVAT | OpenImage | LabelMe | PascalVOC | YOLO | TXT |
|---|---|---|---|---|---|---|---|
| Parsing | 0.22s | 0.12s | 0.44s | 0.60s | 0.97s | 1.45s | 1.12s |
| Saving | 0.32s | 0.17s | 0.14s | 1.06s | 1.08s | 0.91s | 0.85s |
AnnotationSet.show_stats(): 0.02 s- Evalaution: 0.30 s
- Basic data structures and utilities
- Parsers (ImageNet, COCO, YOLO, Pascal, OpenImage, CVAT, LabelMe)
- Parser tests
- Database summary and stats
- Database converters
- Visualization options
- COCO Evaluation
- Tests with a huge load (5k images)
- CLI interface
- Make
image_sizeoptional and raise err when required (bbox conversion) - Make file saving atomic with a temporary to avoid file corruption
- Pip package!
- PascalVOC Evaluation
- Parsers for TFRecord and TensorFlow
- UI interface?
This repo is based on the work ofRafael Padilla.
Feel free to contribute, any help you can offer with this project is most welcome.
About
A package to read and convert object detection datasets (COCO, YOLO, PascalVOC, LabelMe, CVAT, OpenImage, ...) and evaluate them with COCO and PascalVOC metrics.
Topics
Resources
License
Code of conduct
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Uh oh!
There was an error while loading.Please reload this page.
Contributors7
Uh oh!
There was an error while loading.Please reload this page.