How It Works & How It Was Made

⚙️ How It Works — 4‑Model Pipeline

The system processes a single image through four stages: detection, segmentation, measurement, and specification matching.

🔍 Model A – Object Detection (YOLOv8 HBB)

YOLOv8 detects each fastener with a horizontal bounding box, classifies it as bolt, nut, washer, or screw, and provides coordinates. Trained to mAP@0.5 = 0.9687.

🎭 Model B – Instance Segmentation (YOLOv8‑Seg)

For every detected object, a pixel‑perfect mask is generated. This removes background noise and provides exact boundaries for accurate measurement (mask mAP@0.5 = 0.9671).

📐 Model C – Classical Measurement Algorithms

A 20 mm reference square in the scene gives the pixel‑to‑millimeter conversion. Then component‑specific OpenCV routines run:

• Bolts / Screws – cv2.minAreaRect() on the mask → length + head/corner diameter.
• Nuts – Rotated rectangle extracts across‑flats (AF) dimension.
• Washers – Outer diameter via minEnclosingCircle(), inner via Hough Circle detection.

📸 Measurement Evolution – From Raw Input to Final Annotated Output

📊 Model D – Specification Matching (ISO Database)

Measured dimensions are compared against CSV files containing ISO 4017 / 4032 / 7089 / 7045 standard tables. The system finds the nearest nominal size using a ±0.5 mm tolerance and returns the matched specification with deviation analysis.

🛠️ How It Was Made

📸 Dataset Collection & Annotation

4,943 images collected from Google Images, Kaggle, and Roboflow. Each image annotated twice:

• Bounding boxes (Roboflow) – for YOLO detection (Model A).
• Polygon masks – generated automatically from bounding boxes using Meta’s SAM2 (Segment Anything Model), then manually refined on Roboflow.

Final dataset: 27,134 annotated instances across 4 classes. Train/val/test split = 70/20/10.

🧠 Model Training

Trained two YOLOv8‑medium models on Kaggle GPUs:

• Detection model – 100 epochs, mAP@0.5 = 0.9687
• Segmentation model – 50 epochs, mask mAP@0.5 = 0.9671

📏 Measurement Algorithms (Classical CV)

Instead of training another ML model, I designed deterministic OpenCV routines for each fastener type. This makes measurements consistent, fast, and easy to debug. The reference square calibration ensures the system works at any camera distance.

✅ Validation & Accuracy – 95%+ Pass Rate

Experimental Setup: Smartphone camera fixed at ~40 cm overhead, diffuse LED lighting (~600–800 lux), uniform white background, 20×20 mm blue reference square. 62 samples tested across all component types.

Why 95%+ accuracy? Because the natural measurement variation (0.1–0.5 mm) is much smaller than the gap between standard sizes (e.g., M4 vs M6 differ by 2 mm). Coupled with automatic reference calibration, the system confidently distinguishes between sizes. Default acceptance thresholds: absolute error ≤1.0 mm, relative error ≤5% (washer ID allowed ≤1.5 mm due to detection difficulty).

Results: 12/12 nominal size identification correct (bolts, nuts, washers, screws). Overall pass rate >95%. Remaining minor errors come from calibration pixel resolution, mask boundary deviation (±1–2 px), and manual caliper uncertainty (±0.05 mm) — all within acceptable limits for fasteners sorting.

🔧 Tech Stack

Full source code, trained models, and CSV databases are available in the GitHub repository.

🔗 Try It Yourself

Head back to the Live Demo to upload your own hardware image and see the pipeline in action. The backend runs on Hugging Face Spaces, and the frontend is deployed on Netlify.