Field notes & progress

Building an AI that sees the litter we walk past

March 2026

We're building an open-source system that uses two types of cameras — a clip-on device mounted on the grabber and AI-powered glasses worn by volunteers — to automatically detect every piece of litter picked up, classify it in detail, and generate a full report so communities can see the story behind every cleanup.

The goal: turning cleanup walks into data

Every weekend, volunteers go out with grabbers and trash bags to clean up their neighborhoods and highway green-belts. They pick up hundreds of items — 10 to 25 bags of litter each time — but when the walk is over, all that information disappears into a landfill. What if every piece of litter told a story?

That's what we're building — no manual counting, no clipboards. Just walk, clean, and get the data.

How it works: a two-tier AI system

Tier 1 — The Spotter (on-device detection). A lightweight object detection model runs directly on the camera hardware in real time. It spots every piece of litter in the video frame and crops it out for closer analysis.

• Clip-on camera — mounted on the grabber itself, pointing at the grabber head. It captures close-up images at the moment of pickup, with the grabber jaws (~8 cm wide) serving as a built-in size reference.
• AI glasses — worn by the volunteer, capturing a first-person egocentric view of the cleanup scene as they walk and pick up items.

Tier 2 — The Classifier (vision-language model). Cropped images are sent to a more powerful AI — a vision-language model — that determines the category, size, material, condition, whether it's hazardous, and writes a plain-English description of each piece.

Building the dataset

The images span 14 separate cleanup sessions from December 2023 through January 2025. Each image is annotated with bounding boxes and labeled with a detailed category.

Benchmarking the classifiers

We ran a head-to-head benchmark on 241 carefully selected litter images across three leading AI services:

Gemini won on both accuracy and cost, making it our primary cloud classifier.

Performance: where we stand

Object detection (Tier 1) — our YOLO26 detection model achieves a mAP50 of 0.905, correctly identifying and locating litter in the frame about 90% of the time.

Classification (Tier 2) — using Google Gemini, we processed all 4,499 images with a 100% success rate at a total cost of about $4.21. Size accuracy 83%, broad category match 73%, material accuracy 37%.

Training a local model — we fine-tuned a vision-language model (Qwen3-VL-8B) using QLoRA on a single consumer GPU (RTX 3080 Ti). After four iterations, our v4 model achieved an eval loss of 0.024 with no overfitting. A working local model would reduce per-image costs by 97%.

What's next

• Deploy on edge devices — shrink the model for real-time inference on the clip-on camera and AI glasses via knowledge distillation and optimized model export
• Connect the end-to-end flow — from data collection with GPS tagging, through detection and classification, to automated report generation for communities and city councils
• Evaluate the fine-tuned model — head-to-head comparison of our v4 local model against Gemini on unseen images

Get involved

• Cleanup organizations — want to pilot the system on your walks? We'll provide the camera setup and handle the AI.
• ML engineers — the codebase is open source, with interesting problems in fine-tuning, edge deployment, and report generation.
• Community leaders — help us understand what data would be most useful for your advocacy and planning.

Reach out at contact@npotechnologies.com — and if you're early-career and want in, see the 2026 internship.