Protocol — Camera Side-by-Side Test (MISSION 1 PRO vs. current GoPro)
Objective: decide, on evidence from our OR rather than spec sheets, whether to switch the capture camera to the GoPro MISSION 1 PRO (1″ sensor, GP3 low-light) from the current GoPro. The bigger sensor should help in low light; the open questions are whether depth of field holds at our working distance and whether on-camera AI processing alters instrument appearance enough to hurt detection. This test answers both. See camera-data-source-assessment for the rationale.
Decision this produces: GO (switch before P4) / NO-GO (keep current) / CONDITIONAL (switch only with specific settings).
Run this before the P4 multi-surgeon collection. Do not switch cameras mid-dataset. Keep the existing 16 cases as the original-domain corpus.
1. Setup
- Both cameras see the same scene. Mount the MISSION 1 PRO and the current GoPro side-by-side on the tower so their fields overlap as closely as possible. (If simultaneous mounting is awkward, record the same staged scene back-to-back without moving anything else.)
- Scene: a phantom, a cadaver, or a consented live case — one good session is enough. Stage it to include the moments that matter: instrument pickups/handoffs, the dark parts of the field, and quick hand/instrument motion.
- Match what you can, vary only the camera. Same mount distance, same OR lighting, same working distance, same instruments. The camera is the only variable.
- Capture the weak classes on purpose: make sure nav probe, suction bovie, nav suction (your thinnest / most-confused instruments) appear clearly — these are where sensor/DOF differences will show up most.
- MISSION settings to capture (test the variable that matters): record at least two MISSION takes — one with AI Low-Light mode ON, one with a flat/neutral profile, minimal processing — so you can isolate whether the AI processing helps or hurts.
2. What to capture (shot list)
| Shot | Why |
|---|---|
| Static field, lights typical | Baseline image quality + noise |
| Static field, lights dimmed / worst-case | Low-light noise — the headline question |
| Instruments at near and far working distance in one frame | Depth of field |
| Instrument pickup / handoff with scrub tech | Does it catch the exchange (swap moment) |
| Quick hand / instrument motion | Motion blur |
| Thin instruments (nav probe, suction bovie) held in field | Weak-class legibility |
3. Scoring
Two layers: a quick human visual score, and the one that actually decides it — run the YOLO model on both clips and compare per-class recall. Don’t decide on eyeballing alone; the model is the consumer of this footage.
A. Visual rubric (1 = worse, 5 = better) — fill in per camera/setting
| Criterion | Current GoPro | MISSION (AI low-light) | MISSION (flat) | Notes |
|---|---|---|---|---|
| Low-light noise (dark field) | ||||
| Dynamic range (highlight/shadow split from endoscope light) | ||||
| Depth of field (near + far both sharp) | ||||
| Detail on thin instruments | ||||
| Motion blur on quick moves | ||||
| Instrument appearance consistency frame-to-frame (AI artifacts?) | ||||
| Mount fit / ergonomics at working distance |
B. Model-based score (the deciding metric) — run current YOLO on each clip
| Metric | Current GoPro | MISSION (AI low-light) | MISSION (flat) |
|---|---|---|---|
| Overall recall | |||
| Overall precision | |||
| Recall — nav suction | |||
| Recall — nav probe | |||
| Recall — suction bovie | |||
| % unlabelled frames | |||
| Per-class confusion (suction family) |
Note: the current model was trained on current-GoPro footage, so MISSION clips face a mild domain shift — a small recall dip on MISSION is expected and not disqualifying. What you’re looking for is whether the MISSION’s image quality clearly helps on the dark/thin-instrument shots despite that handicap. If it ties or wins untrained, that’s a strong signal.
4. Decision rule
- GO (switch): MISSION (in its better setting) clearly improves low-light noise and thin-instrument recall, and depth of field holds across the working distance, and the AI processing doesn’t introduce frame-to-frame inconsistency. Switch before P4; re-shoot/relabel a calibration case on the new camera; update the capture SOP.
- CONDITIONAL: MISSION wins only with the flat profile (AI low-light hurts consistency) → switch but lock the flat profile in the SOP and do denoise in post where you control it.
- NO-GO (keep current): shallow DOF loses instruments at working distance, or AI artifacts hurt detection consistency, and the low-light gain doesn’t offset it. Stay on the current GoPro; revisit settings/lighting instead.
Whatever the outcome, record the decision + the numbers here and freeze the choice in the capture SOP.
Links
- camera-data-source-assessment — why this test exists (sensor, DOF, AI-processing trade-offs)
- validation-plan — P4 is the deadline; lock the camera before multi-surgeon collection
- YOLO Model Improvement Analysis — the model + weak classes used for the model-based score
- FESS Cases Clean Dataset — original-domain corpus to keep separate from any new-camera data