You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Part of #818 (Track B2). Follow-up to #1015 (vehicle plumbing shipped, model trained on synthetic blobs only) and #1050 (vehicle skeleton template + rigid one-bone binding).
Why now
meshseg_vehicle.onnx (vehicle_body / wheel / window / wing / rotor) is trained exclusively on the synthetic make_vehicle generator in scripts/export-meshseg-onnx.py — boxes with blob wheels. Real cars (fused low-poly bodies like ugly_car.obj, detailed multi-part models like the Buick) look nothing like that, and the model has never seen one. Two things changed that make a real retrain cheap:
We can generate vehicles on demand.qtmesh generate3d --prompt "…" (TRELLIS.2 / TripoSG / TripoSR) produces cars, trucks, buses, planes, helicopters, motorbikes, wheels-only, wing-only meshes — unlimited, varied, and license-clean (our own outputs of MIT-licensed models; no ShapeNet/PartNet non-commercial data).
The vehicle rig gives EXACT body/wheel labels for free.--skeleton vehicle binds every vertex to exactly one of Chassis / FrontAxle / RearAxle / 4 wheel bones (feat(#1013): vehicle skeleton template + rigid one-bone binding #1050). That is a per-vertex label. The existing rig-prior miner (qtmesh segment --dump-training-data, AutoRig::rigPriorPartLabels → MeshSegmenter::partForBoneName) already turns bone weights into exact training samples — it only knows body-part bone names today.
Plan
Miner: extend partForBoneName with the vehicle vocabulary — Chassis/*Axle → vehicle_body, *Wheel → wheel (also common artist names: wheel_fl, tire, rim, hub; wing, rotor, propeller, blade). Make --dump-training-data write the sample's category (schema bump qtmesh-meshseg-training-v2, v1 = body) so the export script routes it to the right model.
Pipeline scriptscripts/mine-vehicle-segdata.py: prompt list × seeds → generate3d (TRELLIS.2 when available, else TripoSG; --target-tris 25000) → rig --skeleton vehicle --rigid → segment --dump-training-data. Reject samples whose wheel snap failed (a wheel bone with < N vertices, or wheels not at the four corners — reuse the fitVehicle quadrant check). Also mine the CC0 vehicle packs already used for the body miner (Quaternius) and the two local test cars.
Parts as standalone meshes: run PartOps split (segment --split-parts) on the rigged cars and export each part alone (wheel, body) — the Auto category classifier (meshseg_category.onnx) must still say vehicle for a lone wheel or a wing, which is the verified failure mode noted in the export script (detached parts at odd offsets).
Window / wing / rotor labels: the rig gives body vs wheel only. Options, in order of preference: (a) keep the synthetic generator for these three classes and mix; (b) label windows on generated cars via the baked texture (dark/low-roughness glass texels → window) — cheap but noisy, gate it; (c) SAM 2 multiview (AI v2 B2: sam2seg — zero-shot mesh segmentation via multiview SAM 2 #1014) if it lands first. Do NOT block the retrain on this — body/wheel is where the real-mesh gap is.
Retrainmeshseg_vehicle.onnx with --real-data ./mined_vehicle/ mixed into the synthetic set (yaw/tilt/jitter aug as for body) and retrain the category classifier with the generated vehicles + lone parts. Upload via the existing HF flow (segment/ on QtMeshEditor-models), update THIRD_PARTY_AI_MODELS.md provenance (generated-data statement).
Evaluation (held out, never trained on): ugly_car.obj, the Buick, 3–5 Quaternius vehicles, 3 generated vehicles from prompts not in the training list. Metrics: per-vertex accuracy + wheel IoU vs the rig-prior labels; category classifier accuracy on whole vehicles and on lone parts. Record the numbers in docs/MESH_SEGMENTATION_STRATEGY.md. Acceptance: wheel IoU ≥ 0.8 on the two real cars (today the geometric fallback's "lowest 30 %" rule is the bar to beat).
Verify in-app: Edit Mode → "Select by Part (AI)" on an unrigged generated car selects a wheel cleanly; qtmesh segment car.glb --category vehicle per-part counts look right.
Notes / risks
Generated meshes are single fused shells: wheel arches merge into the body, exactly like ugly_car.obj — good, that IS the hard case the synthetic set lacks. Discard generations where the rig's wheel snap fails rather than training on bad labels.
Training samples are point clouds + labels, so TRELLIS textures are irrelevant except for the optional window heuristic.
Keep the one-model-per-category design (decision record in docs/MESH_SEGMENTATION_STRATEGY.md); this retrains two files, adds none.
Vehicle-rig-derived labels only cover 4-wheel ground vehicles; planes/helicopters/bikes need the synthetic classes or SAM 2 until a matching rig template exists.
Part of #818 (Track B2). Follow-up to #1015 (vehicle plumbing shipped, model trained on synthetic blobs only) and #1050 (vehicle skeleton template + rigid one-bone binding).
Why now
meshseg_vehicle.onnx(vehicle_body / wheel / window / wing / rotor) is trained exclusively on the syntheticmake_vehiclegenerator inscripts/export-meshseg-onnx.py— boxes with blob wheels. Real cars (fused low-poly bodies likeugly_car.obj, detailed multi-part models like the Buick) look nothing like that, and the model has never seen one. Two things changed that make a real retrain cheap:qtmesh generate3d --prompt "…"(TRELLIS.2 / TripoSG / TripoSR) produces cars, trucks, buses, planes, helicopters, motorbikes, wheels-only, wing-only meshes — unlimited, varied, and license-clean (our own outputs of MIT-licensed models; no ShapeNet/PartNet non-commercial data).--skeleton vehiclebinds every vertex to exactly one of Chassis / FrontAxle / RearAxle / 4 wheel bones (feat(#1013): vehicle skeleton template + rigid one-bone binding #1050). That is a per-vertex label. The existing rig-prior miner (qtmesh segment --dump-training-data,AutoRig::rigPriorPartLabels→MeshSegmenter::partForBoneName) already turns bone weights into exact training samples — it only knows body-part bone names today.Plan
partForBoneNamewith the vehicle vocabulary —Chassis/*Axle→vehicle_body,*Wheel→wheel(also common artist names:wheel_fl,tire,rim,hub;wing,rotor,propeller,blade). Make--dump-training-datawrite the sample's category (schema bumpqtmesh-meshseg-training-v2, v1 = body) so the export script routes it to the right model.scripts/mine-vehicle-segdata.py: prompt list × seeds →generate3d(TRELLIS.2 when available, else TripoSG;--target-tris 25000) →rig --skeleton vehicle --rigid→segment --dump-training-data. Reject samples whose wheel snap failed (a wheel bone with < N vertices, or wheels not at the four corners — reuse thefitVehiclequadrant check). Also mine the CC0 vehicle packs already used for the body miner (Quaternius) and the two local test cars.segment --split-parts) on the rigged cars and export each part alone (wheel, body) — the Auto category classifier (meshseg_category.onnx) must still sayvehiclefor a lone wheel or a wing, which is the verified failure mode noted in the export script (detached parts at odd offsets).window) — cheap but noisy, gate it; (c) SAM 2 multiview (AI v2 B2: sam2seg — zero-shot mesh segmentation via multiview SAM 2 #1014) if it lands first. Do NOT block the retrain on this — body/wheel is where the real-mesh gap is.meshseg_vehicle.onnxwith--real-data ./mined_vehicle/mixed into the synthetic set (yaw/tilt/jitter aug as for body) and retrain the category classifier with the generated vehicles + lone parts. Upload via the existing HF flow (segment/onQtMeshEditor-models), updateTHIRD_PARTY_AI_MODELS.mdprovenance (generated-data statement).ugly_car.obj, the Buick, 3–5 Quaternius vehicles, 3 generated vehicles from prompts not in the training list. Metrics: per-vertex accuracy + wheel IoU vs the rig-prior labels; category classifier accuracy on whole vehicles and on lone parts. Record the numbers indocs/MESH_SEGMENTATION_STRATEGY.md. Acceptance: wheel IoU ≥ 0.8 on the two real cars (today the geometric fallback's "lowest 30 %" rule is the bar to beat).qtmesh segment car.glb --category vehicleper-part counts look right.Notes / risks
ugly_car.obj— good, that IS the hard case the synthetic set lacks. Discard generations where the rig's wheel snap fails rather than training on bad labels.docs/MESH_SEGMENTATION_STRATEGY.md); this retrains two files, adds none.