diff --git a/README.md b/README.md index 5422049..e295ea8 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,10 @@ Built by parent company [Stitchable](https://stitchable.ai). For a complete vide [Still image](docs/media/editor-still.png) · [How this demo was captured](docs/media/README.md) +## Edit a 3D scene locally + +Select actors, move XYZ arrows, rotate, scale, adjust lights and camera framing, and save clean reference images. The [Scene Studio example](examples/scene-editor/README.md) includes drama blocking and a procedural robot-versus-ape scene, with keyframes and reversible edits. Run `npm run build:scene` then `npm run dev:scene`. This standalone editor is an opt-in integration example; native Stitchable integration remains separate. + ## One framework, many visual styles Actual output from the included studies, effects and device viewer. Click any image to see it larger. Run the local studio to play and edit the examples. diff --git a/docs/3d-scene-adjustment-requirements.md b/docs/3d-scene-adjustment-requirements.md new file mode 100644 index 0000000..f899fde --- /dev/null +++ b/docs/3d-scene-adjustment-requirements.md @@ -0,0 +1,542 @@ +# Editable 3D scenes: requirements and implementation design + +Status: **standalone implementation available; native host integration pending**. +The [Scene Studio example](../examples/scene-editor/README.md) implements an opt-in +Three.js editor with spatial controls, keyframes, reversible transforms, local +persistence and committed-state reference capture. The reusable core/adapter +modules are included. Native Stitchable transport, Library and KDV integration +remain pending; this document is the full target contract, not a claim that every +host acceptance fixture has shipped. The baseline source audit below describes +EditableFrames `27f26c9158dc87b729d554dae9accba7e293ab44` (2026-10-26), before this PR. + +## 1. Outcome and scope + +A person selects an exposed character, prop, camera or light, moves it with +**X/Y/Z arrows**, adjusts its orientation and size, and sees the result without +regenerating scene code. The inspector, viewport and agent edit the same declared +values through the same revision-checked history. Existing screen-plane dragging, +uniform resize, object selection, numeric controls and animation tracks continue +to work. A scene without spatial bindings remains fully renderable. + +The initial use cases are drama blocking with rigged characters, action blocking +with procedural figures, and product/set composition. These scenes may later +supply still or motion references to video generation. Editing must work locally +without a model call, account, paid service or Blender installation. + +Required scope is object placement, rotation, uniform scale, scene cameras and +lights. Character root placement is required; arbitrary skeletal posing, walking, +retargeting, full-body IK, physical contact and simulation are separate extensions. +A hand target can use the same spatial interface when an authored solver exists; +exposing a target does not supply that solver. Mesh modelling, rig construction, +multi-object transforms, animated reparenting and negative/nonuniform scale are +not initial release requirements. + +## 2. What exists and what must change + +These are verified **baseline** source observations, before the implementation in this PR: + +- [Controls](../packages/core/controls.mjs) support typed fields, stable bindings, + units and animation. Spatial roles are limited to position (`vec3`) and positive + scalar scale. The normalizer accepts controls version 1 only; rotations are not + a spatial role. [Core sampling](../packages/core/index.mjs) interpolates vec3 + components; it does not provide quaternion rotation tracks through portable controls. +- [Object metadata](../packages/core/objects.mjs) supplies a semantic hierarchy, + pivots and inherited selection policy. This hierarchy is **not** the Three.js + transform hierarchy. [The current spatial adapter](../packages/core/manipulation.mjs) + picks projected rectangles in reverse binding order; this is insufficient for + occluded 3D geometry. +- [The Three helper](../packages/three/manipulation.mjs) freezes a camera-facing + plane and converts pointer intersections to parent-local coordinates. It is a + useful screen-plane move/resize helper, not an axis-constrained gizmo. Its + projected Box3 bounds need special treatment for behind-camera/near-plane cases. +- [Gesture sessions](../packages/core/interaction.mjs) already validate bounded, + sparse absolute proposals, selected-anchor bindings and revisions. The Studio + consumer produces one reversible batch. However, `gestureEvent` currently maps + every operation except `resize` to `move`: new operations must not pass through + that legacy coercion. +- [The portable mount](../packages/runtime/renderer-host.mjs) forwards `pick` and + `gesture`, but does not expose the full bounds/capabilities/edit-overlay surface. + [The interaction contract](interaction-contract.md) explicitly does not claim a + universal drag widget in the portable editor. Both the framework host and native + host require implementation, not just a new scene helper. +- [The package](../package.json) pins Three.js **0.180.0**. The inspected Stitchable + integration uses **0.170.0**, a vendored framework snapshot, admitted libraries + and its own gesture transport. This is a consumer compatibility observation, + not a new framework dependency; verify it again when implementing that consumer. + +## 3. User-visible requirements + +### R1. Selection and coexistence + +Select an authored object by mesh hit or object list. A mesh maps to a stable +anchor, never a generated Three UUID, mesh index or ambiguous node name. Selecting +any body part normally selects the character placement root; optional pose handles +are a distinct mode. Support selection of lights and cameras through editor-only +icons and the object list. Honour inherited hidden/locked selection policy. + +Keep the existing 2D clip-level transform handles distinct from scene-object +handles. Object mode offers Translate, Rotate and Scale only when the selected +binding implements them. Retain screen-plane dragging as an explicit affordance. +Selection, changing modes and viewing the scene must not create keys or history. +The inspector remains usable when a gizmo is unavailable and explains why. + +### R2. Translation, rotation and scale + +Translation shows labelled X/Y/Z arrows, with conventional red/green/blue colours +and labels so colour is not the only cue. Offer XY/XZ/YZ plane handles, world/local +axis orientation, numeric entry, keyboard adjustment, optional snapping and reset. +A Z drag must actually change scene depth; a decorative overlay is not sufficient. + +Rotation uses axis rings and numeric angles. Uniform scale uses a distinct handle; +its positive scalar meaning stays compatible with existing scenes. Do not display +axis stretch handles when only uniform scale is supported. Keep handles legible +across zoom levels with a bounded screen size. Show active axis, value and units. +Snap increments and numeric precision are host preferences, not hardcoded metres. + +Use one selected object initially. Clicking a gizmo takes priority over object +selection and camera navigation. Escape and unexpected capture loss cancel before +publication; normal capture loss after release follows the finalization rules in +§5D. Never leave an invisible or occluded handle intercepting input. + +### R3. Cameras, lighting and scene editing + +Separate **editor view** from **shot camera**. Orbit/pan/dolly in editor view changes +no authored property and no export. Provide Return to shot and an explicit Use +this view as shot action, which commits supported camera fields atomically. +Manipulating a shot camera from another editor view uses its own selectable proxy. +Navigation and object manipulation cannot own the same pointer simultaneously. +Navigation requires an opt-in renderer capability for transient view state, +separate from authored props. Subject rendering, helpers, picking and constraints +must use the same accepted editor camera and viewport. A view change requests a +render even at unchanged source time; it invalidates outstanding picks. Renderers +without this capability remain renderable and report navigation as unsupported. + +Expose declared camera position, orientation OR look-at target, perspective FOV +(or orthographic size), and relevant clipping bounds. Do not simultaneously drive +orientation and look-at with conflicting controls. Focal length needs an explicit +film/sensor convention; avoid an unexplained conversion from FOV. Camera changes +must update projection matrices and reproduce in exports at the same source time. + +Expose declared light position, target/direction, colour, intensity and appropriate +shadow/shape parameters. A point light has no useful aim control; an environment +light has no useful position arrow. Renderer-specific intensity units and colour +handling must be labelled. Camera/light target controls that must change together +should share one semantic anchor so single-anchor validation can commit them +atomically. Initially there is one placement gizmo binding per anchor; aim targets +may remain numeric controls. Additional target gizmos need explicit binding IDs +and control-key mappings, not another ambiguous position role. Props, backgrounds +and additional characters reuse the object contract rather than a fixed catalogue. + +### R4. Animation and history + +Freeze the child-confirmed painted presentation when a drag is admitted. It +identifies renderer instance, document/source revision, source clock, active shot, +editor-view revision and viewport-mapping revision. Rendering, picking, handle +hit-testing and gesture initialization must agree on that presentation. Freeze its +editor-camera pose/projection separately from the authored shot camera, plus parent +transform, pivot and sampled authoring values. Suspend navigation, including +inertia, during manipulation. External view/mapping changes invalidate the gesture. Unanimated +fields with Auto-key off change the base value; an animated field or Auto-key on +inserts/updates the source-time key, preserving existing easing and rational timing. +Follow [current key timing](object-manipulation.md); do not write object motion to +outer clip transform tracks. One completed drag is one undo entry, no matter how +many pointer updates arrive. A no-op produces no entry. Cancellation before the +publication boundary produces no entry; after it, reconcile the outcome (§5D). + +After edits, backward/random seeks and export must evaluate from source time and +saved controls, without accumulating pointer deltas or animation playback state. +Cancel before publication on a camera cut, seek, source rebind, selected-object +replacement, viewport mapping change or conflicting writer. Do not silently rebase +a gesture; after publication begins, report its authoritative outcome. + +## 4. Proposed portable interfaces + +The following is an **API sketch, not valid current v1 syntax**. Final naming can +change. The requirements are explicit negotiation, typed bindings and bounded data. + +```js +// Proposed controls v2: preserve v1 as a supported input format. +controls: { + version: 2, + objects: [{id: 'actor', label: 'Actor', pivot: [0, 0, 0]}], + fields: [ + {key: 'actor.position', anchor: 'actor', property: 'position', + path: '/actor/position', type: 'vec3', role: 'position', + coordinateSpace: 'local', unit: 'world-unit', animatable: true}, + {key: 'actor.orientation', anchor: 'actor', property: 'orientation', + path: '/actor/orientation', type: 'quaternion', role: 'rotation', + coordinateSpace: 'local', animatable: true, interpolation: 'slerp'}, + {key: 'actor.scale', anchor: 'actor', property: 'scale', + path: '/actor/scale', type: 'number', role: 'scale', + min: 0.001, animatable: true} + ] +} + +// Proposed adapter capabilities: independent of controls schema version. +spatialCapabilities({anchor: 'actor', presentationId}) // => +({gestureProtocols: [2], bindings: [{id: 'placement', operations: { + translate: {handles: ['x','y','z','xy','xz','yz','screen'], + spaces: ['world','local'], keys: ['actor.position']}, + rotate: {handles: ['x','y','z'], spaces: ['world','local'], + keys: ['actor.orientation']}, + scale: {handles: ['uniform'], spaces: ['local'], keys: ['actor.scale']} +}}]}) + +// New fields extend the existing begin/update/commit/cancel envelope. +// The host retains its admission, identity and ordered response state. +gesture({version: 2, phase: 'update', id, anchor: 'actor', + bindingId: 'placement', operation: 'translate', handle: 'z', space: 'world', + start, point, clock}) // => {values: {'actor.position': [1, 0, 2]}} +``` + +Renderer protocol, controls schema and gesture protocol are **independent**. +Controls-v1 scenes can use a v2 manipulation adapter; new quaternion fields require +a controls schema that understands them. Before admission, every intervening host, +transport and adapter must explicitly support the selected gesture protocol. +Missing negotiation means legacy-only: send only existing move/resize envelopes on +that path. Never probe v2 operations through the legacy normalizer. A v2-only +adapter rejects unversioned input. Unsupported combinations leave saved state +unchanged; old hosts reject unsupported schemas without resetting history. + +Keep identity and authority outside authored code. Validate and freeze the binding, +operation, handle, space, permitted control keys and snapping configuration at +admission. Revalidate them before publication. A translate proposal cannot alter +rotation or another field just because it shares the anchor. Coupled transforms +must explicitly declare all permitted keys and publish atomically. Capability +advertisements and renderer hit results are proposals, not write authorization. + +The edit surface also needs a versioned handle-hit query, for example +`hitTestHandle({point, presentationId}) -> {anchor, bindingId, operation, handle}`. +Only bounded semantic data crosses the host bridge. The host checks it against +the current capabilities and accepted presentation; stale results are discarded. +A busy/replaced renderer is distinct from a confirmed empty hit. Expose disabled +reasons for unsupported transforms. Presentation IDs are host-generated correlation +values; neither they nor a renderer message grant permission to commit. + +The optional runtime view/overlay service must deliver transient view state to +subject rendering as well as the adapter; it supplies an editor-only surface and +invalidates it with the same presentation. It is absent during content capture. +The exact facade is a spike deliverable, but both host implementations must share +these semantics. Existing content rendering remains usable without that service. + +Add normalized quaternion validation and sampling through the control normalizer, +core value type, template mapping, keyframes, inspector, serialization and consumers. +Use `[x,y,z,w]` unit quaternions, finite/nonzero validation, documented tolerance, +shortest-path spherical interpolation and sign-equivalence-aware no-op comparison. +Euler degrees are an inspector presentation with an explicit order; do not +component-interpolate them as if they were ordinary positions. Preserve deliberate +multi-turn animation through a separate authored angle/spin control; shortest-path +quaternions alone cannot encode winding. [Three quaternion reference](https://threejs.org/docs/pages/Quaternion.html). + +Sampling must be invariant under negating either endpoint. At an exact zero-dot +(180-degree) tie, choose representatives whose first nonzero component in +`[w,x,y,z]` is positive before interpolation; otherwise choose the shortest +hemisphere. Apply existing temporal easing to the interpolation parameter. Do not +rewrite a track for a sign-only no-op. Test intermediate orientations for all four +endpoint-sign combinations, not only equality at the keys. + +Separate **persistent spatial-semantics identity** from **transient binding +generation**. The persistent identity is deterministic across mounts and includes +authored mappings, units and transform/pivot conventions; it excludes runtime +instance IDs and generation counters. Preserve legacy v1 identity and history; +opening a project does not implicitly migrate it. Incompatible authored semantics +require explicit migration/reset, while remounting alone never does. + +The host captures transient binding generation at admission and checks it when +accepting results and immediately before publication. Replacing the bound root, +camera object or conversion mapping invalidates that generation; ordinary draft +repaints do not. View-pose changes separately invalidate presentation identity. +Existing component rounding must not invalidate unit quaternions: normalize after +permitted rounding, and compare orientations rather than raw component signs. + +A runtime-only Three adapter binds an anchor to its editable root, camera, picking +objects and conversions between rendered transforms and declared controls. Persist +only values/metadata; never serialize Object3D instances, callbacks or DOM nodes. +The same contract must remain implementable by other renderers. + +## 5. How to implement the gizmo without breaking editing + +### A. Reuse Three geometry and interaction through a controlled adapter + +Evaluate pinned Three `TransformControls` as the gizmo engine, not a second source +of project state. Its documented features include translate/rotate/scale modes, +local/world space, snapping and `getHelper()`. `OrbitControls` supplies editor-view +navigation. These do not supply EditableFrames history or authorization. +[TransformControls](https://threejs.org/docs/pages/TransformControls.html), +[OrbitControls](https://threejs.org/docs/pages/OrbitControls.html). + +Attach the gizmo to a **transient proxy**, not the authoritative rig or sampled +props. The adapter reads proxy transforms, converts them to sparse absolute +control values and submits proposals. Only validated draft props drive the subject +preview. Commit publishes one host transaction; cancel rebuilds from current +authoritative state. Avoid feedback loops between sampling, proxy updates and +`objectChange` callbacks. The pinned [r180 implementation](https://github.com/mrdoob/three.js/blob/r180/examples/jsm/controls/TransformControls.js) +is the implementation reference; current online docs may describe newer APIs. + +Host-owned pointer capture/admission must remain authoritative. An authored +`mouseUp` or synthetic event is not permission to save. Prototype a pinned adapter +that accepts the host's normalized pointer stream, with automatic DOM listeners +unconnected, before adopting this dependency. Do not depend on undocumented methods +without a version-pinned facade and conformance tests. If that integration fails, +retain the portable protocol and implement an adapter-specific solver; do not +bypass the admission layer to make the arrows work. + +### B. Coordinate and pivot rules + +Define the chain explicitly: client CSS pixels → host stage/clip inverse transform +→ renderer-local pixels → active viewport NDC → camera ray → constrained world +transform → declared authoring values. Account for letterboxing, nested clip +transforms, CSS zoom, device pixel ratio and sub-viewports exactly once. Pointer +coordinates are logical pixels; do not multiply by DPR twice. + +Gizmo axis orientation and storage space are different. A world-X drag may produce +several changed components in a parent-local position field. Capture the parent's +world matrix `P` and inverse at begin. Here `local` position means parent-local, +not object-axis coordinates. For the explicit pivot convention +`M = T(t) T(p) R S T(-p)`, recover `t = P⁻¹(worldPivot) − p`. +For ordinary Three `M = T(t) R S`, instead recover +`t = P⁻¹(worldPivot) − R S p`. The binding must declare which convention it consumes +or provide explicit conversions. A noncentral pivot may therefore require a +rotation proposal to include a compensating position field. Declare that coupled +key set in capabilities; never move the visible pivot accidentally. Do not use a +changing animated bounding-box centre as the gesture pivot. + +Prefer a proxy beneath an identity-transform editor root, expressed at the frozen +world pivot with world orientation. Convert its output back through the captured +parent and declared convention; do not copy proxy position directly to a local +field. A different proxy hierarchy must prove the same round-trip invariants. + +Adopt explicit authoring units, handedness and up axis in each adapter; imported +asset normalization belongs below a stable placement root. Do not refit/recentre +a GLB each frame or reparent its skeleton to attach a gizmo. Preserve bones and +skins under the placement root. Procedural animation and user transforms need an +explicit composition order; editing a placement layer is the default. Arbitrary +noninvertible code-derived transforms can decline a capability. + +Initially support editable roots under invertible rigid or positive uniformly +scaled ancestry. Detect and explain nonuniform scale, shear, reflections and +singular ancestry rather than approximating rotations. A future adapter may +advertise tested support for more cases. Matrix decomposition and reparenting are +not universally safe under nonuniform scale; [Three's Object3D documentation](https://threejs.org/docs/pages/Object3D.html) +notes an explicit limitation for `attach`. + +When an axis projects almost to a point or a ray is nearly parallel to the chosen +drag plane, keep values finite and stable: disable that handle with guidance to +orbit or use numeric input, or use a tested frozen alternative. Never change the +constraint halfway through a drag. Freeze snapping at admission. Initial snapping +uses **displacement from the starting pivot**, with zero displacement preserving +the starting pose; absolute grid snapping is a separately declared option. World +increments use world units along frozen world axes. Local increments use authored +placement units along frozen object-oriented axes, with ancestor uniform scale +included in conversion and the object's own visual scale excluded. Report those +units in the inspector. Rotation uses angular displacement; scale uses the authored +positive scalar. Preserve coordinates outside the selected axes. Do not assume +r180 `translationSnap` handles rotated/scaled ancestry: its world branch adds +parent translation but does not perform a full parent-space conversion. Implement +the declared snap frame in the adapter and cover it with numeric fixtures. + +Enforce storage-space bounds while retaining the frozen axis/plane constraint. +For an axis, derive the feasible interval for scalar displacement from field bounds; +do not independently clamp converted X/Y/Z components. For a plane, project the +candidate onto the feasible region in its two declared coordinates. Bounds take +precedence over snapping: allow an off-grid boundary value rather than leave the +constraint or fail the gesture. Reaching a limit must permit reverse travel. + +An explicitly enabled surface-placement mode may add a final world-up correction +after a translation drag. This is separate from axis/grid snapping: the host must +label that it settles on release, freeze the preference at admission, and publish +the correction with the move in one transaction. It must respect stored-value +bounds, preserve rotation, exclude the moving object from support tests, leave +stationary clicks unchanged and remain cancellable before publication. Numeric +entry stays exact. Scene Studio opts actors into this mode with admitted support +meshes; this is approximate placement, not collision resolution or foot IK. + +Uniform scale saturates at a documented positive lower bound (and declared upper +bound), preserving equal components. Pivot crossing must not create a negative +scale, cancel an otherwise valid drag or prevent reversing direction. A degenerate +starting radius uses a tested fallback or refuses admission with guidance. +Snapping cannot emit zero/negative scale; no nonfinite value reaches a proposal. + +### C. Picking, overlay and disposal + +Raycast eligible subject meshes with the active editor camera and map hits to +anchors; use a declared proxy where deformation or custom shaders make mesh hits +inaccurate. Define nearest eligible hit, transparent/helper filtering and an +object-list escape hatch. Do not let a large projected rectangle select a hidden +object in front of the visible one. For skinned meshes, update pose-dependent bounds +as needed and test actual picking against the supported Three version. + +Render gizmos, selection outlines and camera/light helpers in an editor-only +surface; a separate overlay pass/canvas shares the accepted view and viewport. +Specify its occlusion policy. DOM ancestry alone cannot exclude editor pixels: +[the current exporter](../scripts/render.mjs) screenshots an element's visible +rectangle, which can contain overlapping sibling overlays. Use an isolated content +render, or an acknowledged capture transition that excludes editor pixels from the +actual captured image and restores the editor afterwards. + +Direct stills, movies and generation references sample an immutable **committed** +revision, requested source time and its authored shot camera. Exclude transient +pose drafts and editor-view state. Capture concurrent with publication chooses +one complete committed revision, records its identity and never mixes revisions. +Capture during a drag does not commit or cancel it. An optional draft-capture +feature would need explicit draft identity; it is not implied by this contract. +Authored captions and other content overlays remain included. Dispose editor +resources, listeners and pending proposals on unmount/rebind; no second uncontrolled +animation loop or stale GPU context may survive. + +### D. Async host transaction + +Use an explicit lifecycle: admitted/preparing → active → finalizing → publishing +→ committed/failed/uncertain. Canceled is terminal before publishing. Keep the +existing gesture validator, but extend its host consumer for this lifecycle. + +- Trusted begin freezes the accepted presentation, binding generation and identity. + Keep at most one executing update and one latest pending point. Serialize or + number responses; late work cannot replace a newer draft. +- A matching trusted release fixes the final point and enters finalizing, including + when begin is still pending. Expected capture loss after release is not cancel; + unexpected loss before release is. Disable new updates and drain/evaluate that + final point. If its evaluation fails, do not publish an earlier draft. +- Escape/invalidation can cancel until the host atomically consumes admission for + publication. Immediately before that boundary, recheck cancellation, identity, + generation, allowed fields and the final proposal. Publish with a document-revision + compare-and-swap, not only an earlier frontend revision check. +- After publication starts, report the authoritative transaction outcome. Do not + announce cancellation if a dispatched save may still succeed. A lost acknowledgment + is uncertain: reconcile by a stable host transaction ID/receipt before retrying. + Never duplicate an undo entry or overwrite a newer revision. A later reversal of + a successful publication is a separate normal undo, not gesture cancellation. + +Retain byte/key/time budgets and lease renewal through finalization. A deadline +before publication cancels; after dispatch it may require reconciliation. Neither +renderer callbacks nor transport tokens can consume the host's write admission. + +No LLM, filesystem write, asset download or provider call occurs per pointer move. +Benchmark drag-to-paint latency on representative hardware; a proposed target is +p95 below 50 ms for a declared reference scene/device. Report actual measurements, +scene complexity and degradation rather than promising this on arbitrary scenes. + +## 6. Ownership and delivery slices + +**EditableFrames owns** portable schemas, transform semantics, adapters, gesture +proposal validation, deterministic sampling, standalone editor integration, +examples and conformance fixtures. Keep Three imports out of renderer-independent +core modules. Optional capabilities must not narrow arbitrary creative source. + +**Stitchable owns** its KDV selected-clip panel and pointer admission, native +transport, Library import/custody, runtime-library admission and revisioned timeline +writes. Follow-up integration must update all relevant validators/types/consumers, +not only render a new toolbar. Its inspected host currently carries move/resize +operations and rounds spatial values; both require deliberate compatibility work. +Use one matching Three instance for objects, controls and loaders within a runtime; +either support/test both host versions or explicitly upgrade the consumer. Do not +mix r180 addons into an r170 runtime and assume compatibility. + +GLBs remain immutable Library/project assets with stable version references. +EditableFrames loads admitted bytes and instantiates scene objects; it does not +become an asset marketplace or download manager. An LLM may propose imports or +changes through tools, but cannot create a second persistence path. +Persist immutable asset/version references, never resolved runtime URLs. Resolution +and private package bindings stay host-owned and cannot be overridden by gesture +proposals. Asset identity covers every model dependency used in preview/export. +Initial fixtures may require self-contained GLBs; external buffers/textures need +separately admitted, pinned references or explicit rejection. Preserve native +private-binding and self-containment checks. Standalone projects provide equivalent +project-local custody without requiring Stitchable Library. + +1. **Contract and math spike:** negotiate protocol, specify field/rotation identity, + prove proxy input, transformed-parent conversion and overlay-free capture in a + minimal procedural scene. Resolve unsupported ancestry policy before UI polish. +2. **XYZ placement:** ship arrows/planes, world/local orientation, numeric fallback, + screen-plane compatibility, snapping, one-undo gestures and standalone tests. +3. **Orientation, cameras and lights:** quaternion tracks and rings, uniform scale, + editor-view/shot distinction, targeted light controls and reproducible animation. +4. **Consumer integration and character fixture:** import the framework changes into + Stitchable separately; prove native KDV selection, admission, reload, tracks and + export using a licensed rigged figure and procedural fallback. No large asset + pack is required in this PR. + +A pending clip retains its editable scene and revision. A host may derive stills, +first/last frames, appearance references or a clay motion video according to the +selected generation endpoint. These are derived assets, not a replacement for the +scene. An edit invalidates affected reference renders; it must not purchase a new +generation automatically. Provider APIs, budgets and generation jobs are outside +this framework change. + +## 7. Acceptance criteria for implementation + +These are release gates for follow-up code, **not tests claimed to pass in this PR**. + +- [ ] Existing v1 controls, screen-plane moves, resize, hierarchy selection and + 2D/DOM/Canvas scenes retain their behaviour and serialized history. +- [ ] A rotated character root moves on each world/local axis and plane correctly; + a parent rotation/uniform scale, noncentral pivot and off-screen position do not + introduce jumps. Unsupported transforms produce an actionable disabled state. +- [ ] CSS scaling, letterboxing, DPR 1/2, clipped/nested viewport and perspective/ + orthographic cameras preserve picking and drag direction. Near-parallel axes, + behind-camera objects and near-plane bounds never emit NaN or huge jumps. +- [ ] Overlapping meshes select the correct eligible anchor; locked/hidden objects + cannot initiate gestures. A character limb normally selects its placement root. +- [ ] Translate/rotate/scale inspector values match the gizmo and replay after + save/reload, undo/redo and CLI/agent edits. Numeric and keyboard alternatives work. +- [ ] Auto-key off/on, existing tracks, trimmed/rate-adjusted clips and rational + source rates update only the intended key/value. Quaternion sign equivalence, + 180-degree transitions, normalization and explicit spin tracks are covered. +- [ ] Thousands of pointer events remain bounded; one drag creates one transaction. + The final pointer wins. Cancel, timeout, source rewrite, concurrent agent edit, + seek, camera cut, disposal and lost capture cannot commit stale drafts. +- [ ] Renderer-generated events cannot grant save authority. Unknown operation, + handle, space, version, anchor or field fails without modifying the document. +- [ ] Editor orbit creates no keys/history and does not change export; Use this view + creates one undoable camera edit. Object drag cannot also orbit the view. +- [ ] Light position/target/intensity edits reproduce at the same frame; fields that + have no meaning for that light type do not advertise working affordances. +- [ ] Captured content from preview, direct stills, movies and reference renders + contains no editor helpers; the interactive editing surface may display them. + Forward/backward/random seeks yield the same authored transforms; + do not claim cross-GPU pixel identity. +- [ ] An asynchronous pick begun in editor view A is rejected after orbit to B at + the same source time; subject rendering, helper picking and constraints agree. + Editing a shot camera from a different editor view never moves the editor view. +- [ ] Delayed begin, final evaluation and persistence each exercise release, Escape, + expected/unexpected capture loss and conflicting writes. Lost acknowledgment + reconciles without duplicate history; a failed final point never saves old data. +- [ ] Controls-v1 plus v2 gestures works when negotiated; legacy adapters never + receive v2 input. A v2-only adapter rejects unversioned input. Unsupported bridge + versions, operation/key mismatches and stale handle hits leave state unchanged. +- [ ] Remount preserves persistent semantics/history but invalidates old runtime + bindings. A draft repaint does not invalidate its own gesture. +- [ ] Mid-drag capture followed by cancellation matches the captured committed + revision, not the draft; overlapping editor canvases do not enter its pixels. + Capture concurrent with commit identifies either whole revision, never a mixture. +- [ ] Numeric fixtures cover world snapping under rotated/scaled parents, + axis-preserving bounds, plane bounds/snap conflicts, pivot conventions, scale + pivot-crossing/degenerate starts and quaternion 180-degree sign ties. +- [ ] A GLB with an external dependency is either fully pinned/admitted or rejected; + temporary URLs and private bindings cannot be persisted through transform edits. +- [ ] Standalone and native consumer tests report supported runtime versions, + resource cleanup and latency measurements. A browser-only demo is insufficient + evidence for native-host completion. + +Extend existing [interaction tests](../tests/interaction.test.mjs), +[portable control tests](../tests/portable-controls.test.mjs), +[key timing tests](../tests/key-timing.test.mjs) and renderer/browser suites. +Use conformance fixtures shared with consumers for quaternion semantics and gesture +negotiation. Add meaningful end-to-end tests around trusted input, persistence and +capture, not merely a snapshot of the new schema. + +## 8. Decisions to settle during the spike + +The recommended defaults above allow implementation to start. Resolve these with +measured prototypes, rather than presenting them as existing features: + +- Can pinned TransformControls accept the admitted pointer pipeline cleanly, + including cancellation and the final release, without private API coupling? +- Can the standalone and native host share an overlay adapter, or do they need + separate presenters over the same protocol and math? +- Should the consumer upgrade Three first, or support a small explicit version + matrix? Check loader, material, shadow and existing export regressions. +- What default world unit/character normalization best fits existing scenes? + Store that convention; do not silently convert old world-unit values to metres. +- Which two or three reusable pose targets are worthwhile after root manipulation + works? Do not promise full character animation as a side effect of adding arrows. diff --git a/docs/object-manipulation.md b/docs/object-manipulation.md index 27b18f3..3e89707 100644 --- a/docs/object-manipulation.md +++ b/docs/object-manipulation.md @@ -44,3 +44,8 @@ tracks are untouched on open and converted only when edited. Collisions refuse that track edit with migration guidance. Never convert edited X/Y controls into vectors implicitly, or write object motion into clip transform/param tracks. +## Proposed 3D extension + +For requirements covering XYZ arrows, rotation, cameras, lights and safe host +integration, see [Editable 3D scenes](3d-scene-adjustment-requirements.md). This is +a design proposal; the current screen-plane movement contract above is unchanged. diff --git a/examples/scene-editor/README.md b/examples/scene-editor/README.md new file mode 100644 index 0000000..7a137f0 --- /dev/null +++ b/examples/scene-editor/README.md @@ -0,0 +1,58 @@ +# Local 3D Scene Studio + +A standalone consumer of EditableFrames' spatial editing contracts, with two 12-second scenes: **The exchange** (two-person drama blocking) and **Mecha vs Titan** (procedural robot/giant-ape action). Scene geometry, motion and shot cuts remain ordinary authored Three.js code. There is no generation service or Blender dependency. + +```sh +npm ci +npm run build:scene +npm run dev:scene +``` + +Open `http://127.0.0.1:4189`. The build also produces a self-contained `artifacts/scene-editor/scene-studio.html`; it works offline. Use the localhost server for browser persistence. The page checks that Web Locks and browser storage are usable. If either is unavailable or denied (including restricted local-file contexts), editing runs in memory: use **Download scene** before closing or reloading, and **Open scene** to restore that JSON. It never substitutes unlocked writes to shared browser storage. Downloaded JSON is tied to the source/assets binding that created it; keep the HTML and JSON together. A source rebuild can intentionally make an older snapshot incompatible. + +## Try it + +1. Select an actor or set from the object list, or click its visible geometry. +2. **W** moves along X/Y/Z arrows or planes; **E** rotates; **R** scales uniformly. Choose world/local axes. Red is X, green Y, blue Z. Grid uses 0.25 world/local-parent units, 15 degrees, and 0.1 authored scale units. Bounds take precedence over the grid. Scaling respects authored positive bounds; absent a positive minimum, its fallback floor is 0.001 or the smaller positive maximum. + **Surface snap** is on by default for actors: release a move to settle the evaluated character's lowest geometry onto an admitted surface below (floor/table in the drama; ground, rubble or rooftops in battle). Lift above a table first to place an actor on it. Turn it off for floating/free placement. The final world-Y correction intentionally overrides the drag axis and grid height, keeps orientation, and commits with the move as one undo step. Exact numeric edits, stationary clicks, rotation, scale, lights, cameras and the combined set/floor are unaffected. Escape cancels both movement and snapping. The preference is frozen when the drag starts and resets to on when the page opens. +3. Right-drag orbits the independent director camera; Shift + right-drag pans; scroll dollies. **Use this view as shot** commits the view as the output camera. Editing a shot camera starts from its evaluated position, target and lens. A custom camera replaces the authored shot sequence; keyframe it for a custom camera move. +4. Select **Key light** for position, intensity and color. Select **Shot camera** for position, look-at target and FOV. Camera targets are numeric controls in this slice, not a second target gizmo. +5. Scrub the source-time timeline. Enable **Auto-key**, edit at one frame, then at another. Existing keyed properties continue editing their track even when Auto-key is off. Edits use the current source frame, with smooth temporal easing. Each released drag commits one reversible transaction; Escape discards the draft. Undo/redo and JSON retain history. +6. **Save reference PNG** captures an immutable committed revision through the authored/custom shot camera. Director navigation, selection, helper pixels and active gesture drafts are excluded. Capturing does not commit or cancel an active drag. + +Surface placement uses nine downward probes across the visible posed bounding box, takes the highest upward-facing support they hit, starting 0.25 world units above the higher of the initial and dragged base. This catches downward drags through a floor while allowing only a small upward step from an initially grounded pose. A missing support or a correction outside control bounds leaves the dragged placement unchanged. It is approximate: thin features can fall between probes, wide bounding boxes can bridge gaps, and feet are not individually solved or aligned to slopes. Snapping acts at the current frame; it does not keep an animated character grounded at subsequent frames. + +Actor transforms offset the authored motion. They do not re-solve hand contact, feet, collisions or combat choreography. These are blocking/reference scenes, not physical simulations or photorealistic film renders. The robot and ape are original procedural figures, not licensed franchise models. + +## Optional local rigged characters + +The public build uses original procedural stand-ins. To reuse authorized local models, supply a directory containing `Michelle.glb` and `Civilian.glb`: + +```sh +node examples/scene-editor/build.mjs --assets /path/to/authorized/models +``` + +The supplied models are embedded into the local HTML, hashed into its source binding, and recorded in an ignored asset receipt. No model download occurs. External buffers/textures are refused; models must be self-contained. Rig names compatible with the example's authored arm solver enable its hand poses; other rigs still render but require an authored adapter for that posing. Supplying assets does not establish redistribution rights. Do not commit generated HTML or third-party models without their applicable rights and notices. + +## Integration boundaries + +- `packages/core/rotation.mjs`: normalized quaternion values and sign-invariant shortest-path interpolation, including deterministic 180-degree ties. +- `packages/core/spatial-session.mjs`: independent gesture-v2 negotiation, binding/operation key scopes, complete presentation identity, transient proposals, final-point evaluation, cancellation before publication and reconciliation after uncertain publication. Hosts own trusted admission and durable idempotent writes. Legacy gestures never receive v2 operations. +- `packages/three/spatial-editor.mjs`: pinned Three r180 facade, semantic handle hits, selection, separate helper scene and renderer-local proxies. It forwards no raw `Object3D` into saved state. Parent ancestry must be rigid or positively uniformly scaled; shear/reflection/nonuniform ancestry is refused. +- `packages/three/surface-snap.mjs`: optional visible-pose bounds and downward support probing. A Three adapter host supplies `surfaceTargets`, opts each movable binding into `surfaceSnap:true`, passes the toggle to `begin`, and requests `update(point,{final:true})` only for release. Targets must belong to the same frozen source presentation; external scene changes cancel the gesture. Selected descendants, hidden geometry and steep/underside faces are excluded. +- `packages/three/spatial-math.mjs`: explicit pivot conventions, parent conversion, axis/plane-preserving constraints, snap frame and positive scale saturation. +- `app.mjs`: host wiring to the existing Studio history and controls; local Web Locks plus a saved nonce prevent lost updates between cooperating tabs. JSON snapshots remain a separate download/import operation. This is not the general project server's file transaction API, a remote persistence service or a security sandbox. +- `cinema.mjs`: renderer-specific transient view capability and frozen-content capture. New renderer code can expose different bindings; these controls do not restrict drawable scene types. + +Stitchable native gesture transport, renderer-host capability negotiation across IPC, the native Library importer and KDV presentation are still integration work. The existing generic editor retains its earlier controls/gesture behavior; the new spatial UI is opt-in through this consumer. Movie export UI, arbitrary skeleton posing/IK authoring, model downloading, multi-object transforms and automatic snapshot migration are not provided here. + +## Verification + +```sh +npm test +npm run test:scene +# Optional same browser suite with local rigged models: +EF_SCENE_ASSETS=/path/to/authorized/models npm run test:scene +``` + +The scene suite uses real browser mouse gestures for translation, rotation, scale and Escape; verifies keyframe interpolation, undo/redo, reload, light edits, camera continuity, clean capture during a draft, both scenes and disposal on switching. Core tests cover transformed-parent math, strict bounds, skinned picking, quaternion signs, negotiation, stale views, final evaluation and uncertain publication. It records local screenshots and reference PNGs under ignored `artifacts/scene-editor/`. Browser/renderer tests establish behavior on the tested browser/GPU, not cross-GPU pixel identity. diff --git a/examples/scene-editor/app.mjs b/examples/scene-editor/app.mjs new file mode 100644 index 0000000..d131cea --- /dev/null +++ b/examples/scene-editor/app.mjs @@ -0,0 +1,153 @@ +import * as T from 'three'; +import {OrbitControls} from 'three/addons/controls/OrbitControls.js'; +import {cinemaScene,authoredShot} from './cinema.mjs'; +import {createScenePersistence} from './persistence.mjs'; +import {Studio,HZ} from '../../packages/core/index.mjs'; +import {normalizeClipModule,sampledRendererProps} from '../../packages/core/definition.mjs'; +import {writePointer,readPointer} from '../../packages/core/controls.mjs'; +import {sameOrientation} from '../../packages/core/rotation.mjs'; +import {upsertControlKey} from '../../packages/core/key-timing.mjs'; +import {createSpatialSession,negotiateSpatial} from '../../packages/core/spatial-session.mjs'; +import {ThreeSpatialEditor} from '../../packages/three/spatial-editor.mjs'; + +const $=s=>document.querySelector(s),W=1280,H=720,FPS=24,LAST=287; +const assets=JSON.parse($('#assets').textContent),binding=$('#source-binding').textContent; +const persistence=createScenePersistence(); +let studio,definition,scene,instance,props,editor,orbit,session; +let kind='drama',frame=48,playing=false,autoKey=false,selected='lead',mode='translate',space='world',view='editor'; +let editorCamera=new T.PerspectiveCamera(42,W/H,.05,200),viewRevision=0,viewportRevision=0,generation=0,lastTime=0,drag=null,busy=false,booted=false; +let storageNonce=null,storageKey='',savedLabel='Not yet saved',receipts=[],captureBusy=false; +const caches=new Map(); +const asset=id=>assets.models?.[id]; +function fail(error){if(!booted){busy=false;$('#loading').hidden=true;}$('#status').textContent=error.message??String(error);$('#status').classList.add('error');console.error(error);} +function status(message){$('#status').textContent=message;$('#status').classList.remove('error');} +const clip=()=>studio.doc.clips[0]; +const fields=()=>scene.controls.fields; +const identity=()=>({clipId:clip().id,revision:studio.revision,sourceRevision:binding,seekGeneration:generation,rendererInstance:generation,bindingGeneration:generation,viewRevision,viewportRevision,shot:kind,clock:{frame,fps:FPS}}); +function sample(s=studio,at=frame){const c=s.doc.clips[0];const p=sampledRendererProps(scene,definition.template,c,s.values(c,at/FPS*HZ));if(!p.shot.override)Object.assign(p.shot,authoredShot(p.kind,at/FPS,p.camera==='wide'));return p;} +function draftProps(){const p=sample();for(const [key,value] of Object.entries(session?.state?.values??{})){const f=fields().find(f=>f.key===key);if(f)writePointer(p,f.path,value);}return p;} +function render(){if(!instance)return;const next=draftProps();for(const k of Object.keys(props))delete props[k];Object.assign(props,next);instance.editing.setView(view==='editor'?editorCamera:null);instance.render({frame,fps:FPS,width:W,height:H,capture:false});editor.camera=view==='editor'?editorCamera:instance.editing.camera;editor.transform.camera=editor.camera;editor.render(instance.editing.renderer);$('#clay').checked=!!next.clay;$('#shot-name').textContent=instance.editing.shot;$('#time').textContent=(frame/FPS).toFixed(2)+'s';$('#seek').value=frame;$('#play').textContent=playing?'Pause':'Play';$('#revision').textContent='Revision '+studio.seq+' · '+savedLabel;} +function localPoint(event){const r=instance.editing.renderer.domElement.getBoundingClientRect();return {x:(event.clientX-r.left)/r.width*W,y:(event.clientY-r.top)/r.height*H};} +function commands(values,label='Edit scene',keyMode=autoKey){ + const base=sample(),c=clip(),cmds=[]; + const activatingCamera=values['camera.override']===true&&!base.shot.override; + if(activatingCamera)values={'camera.position':base.shot.position,'camera.target':base.shot.target,'camera.fov':base.shot.fov,...values}; + for(const [key,value] of Object.entries(values)){ + const f=fields().find(f=>f.key===key);if(!f)throw Error('Unknown control');const before=readPointer(base,f.path); + if(!(activatingCamera&&['camera.position','camera.target','camera.fov'].includes(key))&&(f.type==='quaternion'?sameOrientation(before,value):JSON.stringify(before)===JSON.stringify(value)))continue; + const node=studio.node(c,f.anchor),keys=node.keys?.[f.property]??[]; + if(f.animatable&&(keyMode||keys.length))cmds.push({type:'keyframes',clip:c.id,anchor:f.anchor,property:f.property,keys:upsertControlKey(keys,frame/FPS,value,'smooth',{num:FPS,den:1})}); + else cmds.push({type:'set',clip:c.id,anchor:f.anchor,property:f.property,value}); + } + return {type:'batch',label,commands:cmds}; +} +function readSaved(){if(!persistence.supported)return null;const text=persistence.storage.getItem(storageKey);return text?JSON.parse(text):null;} +async function transaction(id,expectedRevision,action){ + const run=()=>{ + const old=studio.serialize(),previousNonce=storageNonce; + if(persistence.supported){let saved;try{saved=readSaved();}catch(error){error.definite=true;throw error;}const receipt=saved?.receipts?.find(r=>r.id===id);if(receipt)return receipt;if((saved?.nonce??null)!==storageNonce){const e=Error('Another window changed this scene. Reload before editing.');e.definite=true;throw e;}} + if(studio.revision!==expectedRevision){const e=Error('Scene changed before the edit was saved');e.definite=true;throw e;} + try{const receipt=action();const record={id,revision:studio.revision,noOp:!!receipt?.noOp};if(!record.noOp){const nonce=crypto.randomUUID(),nextReceipts=[...receipts,record].slice(-64);if(persistence.supported)persistence.storage.setItem(storageKey,JSON.stringify({nonce,raw:studio.serialize(),receipts:nextReceipts}));storageNonce=nonce;receipts=nextReceipts;savedLabel=persistence.supported?'Saved in this browser':'Memory only · download scene to keep';}return record;} + catch(error){studio=Studio.restore(old,definition.templates,binding);storageNonce=previousNonce;error.definite=true;throw error;} + }; + return persistence.run(storageKey,run); +} +function cancelGesture(message='Edit cancelled'){ + if(!drag)return; + const d=drag;if(!session.cancel(d.id)){status('Save in progress; wait for the result before undoing.');return;} + drag=null;editor.end();orbit.enabled=view==='editor';try{d.element.releasePointerCapture(d.pointerId);}catch{} + render();refreshInspector();status(message); +} +async function editValues(values,label='Inspector edit'){ + if(drag||busy)return;playing=false;const command=commands(values,label);if(!command.commands.length)return; + const rev=studio.revision;busy=true;try{await transaction(crypto.randomUUID(),rev,()=>studio.execute(command,rev));render();refreshInspector();status((persistence.supported?'Saved · ':'Applied · ')+label);}catch(e){fail(e);}finally{busy=false;} +} +function setSelection(anchor){if(drag)return;playing=false;frame=Math.round(frame);if(anchor==='camera'&&view==='shot')setView('editor');selected=anchor;editor.setSelection(anchor);refreshInspector();render();} +function setMode(next){if(drag)return;mode=next;editor.setMode(mode,space);refreshInspector();render();} +function setView(next){if(drag)return;if(next==='shot'&&selected==='camera'){selected='lead';editor.setSelection(selected);refreshInspector();}view=next;viewRevision++;orbit.enabled=view==='editor';editor.setCamera(view==='editor'?editorCamera:instance.editing.camera);$('#view').value=view;render();} +function fitView(){if(drag)return;setView('editor');editorCamera.position.fromArray(kind==='drama'?[3.6,2.5,5.6]:[10,6.6,15]);orbit.target.fromArray(kind==='drama'?[0,1.1,0]:[0,2.6,0]);editorCamera.fov=kind==='drama'?42:43;editorCamera.updateProjectionMatrix();orbit.update();viewRevision++;render();} +function refreshInspector(){ + if(!instance)return;const root=$('#inspector');root.replaceChildren();const object=scene.controls.objects.find(o=>o.id===selected);$('#selection-title').textContent=object?.label??'Select an object'; + $('#objects').replaceChildren();for(const o of scene.controls.objects){const b=document.createElement('button');b.textContent=o.label;b.dataset.anchor=o.id;b.className=selected===o.id?'selected':'';b.onclick=()=>setSelection(o.id);$('#objects').append(b);} + const sampled=draftProps(); + for(const f of fields().filter(f=>f.anchor===selected)){ + const wrap=document.createElement('div');wrap.className='field';const label=document.createElement('label');label.textContent=f.property==='orientation'?'Rotation · degrees':f.property+(f.unit?' · '+f.unit:'');wrap.append(label); + const val=readPointer(sampled,f.path);const commit=value=>{const values={[f.key]:value};if(selected==='camera'&&f.key!=='camera.override')values['camera.override']=true;return editValues(values,f.property);}; + if(['vec3','quaternion'].includes(f.type)){ + const row=document.createElement('div');row.className='vector';const vals=f.type==='quaternion'?new T.Euler().setFromQuaternion(new T.Quaternion(...val),'XYZ').toArray().slice(0,3).map(v=>T.MathUtils.radToDeg(v)):val; + for(let i=0;i<3;i++){const cell=document.createElement('label');cell.textContent='XYZ'[i];const input=document.createElement('input');input.type='number';input.step=f.type==='quaternion'?'1':'.1';input.value=Number(vals[i].toFixed(3));input.setAttribute('aria-label',f.key+'.'+'xyz'[i]);input.onchange=()=>{const current=readPointer(sample(),f.path);const next=f.type==='quaternion'?new T.Euler().setFromQuaternion(new T.Quaternion(...current),'XYZ').toArray().slice(0,3).map(T.MathUtils.radToDeg):[...current];next[i]=Number(input.value);commit(f.type==='quaternion'?new T.Quaternion().setFromEuler(new T.Euler(...next.map(T.MathUtils.degToRad),'XYZ')).toArray():next);};cell.append(input);row.append(cell);}wrap.append(row); + }else{const input=document.createElement('input');input.setAttribute('aria-label',f.key);input.type=f.type==='bool'?'checkbox':f.type==='color'?'color':'number';if(f.type==='bool')input.checked=val;else input.value=val;input.step=f.key==='camera.fov'?'1':'.1';if(f.min!==undefined)input.min=f.min;if(f.max!==undefined)input.max=f.max;input.onchange=()=>commit(f.type==='bool'?input.checked:f.type==='color'?input.value:Number(input.value));wrap.append(input);} + if(f.animatable){const keys=studio.node(clip(),f.anchor).keys?.[f.property]??[];const note=document.createElement('small');note.textContent=keys.length?keys.length+' keys · edits update this frame':autoKey?'Auto-key: next edit creates a key':'Base value · no keys';wrap.append(note);}root.append(wrap); + } + $('#undo').disabled=!studio.canUndo;$('#redo').disabled=!studio.canRedo; + for(const b of document.querySelectorAll('[data-mode]'))b.classList.toggle('active',b.dataset.mode===mode); + const cap=editor.capabilities({anchor:selected});$('#capability').textContent=cap.disabledReason??(selected==='lead'||selected==='partner'?'Placement offsets preserve the authored action. Contacts do not automatically re-solve.':'Select a handle or enter an exact value.'); + const lane=$('#keys');lane.replaceChildren();for(const node of clip().nodes)if(node.id===selected)for(const [property,keys] of Object.entries(node.keys??{}))for(const k of keys){const mark=document.createElement('button');mark.className='key';mark.style.left=(k.time/12*100)+'%';mark.title=property+' at '+k.time.toFixed(2)+'s';mark.setAttribute('aria-label',mark.title);mark.onclick=()=>seek(Math.round(k.time*FPS));lane.append(mark);} +} +function seek(at){if(drag)cancelGesture('Drag cancelled by seek');if(session?.state)return;frame=Math.max(0,Math.min(LAST,at));playing=false;render();refreshInspector();} +async function history(method){if(drag){cancelGesture();return;}if(busy)return;playing=false;busy=true;try{const rev=studio.revision;await transaction(crypto.randomUUID(),rev,()=>{studio[method]();return {};});render();refreshInspector();status(method==='undo'?'Undid one edit':'Redid one edit');}catch(e){fail(e);}finally{busy=false;}} +function download(name,content,type='application/json'){const a=document.createElement('a'),url=content.startsWith?.('data:')?content:URL.createObjectURL(new Blob([content],{type}));a.href=url;a.download=name;a.click();if(!url.startsWith('data:'))setTimeout(()=>URL.revokeObjectURL(url),1000);} +async function capture(){ + if(captureBusy)return;captureBusy=true;$('#capture').disabled=true;status('Rendering committed shot…'); + const snapshot=studio.serialize(),at=frame,captureScene=scene,captureDefinition=definition,root=document.createElement('div');let captureInstance; + try{const frozen=Studio.restore(snapshot,captureDefinition.templates,binding),c=frozen.doc.clips[0],p=sampledRendererProps(captureScene,captureDefinition.template,c,frozen.values(c,at/FPS*HZ)); + captureInstance=await captureScene.create({root,props:p,asset});captureInstance.render({frame:at,fps:FPS,width:W,height:H,capture:true});const png=captureInstance.png(); + window.cinema.lastCapture={png,frame:at,raw:snapshot,props:p};download(kind+'-reference-'+at+'.png',png);status('Saved clean PNG from committed shot · frame '+at);return png; + }catch(e){fail(e);}finally{captureInstance?.dispose();captureBusy=false;$('#capture').disabled=false;} +} +async function load(nextKind){ + if(drag||busy||captureBusy)return;busy=true;booted=false;$('#loading').hidden=false;playing=false; + if(studio)caches.set(kind,{raw:studio.serialize(),frame,storageNonce,receipts}); + editor?.dispose();orbit?.dispose();instance?.dispose();$('#canvas').replaceChildren();kind=nextKind;generation++;viewRevision++;frame=caches.get(kind)?.frame??48; + scene=cinemaScene(kind);definition=normalizeClipModule({default:scene});storageKey='editableframes.scene-editor.'+binding+'.'+kind; + let saved=null;try{if(persistence.supported)saved=readSaved();}catch{} + const cached=caches.get(kind);if(cached){studio=Studio.restore(cached.raw,definition.templates,binding);storageNonce=cached.storageNonce;receipts=cached.receipts;} + else if(saved){try{studio=Studio.restore(saved.raw,definition.templates,binding);storageNonce=saved.nonce;receipts=saved.receipts??[];}catch{saved=null;}} + if(!cached&&!saved){studio=new Studio(definition.templates,{binding});studio.doc.clips=[studio.make(scene.id)];studio.base=structuredClone(studio.doc);storageNonce=null;receipts=[];} + props=sample();instance=await scene.create({root:$('#canvas'),props,asset}); + editorCamera=new T.PerspectiveCamera(42,W/H,.05,200);orbit=new OrbitControls(editorCamera,instance.editing.renderer.domElement);orbit.enableDamping=false;orbit.mouseButtons.LEFT=null;orbit.mouseButtons.RIGHT=T.MOUSE.ROTATE;orbit.mouseButtons.MIDDLE=T.MOUSE.DOLLY; + editor=new ThreeSpatialEditor({controls:scene.controls,props,objects:instance.editing.objects,camera:editorCamera,width:W,height:H,surfaceTargets:instance.editing.surfaceTargets});selected='lead';mode='translate';space='world';$('#space').value=space;editor.setSelection(selected);editor.setMode(mode,space); + session=createSpatialSession({controls:scene.controls,readIdentity:identity,publish:async proposal=>{ + if(proposal.anchor==='camera'&&proposal.request.operation==='translate'&&new T.Vector3(...proposal.values['camera.position']).distanceTo(new T.Vector3(...sample().shot.position))<1e-9)return {noOp:true}; + const command=commands(proposal.values,'Transform '+proposal.anchor+' · '+proposal.request.operation,drag?.autoKey??autoKey); + return transaction(proposal.id,proposal.identity.revision,()=>command.commands.length?studio.execute(command,proposal.identity.revision):{noOp:true}); + },reconcile:id=>{const r=readSaved()?.receipts?.find(r=>r.id===id);return r?{status:'committed',receipt:r}:{status:'uncertain'};}}); + const canvas=instance.editing.renderer.domElement; + canvas.addEventListener('pointerdown',event=>{ + if(busy||drag||event.button!==0||!event.isTrusted)return; + const point=localPoint(event),hit=editor.hitHandle(point); + if(hit){ + event.stopImmediatePropagation();event.preventDefault();playing=false;frame=Math.round(frame);orbit.enabled=false; + try{render();const observed=identity(),capabilities=editor.capabilities({anchor:selected});if(!negotiateSpatial({host:[2],transport:[2],adapter:capabilities.gestureProtocols}))throw Error('Spatial protocol unavailable'); + const b=session.begin({trusted:event.isTrusted,identity:observed,anchor:selected,capabilities,...hit});editor.begin(hit,point,{translationSnap:$('#snap').checked?.25:0,rotationSnap:$('#snap').checked?Math.PI/12:0,scaleSnap:$('#snap').checked?.1:0,surfaceSnap:$('#surface-snap').checked});drag={id:b.id,pointerId:event.pointerId,element:canvas,autoKey};canvas.setPointerCapture(event.pointerId);status('Drag '+hit.handle.toUpperCase()+(editor.active.surfaceSnap&&hit.operation==='translate'?' · release to snap to a surface':'')+' · Escape cancels'); + }catch(e){if(session.state)session.cancel(session.state.id);editor.end();orbit.enabled=view==='editor';fail(e);}return; + } + const anchor=editor.pick(point,instance.editing.scene);if(anchor)setSelection(anchor); + },true); + canvas.addEventListener('pointermove',event=>{ + if(!drag||drag.pointerId!==event.pointerId)return;event.stopImmediatePropagation(); + try{session.renew(drag.id);const result=editor.update(localPoint(event));session.update({id:drag.id,values:result.values});render();}catch(e){cancelGesture(e.message);} + },true); + canvas.addEventListener('pointerup',async event=>{ + if(!drag||drag.pointerId!==event.pointerId||!event.isTrusted)return;event.stopImmediatePropagation();const d=drag;d.released=true; + try{const result=await session.release({id:d.id,trusted:event.isTrusted,evaluate:()=>editor.update(localPoint(event),{final:true})});if(result.status==='uncertain'){status('Save outcome uncertain; reload to reconcile before editing.');return;}drag=null;editor.end();orbit.enabled=view==='editor';render();refreshInspector();status(result.status==='cancelled'?'Edit cancelled':(persistence.supported?'Saved one reversible transform':'Applied transform · download scene to keep'));} + catch(e){drag=null;editor.end();orbit.enabled=view==='editor';render();refreshInspector();fail(e);} + finally{try{canvas.releasePointerCapture(event.pointerId);}catch{}} + },true); + canvas.addEventListener('pointercancel',()=>cancelGesture());canvas.addEventListener('lostpointercapture',()=>{if(drag&&!drag.released)cancelGesture('Pointer capture lost');}); + orbit.addEventListener('change',()=>{if(booted&&!drag){viewRevision++;render();}}); + fitView();booted=true;busy=false;$('#loading').hidden=true;$('#scene-kind').value=kind;$('#assets-note').textContent=kind==='battle'?'Original procedural robot and ape · no model downloads':assets.models?.michelle?'Local rigged GLB characters · embedded assets':'Original procedural stand-ins · no external downloads';refreshInspector();render();status('Ready · select an actor and drag an axis'); +} +$('#scene-kind').onchange=e=>load(e.target.value).catch(fail);for(const b of document.querySelectorAll('[data-mode]'))b.onclick=()=>setMode(b.dataset.mode); +$('#space').onchange=e=>{if(drag)return;space=e.target.value;editor.setMode(mode,space);render();};$('#view').onchange=e=>setView(e.target.value);$('#fit').onclick=fitView; +$('#use-view').onclick=()=>{if(view!=='editor')return;editValues({'camera.position':editorCamera.position.toArray(),'camera.target':orbit.target.toArray(),'camera.fov':editorCamera.fov,'camera.override':true},'Use director view as shot');}; +$('#play').onclick=()=>{if(drag)return;playing=!playing;lastTime=performance.now();if(!playing)frame=Math.round(frame);render();refreshInspector();};$('#seek').oninput=e=>seek(Number(e.target.value));$('#auto-key').onchange=e=>{autoKey=e.target.checked;refreshInspector();}; +$('#undo').onclick=()=>history('undo');$('#redo').onclick=()=>history('redo');$('#capture').onclick=capture; +$('#clay').onchange=e=>editValues({'look.clay':e.target.checked},'Reference look'); +$('#save').onclick=()=>download(kind+'-scene.json',studio.serialize());$('#open').onchange=async e=>{if(drag||busy){e.target.value='';return;}busy=true;playing=false;try{const raw=await e.target.files[0].text();const restored=Studio.restore(raw,definition.templates,binding);if(restored.doc.clips.length!==1||restored.doc.clips[0].template!==scene.id)throw Error('Choose a snapshot for this scene');const rev=studio.revision;await transaction(crypto.randomUUID(),rev,()=>{studio=restored;return {};});render();refreshInspector();status('Opened saved scene');}catch(error){fail(error);}finally{busy=false;e.target.value='';}}; +window.addEventListener('keydown',e=>{if(e.key==='Escape'&&drag){e.preventDefault();cancelGesture();return;}if(['INPUT','SELECT','TEXTAREA'].includes(e.target.tagName))return;if(e.key==='Escape'){cancelGesture();return;}if((e.ctrlKey||e.metaKey)&&e.key.toLowerCase()==='z'){e.preventDefault();history(e.shiftKey?'redo':'undo');return;}if(e.code==='Space'){e.preventDefault();$('#play').click();}if(!drag&&['w','e','r'].includes(e.key.toLowerCase()))setMode({w:'translate',e:'rotate',r:'scale'}[e.key.toLowerCase()]);}); +window.addEventListener('resize',()=>{viewportRevision++;if(drag)cancelGesture('Viewport changed');}); +window.addEventListener('storage',event=>{if(event.key===storageKey){if(drag)cancelGesture('Another window changed the scene');status('Another window saved this scene. Reload to adopt its changes.');}}); +window.cinema={get ready(){return booted;},get state(){return {kind,frame,selected,mode,view,persistence:persistence.supported?'browser':'memory',props:sample(),draft:session?.state,revision:studio?.revision,history:studio?.history.length,keys:studio?.doc.clips[0]?.nodes,identity:identity()};},select:setSelection,seek,edit:editValues,undo:()=>history('undo'),redo:()=>history('redo'),capture,serialize:()=>studio.serialize(),get editor(){return editor;},get instance(){return instance;}}; +function loop(time){if(booted&&playing&&!drag){frame=(frame+Math.min(.1,(time-lastTime)/1000)*FPS)%LAST;render();}lastTime=time;requestAnimationFrame(loop);}requestAnimationFrame(loop); +persistence.initialize().then(()=>{savedLabel=persistence.supported?'Not yet saved':'Memory only · download scene to keep';return load('drama');}).catch(fail); diff --git a/examples/scene-editor/build.mjs b/examples/scene-editor/build.mjs new file mode 100644 index 0000000..3f6a6e1 --- /dev/null +++ b/examples/scene-editor/build.mjs @@ -0,0 +1,20 @@ +import fs from 'node:fs/promises';import path from 'node:path';import {fileURLToPath} from 'node:url';import {createHash} from 'node:crypto';import {build} from 'esbuild'; +const root=path.resolve(path.dirname(fileURLToPath(import.meta.url)),'../..'); +const dir=path.join(root,'examples/scene-editor'),out=path.join(root,'artifacts/scene-editor'); +export function validateGLB(bytes){ + if(bytes.readUInt32LE(0)!==0x46546c67||bytes.readUInt32LE(4)!==2||bytes.readUInt32LE(8)!==bytes.length)throw Error('Invalid GLB'); + const size=bytes.readUInt32LE(12);if(bytes.readUInt32LE(16)!==0x4e4f534a||20+size>bytes.length)throw Error('Invalid GLB JSON'); + const data=JSON.parse(bytes.subarray(20,20+size).toString('utf8')); + const visit=v=>{if(v&&typeof v==='object')for(const [k,x] of Object.entries(v)){if(k==='uri'&&(typeof x!=='string'||!x.startsWith('data:')))throw Error('Model has an external dependency');visit(x);}};visit(data);return data; +} +export async function buildScene({assetDirectory}={}){ + const result=await build({entryPoints:[path.join(dir,'app.mjs')],bundle:true,format:'iife',target:'es2022',minify:true,write:false,metafile:true}); + const models={},hashes={}; + if(assetDirectory)for(const [id,file] of [['michelle','Michelle.glb'],['civilian','Civilian.glb']]){const bytes=await fs.readFile(path.join(assetDirectory,file));validateGLB(bytes);models[id]='data:model/gltf-binary;base64,'+bytes.toString('base64');hashes[id]=createHash('sha256').update(bytes).digest('hex');} + const js=result.outputFiles[0].text,css=await fs.readFile(path.join(dir,'style.css'),'utf8'),template=await fs.readFile(path.join(dir,'index.html'),'utf8'); + const binding=createHash('sha256').update(js).update(css).update(template).update(JSON.stringify(hashes)).digest('hex'); + const html=template.replace('__STYLE__',css).replace('__ASSETS__',JSON.stringify({models,hashes}).replaceAll('<','\\u003c')).replace('__BINDING__',binding).replace('__SCRIPT__',()=>js.replaceAll('=0?process.argv[index+1]:undefined});} diff --git a/examples/scene-editor/cinema.mjs b/examples/scene-editor/cinema.mjs new file mode 100644 index 0000000..eb5f468 --- /dev/null +++ b/examples/scene-editor/cinema.mjs @@ -0,0 +1,178 @@ +import {defineRenderer} from '../../packages/core/renderer.mjs'; +import * as T from 'three'; +import {GLTFLoader} from 'three/addons/loaders/GLTFLoader.js'; +const W=1280,H=720,FPS=24,N=288; +const clamp=T.MathUtils.clamp,mix=T.MathUtils.lerp; +const smooth=(a,b,t)=>{const x=clamp((t-a)/(b-a),0,1);return x*x*(3-2*x)}; +const pulse=(a,b,c,t)=>smooth(a,b,t)*(1-smooth(b,c,t)); +const V=(x=0,y=0,z=0)=>new T.Vector3(x,y,z); + + +/** Authored camera values are sampled, not the static fallback props. */ +export function authoredShot(kind,t,wide=false){ + let position,target,fov,label; + if(kind==='drama'){ + if(wide||t<3){position=[2.7-t*.09,1.9,4.3];target=[0,1.15,0];fov=42;label='01 / ESTABLISHING TWO-SHOT';} + else if(t<6){position=[-1.42,1.62,1.15];target=[.55,1.53,0];fov=40;label='02 / OVER HER SHOULDER';} + else if(t<9){position=[1.35,1.58,1.05];target=[-.55,1.47,0];fov=35;label='03 / REVERSE CLOSE-UP';} + else{position=[.1,1.65,1.65];target=[.08,1.15,.3];fov=30;label='04 / INSERT — SLIDING THE EVIDENCE';} + }else{ + if(wide||t<2.8){position=[9-t*.6,5.1,13];target=[0,2.6,0];fov=43;label='01 / REVEAL — ROBOT VS APE';} + else if(t<4.6){position=[-.5,3.3,8.6];target=[0,3.4,0];fov=46;label='02 / SWING → CONTACT → BLOCK';} + else if(t<7.3){position=[-5.6,4.4,9.2];target=[.35,3.8,0];fov=40;label='03 / COUNTERPUNCH → RECOIL';} + else{position=[-8.5+(t-7.3)*.35,4.7,11.5];target=[-.3,2.9,0];fov=43;label='04 / GRAPPLE → SEPARATION';} + }return {position,target,fov,label}; +} + +const placement=()=>({position:[0,0,0],orientation:[0,0,0,1],scale:1}); +export function cinemaScene(kind='drama') { + const names=kind==='drama'?['Maya · lead','Elias · partner']:['Mecha · robot','Titan · giant ape']; + const defaults={kind,clay:false,camera:'edit',soldier:'civilian',michelle:'michelle',actors:{lead:placement(),partner:placement()},set:placement(),key:{position:[-5,10,7],intensity:3.5,color:'#ffdfb8'},shot:{override:false,position:kind==='drama'?[2.7,1.9,4.3]:[9,5.1,13],target:kind==='drama'?[0,1.15,0]:[0,2.6,0],fov:kind==='drama'?42:43}}; + const fields=[]; + for(const [anchor,path] of [['lead','/actors/lead'],['partner','/actors/partner'],['set','/set']])fields.push( + {key:anchor+'.position',anchor,property:'position',path:path+'/position',type:'vec3',role:'position',coordinateSpace:'local',unit:'world-unit',animatable:true}, + {key:anchor+'.orientation',anchor,property:'orientation',path:path+'/orientation',type:'quaternion',role:'rotation',coordinateSpace:'local',animatable:true}, + {key:anchor+'.scale',anchor,property:'scale',path:path+'/scale',type:'number',role:'scale',min:.05,max:5,animatable:true}); + fields.push( + {key:'key.position',anchor:'key',property:'position',path:'/key/position',type:'vec3',role:'position',coordinateSpace:'local',unit:'world-unit',animatable:true}, + {key:'key.intensity',anchor:'key',property:'intensity',path:'/key/intensity',type:'number',min:0,max:15,animatable:true}, + {key:'key.color',anchor:'key',property:'color',path:'/key/color',type:'color'}, + {key:'camera.position',anchor:'camera',property:'position',path:'/shot/position',type:'vec3',role:'position',coordinateSpace:'local',unit:'world-unit',animatable:true}, + {key:'camera.target',anchor:'camera',property:'target',path:'/shot/target',type:'vec3',unit:'world-unit',animatable:true}, + {key:'camera.fov',anchor:'camera',property:'fov',path:'/shot/fov',type:'number',min:15,max:100,unit:'°',animatable:true}, + {key:'camera.override',anchor:'camera',property:'override',path:'/shot/override',type:'bool'}, + {key:'look.clay',anchor:'look',property:'clay',path:'/clay',type:'bool'}); + const controls={version:2,spatialSemantics:'cinema-placement-pivot-v1',objects:[{id:'lead',label:names[0]},{id:'partner',label:names[1]},{id:'set',label:'Set & props'},{id:'key',label:'Key light'},{id:'camera',label:'Shot camera'},{id:'look',label:'Reference look'}],fields}; + return defineRenderer({id:'character-cinema-'+kind,renderer:'three',width:W,height:H,fps:FPS,durationInFrames:N,controls, + defaultProps:defaults, + async create({root,props,asset}){ + const renderer=new T.WebGLRenderer({antialias:true,preserveDrawingBuffer:true});renderer.setSize(W,H);renderer.outputColorSpace=T.SRGBColorSpace;renderer.toneMapping=T.ACESFilmicToneMapping;renderer.toneMappingExposure=1.2;renderer.shadowMap.enabled=true;renderer.shadowMap.type=T.PCFSoftShadowMap;renderer.domElement.style.cssText='width:100%;height:100%;display:block';root.append(renderer.domElement); + const scene=new T.Scene(),camera=new T.PerspectiveCamera(42,W/H,.05,160);scene.background=new T.Color('#101e2b');scene.fog=new T.FogExp2('#182c3c',.018); + const mat=(c,metal=0,rough=.6)=>new T.MeshStandardMaterial({color:c,metalness:metal,roughness:rough}); + const glow=(c,n=3)=>new T.MeshStandardMaterial({color:c,emissive:c,emissiveIntensity:n,roughness:.3}); + const mesh=(g,m,p=scene,pos=V(),scale=V(1,1,1))=>{const o=new T.Mesh(g,m);o.position.copy(pos);o.scale.copy(scale);o.castShadow=true;o.receiveShadow=true;p.add(o);return o}; + const box=(w,h,d,m,p=scene,x=0,y=0,z=0)=>mesh(new T.BoxGeometry(w,h,d),m,p,V(x,y,z)); + const ell=(x,y,z,m,p=scene,a=0,b=0,c=0)=>mesh(new T.SphereGeometry(1,24,16),m,p,V(a,b,c),V(x,y,z)); + const cylinder=(r,h,m,p=scene,x=0,y=0,z=0)=>mesh(new T.CylinderGeometry(r,r,h,24),m,p,V(x,y,z)); + function link(a,b,r1,r2,m,parent){const o=mesh(new T.CylinderGeometry(r2,r1,1,12),m,parent);o.position.copy(a).add(b).multiplyScalar(.5);o.scale.y=a.distanceTo(b);o.quaternion.setFromUnitVectors(V(0,1,0),b.clone().sub(a).normalize());return o;} + // Deterministic two-link IK. Targets and elbow poles are in the same coordinate space. + function elbow(s,target,l1,l2,pole){const d=target.clone().sub(s),len=clamp(d.length(),.001,l1+l2-.001);d.normalize();const a=(l1*l1-l2*l2+len*len)/(2*len),h=Math.sqrt(Math.max(0,l1*l1-a*a));const perpendicular=pole.clone().sub(s).addScaledVector(d,-pole.clone().sub(s).dot(d)).normalize();return s.clone().addScaledVector(d,a).addScaledVector(perpendicular,h);} + const hemi=new T.HemisphereLight('#a4c4e6','#241a16',1.4);scene.add(hemi); + const key=new T.DirectionalLight('#ffdfb8',3.5);key.position.set(-5,10,7);key.castShadow=true;key.shadow.mapSize.set(2048,2048);Object.assign(key.shadow.camera,{left:-14,right:14,top:18,bottom:-10});key.shadow.normalBias=.02;scene.add(key); + const rim=new T.DirectionalLight('#518dde',3);rim.position.set(8,7,-5);scene.add(rim); + // A cheap, local lighting environment gives metal readable reflections without a fetched HDRI. + const env=new T.Scene();env.background=new T.Color('#455260');for(const [x,y,z,c] of [[-5,5,0,'#fff0d0'],[5,2,1,'#7bbbdc'],[0,6,-5,'#c4d3dc']]){const panel=box(4,4,.1,new T.MeshBasicMaterial({color:c}),env,x,y,z);panel.lookAt(0,0,0);}const pm=new T.PMREMGenerator(renderer),envTarget=pm.fromScene(env,.02);scene.environment=envTarget.texture; + let animate,shotName=''; + const editable={},placements=[],surfaceTargets=[]; + const support=object=>{surfaceTargets.push(object);return object;}; + const register=(id,object)=>{const wrapper=new T.Group();object.parent.add(wrapper);wrapper.add(object);const pivot=V();editable[id]={object:wrapper,surfaceSnap:true,pivot:()=>pivot.toArray(),convention:'pivot'};placements.push({id,object,wrapper,pivot});}; + if(props.kind==='drama'){ + scene.fog.density=.035;renderer.toneMappingExposure=1.1; + const ground=mat('#293338',.15,.45),wall=mat('#334854');support(box(20,.1,15,ground,scene,0,-.08,0));box(12,4,.15,wall,scene,0,2,-2.4); + // Concrete station room, night city and practical lighting. + box(4.8,2.5,.2,mat('#091828'),scene,0,2.1,-2.28); + for(let x=-2.4;x<=2.4;x+=1.2)box(.055,2.5,.16,mat('#71818b',.5),scene,x,2.1,-2.13); + for(let i=0;i<28;i++){const x=-2.2+(i%7)*.65,y=1.1+Math.floor(i/7)*.5;box(.1,.13,.02,glow(i%3?'#5bafbd':'#efb36c',1.4),scene,x,y,-2.14);} + for(const x of [-3.0,3.0]){box(.09,2.8,.12,glow('#e2ad6b',2),scene,x,1.7,-2.08);const light=new T.PointLight('#f7ad65',12,8,2);light.position.set(x,2,-1);scene.add(light);} + support(box(2.2,.12,.9,mat('#593f32',.1,.4),scene,0,1.0,.58));for(const x of [-1,1])box(.1,1,.65,mat('#17222a',.6),scene,x,.5,.58);const fill=new T.PointLight('#d7e9ff',11,10,2);fill.position.set(0,2.8,3);scene.add(fill); + const device=box(.13,.018,.24,mat('#0b1219',.6),scene,0,1.12,.1);box(.11,.003,.20,glow('#76c8dd',.8),device,0,.012,0); + const loader=new GLTFLoader(); + function makeStandIn(id){const root=new T.Group(),cloth=mat(id==='michelle'?'#8f4f40':'#354a62'),skin=mat('#c99677'),hair=mat('#292124');ell(.24,.43,.15,cloth,root,0,1.18,0);ell(.15,.21,.15,skin,root,0,1.78,0);ell(.16,.12,.16,hair,root,0,1.92,-.02);for(const side of [-1,1]){link(V(side*.13,.92,0),V(side*.14,.2,0),.085,.1,cloth,root);ell(.09,.07,.16,hair,root,side*.14,.1,.05);link(V(side*.26,1.45,0),V(side*.34,.95,.12),.07,.09,cloth,root);ell(.065,.1,.06,skin,root,side*.34,.88,.12);}return root;} + async function loadCharacter(id,targetHeight,x,yaw){const url=asset?.(id);const gltf=url?await loader.loadAsync(url):{scene:makeStandIn(id),animations:[]};const model=gltf.scene;model.updateMatrixWorld(true);const bounds=new T.Box3().setFromObject(model);const s=targetHeight/bounds.getSize(V()).y;model.scale.multiplyScalar(s);model.updateMatrixWorld(true);const b=new T.Box3().setFromObject(model);model.position.y-=b.min.y;const group=new T.Group();group.add(model);group.position.x=x;group.rotation.y=yaw;scene.add(group);model.traverse(o=>{if(o.isMesh){o.castShadow=true;o.receiveShadow=true;}});const bones={};model.traverse(o=>{if(o.isBone)bones[o.name.replace('mixamorig:','').replace('mixamorig','')]=o;});const rest=new Map(Object.values(bones).map(b=>[b,b.quaternion.clone()]));return {group,model,bones,rest,gltf};} + const woman=await loadCharacter(String(props.michelle),1.72,-.75,Math.PI/2),man=await loadCharacter(String(props.soldier),1.87,.75,-Math.PI/2); + register('lead',woman.group);register('partner',man.group); + const manPositions=new Map(Object.values(man.bones).map(b=>[b,b.position.clone()])); + // Preserve skinned human geometry; author poses by joint targets rather than primitive mannequins. + function aimBone(bone,child,target){if(!bone||!child)return;scene.updateMatrixWorld(true);const p=bone.getWorldPosition(V()),old=child.getWorldPosition(V()).sub(p).normalize(),desired=target.clone().sub(p).normalize();const q=new T.Quaternion().setFromUnitVectors(old,desired).multiply(bone.getWorldQuaternion(new T.Quaternion()));bone.quaternion.copy(bone.parent.getWorldQuaternion(new T.Quaternion()).invert().multiply(q));scene.updateMatrixWorld(true);} + function arm(a,side,target,pole){const upper=a.bones[side+'Arm'],fore=a.bones[side+'ForeArm'],hand=a.bones[side+'Hand'];if(!upper||!fore||!hand)return;scene.updateMatrixWorld(true);const s=upper.getWorldPosition(V()),e=fore.getWorldPosition(V()),h=hand.getWorldPosition(V());const bend=elbow(s,target,s.distanceTo(e),e.distanceTo(h),pole);aimBone(upper,fore,bend);aimBone(fore,hand,target);} + const initialPositions=new Map(Object.values(woman.bones).map(b=>[b,b.position.clone()])); + animate=t=>{ + for(const [b,q] of woman.rest){b.quaternion.copy(q);b.position.copy(initialPositions.get(b));} + for(const [b,q] of man.rest){b.quaternion.copy(q);b.position.copy(manPositions.get(b));} + woman.group.position.set(-.55,0,0);woman.group.rotation.y=Math.PI/2; + man.group.position.set(.55,0,0);man.group.rotation.y=-Math.PI/2; + scene.updateMatrixWorld(true); + const offer=smooth(0,3,t),take=smooth(9,11.5,t); + const phoneX=mix(-.2,.2,take);const hand=V(mix(-.48,phoneX,offer),mix(.95,1.14,offer),.33); + arm(woman,'Right',hand,V(-.8,.85,.75));arm(woman,'Left',V(-.64,.85,-.15),V(-1.1,.95,-.55)); + arm(man,'Left',V(mix(.50,.20,take),mix(.85,1.14,take),.34),V(.8,.95,.7));arm(man,'Right',V(.60,.85,-.1),V(1.2,1,-.5)); + woman.bones.Head?.rotateY(.10*Math.sin(t*.75));woman.bones.Head?.rotateX(-.07*smooth(8,10,t)); + man.bones.Head?.rotateX(.10*smooth(8,10,t)); + scene.updateMatrixWorld(true);device.position.set(phoneX,1.075,.39);device.rotation.y=.2;man.model.traverse(o=>{if(o.morphTargetDictionary){for(const [name,index] of Object.entries(o.morphTargetDictionary)){if(name==='mouthOpen')o.morphTargetInfluences[index]=t>3&&t<6?.07*Math.max(0,Math.sin(t*7)):0;}}}); + + }; + }else{ + renderer.toneMappingExposure=1.35;hemi.intensity=1.6;scene.fog.density=.022; + const asphalt=mat('#30383c',.2,.75);support(box(90,.2,80,asphalt,scene,0,-.15,0)); + for(let i=0;i<20;i++){const x=(i%2?1:-1)*(8+(i%4)*2),z=-8-Math.floor(i/2)*3,h=5+(i*7%9);support(box(3,h,2.5,mat(i%2?'#273743':'#33404b'),scene,x,h/2,z));for(let y=1;yl.l1+l.l2-.001)target=l.shoulder.clone().add(delta.setLength(l.l1+l.l2-.001));const e=elbow(l.shoulder,target,l.l1,l.l2,pole);for(const [o,a,b] of [[l.upper,l.shoulder,e],[l.lower,e,target]]){o.position.copy(a).add(b).multiplyScalar(.5);o.scale.y=a.distanceTo(b);o.quaternion.setFromUnitVectors(V(0,1,0),b.clone().sub(a).normalize());}l.joint.position.copy(e);l.hand.position.copy(target);} + const robotArms=[makeLimb(robot,-1),makeLimb(robot,1)];for(const limb of robotArms){box(.68,.68,.52,orange,limb.upper,0,.02,.08);box(.68,.65,.58,blue,limb.lower,0,0,.08);for(let i=0;i<3;i++)box(.5,.05,.08,steel,limb.lower,0,-.18+i*.16,.4);} + const robotLegs=[];for(const s of [-1,1]){const leg=new T.Group();leg.position.set(s*.48,2.05,0);robot.add(leg);box(.6,.95,.6,steel,leg,0,-.4,0);box(.7,.9,.7,blue,leg,0,-1.3,.1);box(.74,.32,1.12,black,leg,0,-1.9,.3);box(.5,.3,.1,orange,leg,0,-.85,.4);robotLegs.push(leg);} + const fur=mat('#251b16',.05,.9),skin=mat('#49392d',0,.85),mouth=mat('#170e0c'),ivory=mat('#dac5a0'); + const ape=new T.Group();scene.add(ape);ell(1.14,1.7,.85,fur,ape,0,2.65,-.1);ell(.92,1.03,.2,skin,ape,0,2.8,.72);ell(1.1,.8,.78,fur,ape,0,3.7,.05); + const apeHead=new T.Group();apeHead.position.set(0,4.05,.35);ape.add(apeHead);ell(.7,.86,.65,fur,apeHead,0,.35,0);ell(.59,.5,.31,skin,apeHead,0,.2,.5);ell(.45,.15,.2,skin,apeHead,0,.65,.57);ell(.28,.17,.18,mouth,apeHead,0,.32,.79);for(const s of [-1,1]){ell(.13,.09,.07,ivory,apeHead,s*.27,.59,.67);ell(.055,.065,.055,mouth,apeHead,s*.25,.59,.73);ell(.11,.07,.05,mouth,apeHead,s*.12,.32,.95);} + const jaw=ell(.35,.26,.16,mouth,apeHead,0,.03,.77);for(const s of [-1,1]){const tooth=mesh(new T.ConeGeometry(.08,.23,8),ivory,apeHead,V(s*.22,.10,.90));tooth.rotation.z=Math.PI;} + // Fur tufts add a readable gorilla silhouette, all locally authored geometry. + const tuftGeo=new T.ConeGeometry(.045,.09,4);const tufts=new T.InstancedMesh(tuftGeo,fur,440),dummy=new T.Object3D();for(let i=0;i<440;i++){const y=-1+2*(i+.5)/440,a=i*2.399963,r=Math.sqrt(1-y*y);const n=V(Math.cos(a)*r,y,Math.sin(a)*r);dummy.position.set(n.x*1.15,2.7+n.y*1.65,n.z*.86-.1);dummy.quaternion.setFromUnitVectors(V(0,1,0),n);dummy.scale.setScalar(.6+(i%7)/10);dummy.updateMatrix();tufts.setMatrixAt(i,dummy.matrix);}tufts.castShadow=true;ape.add(tufts); + for(const s of [-1,1]){ell(.48,.87,.5,fur,ape,s*.56,1.05,-.05);ell(.44,.28,.68,skin,ape,s*.62,.28,.32);for(let i=0;i<4;i++)ell(.12,.15,.27,skin,ape,s*.62+(i-1.5)*.17,.18,.78);} + const apeArms=[makeLimb(ape,-1,true),makeLimb(ape,1,true)]; + register('lead',robot);register('partner',ape); + const particles=[];for(let i=0;i<45;i++){const o=mesh(new T.IcosahedronGeometry(.06+(i%4)*.035,0),i%3?mat('#766954'):glow('#ffc16a',2),scene);particles.push(o);} + function local(root,p){root.updateMatrixWorld(true);return root.worldToLocal(p.clone());} + animate=t=>{ + const deploy=smooth(0,1.8,t),approach=smooth(1.3,3,t),hit=pulse(4.6,5.4,6.5,t),recoil=pulse(5.25,5.8,7.3,t),push=pulse(7.2,8.7,10.7,t); + robot.position.set(mix(-3.3,-1.65,approach)+hit*.75-push*.6,0,0);robot.rotation.y=Math.PI/2;robot.scale.y=mix(.67,1,deploy); + ape.position.set(mix(3.2,1.7,approach)+recoil*.7,0,0);ape.rotation.y=-Math.PI/2;ape.rotation.z=recoil*.15; + wings.forEach(({wing,s})=>{wing.rotation.z=s*mix(1.3,.18,deploy);wing.rotation.y=s*.35});robotLegs.forEach((l,i)=>l.rotation.x=(i?1:-1)*.12*Math.sin(t*4)*(1-approach)); + apeHead.rotation.x=-.15*pulse(.2,1.1,2.2,t)+recoil*.25;jaw.scale.y=1+1.7*pulse(.2,1,2,t); + scene.updateMatrixWorld(true); + let rr=robot.localToWorld(V(.95,2.9,.65)),rl=robot.localToWorld(V(-.95,3.0,.75)),ar=ape.localToWorld(V(1.3,1.7,.7)),al=ape.localToWorld(V(-1.3,1.7,.7)); + // Ape swing meets the robot's block; later the robot fist reaches the ape's face. + const swing=pulse(2.4,3.6,4.5,t),block=pulse(2.2,3.5,4.5,t);ar.lerp(V(-.05,3.85,.48),swing);rl.lerp(V(-.12,3.85,.48),block); + rl.lerp(ape.localToWorld(V(0,4.35,.95)),hit);al.lerp(V(-.1-push*.3,3.5,-.45),push);rl.lerp(V(-.1-push*.3,3.5,-.45),push); + for(const [l,target,s] of [[robotArms[0],rl,-1],[robotArms[1],rr,1]])setLimb(l,local(robot,target),V(s*2,3.1,-.5)); + for(const [l,target,s] of [[apeArms[0],al,-1],[apeArms[1],ar,1]])setLimb(l,local(ape,target),V(s*2.2,2.8,-.4)); + const impact=Math.max(pulse(3.5,3.65,4.2,t),pulse(5.35,5.5,6.2,t));particles.forEach((o,i)=>{o.visible=impact>.01;const a=i*2.399;o.position.set((t<4.5?0:1.35)+Math.sin(a)*(1-impact)*1.8,3.8+Math.cos(a)*(1-impact)*1.2,(i%5-.2)*.12);o.scale.setScalar(impact);}); + + }; + } + const clay=new T.MeshStandardMaterial({color:'#b8b0a2',roughness:.85}); + const setRoot=new T.Group(); + for(const object of [...scene.children])if(!object.isLight&&!placements.some(p=>p.wrapper===object))setRoot.add(object); + scene.add(setRoot);editable.set={object:setRoot,pivot:[0,0,0],convention:'pivot'}; + editable.key={object:key,kind:'light',convention:'trs'}; + editable.camera={object:camera,kind:'camera',convention:'trs',extraKeys:['camera.override','camera.target','camera.fov'],extraValues:()=>({'camera.override':true,'camera.target':props.shot.target,'camera.fov':props.shot.fov})}; + scene.add(camera); + let editorCamera=null,lastClock=null; + function applyPlacement(object,value,pivot){object.quaternion.fromArray(value.orientation);object.scale.setScalar(value.scale);object.position.fromArray(value.position).add(pivot).sub(pivot.clone().multiplyScalar(value.scale).applyQuaternion(object.quaternion));object.updateMatrixWorld(true);} + return { + editing:{scene,camera,renderer,objects:editable,surfaceTargets,setView(view){editorCamera=view;},get shot(){return shotName;}}, + render(clock){ + lastClock=clock; + // Evaluate authored action with identity placement, then apply editable placement. + // Moving a character does not rewrite its skeleton or accumulate animation state. + for(const p of placements){p.wrapper.position.set(0,0,0);p.wrapper.quaternion.identity();p.wrapper.scale.setScalar(1);} + setRoot.position.set(0,0,0);setRoot.quaternion.identity();setRoot.scale.setScalar(1);scene.updateMatrixWorld(true); + animate(clock.frame/clock.fps); + const shot=authoredShot(props.kind,clock.frame/clock.fps,props.camera==='wide');camera.position.fromArray(shot.position);camera.lookAt(...shot.target);camera.fov=shot.fov;camera.updateProjectionMatrix();shotName=shot.label; + for(const p of placements){p.pivot.copy(p.object.position);applyPlacement(p.wrapper,props.actors[p.id],p.pivot);} + applyPlacement(setRoot,props.set,V());key.position.fromArray(props.key.position);key.intensity=props.key.intensity;key.color.set(props.key.color); + if(props.shot.override){camera.position.fromArray(props.shot.position);camera.lookAt(...props.shot.target);camera.fov=props.shot.fov;camera.updateProjectionMatrix();shotName='CUSTOM / SHOT CAMERA';} + scene.overrideMaterial=props.clay?clay:null;scene.userData.shot=shotName;renderer.domElement.dataset.shot=shotName; + renderer.render(scene,clock.capture?camera:(editorCamera??camera)); + }, + png(){return renderer.domElement.toDataURL('image/png');}, + dispose(){const geometries=new Set(),materials=new Set(),textures=new Set();for(const graph of [scene,env])graph.traverse(o=>{if(o.geometry)geometries.add(o.geometry);for(const m of o.material?(Array.isArray(o.material)?o.material:[o.material]):[]){materials.add(m);for(const v of Object.values(m))if(v?.isTexture)textures.add(v);}});geometries.forEach(g=>g.dispose());materials.forEach(m=>m.dispose());textures.forEach(t=>t.dispose());clay.dispose();envTarget.dispose();pm.dispose();renderer.dispose();renderer.forceContextLoss();renderer.domElement.remove();} + }; + } +}); +} diff --git a/examples/scene-editor/index.html b/examples/scene-editor/index.html new file mode 100644 index 0000000..75c904b --- /dev/null +++ b/examples/scene-editor/index.html @@ -0,0 +1,10 @@ +EditableFrames · Scene Studio +
EF
EditableFramesSCENE STUDIO / LOCAL
CURRENT SCENE
+
+
+
+
Preparing the scene…
THE EXCHANGE
3D blocking / editable reference
+
2.00s/ 12.00s · 24 fps
◆ selected-object keys
00:0000:0300:0600:0900:12
+
Loading…
+
+ diff --git a/examples/scene-editor/persistence.mjs b/examples/scene-editor/persistence.mjs new file mode 100644 index 0000000..e032e21 --- /dev/null +++ b/examples/scene-editor/persistence.mjs @@ -0,0 +1,34 @@ +/** Browser persistence needs usable locks and storage, not just API presence. + * Without them, edits stay in memory and the host offers explicit JSON export. */ +export function createScenePersistence({getLocks=()=>navigator.locks,getStorage=()=>localStorage}={}) { + let locks=null,storage=null,supported=false; + return { + get supported(){return supported;}, + get storage(){return storage;}, + async initialize(){ + try{ + locks=getLocks();if(typeof locks?.request!=='function')return false; + await locks.request('editableframes.scene-editor.storage-probe',()=>{ + const candidate=getStorage(),key='editableframes.scene-editor.probe.'+crypto.randomUUID(); + candidate.setItem(key,'1');candidate.removeItem(key);storage=candidate; + }); + supported=true; + }catch{supported=false;storage=null;} + return supported; + }, + async run(key,action){ + if(!supported)return action(); + let entered=false; + try{return await locks.request(key,()=>{entered=true;return action();});} + catch(error){ + // Only fall back if no transaction callback ran. Never retry a potentially + // published edit merely because the lock promise rejected afterwards. + if(!entered&&['SecurityError','NotAllowedError'].includes(error?.name)){ + supported=false;storage=null;return action(); + } + if(!entered)error.definite=true; + throw error; + } + } + }; +} diff --git a/examples/scene-editor/serve.mjs b/examples/scene-editor/serve.mjs new file mode 100644 index 0000000..66d085d --- /dev/null +++ b/examples/scene-editor/serve.mjs @@ -0,0 +1,5 @@ +import http from 'node:http';import fs from 'node:fs/promises';import path from 'node:path';import {fileURLToPath} from 'node:url'; +const file=path.resolve(path.dirname(fileURLToPath(import.meta.url)),'../../artifacts/scene-editor/scene-studio.html'); +await fs.access(file);const port=Number(process.env.PORT)||4189; +const server=http.createServer(async(req,res)=>{if(!['127.0.0.1:'+port,'localhost:'+port].includes(req.headers.host)||!['GET','HEAD'].includes(req.method)){res.writeHead(403).end();return;}if(!['/','/scene-studio.html'].includes(req.url)){res.writeHead(404).end();return;}res.writeHead(200,{'Content-Type':'text/html; charset=utf-8','Cache-Control':'no-store'});try{res.end(req.method==='HEAD'?undefined:await fs.readFile(file));}catch{res.destroy();}}); +server.listen(port,'127.0.0.1',()=>console.log('Scene Studio: http://127.0.0.1:'+port)); diff --git a/examples/scene-editor/style.css b/examples/scene-editor/style.css new file mode 100644 index 0000000..4e83727 --- /dev/null +++ b/examples/scene-editor/style.css @@ -0,0 +1 @@ +:root{color-scheme:dark;font-family:Inter,ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;color:#dce3e5;background:#101315;--line:#2a3032;--muted:#899497;--accent:#a2e0d4}*{box-sizing:border-box}body{margin:0}button,select,input{font:inherit}button,.button,select{background:#212729;border:1px solid #343c3e;color:#dce3e5;border-radius:6px;padding:8px 11px;cursor:pointer;font-size:12px}button:hover,.button:hover{background:#303a3c;border-color:#627777}button:disabled{opacity:.35;cursor:default}button:focus-visible,input:focus-visible,select:focus-visible{outline:2px solid var(--accent);outline-offset:2px}button.primary{background:var(--accent);border-color:var(--accent);color:#112823;font-weight:650}header{height:86px;border-bottom:1px solid var(--line);display:flex;gap:30px;align-items:center;padding:0 25px;background:#161b1d}.brand{display:flex;align-items:center;gap:12px;min-width:230px}.brand-mark{border:1px solid #91b7ac;width:36px;height:36px;border-radius:9px;display:grid;place-content:center;font-size:16px;font-weight:700;color:var(--accent)}.brand-mark span{font-size:11px}.brand strong{font-size:17px;letter-spacing:-.4px}.brand small{display:block;color:var(--muted);font-size:9px;letter-spacing:2px;margin-top:5px}.scene-switch .eyebrow{display:block;font-size:9px;color:#7c8c8f;letter-spacing:1.8px;margin-bottom:6px}.scene-switch select{min-width:210px;background:transparent;border:0;padding:0;font-size:14px;font-weight:550}.header-actions{margin-left:auto;display:flex;gap:8px;align-items:center}main{display:grid;grid-template-columns:195px minmax(400px,1fr) 255px;min-height:calc(100vh - 86px)}aside{background:#171c1e;padding:22px 15px}.outliner{border-right:1px solid var(--line)}.properties{border-left:1px solid var(--line);padding:22px 19px;overflow:auto;max-height:calc(100vh - 86px)}.section-label{font-size:10px;letter-spacing:1.6px;color:#859497;font-weight:600;margin-bottom:22px}.section-label span{float:right}#objects{display:flex;flex-direction:column;gap:6px}#objects button{text-align:left;background:transparent;border:1px solid transparent;padding:11px 10px;border-radius:7px;font-size:12px;color:#9eadaf}#objects button.selected{background:#213531;border-color:#3b5c53;color:#c0eee1}#objects button:hover{background:#253031}.outliner-note{margin-top:28px;border-top:1px solid var(--line);padding-top:22px;font-size:11px;color:#c3d0ce;line-height:1.7}.outliner-note p{color:var(--muted);font-size:10px}.dot{display:inline-block;width:6px;height:6px;background:#a6dac5;border-radius:50%;margin-right:6px}.shortcuts{margin-top:28px;font-size:11px;color:#8b9a9d;line-height:1.8}.shortcuts b{color:#c6d0d1;font-weight:500}.workbench{min-width:0;display:flex;flex-direction:column;padding:0 20px;background:#101416}.toolbar{display:flex;align-items:center;gap:9px;min-height:66px;flex-wrap:wrap;padding:8px 0;white-space:nowrap}.toolbar .modes{display:flex;gap:3px}.modes button{border:0;background:transparent;color:#92a1a3;padding:9px}.modes button.active{background:#2c3c39;color:#bde8dc}kbd{font-family:inherit;font-size:9px;color:#738481;margin-left:3px;border:1px solid #43524f;border-radius:3px;padding:1px 3px}.spacer{flex:1}.check{display:flex;align-items:center;gap:6px;white-space:nowrap;font-size:11px;color:#a1aeaf}input[type=checkbox]{accent-color:#a2dfd3;width:13px;height:13px}.viewbar{display:flex;align-items:center;gap:8px;padding-bottom:14px}.viewbar select{margin-right:auto;font-size:11px;background:#1b2224}.viewbar button{font-size:10px;background:transparent;padding:6px 9px}.stage-shell{position:relative;background:#192226;aspect-ratio:16/9;width:100%;border:1px solid #384447;border-radius:9px;overflow:hidden;box-shadow:0 16px 50px #0004}#canvas,#canvas canvas{display:block;width:100%!important;height:100%!important;touch-action:none}.stage-badge,.stage-caption{position:absolute;pointer-events:none;color:#cbdcd9;font-family:ui-monospace,monospace;font-size:9px;letter-spacing:.8px}.stage-badge{top:14px;left:15px;padding:7px 10px;background:#0d171bc4;border:1px solid #8eb4ad40;border-radius:4px}.stage-caption{bottom:13px;right:16px;color:#b5cbc9;opacity:.65}#loading{position:absolute;inset:0;background:#162023;display:flex;align-items:center;justify-content:center;gap:13px;color:#b4cfca;font-size:13px;z-index:5}#loading[hidden]{display:none}.spinner{width:18px;height:18px;border:2px solid #5c7773;border-top-color:#b2e9dc;border-radius:50%;animation:spin 1s linear infinite}@keyframes spin{to{transform:rotate(360deg)}}.timeline{padding-top:22px}.timeline-head{display:flex;gap:10px;align-items:center;font-size:12px;color:#d0dfdc}.play{background:#c9d9d3;color:#1a2a25;min-width:65px;font-weight:650}.duration,.key-hint{color:#76888a;font-size:10px}.key-hint{margin-left:10px}#seek{width:100%;margin:0;accent-color:#99d4c6;height:20px;cursor:ew-resize}.ruler{display:flex;justify-content:space-between;font:9px ui-monospace,monospace;color:#607679;margin-top:4px}#keys{position:relative;height:18px;margin-top:8px}.key{position:absolute;width:8px;height:8px;transform:rotate(45deg);padding:0;border-radius:1px;background:#e9c575;border:0;top:5px}footer{display:flex;justify-content:space-between;gap:12px;font-size:10px;color:#708789;border-top:1px solid #263236;margin-top:24px;padding:16px 0}#status{color:#a1c4bb}#status.error{color:#ffae95}#revision{white-space:nowrap}h2{font-size:18px;font-weight:550;letter-spacing:-.4px;margin:0 0 25px}.field{padding-bottom:18px;margin-bottom:4px}.field>label{font-size:11px;display:block;color:#aebcbd;margin-bottom:8px}.field input[type=number]{width:100%;border:1px solid #354144;background:#11191b;color:#d1e2df;border-radius:5px;padding:8px;font-size:12px;min-width:0}.field input[type=color]{background:#263436;border:1px solid #4a5d5d;width:100%;height:30px}.vector{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:5px}.vector>label{font-size:9px;color:#8aaba5;display:block}.vector input{margin-top:5px;padding:7px 4px!important}.field small{display:block;font-size:9px;color:#637c7a;margin-top:7px}#capability{font-size:10px;color:#809694;line-height:1.7;border-top:1px solid #2c393b;padding-top:16px}select{max-width:100%}@media(max-width:1050px){main{grid-template-columns:155px minmax(380px,1fr) 220px}.workbench{padding:0 12px}header{padding:0 15px;gap:10px}.brand{min-width:185px}.header-actions .button{display:none}.toolbar{gap:3px}.toolbar select{max-width:90px}.key-hint{display:none}aside{padding:20px 10px}.scene-switch select{min-width:180px}}@media(max-width:800px){header{height:auto;padding:15px;flex-wrap:wrap}.header-actions{width:100%;justify-content:flex-end}main{grid-template-columns:1fr}.outliner{padding:12px;border:0}.outliner .section-label,.outliner-note,.shortcuts{display:none}#objects{flex-direction:row;overflow:auto}#objects button{white-space:nowrap}.properties{max-height:none;border:0;display:block}.workbench{padding:0 12px}.stage-shell{min-height:220px}.toolbar{flex-wrap:wrap;height:auto;padding:10px 0}.viewbar{flex-wrap:wrap}} diff --git a/package.json b/package.json index cc11f8b..3f258d4 100644 --- a/package.json +++ b/package.json @@ -26,7 +26,10 @@ "./interaction": "./packages/core/interaction.mjs", "./manipulation": "./packages/core/manipulation.mjs", "./three-manipulation": "./packages/three/manipulation.mjs", - "./key-timing": "./packages/core/key-timing.mjs" + "./key-timing": "./packages/core/key-timing.mjs", + "./rotation": "./packages/core/rotation.mjs", + "./spatial-session": "./packages/core/spatial-session.mjs", + "./three-editor": "./packages/three/spatial-editor.mjs" }, "scripts": { "build": "node scripts/build.mjs", @@ -40,7 +43,10 @@ "test:render": "node tests/render-regression.mjs", "build:motion": "node examples/ae-controls/build.mjs", "dev:motion": "node examples/ae-controls/serve.mjs", - "test:renderer": "node tests/renderer-integration.mjs" + "test:renderer": "node tests/renderer-integration.mjs", + "build:scene": "node examples/scene-editor/build.mjs", + "dev:scene": "node examples/scene-editor/serve.mjs", + "test:scene": "node tests/scene-editor-browser.mjs" }, "dependencies": { "esbuild": "0.25.10", diff --git a/packages/core/controls.mjs b/packages/core/controls.mjs index 7a6118d..a080c90 100644 --- a/packages/core/controls.mjs +++ b/packages/core/controls.mjs @@ -1,3 +1,4 @@ +import {isUnitQuaternion} from './rotation.mjs'; import {keyFrameRate} from './key-timing.mjs'; import {normalizeObjectHierarchy} from './objects.mjs'; import {mergeRendererProps as mergeProps} from './renderer.mjs'; @@ -42,6 +43,7 @@ export function validateControlValue(field, value, {reset = false} = {}) { const finite = n => Number.isFinite(n) && (field.min === undefined || n >= field.min) && (field.max === undefined || n <= field.max); const validators = { number: () => typeof value === 'number' && finite(value) && (field.role !== 'scale' || value > 0), + quaternion: () => isUnitQuaternion(value), vec3: () => Array.isArray(value) && value.length === 3 && value.every(finite), color: () => typeof value === 'string' && /^#[\da-f]{6}([\da-f]{2})?$/iu.test(value), string: () => typeof value === 'string', bool: () => typeof value === 'boolean', @@ -53,7 +55,7 @@ export function validateControlValue(field, value, {reset = false} = {}) { } export function normalizeControls(controls) { if (controls == null) return {version: 1, fields: [], presets: []}; - if (controls.version !== 1 || !Array.isArray(controls.fields) || controls.fields.length > 200) throw Error('Invalid controls v1 declaration'); + if (![1,2].includes(controls.version) || !Array.isArray(controls.fields) || controls.fields.length > 200) throw Error('Invalid controls declaration'); if(controls.frameRate)keyFrameRate(controls.frameRate); const keys = new Set(), bindings = new Set(), paths = [], roles = new Set(); const fields = controls.fields.map(raw => { @@ -64,16 +66,16 @@ export function normalizeControls(controls) { const parts = pointerParts(path); if (paths.some(previous => parts.every((v, i) => previous[i] === v) || previous.every((v, i) => parts[i] === v))) throw Error('Overlapping control paths'); paths.push(parts); - if (!['number','vec3','color','string','bool','enum','json','media','model','device'].includes(raw.type)) throw Error('Unknown control type'); + if (![...['number','vec3','color','string','bool','enum','json','media','model','device'],...(controls.version===2?['quaternion']:[])].includes(raw.type)) throw Error('Unknown control type'); for (const bound of ['min','max','step']) if (raw[bound] !== undefined && !Number.isFinite(raw[bound])) throw Error('Invalid control bound'); if (raw.min !== undefined && raw.max !== undefined && raw.min > raw.max) throw Error('Reversed control bounds'); if (raw.step !== undefined && raw.step <= 0) throw Error('Invalid control step'); if (raw.animatable !== undefined && typeof raw.animatable !== 'boolean') throw Error('Invalid animatable flag'); - const interpolation = raw.interpolation ?? (['number','vec3'].includes(raw.type) ? 'continuous' : 'discrete'); - if (!['continuous','discrete'].includes(interpolation) || interpolation === 'continuous' && !['number','vec3'].includes(raw.type)) throw Error('Invalid control interpolation'); + const interpolation = raw.interpolation ?? (raw.type==='quaternion'?'slerp':['number','vec3'].includes(raw.type) ? 'continuous' : 'discrete'); + if (!['continuous','discrete',...(controls.version===2?['slerp']:[])].includes(interpolation) || interpolation === 'continuous' && !['number','vec3'].includes(raw.type) || interpolation==='slerp' && raw.type!=='quaternion') throw Error('Invalid control interpolation'); if (raw.type === 'enum' && (!Array.isArray(raw.options) || !raw.options.length || raw.options.some(v => typeof v !== 'string'))) throw Error('Enum needs string options'); if(raw.role!==undefined){ - if(!['position','scale'].includes(raw.role)||!raw.animatable||raw.role==='position'&&(raw.type!=='vec3'||!['local','world','canvas'].includes(raw.coordinateSpace)||typeof raw.unit!=='string'||!raw.unit)||raw.role==='scale'&&raw.type!=='number'||roles.has(`${anchor}:${raw.role}`))throw Error('Invalid or duplicate spatial role'); + if(!['position','scale',...(controls.version===2?['rotation']:[])].includes(raw.role)||!raw.animatable||raw.role==='position'&&(raw.type!=='vec3'||!['local','world','canvas'].includes(raw.coordinateSpace)||typeof raw.unit!=='string'||!raw.unit)||raw.role==='scale'&&raw.type!=='number'||raw.role==='rotation'&&(raw.type!=='quaternion'||!['local','world'].includes(raw.coordinateSpace))||roles.has(`${anchor}:${raw.role}`))throw Error('Invalid or duplicate spatial role'); roles.add(`${anchor}:${raw.role}`); } const field = {...copy(raw),key,anchor,property,path,animatable:raw.animatable ?? false,interpolation}; @@ -93,11 +95,14 @@ export function normalizeControls(controls) { return copy(preset); }); normalizeObjectHierarchy(controls.objects??[],fields.map(field=>field.anchor)); - return {version:1,fields,presets,...(controls.frameRate?{frameRate:structuredClone(controls.frameRate)}:{}),...(controls.objects!==undefined?{objects:copy(controls.objects)}:{})}; + if(controls.spatialSemantics!==undefined&&(controls.version!==2||typeof controls.spatialSemantics!=='string'||controls.spatialSemantics.length>512))throw Error('Invalid spatial semantics'); + return {version:controls.version,fields,presets,...(controls.spatialSemantics!==undefined?{spatialSemantics:controls.spatialSemantics}:{}),...(controls.frameRate?{frameRate:structuredClone(controls.frameRate)}:{}),...(controls.objects!==undefined?{objects:copy(controls.objects)}:{})}; } export function controlsIdentity(controls) { - return JSON.stringify(controls.fields.map(({key,anchor,property,path,type,unit,coordinateSpace,role,animatable,interpolation}) => - ({key,anchor,property,path,type,unit:unit ?? null,coordinateSpace:coordinateSpace??null,role:role??null,animatable,interpolation})).sort((a,b) => a.key.localeCompare(b.key))); + const fields=controls.fields.map(({key,anchor,property,path,type,unit,coordinateSpace,role,animatable,interpolation}) => + ({key,anchor,property,path,type,unit:unit ?? null,coordinateSpace:coordinateSpace??null,role:role??null,animatable,interpolation})).sort((a,b) => a.key.localeCompare(b.key)); + if(controls.version===1)return JSON.stringify(fields); + return JSON.stringify({version:2,fields,spatialSemantics:controls.spatialSemantics??'',objects:(controls.objects??[]).map(({id,parentId,pivot})=>({id,parentId:parentId??null,pivot:pivot??null})).sort((a,b)=>a.id.localeCompare(b.id))}); } /** Unexposed props remain ordinary authored data; null means reset, not a value. */ export function projectControlProps(defaultProps, instanceProps, controls, nodes, {resetControls=[]}={}) { diff --git a/packages/core/index.mjs b/packages/core/index.mjs index c2c6f74..e3cbe03 100644 --- a/packages/core/index.mjs +++ b/packages/core/index.mjs @@ -1,3 +1,4 @@ +import {isUnitQuaternion,slerpQuaternion} from './rotation.mjs'; import {keySeconds,canonicalTrack} from './key-timing.mjs'; import {normalizeObjectHierarchy} from './objects.mjs'; import {easeProgress, validateEasing} from './animation.mjs'; @@ -13,6 +14,7 @@ const valueTypes=new Map(); export function registerValueType(name,spec){if(!safe(name)||valueTypes.has(name)||typeof spec.validate!=='function')fail('Invalid or duplicate value type');valueTypes.set(name,spec);} registerValueType('number',{validate:(v,s)=>Number.isFinite(v)&&(s.role!=='scale'||v>0)&&(s.min===undefined||v>=s.min)&&(s.max===undefined||v<=s.max)&&(!s.integer||Number.isInteger(v)),interpolate:(a,b,u)=>a+(b-a)*u}); registerValueType('vec3',{validate:(v,s)=>Array.isArray(v)&&v.length===3&&v.every(n=>Number.isFinite(n)&&(s.min===undefined||n>=s.min)&&(s.max===undefined||n<=s.max)),interpolate:(a,b,u)=>a.map((v,i)=>v+(b[i]-v)*u)}); +registerValueType('quaternion',{validate:isUnitQuaternion,interpolate:slerpQuaternion}); registerValueType('color',{validate:v=>typeof v==='string'&&/^#[\da-f]{6}([\da-f]{2})?$/i.test(v)}); registerValueType('text',{validate:(v,s)=>typeof v==='string'&&v.length<=(s.maxLength??160)}); registerValueType('enum',{validate:(v,s)=>s.values.includes(v)}); diff --git a/packages/core/rotation.mjs b/packages/core/rotation.mjs new file mode 100644 index 0000000..2d13da1 --- /dev/null +++ b/packages/core/rotation.mjs @@ -0,0 +1,31 @@ +/** Portable rotation values. No renderer or GPU dependency. */ +export function isUnitQuaternion(value) { + return Array.isArray(value) && value.length === 4 && value.every(Number.isFinite) && + Math.abs(Math.hypot(...value) - 1) <= 1e-6; +} +export function normalizeQuaternion(value) { + if (!Array.isArray(value) || value.length !== 4 || !value.every(Number.isFinite)) throw Error('Invalid quaternion'); + const length = Math.hypot(...value); + if (!Number.isFinite(length) || length < 1e-12) throw Error('Quaternion has no orientation'); + return value.map(v => v / length); +} +export function sameOrientation(a, b, tolerance = 1e-10) { + if (!isUnitQuaternion(a) || !isUnitQuaternion(b)) return false; + return Math.min(Math.hypot(...a.map((v,i)=>v-b[i])), Math.hypot(...a.map((v,i)=>v+b[i]))) <= tolerance; +} +function canonical(q) { + const first = [q[3], q[0], q[1], q[2]].find(v => v !== 0); + return first < 0 ? q.map(v => -v) : q; +} +export function slerpQuaternion(a, b, t) { + a = normalizeQuaternion(a); b = normalizeQuaternion(b); + let dot = a.reduce((n,v,i)=>n+v*b[i],0); + // Exact 180-degree ties have two shortest paths. Choose a sign-invariant one. + if (dot === 0) { a = canonical(a); b = canonical(b); } + else if (dot < 0) { b = b.map(v=>-v); dot = -dot; } + if (t <= 0) return a; + if (t >= 1) return b; + if (dot > .9995) return normalizeQuaternion(a.map((v,i)=>v+(b[i]-v)*t)); + const theta = Math.acos(Math.min(1,Math.max(-1,dot))), denominator = Math.sin(theta); + return normalizeQuaternion(a.map((v,i)=>(v*Math.sin((1-t)*theta)+b[i]*Math.sin(t*theta))/denominator)); +} diff --git a/packages/core/spatial-session.mjs b/packages/core/spatial-session.mjs new file mode 100644 index 0000000..2236931 --- /dev/null +++ b/packages/core/spatial-session.mjs @@ -0,0 +1,101 @@ +import {createGestureSession,gestureResult} from './interaction.mjs'; +import {normalizeControls} from './controls.mjs'; + +const same=(a,b)=>JSON.stringify(a)===JSON.stringify(b); +const copy=v=>structuredClone(v); +const safe=s=>typeof s==='string'&&s.length>0&&s.length<=128&&!['__proto__','constructor','prototype'].includes(s); +/** All layers opt in independently. Controls-v1 does not imply legacy gestures. */ +export function negotiateSpatial({host=[],transport=[],adapter=[]}) { + return [host,transport,adapter].every(v=>Array.isArray(v)&&v.includes(2)) ? 2 : null; +} +export function spatialOperation(capabilities,{bindingId,operation,handle,space},controls,anchor) { + if(!capabilities?.gestureProtocols?.includes(2)||!safe(bindingId)||!['translate','rotate','scale'].includes(operation))throw Error('Unsupported spatial protocol or operation'); + const bindings=capabilities.bindings; + if(!Array.isArray(bindings)||bindings.length>32||bindings.filter(b=>b.id===bindingId).length!==1)throw Error('Invalid spatial binding'); + const op=bindings.find(b=>b.id===bindingId).operations?.[operation]; + if(!op||!Array.isArray(op.handles)||!Array.isArray(op.spaces)||!op.handles.includes(handle)||!op.spaces.includes(space))throw Error('Unsupported spatial handle or space'); + if(!Array.isArray(op.keys)||!op.keys.length||op.keys.length>64||new Set(op.keys).size!==op.keys.length)throw Error('Invalid spatial key set'); + for(const key of op.keys)if(!safe(key)||!controls.fields.some(f=>f.key===key&&f.anchor===anchor&&!['media','model','device'].includes(f.type)))throw Error('Spatial key is outside the selected binding'); + return Object.freeze({keys:Object.freeze([...op.keys])}); +} +/** Renderer proposals cannot admit or publish a transaction. A trusted host calls + * begin/release. Publication must be revision-checked and idempotent by proposal.id. + * publish errors are uncertain unless explicitly marked definite by the host. */ +export function createSpatialSession({controls,readIdentity,publish,reconcile,now,limits}) { + if(typeof publish!=='function'||typeof readIdentity!=='function')throw Error('Spatial session needs a host'); + const schema=normalizeControls(controls); + let active=null; + function identity() { + const i=copy(readIdentity()); + for(const key of ['rendererInstance','bindingGeneration','viewRevision','viewportRevision','shot'])if(i[key]===undefined)throw Error('Incomplete spatial presentation: '+key); + if(!i.clock||!Number.isFinite(i.clock.frame)||!Number.isFinite(i.clock.fps)||i.clock.fps<=0)throw Error('Invalid presentation clock'); + return i; + } + function check(id) { + if(!active||active.id!==id)throw Error('Spatial gesture is no longer active'); + if(!same(active.identity,identity()))throw Error('Stale spatial presentation'); + return active; + } + function cancel(id) { + if(!active||active.id!==id)return false; + if(['publishing','uncertain'].includes(active.phase))return false; + active.inner.cancel({id:active.innerId});active=null;return true; + } + const api={ + get state(){return active?copy({id:active.id,phase:active.phase,identity:active.identity,values:active.values,request:active.request}):null;}, + begin({trusted,identity:observed,anchor,capabilities,...request}) { + if(trusted!==true)throw Error('A host-admitted pointer is required'); + if(active)throw Error('A spatial gesture is already active'); + const current=identity();if(!same(observed,current))throw Error('Stale spatial presentation'); + const allowed=spatialOperation(capabilities,request,schema,anchor); + const inner=createGestureSession({controls:schema,readIdentity,now,limits}); + const b=inner.begin({identity:current,anchor}); + active={id:crypto.randomUUID(),phase:'active',identity:current,anchor,request:copy(request),allowed,inner,innerId:b.id,values:{}}; + return api.state; + }, + update({id,values}) { + let a;try{a=check(id);}catch(error){cancel(id);throw error;} + if(!['active','finalizing'].includes(a.phase))throw Error('Gesture is already publishing'); + const checked=gestureResult({values}).values; + if(Object.keys(checked).some(key=>!a.allowed.keys.includes(key)))throw Error('Proposal changes a field outside this operation'); + a.inner.update({id:a.innerId,identity:a.identity,values:checked}); + Object.assign(a.values,checked);return api.state; + }, + renew(id){const a=check(id);a.inner.renew({id:a.innerId,identity:a.identity});}, + cancel, + async release({id,trusted,evaluate}) { + if(trusted!==true)throw Error('A host-admitted release is required'); + let a;try{a=check(id);}catch(error){cancel(id);throw error;} + if(a.phase!=='active')throw Error('Gesture already released'); + a.phase='finalizing'; + try { + const result=await evaluate(); + if(active!==a)return {status:'cancelled'}; + check(id);if(!result||!result.values||typeof result.values!=='object'||Array.isArray(result.values))throw Error('Missing final gesture evaluation'); + const finalKeys=Object.keys(result.values); + if(Object.keys(a.values).some(key=>!finalKeys.includes(key)))throw Error('Final evaluation omits a previously proposed field'); + api.update({id,values:result.values}); + check(id); + const inner=a.inner.commit({id:a.innerId,identity:a.identity}); + a.proposal=Object.freeze({id:a.id,identity:copy(a.identity),anchor:a.anchor,values:copy(inner.values),request:copy(a.request)}); + // No await between this final check and consuming the write admission. + a.phase='publishing'; + } catch(error){cancel(id);throw error;} + try { + const receipt=await publish(a.proposal);if(active===a)active=null; + return {status:'committed',receipt}; + } catch(error) { + if(error?.definite===true){if(active===a)active=null;throw error;} + a.phase='uncertain';return {status:'uncertain',transactionId:a.id}; + } + }, + async reconcile() { + const a=active;if(!a||a.phase!=='uncertain'||typeof reconcile!=='function')throw Error('No reconcilable spatial transaction'); + const result=await reconcile(a.id); + if(active!==a)throw Error('Spatial transaction replaced'); + if(['committed','failed'].includes(result?.status))active=null; + return result; + } + }; + return Object.freeze(api); +} diff --git a/packages/runtime/editor.mjs b/packages/runtime/editor.mjs index 3c82b5b..ae89fe9 100644 --- a/packages/runtime/editor.mjs +++ b/packages/runtime/editor.mjs @@ -41,7 +41,7 @@ export async function mount({templates,createRenderer,scene,binding,initial,medi if(f.when&&studio.controlValue(c,studio.node(c,f.when.anchor),f.when.property)!==f.when.equals)continue; if(f.kind==='preset'){const b=document.createElement('button');b.textContent=f.label;b.onclick=()=>edit({type:'batch',label:f.label,commands:f.operations.map(o=>({type:'set',clip:selected,...o}))}).catch(fail);$('#controls').append(b);continue} const macro=f.kind==='macro',node=macro?null:studio.node(c,f.anchor),s=macro?{type:'number',min:f.min,max:f.max}:studio.schema(c,node)[f.property]; - const label=document.createElement('label');label.textContent=f.label;label.dataset.anchor=f.anchor??'';label.dataset.property=f.property??'';const jsonInput=['json','vec3'].includes(s.type);const input=document.createElement(s.type==='enum'?'select':jsonInput?'textarea':'input');input.setAttribute('aria-label',f.label); + const label=document.createElement('label');label.textContent=f.label;label.dataset.anchor=f.anchor??'';label.dataset.property=f.property??'';const jsonInput=['json','vec3','quaternion'].includes(s.type);const input=document.createElement(s.type==='enum'?'select':jsonInput?'textarea':'input');input.setAttribute('aria-label',f.label); if(s.type==='enum')for(const v of s.values)input.add(new Option(v,v));else if(!jsonInput)input.type=s.type==='number'?'range':s.type==='boolean'?'checkbox':s.type==='color'?'color':'text'; if(s.type==='number'){input.min=s.min??0;input.max=s.max??100;input.step=s.integer?1:s.step??.01} const shown=macro?f.min:studio.controlValue(c,node,f.property);input.value=jsonInput?JSON.stringify(shown):shown??'';input.checked=!!shown; diff --git a/packages/three/spatial-editor.mjs b/packages/three/spatial-editor.mjs new file mode 100644 index 0000000..e164317 --- /dev/null +++ b/packages/three/spatial-editor.mjs @@ -0,0 +1,148 @@ +import * as T from 'three'; +import {TransformControls} from 'three/addons/controls/TransformControls.js'; +import {normalizeControls,readPointer} from '../core/controls.mjs'; +import {normalizeObjectHierarchy,selectableObject} from '../core/objects.mjs'; +import {normalizeQuaternion} from '../core/rotation.mjs'; +import {rigidParent,worldPivotToPosition,constrainTranslation,positiveScale} from './spatial-math.mjs'; + +import {surfaceBounds,surfaceDrop} from './surface-snap.mjs'; + +const axes={x:new T.Vector3(1,0,0),y:new T.Vector3(0,1,0),z:new T.Vector3(0,0,1)}; +const axisName=h=>h==='uniform'||h==='screen'?'XYZ':h.toUpperCase(); +/** Pinned r180 facade. Never connects TransformControls DOM listeners. A host + * forwards admitted normalized pointers; only sparse values leave this adapter. + * Helpers occupy an independent scene; content captures never render it. */ +export class ThreeSpatialEditor { + constructor({controls,props,objects,camera,width,height,surfaceTargets=[]}) { + this.controls=normalizeControls(controls);this.props=props;this.objects=objects; + this.rows=normalizeObjectHierarchy(this.controls.objects??[],this.controls.fields.map(f=>f.anchor)); + this.surfaceTargets=surfaceTargets;this.camera=camera;this.width=width;this.height=height; + this.overlay=new T.Scene();this.proxy=new T.Object3D();this.overlay.add(this.proxy); + this.transform=new TransformControls(camera,null);this.helper=this.transform.getHelper();this.overlay.add(this.helper); + const updateGizmo=this.transform._gizmo.updateMatrixWorld.bind(this.transform._gizmo); + this.transform._gizmo.updateMatrixWorld=(force)=>{updateGizmo(force);for(const group of [this.transform._gizmo.gizmo.scale,this.transform._gizmo.picker.scale])for(const child of group.children)child.visible=child.name==='XYZ';}; + this.transform.size=.8;this.raycaster=new T.Raycaster();this.selected=null;this.mode='translate';this.space='world';this.active=null; + this.outline=new T.Box3Helper(new T.Box3(),0x86ddd1);this.outline.material.depthTest=false;this.outline.visible=false;this.overlay.add(this.outline); + this.icons=[]; + for(const [anchor,b] of Object.entries(objects))if(['light','camera'].includes(b.kind)) { + const icon=new T.Mesh(new T.OctahedronGeometry(.10),new T.MeshBasicMaterial({color:b.kind==='light'?0xffcf7a:0xaeb9ff,depthTest:false})); + icon.userData.anchor=anchor;this.overlay.add(icon);this.icons.push({anchor,icon}); + } + } + fields(anchor){return Object.fromEntries(this.controls.fields.filter(f=>f.anchor===anchor&&f.role).map(f=>[f.role,f]));} + value(f){return structuredClone(readPointer(this.props,f.path)??f.default);} + capabilities({anchor}) { + const b=this.objects[anchor],f=this.fields(anchor);const denied=reason=>({gestureProtocols:[2],bindings:[],disabledReason:reason}); + if(!b||!selectableObject(this.rows,anchor))return denied('Object is not editable'); + try{b.object.updateWorldMatrix(true,true);rigidParent(b.object.parent?.matrixWorld??new T.Matrix4());}catch(e){return denied(e.message);} + const position=f.position?.coordinateSpace==='local'?f.position:null; + const rotation=f.rotation?.coordinateSpace==='local'?f.rotation:null; + const pivot=this.pivot(b),coupled=b.convention==='trs'&&pivot.lengthSq()>0; + const operations={}; + if(position)operations.translate={handles:['x','y','z','xy','xz','yz','screen'],spaces:['world','local'],keys:[position.key,...(b.extraKeys??[])]}; + if(rotation&&(!coupled||position))operations.rotate={handles:['x','y','z'],spaces:['world','local'],keys:[rotation.key,...(coupled?[position.key]:[]),...(b.extraKeys??[])]}; + if(f.scale&&(!coupled||position))operations.scale={handles:['uniform'],spaces:['local'],keys:[f.scale.key,...(coupled?[position.key]:[]),...(b.extraKeys??[])]}; + return {gestureProtocols:[2],bindings:[{id:'placement',operations}]}; + } + pivot(binding){const p=typeof binding.pivot==='function'?binding.pivot():binding.pivot;return new T.Vector3(...(p??[0,0,0]));} + setSelection(anchor){if(this.active)throw Error('Finish the gesture before changing selection');this.selected=this.objects[anchor]&&selectableObject(this.rows,anchor)?anchor:null;this.sync();} + setMode(mode,space=this.space){if(this.active)throw Error('Finish the gesture before changing mode');this.mode=mode;this.space=mode==='scale'?'local':space;this.transform.setMode(mode);this.transform.setSpace(this.space);this.sync();} + setCamera(camera){if(this.active)throw Error('Camera changed during manipulation');this.camera=camera;this.transform.camera=camera;this.sync();} + snapshot() { + const b=this.objects[this.selected],f=this.fields(this.selected);if(!b)throw Error('No selected spatial object'); + b.object.updateWorldMatrix(true,true); + const parent=rigidParent(b.object.parent?.matrixWorld??new T.Matrix4()),pivot=this.pivot(b); + const position=f.position?new T.Vector3(...this.value(f.position)):b.object.position.clone(); + const quaternion=f.rotation?new T.Quaternion(...this.value(f.rotation)):b.object.quaternion.clone(); + const scale=f.scale?this.value(f.scale):b.object.scale.x; + return {b,f,parent,pivot,position,quaternion,scale,worldPivot:pivot.clone().applyMatrix4(b.object.matrixWorld),worldQuaternion:b.object.getWorldQuaternion(new T.Quaternion()),worldScale:b.object.getWorldScale(new T.Vector3())}; + } + sync() { + for(const {anchor,icon} of this.icons){const b=this.objects[anchor];b.object.updateWorldMatrix(true,false);icon.position.copy(b.object.getWorldPosition(new T.Vector3()));icon.scale.setScalar(Math.max(.05,icon.position.distanceTo(this.camera.position)*.012));icon.visible=!!selectableObject(this.rows,anchor);} + if(!this.selected){this.transform.detach();this.outline.visible=false;return;} + const b=this.objects[this.selected]; + if(!b){this.transform.detach();this.outline.visible=false;return;} + if(b.kind!=='light'&&b.kind!=='camera'){this.outline.box.setFromObject(b.object,true);this.outline.visible=!this.outline.box.isEmpty();}else this.outline.visible=false; + if(this.active)return; + const cap=this.capabilities({anchor:this.selected}),op=cap.bindings[0]?.operations[this.mode]; + if(!op){this.transform.detach();return;} + const s=this.snapshot();this.proxy.position.copy(s.worldPivot);this.proxy.quaternion.copy(s.worldQuaternion);this.proxy.scale.copy(s.worldScale); + this.proxy.updateMatrixWorld(true);this.transform.attach(this.proxy);this.overlay.updateMatrixWorld(true); + // r180 has per-axis scale handles although this contract only offers uniform scale. + for(const child of this.transform._gizmo.gizmo.scale.children)child.visible=child.name==='XYZ'; + for(const child of this.transform._gizmo.picker.scale.children)child.visible=child.name==='XYZ'; + } + pointer(point,button=-1){return {x:point.x/this.width*2-1,y:1-point.y/this.height*2,button};} + hitHandle(point) { + if(this.active||!this.selected)return null; + this.sync();if(!this.transform.object)return null; + this.camera.updateMatrixWorld(true);this.overlay.updateMatrixWorld(true); + this.transform.pointerHover(this.pointer(point)); + let axis=this.transform.axis; + if(!axis)return null; + if(this.mode==='scale'&&axis!=='XYZ')return null; + if(this.mode==='rotate'&&!['X','Y','Z'].includes(axis))return null; + return {anchor:this.selected,bindingId:'placement',operation:this.mode,handle:axis==='XYZ'?(this.mode==='scale'?'uniform':'screen'):axis.toLowerCase(),space:this.space}; + } + pick(point,scene) { + this.camera.updateMatrixWorld(true);scene.updateMatrixWorld(true);scene.traverse(o=>{if(o.isSkinnedMesh){o.computeBoundingSphere();if(o.boundingBox)o.computeBoundingBox();}});this.raycaster.setFromCamera(this.pointer(point),this.camera); + const icons=this.raycaster.intersectObjects(this.icons.map(i=>i.icon),false).filter(h=>h.object.visible); + if(icons[0])return icons[0].object.userData.anchor; + for(const hit of this.raycaster.intersectObjects(scene.children,true)) { + let visible=true;for(let o=hit.object;o;o=o.parent)if(!o.visible)visible=false;if(!visible)continue; + for(let o=hit.object;o;o=o.parent)for(const [anchor,b] of Object.entries(this.objects))if(b.object===o)return selectableObject(this.rows,anchor)?anchor:null; + // A nearer opaque set object occludes subjects, even if it has no binding. + if(!hit.object.material?.transparent)return null; + }return null; + } + begin(request,point,{translationSnap=0,rotationSnap=0,scaleSnap=0,surfaceSnap=false}={}) { + if(this.active)throw Error('Adapter gesture is active'); + const op=this.capabilities({anchor:this.selected}).bindings[0]?.operations[request.operation]; + if(!op?.handles.includes(request.handle)||!op.spaces.includes(request.space))throw Error('Unsupported spatial gesture'); + this.sync();const s=this.snapshot(); + this.transform.axis=axisName(request.handle);this.transform.translationSnap=null;this.transform.scaleSnap=null;this.transform.rotationSnap=rotationSnap>0?rotationSnap:null; + this.transform.pointerDown(this.pointer(point,0)); + if(!this.transform.dragging)throw Error('Handle is not available from this camera'); + if(request.operation==='scale'&&this.transform.pointStart.length()<1e-5){this.transform.pointerUp({button:0});throw Error('Move away from the scale pivot before dragging');} + const cameraOrientation=this.camera.getWorldQuaternion(new T.Quaternion()); + const basis=request.handle==='screen'?[new T.Vector3(1,0,0).applyQuaternion(cameraOrientation),new T.Vector3(0,1,0).applyQuaternion(cameraOrientation)]:[...request.handle].map(a=>axes[a]?.clone()).filter(Boolean); + if(request.space==='local'&&request.handle!=='screen')basis.forEach(v=>v.applyQuaternion(s.worldQuaternion)); + this.active={...s,extraValues:structuredClone(typeof s.b.extraValues==='function'?s.b.extraValues():s.b.extraValues),request:structuredClone(request),translationSnap,scaleSnap,basis,surfaceSnap:surfaceSnap&&s.b.surfaceSnap===true,point:{...point},bounds:surfaceSnap&&s.b.surfaceSnap===true?surfaceBounds(s.b.object):null}; + } + update(point,{final=false}={}) { + const a=this.active;if(!a)throw Error('No adapter gesture'); + this.transform.pointerMove(this.pointer(point)); + const values={...a.extraValues},req=a.request; + if(req.operation==='translate') { + const delta=this.proxy.position.clone().sub(a.worldPivot); + const desired=a.basis.map(v=>delta.dot(v)); + const field=a.f.position; + const nextPosition=constrainTranslation({startLocal:a.position,basisWorld:a.basis,parent:a.parent,desired,min:field.min,max:field.max,snap:a.translationSnap,unitScale:req.space==='local'?a.parent.scale:1}); + // Surface placement is an explicit final world-Y correction after the axis/grid drag. + // A click without pointer travel must remain a no-op. Settings and pose are frozen. + if(final&&a.surfaceSnap&&Math.hypot(point.x-a.point.x,point.y-a.point.y)>.5){ + const delta=nextPosition.clone().sub(a.position).multiplyScalar(a.parent.scale).applyQuaternion(a.parent.quaternion); + const drop=surfaceDrop({bounds:a.bounds,delta,surfaces:this.surfaceTargets,exclude:a.b.object}); + if(drop){ + const corrected=nextPosition.clone().add(new T.Vector3(0,drop.offsetY,0).applyQuaternion(a.parent.quaternion.clone().invert()).divideScalar(a.parent.scale)); + if(corrected.toArray().every(v=>Number.isFinite(v)&&(field.min===undefined||v>=field.min)&&(field.max===undefined||v<=field.max)))nextPosition.copy(corrected); + } + } + values[field.key]=nextPosition.toArray(); + this.proxy.position.copy(a.worldPivot).add(nextPosition.clone().applyMatrix4(a.parent.matrix).sub(a.position.clone().applyMatrix4(a.parent.matrix))); + } else { + const quaternion=req.operation==='rotate'?a.parent.quaternion.clone().invert().multiply(this.proxy.quaternion):a.quaternion; + const scale=req.operation==='scale'?positiveScale(this.proxy.scale.x/a.parent.scale,{min:a.f.scale.min,max:a.f.scale.max,snap:a.scaleSnap}):a.scale; + if(req.operation==='rotate')values[a.f.rotation.key]=normalizeQuaternion(quaternion.toArray()); + else {values[a.f.scale.key]=scale;this.proxy.scale.setScalar(scale*a.parent.scale);} + if(a.b.convention==='trs'&&a.pivot.lengthSq()>0)values[a.f.position.key]=worldPivotToPosition(a.worldPivot,{...a,quaternion,scale,convention:'trs'}).toArray(); + } + return {values}; + } + end(){this.transform.pointerUp({button:0});this.active=null;this.sync();} + render(renderer) { + this.sync();const auto=renderer.autoClear;renderer.autoClear=false;renderer.clearDepth();renderer.render(this.overlay,this.camera);renderer.autoClear=auto; + } + dispose(){this.active=null;this.transform.detach();// No DOM connection was made; r180 dispose() assumes one. Dispose its helper directly. + this.helper.dispose();this.outline.geometry.dispose();this.outline.material.dispose();for(const {icon} of this.icons){icon.geometry.dispose();icon.material.dispose();}this.overlay.clear();} +} diff --git a/packages/three/spatial-math.mjs b/packages/three/spatial-math.mjs new file mode 100644 index 0000000..4a9ec69 --- /dev/null +++ b/packages/three/spatial-math.mjs @@ -0,0 +1,59 @@ +import {Matrix4,Vector3,Quaternion} from 'three'; +const EPS=1e-8; +export function rigidParent(matrix) { + const e=matrix.elements,a=new Vector3(e[0],e[1],e[2]),b=new Vector3(e[4],e[5],e[6]),c=new Vector3(e[8],e[9],e[10]); + const lengths=[a.length(),b.length(),c.length()],scale=lengths[0]; + if(!e.every(Number.isFinite)||scaleMath.abs(l-scale)>EPS*Math.max(1,scale))||Math.max(Math.abs(a.dot(b)),Math.abs(a.dot(c)),Math.abs(b.dot(c)))>EPS*scale*scale)throw Error('Spatial editing requires rigid or positive uniformly scaled ancestry'); + const position=new Vector3(),quaternion=new Quaternion(),scaling=new Vector3();matrix.decompose(position,quaternion,scaling); + return {matrix:matrix.clone(),inverse:matrix.clone().invert(),quaternion,scale}; +} +export function worldPivotToPosition(worldPivot,{parent,pivot,quaternion,scale,convention='pivot'}) { + const offset=convention==='trs'?pivot.clone().multiplyScalar(scale).applyQuaternion(quaternion):pivot; + return worldPivot.clone().applyMatrix4(parent.inverse).sub(offset); +} +export function pivotMatrix(position,quaternion,scale,pivot,convention='pivot') { + const out=new Matrix4().compose(position,quaternion,new Vector3(scale,scale,scale)); + if(convention==='trs')return out; + return new Matrix4().makeTranslation(position.x+pivot.x,position.y+pivot.y,position.z+pivot.z) + .multiply(new Matrix4().compose(new Vector3(),quaternion,new Vector3(scale,scale,scale))) + .multiply(new Matrix4().makeTranslation(-pivot.x,-pivot.y,-pivot.z)); +} +/** Project within the selected axis/plane. Component clamping would change the + * direction under a rotated parent. Plane constraints form a convex polygon. */ +export function constrainTranslation({startLocal,basisWorld,parent,desired,min=-Infinity,max=Infinity,snap=0,unitScale=1}) { + const n=basisWorld.length;if(n<1||n>2)throw Error('Translation needs an axis or plane'); + let point=desired.map(v=>snap>0?Math.round(v/(snap*unitScale))*snap*unitScale:v); + const inv=parent.inverse,origin=new Vector3().applyMatrix4(inv); + const basis=basisWorld.map(v=>v.clone().applyMatrix4(inv).sub(origin).toArray()); + const start=startLocal.toArray(),constraints=[]; + for(let i=0;i<3;i++) { + const row=basis.map(v=>v[i]); + if(Number.isFinite(max))constraints.push([row,max-start[i]]); + if(Number.isFinite(min))constraints.push([row.map(v=>-v),start[i]-min]); + } + const valid=p=>constraints.every(([r,b])=>r.reduce((s,v,i)=>s+v*p[i],0)<=b+1e-8); + if(!valid(point)) { + const candidates=[]; + if(n===1){let lo=-Infinity,hi=Infinity;for(const [[a],b] of constraints){if(Math.abs(a)0)hi=Math.min(hi,b/a);else lo=Math.max(lo,b/a);}if(lo>hi)throw Error('No feasible translation');point=[Math.max(lo,Math.min(hi,point[0]))];} + else { + for(const [r,b] of constraints){const d=r[0]*r[0]+r[1]*r[1];if(d>EPS){const k=(r[0]*point[0]+r[1]*point[1]-b)/d;const p=[point[0]-k*r[0],point[1]-k*r[1]];if(valid(p))candidates.push(p);}} + for(let i=0;iEPS){const p=[(b*c[1]-a[1]*d)/det,(a[0]*d-b*c[0])/det];if(valid(p))candidates.push(p);}} + if(!candidates.length)throw Error('No feasible translation');candidates.sort((a,b)=>Math.hypot(...a.map((v,i)=>v-point[i]))-Math.hypot(...b.map((v,i)=>v-point[i])));point=candidates[0]; + } + } + const local=startLocal.clone();for(let i=0;itolerance)throw Error('Infeasible translation');local.setComponent(i,min);}if(v>max){if(v-max>tolerance)throw Error('Infeasible translation');local.setComponent(i,max);}} + return local; +} +export function positiveScale(value,{min,max=Infinity,snap=0}={}) { + if(!Number.isFinite(value))throw Error('Degenerate scale gesture'); + const upper=max??Infinity; + if(!(upper>0)||min!=null&&!Number.isFinite(min))throw Error('Invalid positive scale bounds'); + // Authored positive bounds take precedence over the fallback floor. If only a + // smaller upper bound is declared, the fallback must still fit that interval. + const lower=min>0?min:Math.min(.001,upper); + if(lower>upper)throw Error('Invalid positive scale bounds'); + return Math.max(lower,Math.min(upper,snap>0?Math.round(value/snap)*snap:value)); +} diff --git a/packages/three/surface-snap.mjs b/packages/three/surface-snap.mjs new file mode 100644 index 0000000..5b8439f --- /dev/null +++ b/packages/three/surface-snap.mjs @@ -0,0 +1,43 @@ +import * as T from 'three'; + +const up=new T.Vector3(0,1,0); +function visible(object){for(let o=object;o;o=o.parent)if(!o.visible)return false;return true;} +function within(object,root){for(let o=object;o;o=o.parent)if(o===root)return true;return false;} + +/** Visible geometry only, including the evaluated skeletal pose. Frozen at admission. */ +export function surfaceBounds(root){ + root.updateWorldMatrix(true,true);const bounds=new T.Box3(),point=new T.Vector3(); + root.traverse(object=>{ + if(!object.isMesh||!visible(object))return; + if(object.isInstancedMesh){object.computeBoundingBox();bounds.union(object.boundingBox.clone().applyMatrix4(object.matrixWorld));return;} + const position=object.geometry.attributes.position;if(!position)return; + for(let i=0;i{if(o.isMesh&&visible(o)&&!within(o,exclude))meshes.add(o);});} + if(!meshes.size)return null; + for(const mesh of meshes)if(mesh.isSkinnedMesh){mesh.computeBoundingSphere();if(mesh.boundingBox)mesh.computeBoundingBox();} + let best=null; + for(const x of [box.min.x,(box.min.x+box.max.x)/2,box.max.x])for(const z of [box.min.z,(box.min.z+box.max.z)/2,box.max.z]){ + ray.set(new T.Vector3(x,ceiling+1e-6,z),new T.Vector3(0,-1,0)); + for(const hit of ray.intersectObjects([...meshes],false)){ + if(!hit.face)continue; + const matrix=hit.object.matrixWorld.clone(); + if(hit.instanceId!==undefined){const instance=new T.Matrix4();hit.object.getMatrixAt(hit.instanceId,instance);matrix.multiply(instance);} + const normal=hit.face.normal.clone().applyMatrix3(normalMatrix.getNormalMatrix(matrix)).normalize(); + if(normal.dot(up)ceiling+1e-7)continue; + if(!best||hit.point.y>best.height)best={height:hit.point.y,object:hit.object}; + } + } + return best?{...best,offsetY:best.height-box.min.y}:null; +} diff --git a/scripts/project.mjs b/scripts/project.mjs index f1f5ed9..7a32ac4 100644 --- a/scripts/project.mjs +++ b/scripts/project.mjs @@ -8,7 +8,7 @@ export async function definition(dir){ dir=await fs.realpath(dir);const file=path.join(dir,'clip.mjs'); const bundled=await build({entryPoints:[file],bundle:true,format:'esm',platform:'browser',write:false,minify:true,nodePaths:[path.join(root,'node_modules')],plugins:[publicPackagePlugin()]}); const assets=await readAssets(dir);const assetIdentity=Object.fromEntries(Object.entries(assets).map(([id,{file,...rest}])=>[id,rest])); - const runtimeFiles=['packages/core/key-timing.mjs','packages/core/manipulation.mjs','packages/three/manipulation.mjs','packages/core/index.mjs','packages/core/animation.mjs','packages/core/host.mjs','packages/core/renderer.mjs','packages/core/controls.mjs','packages/core/objects.mjs','packages/core/interaction.mjs','packages/core/definition.mjs','packages/runtime/renderer-host.mjs','packages/runtime/renderer-compositor.mjs','packages/runtime/editor.mjs','packages/runtime/compositor.mjs','packages/media/ports.mjs','packages/media/browser.mjs','packages/media/audio-plan.mjs','packages/media/playback.mjs','scripts/render.mjs','scripts/audio.mjs','package-lock.json']; + const runtimeFiles=['packages/core/rotation.mjs','packages/core/spatial-session.mjs','packages/three/spatial-math.mjs','packages/three/spatial-editor.mjs','packages/core/key-timing.mjs','packages/core/manipulation.mjs','packages/three/manipulation.mjs','packages/core/index.mjs','packages/core/animation.mjs','packages/core/host.mjs','packages/core/renderer.mjs','packages/core/controls.mjs','packages/core/objects.mjs','packages/core/interaction.mjs','packages/core/definition.mjs','packages/runtime/renderer-host.mjs','packages/runtime/renderer-compositor.mjs','packages/runtime/editor.mjs','packages/runtime/compositor.mjs','packages/media/ports.mjs','packages/media/browser.mjs','packages/media/audio-plan.mjs','packages/media/playback.mjs','scripts/render.mjs','scripts/audio.mjs','package-lock.json']; const runtimeIdentity=Object.fromEntries(await Promise.all(runtimeFiles.map(async file=>[file,hash(await fs.readFile(path.join(root,file)))]))); const binding=hash(Buffer.concat([Buffer.from(bundled.outputFiles[0].contents),Buffer.from(JSON.stringify(runtimeIdentity)),Buffer.from(JSON.stringify(assetIdentity))])); const normalized=normalizeClipModule(await import('data:text/javascript;base64,'+Buffer.from(bundled.outputFiles[0].contents).toString('base64'))); @@ -69,7 +69,7 @@ export async function preview(dir,{port=8791,editable=true,signal,snapshot}={}){ } export async function snapshotSource(d){ - const dest=path.join(d.dir,'.editableframe','sources',d.binding);await fs.mkdir(dest,{recursive:true});await fs.writeFile(path.join(dest,'clip.bundle.mjs'),d.bundledSource);await fs.copyFile(d.file,path.join(dest,'clip.mjs'));await fs.copyFile(path.join(root,'packages/core/index.mjs'),path.join(dest,'core.mjs'));await fs.copyFile(path.join(root,'packages/core/animation.mjs'),path.join(dest,'animation.mjs'));for(const file of ['key-timing.mjs','manipulation.mjs','renderer.mjs','controls.mjs','objects.mjs','interaction.mjs','definition.mjs'])await fs.copyFile(path.join(root,'packages/core',file),path.join(dest,file));await fs.writeFile(path.join(dest,'runtime.json'),JSON.stringify(d.runtimeIdentity,null,2));await fs.writeFile(path.join(dest,'assets.json'),JSON.stringify({version:1,assets:d.assetIdentity},null,2)); + const dest=path.join(d.dir,'.editableframe','sources',d.binding);await fs.mkdir(dest,{recursive:true});await fs.writeFile(path.join(dest,'clip.bundle.mjs'),d.bundledSource);await fs.copyFile(d.file,path.join(dest,'clip.mjs'));await fs.copyFile(path.join(root,'packages/core/index.mjs'),path.join(dest,'core.mjs'));await fs.copyFile(path.join(root,'packages/core/animation.mjs'),path.join(dest,'animation.mjs'));for(const file of ['rotation.mjs','spatial-session.mjs','key-timing.mjs','manipulation.mjs','renderer.mjs','controls.mjs','objects.mjs','interaction.mjs','definition.mjs'])await fs.copyFile(path.join(root,'packages/core',file),path.join(dest,file));await fs.writeFile(path.join(dest,'runtime.json'),JSON.stringify(d.runtimeIdentity,null,2));await fs.writeFile(path.join(dest,'assets.json'),JSON.stringify({version:1,assets:d.assetIdentity},null,2)); } export async function rebind(dir,{reset=false}={}){ const lock=path.join(dir,'.editableframe-edit.lock'),fd=await fs.open(lock,'wx'); diff --git a/tests/scene-editor-browser.mjs b/tests/scene-editor-browser.mjs new file mode 100644 index 0000000..4328ba3 --- /dev/null +++ b/tests/scene-editor-browser.mjs @@ -0,0 +1,80 @@ +import assert from 'node:assert/strict'; +import fs from 'node:fs/promises'; +import http from 'node:http'; +import path from 'node:path'; +import {pathToFileURL} from 'node:url'; +import {chromium} from 'playwright'; +import {buildScene} from '../examples/scene-editor/build.mjs'; + +const {html,out}=await buildScene({assetDirectory:process.env.EF_SCENE_ASSETS}); +const server=http.createServer((req,res)=>res.writeHead(200,{'Content-Type':'text/html'}).end(html)); +await new Promise(resolve=>server.listen(0,'127.0.0.1',resolve)); +const url='http://127.0.0.1:'+server.address().port; +let browser;const checks=[],errors=[]; +try{ + browser=await chromium.launch({headless:true,args:['--enable-webgl',process.platform==='darwin'?'--use-angle=metal':'--use-angle=swiftshader']}); + const context=await browser.newContext({viewport:{width:1440,height:1000}}),p=await context.newPage(); + p.on('pageerror',e=>errors.push(e.message)); + p.on('request',r=>{if(!r.url().startsWith(url)&&!r.url().startsWith('data:')&&!r.url().startsWith('blob:'))errors.push('External request: '+r.url());}); + const ready=()=>p.waitForFunction(()=>window.cinema?.ready); + const state=()=>p.evaluate(()=>cinema.state); + const seek=at=>p.evaluate(at=>cinema.seek(at),at); + const select=anchor=>p.locator('[data-anchor="'+anchor+'"]').click(); + const mode=next=>p.locator('[data-mode="'+next+'"]').click(); + // Discover rendered semantic handles, then use trusted browser mouse input. + async function handle(name,page=p){ + const point=await page.evaluate(name=>{ + const e=cinema.editor;e.sync();const center=e.proxy.position.clone().project(e.camera),cx=(center.x+1)/2*e.width,cy=(1-center.y)/2*e.height; + const sync=e.sync;e.sync=()=>{};let found; + try{for(let radius=12;radius<180&&!found;radius+=4)for(let angle=0;angle!!cinema.state.draft); + return point; + } + async function drag(name,dx=60,dy=-25,page=p){const point=await handle(name,page);await page.mouse.move(point.x+dx,point.y+dy,{steps:6});await page.mouse.up();await page.waitForFunction(()=>!cinema.state.draft);return page.evaluate(()=>cinema.state);} + await p.goto(url);await ready();const initial=await state();assert.equal(initial.kind,'drama');assert.equal(await p.locator('#surface-snap').isChecked(),true);await p.locator('#surface-snap').uncheck(); + const moved=await drag('x');assert.notEqual(moved.props.actors.lead.position[0],initial.props.actors.lead.position[0]);assert.deepEqual(moved.props.actors.lead.position.slice(1),initial.props.actors.lead.position.slice(1));assert.equal(moved.history,initial.history+1); + await p.locator('#undo').click();await p.waitForFunction(()=>cinema.state.props.actors.lead.position[0]===0);assert.deepEqual((await state()).props.actors.lead.position,initial.props.actors.lead.position);await p.locator('#redo').click();await p.waitForFunction(x=>cinema.state.props.actors.lead.position[0]===x,moved.props.actors.lead.position[0]);assert.deepEqual((await state()).props.actors.lead.position,moved.props.actors.lead.position);checks.push('Trusted axis drag, single transaction, undo and redo'); + const point=await handle('y');await p.mouse.move(point.x+20,point.y-60,{steps:4});assert.ok((await state()).draft);await p.keyboard.press('Escape');await p.mouse.up();assert.deepEqual((await state()).props,moved.props);assert.equal((await state()).history,moved.history);checks.push('Escape cancels without a history entry'); + await mode('rotate');const rotated=await drag('y',35,30);assert.notDeepEqual(rotated.props.actors.lead.orientation,moved.props.actors.lead.orientation); + await mode('scale');const scaled=await drag('uniform',45,45);assert.ok(scaled.props.actors.lead.scale>0);assert.notEqual(scaled.props.actors.lead.scale,rotated.props.actors.lead.scale);checks.push('Quaternion rotation and positive uniform scaling'); + await p.locator('#auto-key').check();await seek(0);await p.evaluate(()=>cinema.edit({'lead.position':[0,0,0]}));await seek(96);await p.evaluate(()=>cinema.edit({'lead.position':[2,0,0]}));await seek(48);assert.ok(Math.abs((await state()).props.actors.lead.position[0]-1)<1e-8);await seek(96);assert.equal((await state()).props.actors.lead.position[0],2);await seek(48);checks.push('Frame-based keyframes interpolate and repeat after seeking'); + const snapContext=await browser.newContext({viewport:{width:1440,height:1000}}),snapPage=await snapContext.newPage();snapPage.on('pageerror',e=>errors.push(e.message));await snapPage.goto(url);await snapPage.waitForFunction(()=>window.cinema?.ready); + const bottom=()=>snapPage.evaluate(()=>cinema.editor.outline.box.min.y); + await snapPage.evaluate(()=>cinema.edit({'lead.position':[0,2,0]})); + const elevated=await snapPage.evaluate(()=>cinema.state),stationaryPoint=await handle('x',snapPage);await snapPage.mouse.up();await snapPage.waitForFunction(()=>!cinema.state.draft);assert.equal((await snapPage.evaluate(()=>cinema.state)).history,elevated.history); + // Lifted clear of the tabletop, then release over it: upper face is 1.06. + const snapped=await drag('x',15,0,snapPage);assert.ok(Math.abs(await bottom()-1.06)<1e-5);assert.equal(snapped.history,elevated.history+1); + await snapPage.evaluate(()=>cinema.undo());assert.deepEqual((await snapPage.evaluate(()=>cinema.state)).props.actors.lead.position,elevated.props.actors.lead.position);await snapPage.evaluate(()=>cinema.redo());assert.ok(Math.abs(await bottom()-1.06)<1e-5); + await snapPage.evaluate(()=>cinema.edit({'lead.position':[0,2,-2]}));await drag('x',15,0,snapPage);assert.ok(Math.abs(await bottom()+.03)<1e-5); + await snapPage.locator('#surface-snap').uncheck();await snapPage.evaluate(()=>cinema.edit({'lead.position':[0,2,-2]}));await drag('x',15,0,snapPage);assert.equal((await snapPage.evaluate(()=>cinema.state)).props.actors.lead.position[1],2); + await snapPage.locator('#surface-snap').check();const cancelBefore=await snapPage.evaluate(()=>cinema.serialize()),cancelPoint=await handle('x',snapPage);await snapPage.mouse.move(cancelPoint.x+25,cancelPoint.y);await snapPage.keyboard.press('Escape');await snapPage.mouse.up();await snapPage.waitForFunction(()=>!cinema.state.draft);assert.equal(await snapPage.evaluate(()=>cinema.serialize()),cancelBefore); + await snapPage.locator('#scene-kind').selectOption('battle');await snapPage.waitForFunction(()=>cinema.ready&&cinema.state.kind==='battle');await snapPage.evaluate(()=>cinema.edit({'lead.position':[0,2,0]}));await drag('x',15,0,snapPage);assert.ok(Math.abs(await bottom()+.05)<1e-5);await snapContext.close();checks.push('Surface snap settles onto table/floor in both scenes, toggles off, ignores clicks, cancels and undoes atomically'); + const saved=await p.evaluate(()=>cinema.serialize());await p.reload();await ready();await seek(48);assert.equal(await p.evaluate(()=>cinema.serialize()),saved);assert.equal((await state()).props.actors.lead.position[0],1);checks.push('Reload restores revision, history and animation'); + await select('key');await p.locator('[aria-label="key.intensity"]').fill('5');await p.locator('[aria-label="key.intensity"]').press('Enter');await p.waitForFunction(()=>cinema.state.props.key.intensity===5);checks.push('Light inspector edits'); + await select('camera');assert.equal((await state()).view,'editor');await mode('translate');const stationary=await state();await handle('x');await p.mouse.up();await p.waitForFunction(()=>!cinema.state.draft);assert.equal((await state()).history,stationary.history);assert.equal((await state()).props.shot.override,false);checks.push('Stationary camera click leaves authored shots and history unchanged');await drag('x',35,0);assert.equal((await state()).props.shot.override,true);checks.push('Shot camera is editable from a separate director view'); + await select('lead');await mode('translate');await p.locator('#auto-key').uncheck(); + const beforeOrbit=await state(),canvasBox=await p.locator('#canvas canvas').boundingBox();await p.mouse.move(canvasBox.x+canvasBox.width*.75,canvasBox.y+canvasBox.height*.5);await p.mouse.down({button:'right'});await p.mouse.move(canvasBox.x+canvasBox.width*.75+45,canvasBox.y+canvasBox.height*.5+15,{steps:4});await p.mouse.up({button:'right'});assert.equal((await state()).revision,beforeOrbit.revision);assert.ok((await state()).identity.viewRevision>beforeOrbit.identity.viewRevision);checks.push('Director navigation updates presentation without authoring a scene edit'); + const beforeCapture=await state(); + await p.evaluate(()=>cinema.capture());const clean=await p.evaluate(()=>({png:cinema.lastCapture.png,props:cinema.lastCapture.props,raw:cinema.lastCapture.raw})); + const draftPoint=await handle('x');await p.mouse.move(draftPoint.x+70,draftPoint.y-20,{steps:4});assert.ok((await state()).draft);await p.evaluate(()=>cinema.capture()); + const during=await p.evaluate(()=>({png:cinema.lastCapture.png,props:cinema.lastCapture.props,raw:cinema.lastCapture.raw}));assert.equal(during.raw,clean.raw);assert.deepEqual(during.props,clean.props);assert.equal(during.png,clean.png);assert.ok((await state()).draft);await p.keyboard.press('Escape');await p.mouse.up();assert.deepEqual((await state()).props,beforeCapture.props);checks.push('Capture during drag uses committed shot, excludes draft and helpers, leaves gesture untouched'); + await fs.writeFile(path.join(out,'drama-reference.png'),Buffer.from(clean.png.split(',')[1],'base64'));await p.screenshot({path:path.join(out,'drama-editor.png')}); + await p.locator('#scene-kind').selectOption('battle');await p.waitForFunction(()=>cinema.ready&&cinema.state.kind==='battle');await seek(48);await select('camera');await p.locator('#play').click();await p.waitForFunction(()=>cinema.state.frame>=120);await p.locator('#play').click();assert.equal(Number(await p.locator('[aria-label="camera.position.x"]').inputValue()),-5.6);await p.locator('[aria-label="camera.position.x"]').fill('-5.5');await p.locator('[aria-label="camera.position.x"]').press('Enter');await p.waitForFunction(()=>cinema.state.props.shot.override);const shot=await p.evaluate(()=>({props:cinema.state.props.shot,position:cinema.instance.editing.camera.position.toArray(),fov:cinema.instance.editing.camera.fov}));assert.deepEqual(shot.position,[-5.5,4.4,9.2]);assert.equal(shot.fov,40);assert.deepEqual(shot.props.target,[.35,3.8,0]);await p.evaluate(()=>cinema.undo());checks.push('First camera edit preserves the evaluated shot target and lens');await select('lead');await seek(150);const battle=await state();await drag('x',40,-15);assert.notDeepEqual((await state()).props.actors.lead.position,battle.props.actors.lead.position);await p.locator('#undo').click(); + await p.evaluate(()=>cinema.capture());const battlePng=await p.evaluate(()=>cinema.lastCapture.png);await fs.writeFile(path.join(out,'battle-reference.png'),Buffer.from(battlePng.split(',')[1],'base64'));await p.screenshot({path:path.join(out,'battle-editor.png')});checks.push('Mecha-versus-ape scene renders, manipulates and captures'); + await p.locator('#scene-kind').selectOption('drama');await p.waitForFunction(()=>cinema.ready&&cinema.state.kind==='drama');assert.equal((await state()).props.key.intensity,5);checks.push('Scene switching disposes and restores independent edits'); + await select('look');await p.locator('[aria-label="look.clay"]').check();await p.waitForFunction(()=>cinema.state.props.clay&&document.querySelector('#clay').checked);await p.evaluate(()=>cinema.undo());assert.equal(await p.locator('#clay').isChecked(),false);await p.evaluate(()=>cinema.redo());assert.equal(await p.locator('#clay').isChecked(),true); + await p.reload();await ready();assert.equal((await state()).props.clay,true);assert.equal(await p.locator('#clay').isChecked(),true); + await p.locator('#scene-kind').selectOption('battle');await p.waitForFunction(()=>cinema.ready&&cinema.state.kind==='battle');assert.equal(await p.locator('#clay').isChecked(),false);await p.locator('#scene-kind').selectOption('drama');await p.waitForFunction(()=>cinema.ready&&cinema.state.kind==='drama');assert.equal(await p.locator('#clay').isChecked(),true);await p.locator('#clay').uncheck();await p.waitForFunction(()=>!cinema.state.props.clay);checks.push('Clay toolbar follows inspector edits, undo/redo, reload and scene switching'); + const second=await context.newPage();await second.goto(url);await second.waitForFunction(()=>window.cinema?.ready);const old=await state();await second.evaluate(()=>cinema.edit({'key.intensity':6}));await p.evaluate(()=>cinema.edit({'key.intensity':7}));assert.equal((await state()).revision,old.revision);assert.equal((await state()).props.key.intensity,5);assert.match(await p.locator('#status').textContent(),/Another window/);await second.close();checks.push('Concurrent tab edits cannot overwrite a newer saved revision'); + const standalone=await context.newPage();await standalone.goto(pathToFileURL(path.join(out,'scene-studio.html')).href);await standalone.waitForFunction(()=>window.cinema?.ready);assert.equal(await standalone.evaluate(()=>cinema.state.kind),'drama');await standalone.locator('[aria-label="lead.position.x"]').fill('1.25');await standalone.locator('[aria-label="lead.position.x"]').press('Enter');await standalone.waitForFunction(()=>cinema.state.props.actors.lead.position[0]===1.25,{},{timeout:3000});await drag('x',45,-15,standalone);await standalone.evaluate(()=>cinema.undo());assert.equal((await standalone.evaluate(()=>cinema.state.props.actors.lead.position[0])),1.25);await standalone.close();checks.push('Self-contained HTML opens, edits and drags directly from disk'); + const deniedContext=await browser.newContext({viewport:{width:1440,height:1000}});await deniedContext.addInitScript(()=>Object.defineProperty(navigator,'locks',{configurable:true,value:{request(){throw new DOMException("Failed to execute 'request' on 'LockManager': Access to the Locks API is denied in this context.",'SecurityError');}}})); + const denied=await deniedContext.newPage();denied.on('pageerror',e=>errors.push(e.message));await denied.goto(pathToFileURL(path.join(out,'scene-studio.html')).href);await denied.waitForFunction(()=>window.cinema?.ready);assert.equal(await denied.evaluate(()=>cinema.state.persistence),'memory'); + await denied.locator('[aria-label="lead.position.x"]').fill('1.5');await denied.locator('[aria-label="lead.position.x"]').press('Enter');await denied.waitForFunction(()=>cinema.state.props.actors.lead.position[0]===1.5);await drag('x',45,-15,denied);await denied.evaluate(()=>cinema.undo());assert.equal(await denied.evaluate(()=>cinema.state.props.actors.lead.position[0]),1.5);await denied.evaluate(()=>cinema.redo());const retained=await denied.evaluate(()=>cinema.serialize());assert.match(await denied.locator('#revision').textContent(),/Memory only/); + const downloading=denied.waitForEvent('download');await denied.locator('#save').click();const downloaded=await downloading,downloadPath=await downloaded.path();assert.equal(await fs.readFile(downloadPath,'utf8'),retained); + await denied.reload();await denied.waitForFunction(()=>window.cinema?.ready);assert.equal(await denied.evaluate(()=>cinema.state.props.actors.lead.position[0]),0);await denied.locator('#open').setInputFiles(downloadPath);await denied.waitForFunction(()=>cinema.state.props.actors.lead.position[0]!==0);assert.equal(await denied.evaluate(()=>cinema.serialize()),retained);await deniedContext.close();checks.push('Denied Locks API still permits numeric edits, drags, undo/redo and JSON download/reopen'); + assert.deepEqual(errors,[]);await fs.writeFile(path.join(out,'browser-results.json'),JSON.stringify({checks,errors},null,2));console.log(JSON.stringify({checks,errors},null,2)); +}catch(error){console.error(error);throw error;}finally{await browser?.close();await new Promise(resolve=>server.close(resolve));} diff --git a/tests/scene-persistence.test.mjs b/tests/scene-persistence.test.mjs new file mode 100644 index 0000000..75d4d7a --- /dev/null +++ b/tests/scene-persistence.test.mjs @@ -0,0 +1,27 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import {createScenePersistence} from '../examples/scene-editor/persistence.mjs'; +const denied=()=>new DOMException("Failed to execute 'request' on 'LockManager': Access to the Locks API is denied in this context.",'SecurityError'); +const storage=()=>{const entries=new Map();return {entries,setItem:(k,v)=>entries.set(k,v),removeItem:k=>entries.delete(k)};}; +test('absent or inaccessible Locks API uses memory without accessing storage',async()=>{ + for(const getLocks of [()=>undefined,()=>{throw denied();}]){let actions=0;const p=createScenePersistence({getLocks,getStorage:()=>{throw Error('Must not access storage');}});assert.equal(await p.initialize(),false);assert.equal(await p.run('scene',()=>++actions),1);assert.equal(actions,1);} +}); +test('exposed but denied lock requests fall back for synchronous and asynchronous errors',async()=>{ + for(const asyncFailure of [false,true]){let requests=0;const p=createScenePersistence({getLocks:()=>({request(){requests++;if(asyncFailure)return Promise.reject(denied());throw denied();}}),getStorage:()=>{throw Error('Must not access storage');}});assert.equal(await p.initialize(),false);let actions=0;await p.run('scene',()=>actions++);assert.equal(actions,1);assert.equal(requests,1);assert.equal(p.storage,null);} +}); +test('unavailable storage also chooses memory and successful probes remove their key',async()=>{ + const store=storage(),locks={request:async(_key,action)=>action()};const good=createScenePersistence({getLocks:()=>locks,getStorage:()=>store});assert.equal(await good.initialize(),true);assert.equal(store.entries.size,0);assert.equal(good.storage,store); + const bad=createScenePersistence({getLocks:()=>locks,getStorage:()=>({setItem(){throw denied();}})});assert.equal(await bad.initialize(),false);assert.equal(bad.storage,null); +}); +test('permission revoked before a transaction callback switches to memory once',async()=>{ + let requests=0,actions=0;const p=createScenePersistence({getLocks:()=>({request:async(_key,action)=>{if(++requests>1)throw denied();return action();}}),getStorage:storage});assert.equal(await p.initialize(),true); + await p.run('scene',()=>{actions++;assert.equal(p.supported,false);assert.equal(p.storage,null);});await p.run('scene',()=>actions++);assert.equal(actions,2);assert.equal(requests,2); +}); +test('callback failures never fall back or replay a transaction',async()=>{ + const p=createScenePersistence({getLocks:()=>({request:async(_key,action)=>action()}),getStorage:storage});await p.initialize();let actions=0; + await assert.rejects(p.run('scene',()=>{actions++;throw denied();}),{name:'SecurityError'});assert.equal(actions,1);assert.equal(p.supported,true); +}); +test('rejection after the callback ran preserves the uncertain outcome without replay',async()=>{ + let requests=0,actions=0;const p=createScenePersistence({getLocks:()=>({request:async(_key,action)=>{const value=action();if(++requests>1)throw denied();return value;}}),getStorage:storage});await p.initialize(); + await assert.rejects(p.run('scene',()=>++actions),{name:'SecurityError'});assert.equal(actions,1);assert.equal(p.supported,true); +}); diff --git a/tests/spatial.test.mjs b/tests/spatial.test.mjs new file mode 100644 index 0000000..157c52b --- /dev/null +++ b/tests/spatial.test.mjs @@ -0,0 +1,106 @@ +import test from 'node:test';import assert from 'node:assert/strict'; +import {Matrix4,Vector3,Quaternion} from 'three'; +import {normalizeControls,controlsIdentity} from '../packages/core/controls.mjs'; +import {slerpQuaternion,sameOrientation} from '../packages/core/rotation.mjs'; +import {negotiateSpatial,createSpatialSession} from '../packages/core/spatial-session.mjs'; +import {rigidParent,pivotMatrix,worldPivotToPosition,constrainTranslation,positiveScale} from '../packages/three/spatial-math.mjs'; +import {cinemaScene} from '../examples/scene-editor/cinema.mjs'; +import {normalizeClipModule,sampledRendererProps} from '../packages/core/definition.mjs'; +import {Studio} from '../packages/core/index.mjs'; +const schema={version:1,fields:[{key:'p',anchor:'a',type:'vec3',default:[0,0,0],role:'position',unit:'world-unit',coordinateSpace:'local',animatable:true},{key:'other',anchor:'a',type:'number',default:0}]}; +const identity=()=>({clipId:'clip',revision:'revision',sourceRevision:'source',seekGeneration:0,rendererInstance:1,bindingGeneration:1,viewRevision:0,viewportRevision:0,shot:'wide',clock:{frame:0,fps:24}}); +const capabilities={gestureProtocols:[2],bindings:[{id:'placement',operations:{translate:{handles:['x'],spaces:['world'],keys:['p']}}}]}; +const begin=(session,i)=>session.begin({trusted:true,identity:i,anchor:'a',capabilities,bindingId:'placement',operation:'translate',handle:'x',space:'world'}); +test('v1 identity stays byte-compatible; v2 rotation sampling survives persistence',()=>{ + const v1=normalizeControls(schema);assert.equal(controlsIdentity(v1),JSON.stringify(v1.fields.map(({key,anchor,property,path,type,unit,coordinateSpace,role,animatable,interpolation})=>({key,anchor,property,path,type,unit:unit??null,coordinateSpace:coordinateSpace??null,role:role??null,animatable,interpolation})).sort((a,b)=>a.key.localeCompare(b.key)))); + const scene=cinemaScene('battle'),d=normalizeClipModule({default:scene}),studio=new Studio(d.templates,{binding:'test'});studio.doc.clips=[studio.make(scene.id)];studio.base=structuredClone(studio.doc);const clip=studio.doc.clips[0]; + studio.execute({type:'keyframes',clip:clip.id,anchor:'lead',property:'orientation',keys:[{time:0,value:[0,0,0,1],easing:'linear'},{time:1,value:[0,1,0,0],easing:'linear'}]}); + const restored=Studio.restore(studio.serialize(),d.templates,'test');const props=sampledRendererProps(scene,d.template,restored.clip(clip.id),restored.values(restored.clip(clip.id),60000));assert.ok(sameOrientation(props.actors.lead.orientation,[0,Math.SQRT1_2,0,Math.SQRT1_2])); + assert.throws(()=>normalizeControls({...schema,fields:[{...schema.fields[0],type:'quaternion'}]})); +}); +test('180-degree rotation ties are invariant to all endpoint signs',()=>{ + const a=[0,0,0,1],b=[0,1,0,0],expected=slerpQuaternion(a,b,.5); + for(const x of [a,a.map(v=>-v)])for(const y of [b,b.map(v=>-v)])assert.ok(sameOrientation(expected,slerpQuaternion(x,y,.5))); +}); +test('pivot conventions round trip under rotated scaled ancestry',()=>{ + const parent=rigidParent(new Matrix4().compose(new Vector3(3,2,1),new Quaternion().setFromAxisAngle(new Vector3(0,0,1),.6),new Vector3(2,2,2))); + const pivot=new Vector3(1,.2,0),position=new Vector3(.3,.4,.5),quaternion=new Quaternion().setFromAxisAngle(new Vector3(0,1,0),.8),scale=1.4; + for(const convention of ['pivot','trs']){const worldPivot=pivot.clone().applyMatrix4(pivotMatrix(position,quaternion,scale,pivot,convention)).applyMatrix4(parent.matrix);assert.ok(worldPivotToPosition(worldPivot,{parent,pivot,quaternion,scale,convention}).distanceTo(position)<1e-10);} + assert.throws(()=>rigidParent(new Matrix4().makeScale(1,2,1)));assert.throws(()=>rigidParent(new Matrix4().makeScale(-1,1,1))); +}); +test('bounds and snapping preserve world-axis direction under transformed parents',()=>{ + const parent=rigidParent(new Matrix4().compose(new Vector3(10,3,0),new Quaternion().setFromAxisAngle(new Vector3(0,0,1),Math.PI/4),new Vector3(2,2,2))),startLocal=new Vector3(.3,.2,0),basisWorld=[new Vector3(1,0,0)]; + const end=constrainTranslation({startLocal,basisWorld,parent,desired:[1],min:0,max:1,snap:.5});const delta=end.clone().applyMatrix4(parent.matrix).sub(startLocal.clone().applyMatrix4(parent.matrix));assert.ok(Math.abs(delta.y)<1e-9);assert.ok(delta.x>0&&delta.x<1);assert.ok(end.y>=-1e-9); + const plane=constrainTranslation({startLocal:new Vector3(.5,.5,0),basisWorld:[new Vector3(1,0,0),new Vector3(0,1,0)],parent:rigidParent(new Matrix4()),desired:[2,2],min:0,max:1,snap:.7});assert.deepEqual(plane.toArray(),[1,1,0]); + assert.equal(positiveScale(-2),.001);assert.equal(positiveScale(1.2,{snap:.5}),1);assert.throws(()=>positiveScale(NaN)); +}); +test('v1 controls negotiate v2 independently; operation keys and presentation are enforced',()=>{ + assert.equal(negotiateSpatial({host:[2],transport:[1],adapter:[2]}),null);let i=identity(),count=0;const s=createSpatialSession({controls:schema,readIdentity:()=>i,publish:()=>count++});const b=begin(s,i);assert.throws(()=>s.update({id:b.id,values:{other:1}}));i={...i,viewRevision:1};assert.throws(()=>s.update({id:b.id,values:{p:[1,0,0]}}),/Stale/);assert.equal(s.state,null);assert.equal(count,0); +}); +test('release fixes final value; cancellation and uncertain publication have distinct boundaries',async()=>{ + let i=identity(),published=[],resolve;const s=createSpatialSession({controls:schema,readIdentity:()=>i,publish:async p=>{published.push(p);return {id:p.id}}});let b=begin(s,i);s.update({id:b.id,values:{p:[1,0,0]}});const releasing=s.release({id:b.id,trusted:true,evaluate:()=>new Promise(r=>resolve=r)});assert.equal(s.state.phase,'finalizing');assert.ok(s.cancel(b.id));resolve({values:{p:[2,0,0]}});assert.equal((await releasing).status,'cancelled');assert.equal(published.length,0); + b=begin(s,i);await s.release({id:b.id,trusted:true,evaluate:()=>({values:{p:[3,0,0]}})});assert.equal(published.length,1);assert.deepEqual(published[0].values.p,[3,0,0]); + let receipt;const u=createSpatialSession({controls:schema,readIdentity:()=>i,publish:p=>{receipt={id:p.id};throw Error('lost acknowledgment')},reconcile:()=>({status:'committed',receipt})});b=begin(u,i);assert.equal((await u.release({id:b.id,trusted:true,evaluate:()=>({values:{p:[4,0,0]}})})).status,'uncertain');assert.equal(u.cancel(b.id),false);assert.equal((await u.reconcile()).status,'committed');assert.equal(u.state,null); +}); +test('failed final evaluation never commits an earlier draft',async()=>{let i=identity(),count=0;const s=createSpatialSession({controls:schema,readIdentity:()=>i,publish:()=>count++});const b=begin(s,i);s.update({id:b.id,values:{p:[1,0,0]}});await assert.rejects(s.release({id:b.id,trusted:true,evaluate:()=>{throw Error('bad final point')}}));assert.equal(count,0);assert.equal(s.state,null);}); + +test('missing or incomplete final values cannot publish an earlier draft',async()=>{ + for(const result of [null,undefined,{}, {values:{} }]){let count=0;const i=identity(),s=createSpatialSession({controls:schema,readIdentity:()=>i,publish:()=>count++}),b=begin(s,i);s.update({id:b.id,values:{p:[1,0,0]}});await assert.rejects(s.release({id:b.id,trusted:true,evaluate:()=>result}));assert.equal(count,0);assert.equal(s.state,null);} +}); +test('translation boundary roundoff passes strict control validation and preserves world axis',async()=>{ + const {validateControlValue}=await import('../packages/core/controls.mjs'); + const parent=rigidParent(new Matrix4().compose(new Vector3(10,3,0),new Quaternion().setFromAxisAngle(new Vector3(0,0,1),15/37),new Vector3(2,2,2))),startLocal=new Vector3(.3,.2,.1); + const end=constrainTranslation({parent,startLocal,basisWorld:[new Vector3(1,0,0)],desired:[2],min:0,max:1});validateControlValue({key:'p',type:'vec3',min:0,max:1},end.toArray()); + const delta=end.clone().applyMatrix4(parent.matrix).sub(startLocal.clone().applyMatrix4(parent.matrix));assert.ok(Math.abs(delta.y)<1e-12&&Math.abs(delta.z)<1e-12); + const reverse=constrainTranslation({parent,startLocal,basisWorld:[new Vector3(1,0,0)],desired:[.1],min:0,max:1});assert.ok(reverse.y>0); +}); +test('picking follows a deformed skinned mesh after its initial raycast',async()=>{ + const T=await import('three'),{ThreeSpatialEditor}=await import('../packages/three/spatial-editor.mjs'),{normalizeObjectHierarchy}=await import('../packages/core/objects.mjs'); + const geometry=new T.BoxGeometry(1,1,1),n=geometry.attributes.position.count,indices=new Uint16Array(n*4),weights=new Float32Array(n*4);for(let i=0;i{ + const {validateGLB}=await import('../examples/scene-editor/build.mjs'); + const glb=data=>{const text=Buffer.from(JSON.stringify(data)),padded=Buffer.alloc(Math.ceil(text.length/4)*4,32);text.copy(padded);const b=Buffer.alloc(20+padded.length);b.writeUInt32LE(0x46546c67,0);b.writeUInt32LE(2,4);b.writeUInt32LE(b.length,8);b.writeUInt32LE(padded.length,12);b.writeUInt32LE(0x4e4f534a,16);padded.copy(b,20);return b;}; + assert.doesNotThrow(()=>validateGLB(glb({asset:{version:'2.0'},buffers:[{byteLength:4}],images:[{uri:'data:image/png;base64,AA=='}]}))); + assert.throws(()=>validateGLB(glb({images:[{uri:'https://example.com/texture.png'}]})),/external/);assert.throws(()=>validateGLB(glb({buffers:[{uri:'model.bin'}]})),/external/); +}); + +test('stale release cancels the draft and permits the next gesture',async()=>{ + let i=identity(),published=0,evaluated=0;const s=createSpatialSession({controls:schema,readIdentity:()=>i,publish:()=>published++}); + const b=begin(s,i);s.update({id:b.id,values:{p:[1,0,0]}});i={...i,viewRevision:1}; + await assert.rejects(s.release({id:b.id,trusted:true,evaluate:()=>{evaluated++;return {values:{p:[2,0,0]}};}}),/Stale/); + assert.equal(s.state,null);assert.equal(published,0);assert.equal(evaluated,0); + const next=begin(s,i);await s.release({id:next.id,trusted:true,evaluate:()=>({values:{p:[3,0,0]}})});assert.equal(published,1); +}); +test('duplicate release does not cancel the original final evaluation',async()=>{ + const i=identity();let resolve,published=0;const s=createSpatialSession({controls:schema,readIdentity:()=>i,publish:()=>published++}),b=begin(s,i); + const releasing=s.release({id:b.id,trusted:true,evaluate:()=>new Promise(r=>resolve=r)}); + await assert.rejects(s.release({id:b.id,trusted:true,evaluate:()=>({values:{p:[2,0,0]}})}),/already released/); + assert.equal(s.state.phase,'finalizing');resolve({values:{p:[1,0,0]}});assert.equal((await releasing).status,'committed');assert.equal(published,1); +}); +test('screen-plane drag follows a parented camera in both manipulation spaces',async()=>{ + const T=await import('three'),{ThreeSpatialEditor}=await import('../packages/three/spatial-editor.mjs'); + for(const space of ['world','local']){ + const object=new T.Object3D(),scene=new T.Scene(),rig=new T.Group(),camera=new T.PerspectiveCamera(45,1280/720,.1,100);scene.add(object,rig);rig.rotation.set(.2,Math.PI/4,.1);rig.add(camera);camera.position.z=5;scene.updateMatrixWorld(true); + const e=new ThreeSpatialEditor({controls:schema,props:{p:[0,0,0]},objects:{a:{object}},camera,width:1280,height:720}); + try{e.setSelection('a');e.setMode('translate',space);e.begin({operation:'translate',handle:'screen',space},{x:640,y:360});const delta=new T.Vector3(...e.update({x:740,y:360}).values.p); + const worldQ=camera.getWorldQuaternion(new T.Quaternion()),right=new T.Vector3(1,0,0).applyQuaternion(worldQ),forward=camera.getWorldDirection(new T.Vector3()); + assert.ok(delta.length()>.1);assert.ok(Math.abs(delta.dot(forward))<1e-10);assert.ok(delta.clone().normalize().distanceTo(right)<1e-10); + }finally{e.end();e.dispose();} + } +}); +test('small authored scale intervals remain valid through drag, snap and reversal',async()=>{ + const T=await import('three'),{ThreeSpatialEditor}=await import('../packages/three/spatial-editor.mjs'),{validateControlValue}=await import('../packages/core/controls.mjs'); + const field={key:'s',anchor:'a',path:'/s',type:'number',role:'scale',min:.0001,max:.0005,animatable:true},object=new T.Object3D(),scene=new T.Scene(),camera=new T.PerspectiveCamera(45,1280/720,.1,100);scene.add(object);object.scale.setScalar(.0002);camera.position.z=5;camera.updateMatrixWorld(true); + const e=new ThreeSpatialEditor({controls:{version:1,fields:[field]},props:{s:.0002},objects:{a:{object}},camera,width:1280,height:720}); + try{e.setSelection('a');e.setMode('scale','local');e.begin({operation:'scale',handle:'uniform',space:'local'},{x:650,y:360}); + for(const [x,expected] of [[652,.00024],[630,.0001],[652,.00024],[680,.0005]]){const value=e.update({x,y:360}).values.s;validateControlValue(field,value);assert.ok(Math.abs(value-expected)<1e-12);} + }finally{e.end();e.dispose();} + assert.equal(positiveScale(.0002,{min:.0001,max:.0005,snap:.1}),.0001); + assert.equal(positiveScale(-1,{max:.0005}),.0005);assert.equal(positiveScale(-1),.001); + assert.throws(()=>positiveScale(1,{min:.2,max:.1}),/bounds/);assert.throws(()=>positiveScale(1,{max:0}),/bounds/); +}); diff --git a/tests/surface-snap.test.mjs b/tests/surface-snap.test.mjs new file mode 100644 index 0000000..9ffad54 --- /dev/null +++ b/tests/surface-snap.test.mjs @@ -0,0 +1,54 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import * as T from 'three'; +import {surfaceBounds,surfaceDrop} from '../packages/three/surface-snap.mjs'; +import {ThreeSpatialEditor} from '../packages/three/spatial-editor.mjs'; +const v=(x,y,z)=>new T.Vector3(x,y,z); +const floor=()=>{const m=new T.Mesh(new T.BoxGeometry(20,.2,20),new T.MeshBasicMaterial());m.position.y=-.1;return m;}; +const bounds=new T.Box3(v(-.5,2,-.5),v(.5,4,.5)); +const near=(a,b)=>assert.ok(Math.abs(a-b)<1e-8,`${a} != ${b}`); +test('surface drop finds floor and highest support under the footprint, not overhead',()=>{ + const ground=floor(),table=new T.Mesh(new T.BoxGeometry(2,.2,2),ground.material);table.position.y=1; + near(surfaceDrop({bounds,surfaces:[ground]}).offsetY,-2); + near(surfaceDrop({bounds,surfaces:[ground,table]}).height,1.1); + table.position.y=6;near(surfaceDrop({bounds,surfaces:[ground,table]}).height,0); + assert.equal(surfaceDrop({bounds,delta:v(30,0,0),surfaces:[ground]}),null); + table.position.set(.9,1,0);near(surfaceDrop({bounds,surfaces:[ground,table]}).height,1.1); +}); +test('surface drop repairs small penetration, ignores hidden/self meshes and steep faces',()=>{ + const ground=floor(),root=new T.Group();root.add(ground);root.visible=false;assert.equal(surfaceDrop({bounds,surfaces:[ground]}),null);root.visible=true; + assert.equal(surfaceDrop({bounds,surfaces:[root],exclude:root}),null); + near(surfaceDrop({bounds,delta:v(0,-2.1,0),surfaces:[ground]}).offsetY,.1); + near(surfaceDrop({bounds,delta:v(0,-5,0),surfaces:[ground]}).offsetY,3); + const slope=new T.Mesh(new T.PlaneGeometry(10,10),new T.MeshBasicMaterial({side:T.DoubleSide}));slope.rotation.x=-Math.PI/6; + assert.equal(surfaceDrop({bounds,surfaces:[slope]}),null); +}); +test('support height respects transformed surfaces and visible posed bounds',()=>{ + const root=new T.Group(),ground=floor();root.add(ground);root.position.y=.7;root.scale.setScalar(2);near(surfaceDrop({bounds,surfaces:[ground]}).height,.7); + const actor=new T.Group(),body=new T.Mesh(new T.BoxGeometry(1,2,1),ground.material),hidden=new T.Mesh(new T.BoxGeometry(1,10,1),ground.material);body.position.y=1;hidden.visible=false;actor.add(body,hidden);actor.position.y=3; + near(surfaceBounds(actor).min.y,3);near(surfaceBounds(actor).max.y,5); +}); +function editorFixture({enabled=true,min,max,parented=false,groundY=0}={}){ + const scene=new T.Scene(),parent=new T.Group(),object=new T.Group(),body=new T.Mesh(new T.BoxGeometry(1,2,1),new T.MeshBasicMaterial());body.position.y=1;object.add(body);parent.add(object);scene.add(parent);object.position.y=2; + if(parented){parent.rotation.z=.35;parent.scale.setScalar(2);} + const camera=new T.PerspectiveCamera(45,1280/720,.1,100);camera.position.set(0,3,8);camera.lookAt(0,2,0);camera.updateMatrixWorld(true);scene.updateMatrixWorld(true); + const controls={version:1,fields:[{key:'p',anchor:'a',path:'/p',type:'vec3',animatable:true,role:'position',unit:'world-unit',coordinateSpace:'local',min,max}]}; + const ground=floor();ground.position.y+=groundY; + const e=new ThreeSpatialEditor({controls,props:{p:[0,2,0]},objects:{a:{object,surfaceSnap:enabled}},camera,width:1280,height:720,surfaceTargets:[ground]});e.setSelection('a');e.setMode('translate','world'); + const c=e.proxy.position.clone().project(camera),start={x:(c.x+1)*640,y:(1-c.y)*360};e.begin({operation:'translate',handle:'screen',space:'world'},start,{surfaceSnap:true});return {e,start,object,parent}; +} +test('surface snapping applies only on final movement; one click, drafts and opt-out stay untouched',()=>{ + for(const enabled of [true,false]){const {e,start}=editorFixture({enabled});try{ + near(e.update(start,{final:true}).values.p[1],2); + const point={x:start.x+40,y:start.y};near(e.update(point).values.p[1],2); + near(e.update(point,{final:true}).values.p[1],enabled?0:2); + }finally{e.end();e.dispose();}} +}); +test('final grounding converts world Y through rotated scaled parents and respects bounds',()=>{ + const {e,start,parent,object}=editorFixture({parented:true});try{ + const before=surfaceBounds(object),next=e.update({x:start.x+40,y:start.y},{final:true}).values.p; + const delta=v(...next).sub(v(0,2,0)).multiplyScalar(2).applyQuaternion(parent.quaternion); + near(before.min.y+delta.y,0); + }finally{e.end();e.dispose();} + const bounded=editorFixture({min:-1,max:5,groundY:-3});try{near(bounded.e.update({x:bounded.start.x+40,y:bounded.start.y},{final:true}).values.p[1],2);}finally{bounded.e.end();bounded.e.dispose();} +});