Position UI components relative to camera on XR entry
When entering immersive mode, toolbox and input text view now position themselves relative to the user's initial camera position: - Toolbox: 0.5m ahead, 0.5m below, 0.2m to the left - Input text view: 0.5m ahead, 0.5m below (centered) Uses camera world Y position to ensure vertical offset is consistent regardless of head pitch/tilt when entering XR. Also added CLAUDE.md documentation for the codebase. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
58668443c4
commit
cf0f359921
110
CLAUDE.md
Normal file
110
CLAUDE.md
Normal file
@ -0,0 +1,110 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
This is "immersive" - a WebXR/VR diagramming application built with BabylonJS and React. It allows users to create and interact with 3D diagrams in both standard web browsers and VR environments, with real-time collaboration via PouchDB sync.
|
||||||
|
|
||||||
|
## Build and Development Commands
|
||||||
|
|
||||||
|
### Development
|
||||||
|
- `npm run dev` - Start Vite dev server on port 3001 (DO NOT USE per user instructions)
|
||||||
|
- `npm run build` - Build production bundle (includes version bump)
|
||||||
|
- `npm run preview` - Preview production build on port 3001
|
||||||
|
- `npm test` - Run tests with Vitest
|
||||||
|
- `npm run socket` - Start WebSocket server for collaboration (port 8080)
|
||||||
|
- `npm run serverBuild` - Compile TypeScript server code
|
||||||
|
- `npm run havok` - Copy Havok physics WASM files to Vite deps
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
- Run all tests: `npm test`
|
||||||
|
- No single test command is configured; tests use Vitest
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Core Technologies
|
||||||
|
- **BabylonJS 8.x**: 3D engine with WebXR support and Havok physics
|
||||||
|
- **React + Mantine**: UI framework for 2D interface and settings
|
||||||
|
- **PouchDB**: Client-side database with CouchDB sync for collaboration
|
||||||
|
- **Auth0**: Authentication provider
|
||||||
|
- **Vite**: Build tool and dev server
|
||||||
|
|
||||||
|
### Key Architecture Patterns
|
||||||
|
|
||||||
|
#### Singleton Scene Management
|
||||||
|
The application uses a singleton pattern for the BabylonJS Scene via `DefaultScene` (src/defaultScene.ts). Always access the scene through `DefaultScene.Scene` rather than creating new instances.
|
||||||
|
|
||||||
|
#### Observable-Based Event System
|
||||||
|
The application heavily uses BabylonJS Observables for event handling:
|
||||||
|
- **DiagramManager.onDiagramEventObservable**: Central hub for diagram entity changes
|
||||||
|
- **DiagramManager.onUserEventObservable**: User position/state updates for multiplayer
|
||||||
|
- **AppConfig.onConfigChangedObservable**: Application settings changes
|
||||||
|
- **controllerObservable**: VR controller input events
|
||||||
|
|
||||||
|
Event observers use a mask system (`DiagramEventObserverMask`) to distinguish:
|
||||||
|
- `FROM_DB`: Events coming from database sync (shouldn't trigger database writes)
|
||||||
|
- `TO_DB`: Events that should be persisted to database
|
||||||
|
|
||||||
|
#### Diagram Entity System
|
||||||
|
All 3D objects in the scene are represented by `DiagramEntity` types (src/diagram/types/diagramEntity.ts):
|
||||||
|
- Entities have a template reference (e.g., `#image-template`)
|
||||||
|
- Managed by `DiagramManager` which maintains a Map of `DiagramObject` instances
|
||||||
|
- Changes propagate through the Observable system to database and other clients
|
||||||
|
|
||||||
|
#### VR Controller Architecture
|
||||||
|
Controllers inherit from `AbstractController` with specialized implementations:
|
||||||
|
- `LeftController`: Menu interactions, navigation
|
||||||
|
- `RightController`: Object manipulation, selection
|
||||||
|
- Controllers communicate via `controllerObservable` with `ControllerEvent` messages
|
||||||
|
- `Rigplatform` manages the player rig and handles locomotion
|
||||||
|
|
||||||
|
#### Database & Sync
|
||||||
|
- `PouchdbPersistenceManager` (src/integration/database/pouchdbPersistenceManager.ts) handles all persistence
|
||||||
|
- Supports optional encryption via `Encryption` class
|
||||||
|
- Syncs to remote CouchDB via proxy (configured in vite.config.ts)
|
||||||
|
- URL pattern `/db/public/:db` or `/db/private/:db` determines database name
|
||||||
|
- Uses `presence.ts` for broadcasting user positions over WebSocket
|
||||||
|
|
||||||
|
### Project Structure
|
||||||
|
- `src/vrcore/`: Engine initialization and core VR setup
|
||||||
|
- `src/controllers/`: VR controller implementations and input handling
|
||||||
|
- `src/diagram/`: 3D diagram entities, management, and scene interaction
|
||||||
|
- `src/integration/`: Database sync, encryption, and presence system
|
||||||
|
- `src/menus/`: In-VR 3D menus (not React components)
|
||||||
|
- `src/objects/`: Reusable 3D objects (buttons, handles, avatars)
|
||||||
|
- `src/react/`: React UI components for 2D interface
|
||||||
|
- `src/util/`: Shared utilities and configuration
|
||||||
|
- `server/`: WebSocket server for real-time presence
|
||||||
|
|
||||||
|
### Configuration System
|
||||||
|
Two configuration systems exist (being migrated):
|
||||||
|
1. **AppConfig class** (src/util/appConfig.ts): Observable-based config with typed properties
|
||||||
|
2. **ConfigType** (bottom of appConfig.ts): Legacy localStorage-based config
|
||||||
|
|
||||||
|
Settings include snapping values, physics toggles, fly mode, and turn snap angles.
|
||||||
|
|
||||||
|
## Important Development Notes
|
||||||
|
|
||||||
|
### Proxy Configuration
|
||||||
|
The dev and preview servers proxy certain routes to production:
|
||||||
|
- `/sync/*` - Database sync endpoint
|
||||||
|
- `/create-db` - Database creation
|
||||||
|
- `/api/images` - Image uploads
|
||||||
|
|
||||||
|
### Physics System
|
||||||
|
- Uses Havok physics engine (requires WASM file via `npm run havok`)
|
||||||
|
- Physics can be enabled/disabled via AppConfig
|
||||||
|
- `customPhysics.ts` provides helper functions
|
||||||
|
|
||||||
|
### WebGPU Support
|
||||||
|
The engine initializer supports both WebGL and WebGPU backends via the `useWebGpu` parameter.
|
||||||
|
|
||||||
|
### Encryption
|
||||||
|
Databases can be optionally encrypted. The `Encryption` class handles AES encryption with password-derived keys. Salt is stored in metadata document.
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
- `VITE_USER_ENDPOINT`: User authentication endpoint
|
||||||
|
- `VITE_SYNCDB_ENDPOINT`: Remote database sync endpoint
|
||||||
|
|
||||||
|
Check `.env.local` for local configuration.
|
||||||
@ -1,10 +1,11 @@
|
|||||||
import {AbstractMesh, WebXRDefaultExperience, WebXRMotionControllerManager, WebXRState} from "@babylonjs/core";
|
import {AbstractMesh, Vector3, WebXRDefaultExperience, WebXRMotionControllerManager, WebXRState} from "@babylonjs/core";
|
||||||
import log from "loglevel";
|
import log from "loglevel";
|
||||||
import {WebController} from "../../controllers/webController";
|
import {WebController} from "../../controllers/webController";
|
||||||
import {Rigplatform} from "../../controllers/rigplatform";
|
import {Rigplatform} from "../../controllers/rigplatform";
|
||||||
import {DiagramManager} from "../../diagram/diagramManager";
|
import {DiagramManager} from "../../diagram/diagramManager";
|
||||||
import {Spinner} from "../../objects/spinner";
|
import {Spinner} from "../../objects/spinner";
|
||||||
import {getAppConfig} from "../appConfig";
|
import {getAppConfig} from "../appConfig";
|
||||||
|
import {Scene} from "@babylonjs/core";
|
||||||
|
|
||||||
|
|
||||||
export async function groundMeshObserver(ground: AbstractMesh,
|
export async function groundMeshObserver(ground: AbstractMesh,
|
||||||
@ -60,6 +61,8 @@ export async function groundMeshObserver(ground: AbstractMesh,
|
|||||||
logger.debug(event.detail);
|
logger.debug(event.detail);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
// Position components relative to camera on XR entry
|
||||||
|
positionComponentsRelativeToCamera(ground.getScene(), diagramManager);
|
||||||
break;
|
break;
|
||||||
case WebXRState.EXITING_XR:
|
case WebXRState.EXITING_XR:
|
||||||
setTimeout(() => {
|
setTimeout(() => {
|
||||||
@ -76,4 +79,68 @@ export async function groundMeshObserver(ground: AbstractMesh,
|
|||||||
rig.turnSnap = parseFloat(config.snapTurnSnap);
|
rig.turnSnap = parseFloat(config.snapTurnSnap);
|
||||||
const webController = new WebController(ground.getScene(), rig, diagramManager);
|
const webController = new WebController(ground.getScene(), rig, diagramManager);
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
function positionComponentsRelativeToCamera(scene: Scene, diagramManager: DiagramManager) {
|
||||||
|
const logger = log.getLogger('positionComponentsRelativeToCamera');
|
||||||
|
const platform = scene.getMeshByName('platform');
|
||||||
|
if (!platform) {
|
||||||
|
logger.warn('Platform not found, cannot position components');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const camera = scene.activeCamera;
|
||||||
|
if (!camera) {
|
||||||
|
logger.warn('Active camera not found, cannot position components');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get camera world position
|
||||||
|
const cameraWorldPos = camera.globalPosition;
|
||||||
|
|
||||||
|
// Create a horizontal forward direction from camera's world rotation
|
||||||
|
const cameraRotationY = camera.absoluteRotation.toEulerAngles().y;
|
||||||
|
const horizontalForward = new Vector3(
|
||||||
|
Math.sin(cameraRotationY),
|
||||||
|
0,
|
||||||
|
Math.cos(cameraRotationY)
|
||||||
|
);
|
||||||
|
|
||||||
|
// Create a left direction (perpendicular to forward)
|
||||||
|
const horizontalLeft = new Vector3(
|
||||||
|
-Math.cos(cameraRotationY),
|
||||||
|
0,
|
||||||
|
Math.sin(cameraRotationY)
|
||||||
|
);
|
||||||
|
|
||||||
|
// Calculate base target world position: 0.5m ahead horizontally and 0.5m below camera Y
|
||||||
|
const baseTargetWorldPos = new Vector3(
|
||||||
|
cameraWorldPos.x + (horizontalForward.x * 0.5),
|
||||||
|
cameraWorldPos.y - 0.5,
|
||||||
|
cameraWorldPos.z + (horizontalForward.z * 0.5)
|
||||||
|
);
|
||||||
|
|
||||||
|
logger.info('Camera world Y:', cameraWorldPos.y);
|
||||||
|
logger.info('Base target world position:', baseTargetWorldPos);
|
||||||
|
|
||||||
|
// Position toolbox: 0.2m to the left of base position
|
||||||
|
const toolbox = diagramManager.diagramMenuManager.toolbox;
|
||||||
|
if (toolbox && toolbox.handleMesh) {
|
||||||
|
const toolboxWorldPos = new Vector3(
|
||||||
|
baseTargetWorldPos.x + (horizontalLeft.x * 0.2),
|
||||||
|
baseTargetWorldPos.y,
|
||||||
|
baseTargetWorldPos.z + (horizontalLeft.z * 0.2)
|
||||||
|
);
|
||||||
|
const toolboxLocalPos = Vector3.TransformCoordinates(toolboxWorldPos, platform.getWorldMatrix().invert());
|
||||||
|
toolbox.handleMesh.position = toolboxLocalPos;
|
||||||
|
logger.info('Toolbox positioned at:', toolboxLocalPos);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Position input text view: at base position
|
||||||
|
const inputTextView = diagramManager.diagramMenuManager['_inputTextView'];
|
||||||
|
if (inputTextView && inputTextView.handleMesh) {
|
||||||
|
const inputLocalPos = Vector3.TransformCoordinates(baseTargetWorldPos, platform.getWorldMatrix().invert());
|
||||||
|
inputTextView.handleMesh.position = inputLocalPos;
|
||||||
|
logger.info('InputTextView positioned at:', inputLocalPos);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
Loading…
Reference in New Issue
Block a user