Updates search functionality to include source field in results, adds Editor documentation search handler, and removes unused helper methods from TSX parser.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Root cause analysis revealed the previous fix used wrong casing and prefix.
Investigation findings:
- Queried database and found actual IDs use lowercase "documentation_" prefix
- Example: "documentation_features_featuresDeepDive_audio_v2_playingSoundsMusic"
- Previous fix incorrectly used "Documentation_content_" (wrong case + extra "content")
How indexing works:
- Source name: "documentation" (lowercase)
- Source path: ends at .../Documentation/content
- Indexer strips path up to and including /content/
- Prepends source name with underscore: "documentation_"
Correct implementation:
- Strip /content/ from input path (handles both full and relative paths)
- Replace slashes with underscores
- Prepend "documentation_" (lowercase, no "content")
Verified with test:
- Input: "features/featuresDeepDive/audio/v2/playingSoundsMusic"
- Output: "documentation_features_featuresDeepDive_audio_v2_playingSoundsMusic"
- Successfully retrieves document from database
All 152 tests passing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixed pathToDocId to properly match document IDs generated during indexing.
Problem:
- During indexing, doc IDs are generated as: Documentation_content_<path>
- During retrieval, pathToDocId was not prepending the prefix
- This caused get_babylon_doc to fail with "Document not found" errors
Solution:
- Updated pathToDocId to prepend "Documentation_content_" prefix
- Now handles paths with or without "content/" correctly
- Matches the ID format used during indexing
Example:
- User provides: "features/featuresDeepDive/audio/v2/migrateFromV1"
- Now correctly converts to: "Documentation_content_features_featuresDeepDive_audio_v2_migrateFromV1"
All 152 tests passing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Features:
- Implemented SourceCodeIndexer class for indexing TypeScript/JavaScript source files
- Chunks large files into 200-line segments with 20-line overlap
- Extracts imports, exports, and metadata
- Generates semantic embeddings using Xenova/all-MiniLM-L6-v2
- Creates GitHub URLs with line numbers for easy navigation
- Enhanced LanceDBSearch with source code search capabilities
- Added searchSourceCode() method for semantic source code search
- Added getSourceFile() method for retrieving specific files or line ranges
- Supports package filtering and configurable table names
- Fixed score calculation to ensure values between 0-100%
- Added two new MCP tools
- search_babylon_source: Search Babylon.js source code with semantic search
- get_babylon_source: Retrieve full source files or specific line ranges
- Both tools include comprehensive error handling and JSON responses
- Created indexing and testing scripts
- scripts/index-source.ts: Production script for indexing all packages
- scripts/test-source-indexing.ts: Test script for core package only
- scripts/test-source-search.ts: Test script for search functionality
- Updated package.json with comprehensive indexing commands
- npm run index:docs - Index documentation only
- npm run index:api - Index API documentation only
- npm run index:source - Index source code only
- npm run index:all - Master script to index everything
- Created comprehensive README.md
- Complete setup and installation instructions
- Claude Desktop integration guide with configuration examples
- Documentation of all 5 MCP tools with parameters and examples
- Project structure, development commands, and troubleshooting guide
- Architecture overview and disk space requirements
Testing:
- All 118 tests passing
- TypeScript compilation successful
- Source code search verified with real queries
- Successfully indexed 1,561 files into 5,650 searchable chunks
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
## New Features
- Implemented TSDoc extraction using TypeDoc API
- Added API documentation indexing with LanceDB vector search
- Created search_babylon_api MCP tool for querying API docs
- Added 6 indexing and testing scripts
## API Indexing System
- TSDocExtractor: Parses TypeScript source files and extracts documentation
- ApiIndexer: Converts API docs to embeddings and stores in LanceDB
- Support for all Babylon.js packages (core, gui, materials, loaders, etc.)
- Successfully indexed 44,253 API entries from core package
## Bug Fixes
- Fixed TypeScript strict mode errors with exactOptionalPropertyTypes
- Fixed optional property handling in tsConfigPath and returns fields
- Resolved EventEmitter MaxListeners warning in test suite
- Updated all failing handler tests for real implementation
## Test Coverage Improvements
- Added 27 new tests (92 → 119 tests passing)
- Lines: 93.88% (was 82.53%, target 80%) ✓
- Functions: 100% (was 91.17%, target 80%) ✓
- Statements: 93.3% (was 81.58%, target 80%) ✓
- Branches: 69.72% (was 51.37%, target 75%)
## New Test Files
- src/search/lancedb-search.test.ts (15 tests)
- Enhanced handlers.test.ts with API search tests
- Enhanced document-parser.test.ts with edge case tests
## Scripts Added
- scripts/index-api.ts: Index all Babylon.js API documentation
- scripts/test-api-indexing.ts: Test API indexing for core package
- scripts/test-api-search.ts: Test API search functionality
- scripts/get-api-details.ts: Display detailed API documentation
- scripts/search-handmenu-api.ts: Search for HandMenu API examples
## Technical Details
- TypeDoc integration for TSDoc extraction
- Vector embeddings using Xenova/all-MiniLM-L6-v2 model
- Semantic search across 11 Babylon.js packages
- GitHub source links with line numbers in search results
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add LanceDBSearch class for vector-based documentation search
- Implement search() method with category filtering and relevance scoring
- Add getDocumentByPath() with URL lookup and local file fetching
- Fix getDocument() to use .query() instead of .search() for non-vector queries
- Update handlers.ts to integrate LanceDBSearch with MCP tools
- Parse stringified array fields (breadcrumbs, headings, keywords, playgroundIds) in get_babylon_doc
- Fetch fresh content from local repositories (Documentation, Babylon.js, havok)
- Add DocumentParser, LanceDBIndexer and related types for document processing
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
