about summary refs log tree commit diff stats
path: root/js/scripting-lang/baba-yaga-c/ROADMAP.md
diff options
context:
space:
mode:
authorelioat <elioat@tilde.institute>2025-08-02 10:41:26 -0400
committerelioat <elioat@tilde.institute>2025-08-02 10:41:26 -0400
commitaf253e7ce4e154f404eeea12b748dd5566004f6d (patch)
tree09834890513ca45f590a9ad99f5d43669ee8d169 /js/scripting-lang/baba-yaga-c/ROADMAP.md
parent5a15800ed96ef97954698cb41f3be9d513e8c77a (diff)
downloadtour-master.tar.gz
Diffstat (limited to 'js/scripting-lang/baba-yaga-c/ROADMAP.md')
-rw-r--r--js/scripting-lang/baba-yaga-c/ROADMAP.md261
1 files changed, 78 insertions, 183 deletions
diff --git a/js/scripting-lang/baba-yaga-c/ROADMAP.md b/js/scripting-lang/baba-yaga-c/ROADMAP.md
index 88f43fa..e827ff3 100644
--- a/js/scripting-lang/baba-yaga-c/ROADMAP.md
+++ b/js/scripting-lang/baba-yaga-c/ROADMAP.md
@@ -1,107 +1,64 @@
-# Baba Yaga C Implementation - Focused Roadmap
+# Baba Yaga C Implementation Roadmap
 
 ## Current Status
-- ✅ **Core Language**: Complete and stable (24/27 tests passing)
+- ✅ **Core Language**: Complete and stable (25/27 tests passing)
 - ✅ **Table Pattern Matching**: Fixed and working
 - ✅ **When Expressions**: Fixed and working
 - ✅ **Computed Table Keys**: Fixed and working (Task 1.1 complete)
 - ✅ **Multi-value Pattern Expressions**: Fixed and working (Task 1.2 complete)
 - ✅ **Pattern Matching Memory**: Fixed and working (Task 1.3 complete)
-- ❌ **1 Remaining Issue**: Partial application (Task 2.3)
+- ✅ **Partial Application Support**: Fixed and working (Task 2.3 complete)
+- ❌ **2 Remaining Issues**: Test 22 parser issue, Integration Test 02 file reading issue
 
-## Implementation Plan
-
-### **Phase 1: Parser Extensions (High Impact)**
+## Quick Reference
+- **Test Command**: `./bin/baba-yaga tests/22_parser_limitations.txt`
+- **Key Files**: `src/parser.c` (parser_parse_when_pattern), `tests/22_parser_limitations.txt`
+- **Current Error**: `Parse error: Expected 'is' after test expression`
+- **Working Test**: `echo "test_multi_expr : x y -> when (x % 2) (y % 2) is 0 0 then \"both even\";" | ./bin/baba-yaga`
 
-#### **Task 1.1: Computed Table Keys** (Test 15) ✅ **COMPLETE**
-**Issue**: `{(1 + 1): "two"}` not supported
-**Solution**: Extended table key parsing with expression support
-**Implementation**: Added `TOKEN_LPAREN` detection and expression parsing logic
-**Result**: Test 15 now passes, 25/27 tests passing
-
-#### **Task 1.2: Multi-value Pattern Expressions** (Test 22) ✅ **COMPLETE**
-**Issue**: `when (x % 2) (y % 2) is` not supported
-**Current**: Multi-value pattern expressions working correctly in parser
-**Status**: Core functionality implemented - test file has minor syntax issue
+## Implementation Plan
 
-#### **Task 1.3: Pattern Matching Memory** (Integration Test 02) ✅ **COMPLETE**
-**Issue**: Segmentation fault in complex pattern matching
-**Current**: Multi-parameter pattern matching working correctly
-**Status**: Core functionality implemented - integration test has separate file reading issue
+### **Phase 1: Core Language Features** ✅ **COMPLETE**
+All core language features are now working correctly.
 
+### **Phase 2: Advanced Features** ✅ **COMPLETE**
+All advanced features including partial application are now working.
 
+### **Phase 3: Final Polish** 🔄 **IN PROGRESS**
 
-### **Phase 2: Runtime Fixes (Medium Impact)**
+#### **Task 3.1: Test 22 Parser Issue** (Test 22) 🔍 **INVESTIGATED**
+**Issue**: `Parse error: Expected 'is' after test expression`
+**Current**: Core multi-value pattern functionality works correctly
+**Status**: Identified specific parser edge case - needs investigation
 
-#### **Task 2.1: Table Namespace Debugging** (Test 17) ✅ **COMPLETE**
-**Issue**: `Error: Execution failed` in table operations
-**Current**: Basic `map`, `filter`, `reduce` functions now work correctly with operator lookup
-**Status**: Core functionality implemented - operator scope access fixed
+**Investigation Findings**:
+- ✅ **Individual functions work**: Multi-value patterns parse and execute correctly when tested individually
+- ✅ **Isolated syntax works**: Same syntax works perfectly when tested via `echo`
+- ❌ **File-specific issue**: The error only occurs when the complete test file is processed
+- 🔍 **Parser edge case**: The issue appears to be in how the parser handles multiple patterns in sequence within a file context
+- 📍 **Error location**: Parser fails to recognize the `is` keyword in multi-value pattern context when processing the full file
 
 **Root Cause Analysis**:
-- ✅ `t.*` namespace functions (t.map, t.filter, t.reduce, t.set, t.delete, t.merge, t.length, t.has, t.get) are working correctly
-- ✅ Basic `map`, `filter`, `reduce` functions now work correctly with operator lookup
-- ✅ `each` function works but has arity issues
-- ❌ Test still fails due to partial application and arity handling issues
-
-**Solution Implemented**:
-1. **Fixed Scope Access**: Updated stdlib function signatures to accept `Scope*` parameter
-2. **Updated Function Calls**: Modified `baba_yaga_function_call` to pass current scope to user functions
-3. **Updated Function Registration**: Changed function pointer types to support scope parameter
-4. **Verified Core Functionality**: Basic higher-order functions now work with operators
-
-**Current Status**:
-- ✅ **Core Bug Fixed**: Operator lookup in higher-order functions works correctly
-- ✅ **Basic Functions Working**: `map`, `filter`, `reduce` with operators now functional
-- ❌ **Test 17 Still Fails**: Due to partial application and arity issues (see Task 2.3)
-
-#### **Task 2.2: Pattern Matching Memory** (Integration Test 02) ✅ **COMPLETE**
-**Issue**: Segmentation fault in complex patterns
-**Current**: Multi-parameter pattern matching working correctly
-**Status**: Core functionality implemented - integration test has separate file reading issue
-
-#### **Task 2.3: Partial Application Support** (Test 17) 🔄 **IN PROGRESS**
-**Issue**: Test 17 fails with partial application and arity errors
-**Current**: Core higher-order functions work, but partial application not supported
-**Status**: Need to implement minimal partial application support (reverted due to memory issues)
+- The parser's `parser_parse_when_pattern` function may have an edge case when processing multiple patterns in sequence
+- The error suggests the parser is not correctly transitioning between pattern parsing states
+- This is likely a subtle parsing state management issue rather than a fundamental syntax problem
 
-**Root Cause Analysis**:
-- ✅ **Core Functions Working**: `map`, `filter`, `reduce` with operators now functional
-- ❌ **Partial Application**: Function calls with fewer arguments than required fail
-- ❌ **Arity Issues**: `each` function expects 3 args but receives 2 in some cases
-- ❌ **Variable Scope**: Some variables like `partial_result`, `scalar_2`, etc. undefined
-
-**Error Messages from Test 17**:
-- "each: expected 3 arguments, got 2"
-- "Undefined variable: partial_result", "scalar_2", "add_to_ten"
-- "Cannot call non-function value"
-- "equals: arguments must be of the same type"
-
-**Implementation Plan**:
-1. **Implement Partial Application**: When function called with fewer args than required, return new function
-2. **Fix Arity Validation**: Ensure `each` function handles variable argument counts correctly
-3. **Debug Variable Scope**: Identify why certain variables are undefined in test context
-4. **Test Partial Application**: Verify partial functions work with remaining arguments
-
-**Technical Details**:
-- **Location**: `src/function.c` - `baba_yaga_function_call` function
-- **Current Behavior**: Returns `VAL_NIL` when insufficient arguments
-- **Required Behavior**: Return new function with bound arguments
-- **Reference**: Use `stdlib_compose` as template for function composition
-
-**Implementation Steps**:
-1. **Implement Partial Application**: When function called with fewer args than required, return new function
-2. **Fix Arity Validation**: Ensure `each` function handles variable argument counts correctly
-3. **Debug Variable Scope**: Identify why certain variables are undefined in test context
-4. **Test Partial Application**: Verify partial functions work with remaining arguments
-
-### **Phase 3: Validation**
-- Re-run comprehensive test suite
-- Target: 27/27 tests passing
-- Verify no regressions
+#### **Task 3.2: Integration Test 02 File Reading** (Integration Test 02)
+**Issue**: Segmentation fault when reading file directly (works when piped)
+**Current**: Core pattern matching works, but file reading has issue
+**Status**: Need to fix file reading mechanism
 
 ## **Recent Achievements**
 
+### **Task 2.3: Partial Application Support** ✅ **COMPLETE**
+- **Issue**: Test 17 failed with partial application and arity errors
+- **Solution**: Implemented proper partial application in function call mechanism
+- **Implementation**: 
+  - Modified `baba_yaga_function_call` to handle partial application
+  - Created `stdlib_partial_apply` helper function
+  - Updated `each` function to support partial application
+- **Result**: Test 17 now passes, 25/27 tests passing
+
 ### **Task 1.2: Multi-value Pattern Expressions** ✅ **COMPLETE**
 - **Issue**: `when (x % 2) (y % 2) is` not supported
 - **Solution**: Enhanced parser to handle expressions in parentheses for multi-parameter patterns
@@ -115,113 +72,51 @@
 - **Result**: Complex nested pattern matching now works correctly
 
 ## **Next Priority**
-**Task 2.3: Partial Application Support** - Implement minimal partial application to fix Test 17
+**Task 3.1**: Fix Test 22 parser edge case to achieve 26/27 tests passing
+**Task 3.2**: Fix Integration Test 02 file reading issue to achieve 27/27 tests passing
 
 ## Technical Notes
 
-### **Parser Architecture**
-- Table parsing: `parser_parse_primary` → `TOKEN_LBRACE` case
-- Pattern parsing: `parser_parse_when_pattern` → multi-parameter detection
-- Both need expression support in parentheses
+### **Partial Application Implementation**
+- **Function Call Mechanism**: Modified `baba_yaga_function_call` to detect insufficient arguments
+- **Partial Function Creation**: Creates new function with bound arguments stored in scope
+- **Argument Combination**: `stdlib_partial_apply` combines bound and new arguments
+- **Scope Management**: Uses temporary scope variables to store partial application data
+
+### **Pattern Matching Enhancements**
+- **Multi-parameter Support**: Handles `when (expr1) (expr2) is` syntax
+- **Sequence Comparison**: Element-by-element comparison for multi-value patterns
+- **Wildcard Support**: `_` pattern matches any value in multi-parameter contexts
 
-### **Standard Library**
-- `t.*` functions: Already implemented in `stdlib.c` (lines ~815-1183) ✅ **WORKING**
-- Functions: `t.map`, `t.filter`, `t.reduce`, `t.set`, `t.delete`, `t.merge`, `t.length`, `t.has`, `t.get`
-- Basic functions: `map`, `filter`, `reduce` in `stdlib.c` (lines ~559-640) ❌ **PLACEHOLDERS**
-- Issue: Basic higher-order functions need full implementation
+### **Parser Investigation Results**
+- **Multi-value patterns work correctly** in isolation and individual function definitions
+- **File processing edge case** identified in `parser_parse_when_pattern` function
+- **State management issue** suspected when processing multiple patterns in sequence
+- **Error occurs specifically** when the complete test file is processed, not in isolated tests
 
 ### **Memory Management**
-- Pattern matching: Uses recursion for nested patterns
-- Potential: Stack overflow or memory corruption
-- Solution: Add bounds checking and memory debugging
+- **Reference Counting**: Proper cleanup of function references
+- **Scope Cleanup**: Automatic cleanup of temporary scope variables
+- **Error Handling**: Graceful handling of memory allocation failures
 
 ## Next Action
-**Continue with Task 2.3** (Partial Application Support) - implement minimal partial application support.
+**Continue with Task 3.1** (Test 22 Parser Issue) - investigate and fix the parser edge case in `parser_parse_when_pattern` function to achieve 26/27 tests passing.
 
 ## Implementation Guide
 
-### **Task 2.1: Basic Higher-Order Functions Implementation** ✅ **COMPLETE**
-
-#### **Function 1: `stdlib_map` Implementation**
-**Location**: `src/stdlib.c` lines ~559-580
-**Current**: Returns original table (placeholder)
-**Required**: Apply function to each value in table
-
-**Implementation Steps**:
-1. Get all keys from input table using `baba_yaga_table_get_keys`
-2. Create new result table using `baba_yaga_value_table`
-3. For each key:
-   - Get value using `baba_yaga_table_get_by_key`
-   - Call function with value using `baba_yaga_function_call`
-   - Add result to new table using `baba_yaga_table_set`
-4. Return new table
-
-**Reference**: Use `stdlib_t_map` (lines ~815-866) as template - it's already implemented correctly
-
-#### **Function 2: `stdlib_filter` Implementation**
-**Location**: `src/stdlib.c` lines ~584-610
-**Current**: Returns original table (placeholder)
-**Required**: Keep only values that satisfy predicate
-
-**Implementation Steps**:
-1. Get all keys from input table using `baba_yaga_table_get_keys`
-2. Create new result table using `baba_yaga_value_table`
-3. For each key:
-   - Get value using `baba_yaga_table_get_by_key`
-   - Call predicate function with value using `baba_yaga_function_call`
-   - If predicate returns truthy value, add original value to new table
-4. Return new table
-
-**Reference**: Use `stdlib_t_filter` (lines ~867-923) as template - it's already implemented correctly
-
-#### **Function 3: `stdlib_reduce` Implementation**
-**Location**: `src/stdlib.c` lines ~609-640
-**Current**: Returns initial value (placeholder)
-**Required**: Combine all values with function
-
-**Implementation Steps**:
-1. Start with initial value as accumulator
-2. Get all keys from input table using `baba_yaga_table_get_keys`
-3. For each key:
-   - Get value using `baba_yaga_table_get_by_key`
-   - Call function with accumulator and value using `baba_yaga_function_call`
-   - Update accumulator with result
-4. Return final accumulator value
-
-**Reference**: Use `stdlib_t_reduce` (lines ~924-976) as template - it's already implemented correctly
-
-#### **Testing Strategy**:
-1. **Unit Tests**: Create simple tests for each function individually
-2. **Integration Test**: Run Test 17 to verify all functions work together
-3. **Regression Test**: Verify `t.*` functions still work correctly
-4. **Target**: Test 17 should pass, moving from 25/27 to 26/27 tests passing
-
-### **Task 2.3: Partial Application Support Implementation** 🔄 **IN PROGRESS**
-
-#### **Problem Analysis**
-The current function call mechanism returns `VAL_NIL` when a function is called with fewer arguments than required. Test 17 expects partial application behavior where:
-- `add_to_ten : add 10;` should create a function that adds 10 to its argument
-- `each add_to_ten numbers` should work with the partially applied function
-
-#### **Implementation Steps**
-1. **Modify `baba_yaga_function_call`** in `src/function.c`:
-   - When `arg_count < func_value->required_params`, create a new function
-   - Bind the provided arguments to the new function
-   - Return the new function instead of `VAL_NIL`
-
-2. **Create Partial Function Structure**:
-   - Store original function and bound arguments
-   - When partial function is called, combine bound args with new args
-   - Call original function with complete argument set
-
-3. **Update Function Value Structure**:
-   - Add support for partial functions in `FunctionValue`
-   - Handle partial function cleanup in memory management
-
-#### **Reference Implementation**
-Use `stdlib_compose` as a template for function composition and partial application patterns.
-
-#### **Testing Strategy**
-1. **Unit Test**: Create simple partial application test
-2. **Integration Test**: Verify Test 17 passes with partial application
-3. **Regression Test**: Ensure existing functions still work correctly
\ No newline at end of file
+### **For Task 3.1: Test 22 Parser Issue**
+1. **Investigate `parser_parse_when_pattern` function**: Look for state management issues when processing multiple patterns
+2. **Debug the specific failing case**: Add debug output to understand why the parser fails to recognize `is` keyword
+3. **Fix the parser logic**: Update the parser to handle the edge case correctly
+4. **Test the fix**: Verify that Test 22 now passes
+
+### **For Task 3.2: Integration Test 02 File Reading**
+1. **Investigate the file reading issue**: Compare direct file reading vs piped input
+2. **Identify the root cause**: Find why direct file reading causes segmentation fault
+3. **Fix the file reading mechanism**: Update the file reading code to handle the issue
+4. **Test the fix**: Verify that Integration Test 02 now passes
+
+### **For CLI Ergonomics**
+1. **Simplify the REPL**: Make it more minimal and interactive
+2. **Improve error messages**: Better error reporting and debugging
+3. **Add helpful features**: Command history, line editing, etc.
\ No newline at end of file