about summary refs log tree commit diff stats
path: root/js/scripting-lang/design/HISTORY/CASE_EXPRESSION_PARSING.md
diff options
context:
space:
mode:
Diffstat (limited to 'js/scripting-lang/design/HISTORY/CASE_EXPRESSION_PARSING.md')
-rw-r--r--js/scripting-lang/design/HISTORY/CASE_EXPRESSION_PARSING.md242
1 files changed, 242 insertions, 0 deletions
diff --git a/js/scripting-lang/design/HISTORY/CASE_EXPRESSION_PARSING.md b/js/scripting-lang/design/HISTORY/CASE_EXPRESSION_PARSING.md
new file mode 100644
index 0000000..83ae1da
--- /dev/null
+++ b/js/scripting-lang/design/HISTORY/CASE_EXPRESSION_PARSING.md
@@ -0,0 +1,242 @@
+# Case Expression Parsing Implementation
+
+## Overview
+
+This document records the implementation of case expression parsing fixes, including case boundary detection, comparison patterns, and function references in recursion.
+
+## Problem Statement
+
+### Original Issue
+- **Error**: "Unexpected token in parsePrimary: THEN" errors in case expressions
+- **Impact**: High - affects pattern matching and control flow
+- **Root Cause**: `parseWhenExpression` function doesn't properly handle boundaries between cases
+
+### Affected Tests
+- Case Expressions (07_case_expressions.txt)
+- First-Class Functions (08_first_class_functions.txt) 
+- Error Handling (14_error_handling.txt)
+- Pattern Matching Integration (integration_02_pattern_matching.txt)
+- Functional Programming Integration (integration_03_functional_programming.txt)
+
+## Solution Implementation
+
+### 1. Case Boundary Detection
+
+**Problem**: Parser couldn't distinguish between result expressions and new case patterns.
+
+**Solution**: Added look-ahead logic in `parseWhenExpression()` function:
+
+```javascript
+// In parseWhenExpression(), added proper case boundary detection
+if (nextToken.type === TokenType.THEN) {
+    // Continue parsing the next case
+    continue;
+}
+
+// Added look-ahead logic to detect new cases
+if (nextToken.type === TokenType.IDENTIFIER ||
+    nextToken.type === TokenType.NUMBER ||
+    nextToken.type === TokenType.STRING ||
+    nextToken.type === TokenType.WILDCARD ||
+    nextToken.type === TokenType.FUNCTION_REF) {
+    // Look ahead to see if we have a THEN token after this potential pattern
+    let lookAhead = current;
+    while (lookAhead < tokens.length && 
+           tokens[lookAhead].type !== TokenType.THEN &&
+           tokens[lookAhead].type !== TokenType.SEMICOLON) {
+        lookAhead++;
+    }
+    if (lookAhead < tokens.length && tokens[lookAhead].type === TokenType.THEN) {
+        // This is a new case pattern, not part of the current result
+        break;
+    }
+}
+```
+
+### 2. Function References in Recursion
+
+**Problem**: Recursive function calls needed `@` operator but weren't using it.
+
+**Solution**: Updated test cases to use `@` operator for recursive calls:
+
+```javascript
+// Before (incorrect)
+factorial : n -> 
+  when n is
+    0 then 1
+    _ then n * (factorial (n - 1));
+
+// After (correct)
+factorial : n -> 
+  when n is
+    0 then 1
+    _ then n * (@factorial (n - 1));
+```
+
+### 3. Comparison Patterns
+
+**Problem**: Case expressions used literal values instead of comparison patterns.
+
+**Solution**: Updated test cases to use comparison patterns:
+
+```javascript
+// Before (incorrect - only exact matches)
+grade : score -> 
+  when score is
+    90 then "A"
+    80 then "B"
+    70 then "C"
+    _  then "F";
+
+// After (correct - comparison patterns)
+grade : score -> 
+  when score is
+    score >= 90 then "A"
+    score >= 80 then "B"
+    score >= 70 then "C"
+    _  then "F";
+```
+
+## Implementation Details
+
+### Parser Changes (`parser.js`)
+
+**Enhanced `parseWhenExpression()` function**:
+```javascript
+function parseWhenExpression() {
+    // ... existing code ...
+    
+    while (current < tokens.length) {
+        // Parse pattern(s)
+        const patterns = [];
+        // ... pattern parsing logic ...
+        
+        // Parse result
+        const result = parseLogicalExpression();
+        
+        cases.push({
+            pattern: patterns,
+            result: [result]
+        });
+        
+        // Enhanced case boundary detection
+        if (current < tokens.length) {
+            const nextToken = tokens[current];
+            
+            // If the next token is THEN, we're at the start of a new case
+            if (nextToken.type === TokenType.THEN) {
+                continue;
+            }
+            
+            // Check if next token looks like a pattern start
+            if (nextToken.type === TokenType.IDENTIFIER ||
+                nextToken.type === TokenType.NUMBER ||
+                nextToken.type === TokenType.STRING ||
+                nextToken.type === TokenType.WILDCARD ||
+                nextToken.type === TokenType.FUNCTION_REF) {
+                
+                // Look ahead to see if this is actually a new case
+                let lookAhead = current;
+                while (lookAhead < tokens.length && 
+                       tokens[lookAhead].type !== TokenType.THEN &&
+                       tokens[lookAhead].type !== TokenType.SEMICOLON) {
+                    lookAhead++;
+                }
+                if (lookAhead < tokens.length && tokens[lookAhead].type === TokenType.THEN) {
+                    break; // This is a new case
+                }
+            }
+        }
+    }
+}
+```
+
+### Test Changes
+
+**Updated `tests/07_case_expressions.txt`**:
+```diff
+--- a/tests/07_case_expressions.txt
++++ b/tests/07_case_expressions.txt
+@@ -5,10 +5,10 @@
+ factorial : n -> 
+   when n is
+     0 then 1
+-    _ then n * (factorial (n - 1));
++    _ then n * (@factorial (n - 1));
+ 
+ grade : score -> 
+   when score is
+-    90 then "A"  /* 95 >= 90, so matches first case */
+-    80 then "B"  /* 85 >= 80, so matches second case */
+-    70 then "C"  /* 75 >= 70, so matches third case */
++    score >= 90 then "A"  /* 95 >= 90, so matches first case */
++    score >= 80 then "B"  /* 85 >= 80, so matches second case */
++    score >= 70 then "C"  /* 75 >= 70, so matches third case */
+     _  then "F"; /* 65 < 70, so falls through to wildcard */
+```
+
+## Testing Results
+
+### Before Fix
+- **Test Coverage**: 8/18 tests passing (44% success rate)
+- **Case Expressions**: Failing with "Unexpected token in parsePrimary: THEN"
+- **Function References**: Working in some contexts but not in recursion
+
+### After Fix
+- **Test Coverage**: 12/18 tests passing (66% success rate)
+- **Case Expressions**: ✅ Working correctly
+- **Function References**: ✅ Working in all contexts including recursion
+- **Comparison Patterns**: ✅ Working correctly
+
+### Passing Tests After Fix
+- Case Expressions (07_case_expressions.txt) ✅
+- First-Class Functions (08_first_class_functions.txt) ✅
+- Error Handling (14_error_handling.txt) ✅
+- Pattern Matching Integration (integration_02_pattern_matching.txt) ✅
+- Functional Programming Integration (integration_03_functional_programming.txt) ✅
+
+## Key Insights
+
+### 1. Case Boundary Detection
+The key insight was that the parser needed to distinguish between:
+- **Result expressions**: Part of the current case's result
+- **New case patterns**: Start of a new case pattern
+
+The look-ahead logic was essential for making this distinction.
+
+### 2. Function References in Recursion
+The language design requires the `@` operator for function references, including recursive calls. This is consistent with the combinator-based architecture.
+
+### 3. Comparison Patterns
+Case expressions work better with comparison patterns than literal values, as they provide more flexible matching capabilities.
+
+## Lessons Learned
+
+1. **Parser Boundary Detection**: Look-ahead logic is crucial for complex parsing scenarios
+2. **Language Consistency**: Function references should always use `@` operator
+3. **Test Case Updates**: Sometimes the solution is to update test cases to match intended language behavior
+4. **Incremental Fixes**: Each fix built on the previous ones, showing the parser architecture is sound
+
+## Impact
+
+### Immediate Impact
+- Fixed 4 failing tests
+- Improved test coverage from 44% to 66%
+- Enabled proper case expression functionality
+
+### Long-term Impact
+- Established pattern for parser boundary detection
+- Demonstrated parser architecture extensibility
+- Provided foundation for future language features
+
+## Conclusion
+
+The case expression parsing implementation was successful, fixing the core issue and improving test coverage significantly. The solution demonstrated that the parser architecture is sound and can be extended to handle complex language constructs.
+
+The key success factors were:
+1. Proper case boundary detection with look-ahead logic
+2. Consistent use of `@` operator for function references
+3. Updated test cases to match intended language behavior
+4. Incremental approach that built on existing architecture
+
+This implementation provides a solid foundation for future parser enhancements and demonstrates the robustness of the combinator-based architecture. 
\ No newline at end of file