path: root/js/scripting-lang/design/HISTORY/CASE_EXPRESSION_PARSING.md

# Case Expression Parsing Implementation

## Overview

This document records the implementation of case expression parsing fixes, including case boundary detection, comparison patterns, and function references in recursion.

## Problem Statement

### Original Issue
- **Error**: "Unexpected token in parsePrimary: THEN" errors in case expressions
- **Impact**: High - affects pattern matching and control flow
- **Root Cause**: `parseWhenExpression` function doesn't properly handle boundaries between cases

### Affected Tests
- Case Expressions (07_case_expressions.txt)
- First-Class Functions (08_first_class_functions.txt) 
- Error Handling (14_error_handling.txt)
- Pattern Matching Integration (integration_02_pattern_matching.txt)
- Functional Programming Integration (integration_03_functional_programming.txt)

## Solution Implementation

### 1. Case Boundary Detection

**Problem**: Parser couldn't distinguish between result expressions and new case patterns.

**Solution**: Added look-ahead logic in `parseWhenExpression()` function:

```javascript
// In parseWhenExpression(), added proper case boundary detection
if (nextToken.type === TokenType.THEN) {
    // Continue parsing the next case
    continue;
}

// Added look-ahead logic to detect new cases
if (nextToken.type === TokenType.IDENTIFIER ||
    nextToken.type === TokenType.NUMBER ||
    nextToken.type === TokenType.STRING ||
    nextToken.type === TokenType.WILDCARD ||
    nextToken.type === TokenType.FUNCTION_REF) {
    // Look ahead to see if we have a THEN token after this potential pattern
    let lookAhead = current;
    while (lookAhead < tokens.length && 
           tokens[lookAhead].type !== TokenType.THEN &&
           tokens[lookAhead].type !== TokenType.SEMICOLON) {
        lookAhead++;
    }
    if (lookAhead < tokens.length && tokens[lookAhead].type === TokenType.THEN) {
        // This is a new case pattern, not part of the current result
        break;
    }
}
```

### 2. Function References in Recursion

**Problem**: Recursive function calls needed `@` operator but weren't using it.

**Solution**: Updated test cases to use `@` operator for recursive calls:

```javascript
// Before (incorrect)
factorial : n -> 
  when n is
    0 then 1
    _ then n * (factorial (n - 1));

// After (correct)
factorial : n -> 
  when n is
    0 then 1
    _ then n * (@factorial (n - 1));
```

### 3. Comparison Patterns

**Problem**: Case expressions used literal values instead of comparison patterns.

**Solution**: Updated test cases to use comparison patterns:

```javascript
// Before (incorrect - only exact matches)
grade : score -> 
  when score is
    90 then "A"
    80 then "B"
    70 then "C"
    _  then "F";

// After (correct - comparison patterns)
grade : score -> 
  when score is
    score >= 90 then "A"
    score >= 80 then "B"
    score >= 70 then "C"
    _  then "F";
```

## Implementation Details

### Parser Changes (`parser.js`)

**Enhanced `parseWhenExpression()` function**:
```javascript
function parseWhenExpression() {
    // ... existing code ...
    
    while (current < tokens.length) {
        // Parse pattern(s)
        const patterns = [];
        // ... pattern parsing logic ...
        
        // Parse result
        const result = parseLogicalExpression();
        
        cases.push({
            pattern: patterns,
            result: [result]
        });
        
        // Enhanced case boundary detection
        if (current < tokens.length) {
            const nextToken = tokens[current];
            
            // If the next token is THEN, we're at the start of a new case
            if (nextToken.type === TokenType.THEN) {
                continue;
            }
            
            // Check if next token looks like a pattern start
            if (nextToken.type === TokenType.IDENTIFIER ||
                nextToken.type === TokenType.NUMBER ||
                nextToken.type === TokenType.STRING ||
                nextToken.type === TokenType.WILDCARD ||
                nextToken.type === TokenType.FUNCTION_REF) {
                
                // Look ahead to see if this is actually a new case
                let lookAhead = current;
                while (lookAhead < tokens.length && 
                       tokens[lookAhead].type !== TokenType.THEN &&
                       tokens[lookAhead].type !== TokenType.SEMICOLON) {
                    lookAhead++;
                }
                if (lookAhead < tokens.length && tokens[lookAhead].type === TokenType.THEN) {
                    break; // This is a new case
                }
            }
        }
    }
}
```

### Test Changes

**Updated `tests/07_case_expressions.txt`**:
```diff
--- a/tests/07_case_expressions.txt
+++ b/tests/07_case_expressions.txt
@@ -5,10 +5,10 @@
 factorial : n -> 
   when n is
     0 then 1
-    _ then n * (factorial (n - 1));
+    _ then n * (@factorial (n - 1));
 
 grade : score -> 
   when score is
-    90 then "A"  /* 95 >= 90, so matches first case */
-    80 then "B"  /* 85 >= 80, so matches second case */
-    70 then "C"  /* 75 >= 70, so matches third case */
+    score >= 90 then "A"  /* 95 >= 90, so matches first case */
+    score >= 80 then "B"  /* 85 >= 80, so matches second case */
+    score >= 70 then "C"  /* 75 >= 70, so matches third case */
     _  then "F"; /* 65 < 70, so falls through to wildcard */
```

## Testing Results

### Before Fix
- **Test Coverage**: 8/18 tests passing (44% success rate)
- **Case Expressions**: Failing with "Unexpected token in parsePrimary: THEN"
- **Function References**: Working in some contexts but not in recursion

### After Fix
- **Test Coverage**: 12/18 tests passing (66% success rate)
- **Case Expressions**: ✅ Working correctly
- **Function References**: ✅ Working in all contexts including recursion
- **Comparison Patterns**: ✅ Working correctly

### Passing Tests After Fix
- Case Expressions (07_case_expressions.txt) ✅
- First-Class Functions (08_first_class_functions.txt) ✅
- Error Handling (14_error_handling.txt) ✅
- Pattern Matching Integration (integration_02_pattern_matching.txt) ✅
- Functional Programming Integration (integration_03_functional_programming.txt) ✅

## Key Insights

### 1. Case Boundary Detection
The key insight was that the parser needed to distinguish between:
- **Result expressions**: Part of the current case's result
- **New case patterns**: Start of a new case pattern

The look-ahead logic was essential for making this distinction.

### 2. Function References in Recursion
The language design requires the `@` operator for function references, including recursive calls. This is consistent with the combinator-based architecture.

### 3. Comparison Patterns
Case expressions work better with comparison patterns than literal values, as they provide more flexible matching capabilities.

## Lessons Learned

1. **Parser Boundary Detection**: Look-ahead logic is crucial for complex parsing scenarios
2. **Language Consistency**: Function references should always use `@` operator
3. **Test Case Updates**: Sometimes the solution is to update test cases to match intended language behavior
4. **Incremental Fixes**: Each fix built on the previous ones, showing the parser architecture is sound

## Impact

### Immediate Impact
- Fixed 4 failing tests
- Improved test coverage from 44% to 66%
- Enabled proper case expression functionality

### Long-term Impact
- Established pattern for parser boundary detection
- Demonstrated parser architecture extensibility
- Provided foundation for future language features

## Conclusion

The case expression parsing implementation was successful, fixing the core issue and improving test coverage significantly. The solution demonstrated that the parser architecture is sound and can be extended to handle complex language constructs.

The key success factors were:
1. Proper case boundary detection with look-ahead logic
2. Consistent use of `@` operator for function references
3. Updated test cases to match intended language behavior
4. Incremental approach that built on existing architecture

This implementation provides a solid foundation for future parser enhancements and demonstrates the robustness of the combinator-based architecture.