diff options
Diffstat (limited to 'js/scripting-lang/README.md')
| -rw-r--r-- | js/scripting-lang/README.md | 448 |
1 files changed, 177 insertions, 271 deletions
diff --git a/js/scripting-lang/README.md b/js/scripting-lang/README.md index 08334a4..cae6642 100644 --- a/js/scripting-lang/README.md +++ b/js/scripting-lang/README.md @@ -1,311 +1,217 @@ -# Simple Scripting Language - -A functional programming language with immutable variables, first-class functions, and pattern matching, built on a combinator-based foundation. - -## Features - -- **Immutable Variables**: Variables cannot be reassigned -- **First-Class Functions**: Functions can be passed as arguments and stored in data structures -- **Lexical Scoping**: Functions create their own scope -- **Pattern Matching**: Case expressions with wildcard support -- **Table Literals**: Lua-style tables with both array-like and key-value entries -- **Standard Library**: Built-in higher-order functions (`map`, `compose`, `pipe`, `apply`, `filter`, `reduce`, `fold`, `curry`) -- **Combinator Foundation**: All operations are implemented as function calls under the hood -- **IO Operations**: Built-in input/output operations (`..in`, `..out`, `..assert`) -- **Floating Point Arithmetic**: Full support for decimal numbers -- **Unary Minus**: Support for negative numbers (e.g., `-1`, `-3.14`) - -## Current Implementation Status - -### ✅ Completed Features -- **Core Combinators**: All arithmetic, comparison, logical, and higher-order combinators implemented -- **Parser Translation**: All operators translated to combinator function calls -- **Syntax Preservation**: All existing syntax works unchanged -- **Standard Library**: Complete set of higher-order functions -- **Basic Operations**: Arithmetic, comparison, logical operations -- **Function Definitions**: Arrow functions and function declarations -- **Tables**: Table literals and bracket notation access -- **IO Operations**: Input, output, and assertions +# Scripting Language -### 🔄 In Progress -- **Recursive Functions**: Support for functions that call themselves -- **Advanced Pattern Matching**: Extended when expression patterns -- **Dot Notation**: Table access with dot notation (`table.property`) -- **Multi-parameter Cases**: Case expressions with multiple parameters +A combinator-based scripting language with functional programming features, pattern matching, and a comprehensive standard library. -## Syntax +## Overview -### Basic Operations -``` -/* Arithmetic */ -x : 5 + 3; -y : 10 - 2; -z : 4 * 3; -w : 15 / 3; -neg : -5; /* Unary minus */ - -/* Comparisons */ -result : x > y; -equal : a = b; -not_equal : a != b; - -/* Logical */ -and_result : true and false; -or_result : true or false; -``` +This is a functional scripting language that translates all operations into function calls to standard library combinators. The language supports: -### Variables and Functions -``` -/* Immutable variables */ -x : 42; -y : "hello"; +- **Function Definitions**: Arrow syntax with lexical scoping +- **Pattern Matching**: When expressions with wildcards and nested expressions +- **Tables**: Array-like and key-value entries with boolean keys +- **Function References**: @ operator for higher-order programming +- **IO Operations**: Input, output, and assertions +- **Standard Library**: Complete set of arithmetic, comparison, logical, and higher-order combinators +- **Table Enhancements**: APL-inspired element-wise operations and immutable table operations -/* Function definition */ -f : x -> x * 2; +## Quick Start -/* Function call */ -result : f 5; -``` +### Usage +```bash +# Run a script file +node lang.js your-script.txt + +# Or with Bun +bun lang.js your-script.txt +``` + +### Example Script +```plaintext +// Basic arithmetic +result : 5 + 3 * 2; +..out result; + +// Function definition +factorial : n -> + when n is + 0 then 1 + _ then n * (factorial (n - 1)); + +// Pattern matching +classify : x y -> + when x y is + 0 0 then "both zero" + 0 _ then "x is zero" + _ 0 then "y is zero" + _ _ then "neither zero"; + +// Tables +person : {name: "Alice", age: 30, active: true}; +..out person.name; +..out person["age"]; + +// Function composition +double : x -> x * 2; +increment : x -> x + 1; +composed : compose @double @increment 5; +..out composed; // Output: 12 -### Tables +// Table enhancements +numbers : {1, 2, 3, 4, 5}; +doubled : map @double numbers; +..out doubled[1]; // Output: 2 + +// APL-style element-wise operations +table1 : {a: 1, b: 2, c: 3}; +table2 : {a: 10, b: 20, c: 30}; +sum : each @add table1 table2; +..out sum.a; // Output: 11 ``` -/* Table literal */ -table : {1, 2, 3, key: "value"}; -/* Table access */ -first : table[1]; -value : table.key; /* Coming soon */ -nested : table.key.subkey; /* Coming soon */ +## Key Features + +### Function Application +Functions are applied using juxtaposition (space-separated): +```plaintext +f x // Apply function f to argument x +f x y // Apply f to x, then apply result to y +f (g x) // Apply g to x, then apply f to result ``` ### Pattern Matching -``` -/* Case expression */ -result : when x is - 1 then "one" - 2 then "two" - _ then "other"; +Use `when` expressions for pattern matching: +```plaintext +result : when value is + 0 then "zero" + 1 then "one" + _ then "other"; ``` -### IO Operations -``` -/* Output */ -..out "Hello, World!"; +### Tables +Create and access data structures: +```plaintext +// Array-like +numbers : {1, 2, 3, 4, 5}; -/* Input */ -name : ..in; +// Key-value pairs +person : {name: "Alice", age: 30, active: true}; -/* Assertion */ -..assert x = 5; +// Boolean keys +flags : {true: "enabled", false: "disabled"}; ``` -### Standard Library +### Function References +Use `@` to reference functions: +```plaintext +numbers : {1, 2, 3, 4, 5}; +doubled : map @double numbers; ``` -/* Map */ -double : x -> x * 2; -squared : map @double 5; -/* Filter */ -isPositive : x -> x > 0; -filtered : filter @isPositive 5; +## Combinators and Higher-Order Functions -/* Compose */ -f : x -> x + 1; -g : x -> x * 2; -h : compose @f @g; -result : h 5; /* (5 * 2) + 1 = 11 */ -``` +The language provides a comprehensive set of combinators for functional programming: -## Usage +### Core Combinators +- **`map(f, x)`** - Transform elements in collections +- **`filter(p, x)`** - Select elements based on predicates +- **`reduce(f, init, x)`** - Accumulate values into a single result +- **`each(f, x)`** - Multi-argument element-wise operations -### Running Scripts -```bash -node lang.js script.txt -``` +### Function Composition +- **`compose(f, g)`** - Right-to-left composition (mathematical style) +- **`pipe(f, g)`** - Left-to-right composition (pipeline style) +- **`via` operator** - Natural composition syntax: `f via g via h` -### Testing -The project uses a structured testing approach with unit and integration tests: - -#### Unit Tests -Located in `tests/` directory, each focusing on a specific language feature: -- `01_lexer_basic.txt` - Basic lexer functionality ✅ -- `02_arithmetic_operations.txt` - Arithmetic operations ✅ -- `03_comparison_operators.txt` - Comparison operators ✅ -- `04_logical_operators.txt` - Logical operators ✅ -- `05_io_operations.txt` - IO operations ✅ -- `06_function_definitions.txt` - Function definitions ✅ -- `07_case_expressions.txt` - Case expressions and pattern matching 🔄 -- `08_first_class_functions.txt` - First-class function features ✅ -- `09_tables.txt` - Table literals and access ✅ -- `10_standard_library.txt` - Standard library functions ✅ -- `11_edge_cases.txt` - Edge cases 🔄 -- `12_advanced_tables.txt` - Advanced table features 🔄 -- `13_standard_library_complete.txt` - Complete standard library ✅ -- `14_error_handling.txt` - Error handling 🔄 -- `15_multi_parameter_case.txt` - Multi-parameter case expressions 🔄 - -#### Integration Tests -Test combinations of multiple features: -- `integration_01_basic_features.txt` - Basic feature combinations ✅ -- `integration_02_pattern_matching.txt` - Pattern matching with other features 🔄 -- `integration_03_functional_programming.txt` - Functional programming patterns ✅ - -#### Running Tests -```bash -# Run all tests -./run_tests.sh +### Table Operations (`t.` namespace) +- **`t.map`**, **`t.filter`**, **`t.set`**, **`t.delete`**, **`t.merge`**, **`t.get`**, **`t.has`**, **`t.length`** +- All operations are immutable and return new tables -# Run individual tests -node lang.js tests/01_lexer_basic.txt -node lang.js tests/integration_01_basic_features.txt -# or with bun -bun run lang.js tests/01_lexer_basic.txt -``` +### When to Use Which Combinator +- **`map` vs `t.map`**: Use `map` for general collections, `t.map` to emphasize table operations +- **`each` vs `map`**: Use `each` for multi-argument operations, `map` for single-table transformations +- **`compose` vs `pipe`**: Use `compose` for mathematical notation, `pipe` for pipeline notation -## Implementation Details - -### Architecture -- **Lexer**: Tokenizes input into tokens (numbers, identifiers, operators, etc.) -- **Parser**: Builds Abstract Syntax Tree (AST) from tokens, translating operators to combinator calls -- **Interpreter**: Executes AST with scope management and combinator function calls - -### Key Components -- **Token Types**: Supports all basic operators, literals, and special tokens -- **AST Nodes**: Expression, statement, and declaration nodes -- **Scope Management**: Lexical scoping with proper variable resolution -- **Combinator Foundation**: All operations implemented as function calls -- **Error Handling**: Comprehensive error reporting for parsing and execution - -## Recent Fixes - -### ✅ Combinator Foundation Implementation (Latest) -- **Issue**: Parser ambiguity between function application and operator expressions -- **Solution**: Implemented comprehensive combinator foundation -- **Status**: ✅ Completed - All operators now translate to combinator function calls -- **Impact**: Eliminated parsing ambiguity while preserving syntax - -### ✅ Standard Library Function Naming Conflicts -- **Issue**: Test functions using names that conflict with standard library combinators -- **Solution**: Renamed test functions to avoid conflicts (e.g., `add` → `add_func`) -- **Status**: ✅ Resolved - All tests now use unique function names - -### ✅ Parser Ambiguity with Unary Minus Arguments -- **Issue**: `filter @isPositive -3` was incorrectly parsed as binary operation -- **Root Cause**: Parser treating `FunctionReference MINUS` as binary minus operation -- **Solution**: Added special case in `parseExpression()` to handle `FunctionReference MINUS` pattern -- **Status**: ✅ Resolved - Standard library functions now work with negative arguments - -### ✅ Unary Minus Operator -- **Issue**: Stack overflow when parsing negative numbers (e.g., `-1`) -- **Root Cause**: Parser lacked specific handling for unary minus operator -- **Solution**: Added `UnaryMinusExpression` parsing and evaluation -- **Status**: ✅ Resolved - All tests passing - -### ✅ IO Operation Parsing -- **Issue**: IO operations not parsed correctly at top level -- **Solution**: Moved IO parsing to proper precedence level -- **Status**: ✅ Resolved - -### ✅ Decimal Number Support -- **Issue**: Decimal numbers not handled correctly -- **Solution**: Updated lexer and interpreter to use `parseFloat()` -- **Status**: ✅ Resolved - -## Known Issues - -### 🔄 Recursive Function Support -- **Issue**: Functions cannot call themselves recursively -- **Example**: `factorial : n -> when n is 0 then 1 _ then n * factorial (n - 1);` fails -- **Root Cause**: Function not available in global scope when body is evaluated -- **Status**: 🔄 In Progress - Implementing forward declaration pattern -- **Impact**: Recursive algorithms cannot be implemented - -### 🔄 When Expression Pattern Parsing -- **Issue**: When expressions only support basic patterns (identifiers, numbers, strings, wildcards) -- **Example**: `when x is < 0 then "negative" _ then "non-negative"` fails -- **Root Cause**: Parser not handling comparison operators in patterns -- **Status**: 🔄 In Progress - Extending pattern parsing -- **Impact**: Limited pattern matching capabilities - -### 🔄 Dot Notation for Table Access -- **Issue**: Table access only supports bracket notation -- **Example**: `table.property` fails to parse -- **Root Cause**: Parser not handling dot notation -- **Status**: 🔄 In Progress - Adding dot notation support -- **Impact**: Less convenient table access syntax - -### 🔄 Multi-parameter Case Expressions -- **Issue**: Multi-parameter case expressions not parsed correctly -- **Example**: `when x y is 0 0 then "both zero" _ _ then "not both zero"` fails -- **Root Cause**: Parser not handling multiple parameters in case expressions -- **Status**: 🔄 In Progress - Extending case expression parsing -- **Impact**: Limited pattern matching with multiple values - -### 🔄 When Expression Parsing Precedence -- **Issue**: When expressions not parsed correctly in all contexts -- **Example**: Some when expressions fail with "Unexpected token in parsePrimary: WHEN" -- **Root Cause**: Parser precedence not handling when expressions properly -- **Status**: 🔄 In Progress - Adjusting parser precedence -- **Impact**: When expressions may fail in certain contexts +### Standard Library -## Development +The language includes a comprehensive standard library: -### File Structure -``` -. -├── lang.js # Main implementation with combinator foundation -├── parser.js # Parser with operator-to-combinator translation -├── lexer.js # Lexical analyzer -├── tests/ # Unit and integration tests -│ ├── 01_lexer_basic.txt -│ ├── 02_arithmetic_operations.txt -│ ├── ... -│ ├── integration_01_basic_features.txt -│ ├── integration_02_pattern_matching.txt -│ └── integration_03_functional_programming.txt -├── run_tests.sh # Test runner script -├── COMBINATORS.md # Combinator foundation documentation -└── README.md # This file -``` +**Arithmetic**: `add`, `subtract`, `multiply`, `divide`, `modulo`, `power`, `negate` +**Comparison**: `equals`, `notEquals`, `lessThan`, `greaterThan`, `lessEqual`, `greaterEqual` +**Logical**: `logicalAnd`, `logicalOr`, `logicalXor`, `logicalNot` +**Higher-Order**: `map`, `compose`, `pipe`, `apply`, `filter`, `reduce`, `fold`, `curry`, `each` +**Enhanced**: `identity`, `constant`, `flip`, `on`, `both`, `either` +**Table Operations**: `t.map`, `t.filter`, `t.set`, `t.delete`, `t.merge`, `t.get`, `t.has`, `t.length` -### Debugging -Enable debug mode by setting `DEBUG=true`: -```bash -DEBUG=true node lang.js script.txt -``` +## Key Language Takeaways + +- **Function application with negative arguments requires parentheses:** + - Example: `f (-5)` applies `f` to `-5`. +- **Infix minus (`-`) is always parsed as subtraction:** + - Example: `3 - 4` is parsed as `subtract(3, 4)`. +- **Ambiguous syntax like `f -5` is not supported:** + - Use parentheses for negative arguments in function application. +- **Table operations are immutable:** + - All `t.` namespace operations return new tables, never modify existing ones. +- **`each` is for multi-argument operations:** + - Use `map` for single-table transformations, `each` for combining multiple collections. + +These rules ensure that function application and infix operators are unambiguous and match functional language conventions. -## Combinator Foundation +## Architecture -The language is built on a combinator foundation where all operations are implemented as function calls: +The language uses a combinator-based architecture where all operations are translated to function calls: -### Internal Translation -```javascript -// x - y becomes internally: -subtract(x, y) +1. **Lexer**: Converts source code into tokens +2. **Parser**: Translates tokens into AST, converting operators to combinator calls +3. **Interpreter**: Executes combinator functions from the standard library -// filter @isPositive -3 becomes internally: -filter(isPositive, negate(3)) +This approach eliminates parsing ambiguity while preserving syntax and enabling powerful functional programming patterns. -// x + y becomes internally: -add(x, y) +## Testing -// true and false becomes internally: -logicalAnd(true, false) +Run the complete test suite: +```bash +./run_tests.sh ``` -### Benefits -- **Eliminates Parsing Ambiguity**: Every operation is a function call -- **Preserves Syntax**: Zero breaking changes to existing code -- **Functional Foundation**: Everything is a function under the hood -- **Extensible**: Easy to add new combinators and patterns -- **Consistent Semantics**: All operations follow the same pattern +All 23 tests should pass, covering: +- Basic lexer and parser functionality +- Arithmetic and comparison operations +- Function definitions and calls +- Pattern matching and case expressions +- Table literals and access +- Standard library functions +- Error handling and edge cases +- Table enhancements and combinators +- Integration tests -For detailed information about the combinator foundation, see [COMBINATORS.md](COMBINATORS.md). +## Development + +### Project Structure +``` +scripting-lang/ +├── lang.js # Main interpreter and standard library +├── lexer.js # Lexical analysis +├── parser.js # Parsing and AST generation +├── tests/ # Test files (.txt format) +├── design/ # Architecture and design documentation +│ ├── ARCHITECTURE.md +│ ├── README.md +│ └── HISTORY/ # Historical implementation records +└── docs/ # Generated documentation +``` + +### Debug Mode +Enable debug output for development: +```bash +DEBUG=1 node lang.js your-script.txt +``` -## Contributing +### Adding Features +The language is designed to be extensible. To add new features: -1. Create focused unit tests for new features -2. Add integration tests for feature combinations -3. Update documentation -4. Run the full test suite before submitting changes -5. Follow the combinator foundation approach for new operations \ No newline at end of file +1. **Add tokens** in `lexer.js` +2. **Add parsing logic** in `parser.js` +3. **Add evaluation logic** in `lang.js` +4. **Add tests** in `tests/` +5. **Update documentation** \ No newline at end of file |