diff options
Diffstat (limited to 'js/scripting-lang/README.md')
| -rw-r--r-- | js/scripting-lang/README.md | 769 |
1 files changed, 769 insertions, 0 deletions
diff --git a/js/scripting-lang/README.md b/js/scripting-lang/README.md new file mode 100644 index 0000000..296c39f --- /dev/null +++ b/js/scripting-lang/README.md @@ -0,0 +1,769 @@ +# Simple Scripting Language + +A minimal, interpreted scripting language designed for learning language implementation concepts. This language demonstrates the core components needed for an interpreter: lexer, parser, and interpreter. + +## Features + +- **Arithmetic operations**: `+`, `-`, `*`, `/`, `%` (modulo), `^` (power) +- **Comparison operators**: `=`, `<`, `>`, `<=`, `>=`, `!=` +- **Logical operators**: `and`, `or`, `xor`, `not` +- **Variable assignment**: Immutable variables with `:` syntax +- **Function definitions**: Arrow function syntax with pattern matching +- **Pattern matching**: Case expressions with wildcard support +- **Function calls**: Direct function application +- **First-class functions**: Functions can be passed as arguments using `@` syntax +- **Function composition**: Higher-order functions for combining functions +- **Lexical scoping**: Functions create their own scope +- **Input/Output**: Built-in IO operations with `..in`, `..out`, and `..assert` +- **Standard Library**: Built-in functional programming utilities +- **Comments**: C-style block comments with `/* ... */` syntax +- **Tables**: Lua-style immutable tables with array and object capabilities + +## Syntax + +### Variables + +Variables are immutable and assigned using the `:` operator: + +``` +x : 5; +y : 10; +``` + +### Arithmetic Operations + +Arithmetic operations are supported with parentheses grouping: + +``` +sum : x + y; +diff : x - y; +product : x * y; +quotient : x / y; +modulo : 17 % 5; /* Returns 2 */ +power : 2 ^ 3; /* Returns 8 */ +grouped : (5 + 3) * 2; /* Returns 16 */ +nested : ((5 + 3) * 2) + 1; /* Returns 17 */ +``` + +### Function Definitions + +Functions are defined using arrow syntax (`->`): + +``` +add : x y -> x + y; +double : x -> x * 2; +``` + +### Function Calls + +Functions are called by providing arguments directly after the function name: + +``` +result : add 3 4; +doubled : double 5; +``` + +#### Function Calls with Parentheses + +Function calls support parenthesized arguments for complex expressions: + +``` +/* Simple function call */ +result1 : add 3 4; + +/* Function call with parenthesized arithmetic */ +result2 : add (3 + 2) (4 + 1); + +/* Function call with function calls in parentheses */ +result3 : add (double 3) (square 2); + +/* Complex nested function calls with parentheses */ +result4 : add (add 1 2) (add 3 4); +``` + +**Note:** Parentheses provide explicit grouping and are the recommended way to handle complex nested function calls. + +### Tables + +The language supports Lua-style immutable tables that can serve as both arrays and objects: + +#### Table Creation + +Tables are created using curly braces `{}`: + +``` +/* Empty table */ +empty : {}; + +/* Array-like table (1-indexed) */ +numbers : {1, 2, 3, 4, 5}; + +/* Key-value table */ +person : {name: "Alice", age: 30, active: true}; + +/* Mixed table (array-like and key-value) */ +mixed : {1, name: "Bob", 2, active: false}; +``` + +#### Table Access + +Tables support both bracket notation and dot notation for accessing values: + +``` +/* Array access with bracket notation */ +first : numbers[1]; /* Returns 1 */ +second : numbers[2]; /* Returns 2 */ + +/* Object access with dot notation */ +name : person.name; /* Returns "Alice" */ +age : person.age; /* Returns 30 */ + +/* Object access with bracket notation */ +name_bracket : person["name"]; /* Returns "Alice" */ +age_bracket : person["age"]; /* Returns 30 */ + +/* Mixed table access */ +first_mixed : mixed[1]; /* Returns 1 */ +name_mixed : mixed.name; /* Returns "Bob" */ +second_mixed : mixed[2]; /* Returns 2 */ +``` + +#### Table Features + +- **Immutable**: Tables cannot be modified after creation +- **1-indexed arrays**: Array-like tables start indexing at 1 +- **Mixed types**: Tables can contain numbers, strings, booleans, and functions +- **Nested access**: Tables can contain other tables for complex data structures +- **Unified syntax**: Same syntax for arrays and objects + +#### Tables with Case Expressions + +Tables work well with case expressions for pattern matching: + +``` +/* Case expressions with table values */ +person : {name: "Alice", age: 30, active: true}; +result : case person.age of + 0 : 30 : "Person is 30" + 1 : _ : "Person is different age" +; + +/* Case expressions with boolean values */ +status : case person.active of + 0 : true : "Person is active" + 1 : false : "Person is inactive" +; + +/* Case expressions with array-like access */ +numbers : {1: 10, 2: 20, 3: 30}; +element : case numbers[2] of + 0 : 20 : "Second element is 20" + 1 : _ : "Unexpected second element" +; +``` + +**Current Limitations:** +- Function calls from tables (e.g., `table.func args`) are not supported +- Function definitions inside table literals are not supported +- Tables can store function references, but they cannot be called directly + +### First-Class Functions + +Functions can be passed as arguments to other functions using the `@` prefix: + +``` +double : x -> x * 2; +square : x -> x * x; +compose : f g x -> f g x; + +composed : compose @double @square 3; /* double(square(3)) = 18 */ +``` + +**Function Reference Syntax:** +- `@functionName` - Creates a function reference (doesn't execute the function) +- `functionName args` - Calls the function with arguments + +### Pattern Matching + +The language supports pattern matching with case expressions in function bodies: + +``` +compare : x y -> + case x y of + 0 0 : "both zero" + 0 _ : "x is zero" + _ 0 : "y is zero" + _ _ : "neither zero"; +``` + +#### Pattern Matching Syntax + +- **Exact matches**: `0 0` matches when both values are 0 +- **Wildcards**: `_` matches any value +- **Mixed patterns**: `0 _` matches when first value is 0 and second can be anything + +### Comparison and Logical Operations + +The language supports comparison and logical operations: + +``` +/* Comparison operators */ +less : 3 < 5; /* true */ +greater : 10 > 5; /* true */ +equal : 5 = 5; /* true */ +not_equal : 3 != 5; /* true */ +less_equal : 5 <= 5; /* true */ +greater_equal : 5 >= 3; /* true */ + +/* Logical operators */ +and_result : 1 and 1; /* true */ +or_result : 0 or 1; /* true */ +xor_result : 1 xor 0; /* true */ +not_result : not 0; /* true */ +``` + +### Comments + +The language supports C-style block comments: + +``` +/* This is a single line comment */ + +/* This is a multi-line comment + that spans multiple lines */ + +x : 5; /* Comment on same line as code */ + +/* Nested comments are supported */ +/* Outer comment /* Inner comment */ More outer comment */ +``` + +**Comment Features:** +- Block comments start with `/*` and end with `*/` +- Comments can span multiple lines +- Comments can appear on the same line as code +- Nested comments are supported +- Comments are completely ignored by the lexer and parser + +### Input/Output Operations + +The language provides built-in IO operations for reading from stdin and writing to stdout: + +``` +name : ..in; /* Read input from stdin */ +..out name; /* Output to stdout */ +..out "Hello"; /* Output string literal */ +..out 42; /* Output number */ +..assert x = 5; /* Assert that x equals 5 */ +..assert y > 3; /* Assert that y is greater than 3 */ +..assert z != 0; /* Assert that z is not equal to 0 */ +``` + +**IO Functions:** +- `..in` - Reads a line from stdin, returns a number if possible, otherwise a string +- `..out` - Outputs a value to stdout and returns the value +- `..assert` - Asserts that a comparison is true, throws an error if it's false. Supports all comparison operators: `=`, `<`, `>`, `<=`, `>=`, `!=` + +**Note:** The `..` prefix indicates these are impure operations that have side effects. + +## Standard Library + +The language includes a built-in standard library with functional programming utilities: + +### Higher-Order Functions + +- **map**: Apply a function to a value (`map f x = f x`) +- **compose**: Compose two functions (`compose f g x = f(g(x))`) +- **curry**: Explicit currying (`curry f x y = f x y`) +- **apply**: Apply a function to an argument (`apply f x = f x`) +- **pipe**: Compose functions left-to-right (`pipe f g x = g(f(x))`) +- **filter**: Filter based on a predicate (`filter p x = p(x) ? x : 0`) +- **reduce**: Reduce to a single value (`reduce f init x = f(init, x)`) +- **fold**: Same as reduce (`fold f init x = f(init, x)`) + +### Usage Examples + +``` +double : x -> x * 2; +square : x -> x * x; + +/* Using map */ +result1 : map @double 5; /* Returns 10 */ +result2 : map @square 3; /* Returns 9 */ + +/* Using compose */ +composed : compose @double @square 3; /* double(square(3)) = 18 */ + +/* Using pipe */ +piped : pipe @double @square 2; /* square(double(2)) = 16 */ + +/* Using filter */ +isPositive : x -> x > 0; +filtered : filter @isPositive 5; /* Returns 5 (since 5 > 0) */ +``` + + +## Language Components + +### Lexer + +The lexer tokenizes input into meaningful units: +- Numbers: `123` +- Identifiers: `variable_name` +- Operators: `+`, `-`, `*`, `/` +- Keywords: `case`, `of`, `function` +- Symbols: `:`, `->`, `_` + +### Parser + +The parser builds an Abstract Syntax Tree (AST) from tokens, supporting: +- Number literals +- Arithmetic expressions +- Variable assignments +- Function declarations +- Function calls +- Case expressions +- Pattern matching + +### Interpreter + +The interpreter executes the AST with: +- Global scope management +- Function evaluation +- Pattern matching execution +- Arithmetic computation +- Immutable variable semantics + +## Example Programs + +### Basic Arithmetic and Variables + +``` +/* Simple arithmetic with variables */ +x : 5; +y : 10; +sum : x + y; +diff : x - y; +product : x * y; +quotient : y / x; +modulo : 17 % 5; /* Returns 2 */ +power : 2 ^ 3; /* Returns 8 */ + +..out sum; /* 15 */ +..out diff; /* -5 */ +..out product; /* 50 */ +..out quotient; /* 2 */ +..out modulo; /* 2 */ +..out power; /* 8 */ + +/* Grouped expressions with parentheses */ +grouped : (5 + 3) * 2; /* 16 */ +nested : ((5 + 3) * 2) + 1; /* 17 */ +complex : (2 ^ 3) % 3 and 5 > 3; /* true */ +``` + +### Function Definitions and Calls + +``` +/* Basic function definition */ +add : x y -> x + y; +multiply : x y -> x * y; +double : x -> x * 2; +square : x -> x * x; + +/* Function calls */ +result1 : add 3 4; /* 7 */ +result2 : multiply 5 6; /* 30 */ +result3 : double 8; /* 16 */ +result4 : square 4; /* 16 */ + +/* Function calls with parenthesized arguments */ +result5 : add (3 + 2) (4 + 1); /* 15 */ +result6 : add (double 3) (square 2); /* 10 */ +result7 : add (add 1 2) (add 3 4); /* 10 */ +``` + +### Tables and Data Structures + +``` +/* Create various table types */ +empty : {}; +numbers : {1, 2, 3, 4, 5}; +person : {name: "Alice", age: 30, active: true}; +mixed : {1, name: "Bob", 2, active: false}; + +/* Access array elements */ +first : numbers[1]; +second : numbers[2]; +last : numbers[5]; + +/* Access object properties */ +name : person.name; +age : person.age; +active : person.active; + +/* Access mixed table */ +first_mixed : mixed[1]; +name_mixed : mixed.name; +second_mixed : mixed[2]; + +/* Output results */ +..out "First number: "; +..out first; +..out "Person name: "; +..out name; +..out "Mixed first: "; +..out first_mixed; +..out "Mixed name: "; +..out name_mixed; +``` + +### Pattern Matching with Case Expressions + +``` +/* Pattern matching in function bodies */ +compare : x y -> + case x y of + 0 0 : "both zero" + 0 _ : "x is zero" + _ 0 : "y is zero" + _ _ : "neither zero"; + +/* Testing pattern matching */ +test1 : compare 0 0; /* "both zero" */ +test2 : compare 0 5; /* "x is zero" */ +test3 : compare 5 0; /* "y is zero" */ +test4 : compare 5 5; /* "neither zero" */ + +/* Single-parameter case expressions */ +factorial : n -> + case n of + 0 : 1 + _ : n * (factorial (n - 1)); + +/* Single-parameter grade function */ +grade : score -> + case score of + 90 : "A" + 80 : "B" + 70 : "C" + _ : "F"; + +fact5 : factorial 5; /* 120 */ +grade1 : grade 95; /* "A" */ +grade2 : grade 85; /* "B" */ +grade3 : grade 65; /* "F" */ +``` + +### First-Class Functions and Composition + +``` +/* Function references and composition */ +double : x -> x * 2; +square : x -> x * x; +add1 : x -> x + 1; + +/* Using standard library functions */ +composed : compose @double @square 3; /* double(square(3)) = 18 */ +piped : pipe @double @square 2; /* square(double(2)) = 16 */ +applied : apply @double 5; /* double(5) = 10 */ + +/* Higher-order functions */ +mapped1 : map @double 5; /* 10 */ +mapped2 : map @square 3; /* 9 */ + +/* Function references in case expressions */ +getFunction : type -> + case type of + "double" : @double + "square" : @square + _ : @add1; + +func1 : getFunction "double"; /* Function reference */ +func2 : getFunction "square"; /* Function reference */ +``` + +### Recursion and Complex Functions + +``` +/* Recursive factorial function (using single-parameter case) */ +factorial : n -> + case n of + 0 : 1 + _ : n * (factorial (n - 1)); + +/* Recursive countdown function */ +countdown : n -> + case n of + 0 : "done" + _ : countdown (n - 1); + +/* Testing recursive functions */ +fact5 : factorial 5; /* 120 */ +count : countdown 3; /* "done" */ +``` + +### Comparison and Logical Operators + +``` +/* Comparison operators */ +a : 5; +b : 10; + +less : a < b; /* true */ +greater : b > a; /* true */ +equal : a = a; /* true */ +notEqual : a != b; /* true */ +lessEqual : a <= 5; /* true */ +greaterEqual : b >= 10; /* true */ + +/* Logical operators */ +logicalAnd : 1 and 1; /* true */ +logicalOr : 0 or 1; /* true */ +logicalXor : 1 xor 0; /* true */ +logicalNot : not 0; /* true */ + +/* Complex logical expressions */ +complex1 : a < b and b > 5; /* true */ +complex2 : a = 5 or b = 15; /* true */ +complex3 : not (a = b); /* true */ +``` + +### Input/Output and Assertions + +``` +/* Interactive input/output */ +..out "Enter your name: "; +name : ..in; +..out "Hello, "; +..out name; +..out "!"; + +/* Assertions for testing */ +x : 5; +y : 3; +sum : x + y; + +..assert x = 5; /* Passes */ +..assert y = 3; /* Passes */ +..assert sum = 8; /* Passes */ +..assert x > 3; /* Passes */ +..assert y < 10; /* Passes */ +..assert sum != 0; /* Passes */ +..assert (add 3 4) = 7; /* Passes (with parentheses) */ + +/* String comparisons */ +..assert "hello" = "hello"; /* Passes */ +..assert "world" != "hello"; /* Passes */ +``` + +### Standard Library Functions + +``` +/* Using all standard library functions */ +double : x -> x * 2; +square : x -> x * x; +add : x y -> x + y; +isPositive : x -> x > 0; + +/* Map function */ +mapped1 : map @double 5; /* 10 */ +mapped2 : map @square 3; /* 9 */ + +/* Compose function */ +composed : compose @double @square 3; /* 18 */ + +/* Pipe function */ +piped : pipe @double @square 2; /* 16 */ + +/* Apply function */ +applied : apply @double 7; /* 14 */ + +/* Filter function */ +filtered : filter @isPositive 5; /* 5 (since 5 > 0) */ +filtered2 : filter @isPositive -3; /* 0 (since -3 <= 0) */ + +/* Reduce and Fold functions */ +reduced : reduce @add 0 5; /* 5 */ +folded : fold @add 0 5; /* 5 */ + +/* Curry function (explicit currying) */ +curried : curry @add 3 4; /* 7 */ +``` + +### Complete Program Example + +``` +/* A complete program demonstrating multiple features */ +..out "Welcome to the Calculator!"; + +/* Define arithmetic functions */ +add : x y -> x + y; +subtract : x y -> x - y; +multiply : x y -> x * y; +divide : x y -> x / y; + +/* Define utility functions */ +double : x -> x * 2; +square : x -> x * x; +isEven : x -> x % 2 = 0; + +/* Get user input */ +..out "Enter first number: "; +num1 : ..in; +..out "Enter second number: "; +num2 : ..in; + +/* Perform calculations */ +sum : add num1 num2; +diff : subtract num1 num2; +prod : multiply num1 num2; +quot : divide num1 num2; + +/* Display results */ +..out "Sum: "; +..out sum; +..out "Difference: "; +..out diff; +..out "Product: "; +..out prod; +..out "Quotient: "; +..out quot; + +/* Use higher-order functions */ +doubledSum : double sum; +squaredDiff : square diff; + +..out "Doubled sum: "; +..out doubledSum; +..out "Squared difference: "; +..out squaredDiff; + +/* Pattern matching for number classification */ +classify : num -> + case num of + 0 : "zero" + _ : case isEven num of + true : "even" + _ : "odd"; + +classification : classify num1; +..out "First number is: "; +..out classification; + +/* Assertions to verify calculations */ +..assert sum = add num1 num2; +..assert diff = subtract num1 num2; +..assert prod = multiply num1 num2; + +..out "All calculations verified!"; +``` + +## Running Programs + +The language supports file execution mode only: + +### File Execution Mode +Run a script file by providing the file path as an argument: + +```bash +node lang.js script.txt +``` + +**Note:** REPL mode has been removed in the simplified version. The language now only supports file execution. + +## Testing + +The language includes a comprehensive test suite to verify functionality: + +### Test Structure + +- **Unit Tests** (`tests/01_*.txt` to `tests/10_*.txt`): Test individual language features in isolation +- **Integration Tests** (`tests/integration_*.txt`): Test combinations of multiple features +- **Test Runner** (`run_tests.sh`): Automated test execution with pass/fail reporting + +### Running Tests + +**Run all tests:** +```bash +./run_tests.sh +``` + +**Run individual tests:** +```bash +node lang.js tests/01_lexer_basic.txt +node lang.js tests/07_case_expressions.txt +node lang.js tests/integration_01_basic_features.txt +``` + +### Test Coverage + +The test suite covers: +- Basic lexer functionality (numbers, identifiers, operators, keywords) +- Arithmetic operations (all operators, precedence, parentheses) +- Comparison operators (all comparison types, edge cases) +- Logical operators (and, or, xor, not, complex expressions) +- IO operations (output, assertions, string comparisons) +- Function definitions (syntax, parameters, calls, parentheses) +- Case expressions (pattern matching, wildcards, recursion) +- First-class functions (references, higher-order functions) +- Tables (literals, access, mixed types) +- Standard library (all built-in higher-order functions) +- Integration scenarios (feature combinations, complex patterns) + +### Testing Strategy + +The testing approach focuses on: +1. **Isolation**: Each unit test focuses on a single language feature +2. **Integration**: Integration tests verify feature combinations work correctly +3. **Edge Cases**: Tests include boundary conditions and error scenarios +4. **Automation**: Test runner provides systematic execution and reporting + +## Language Design Principles + +- **Simplicity**: Minimal syntax for maximum clarity +- **Immutability**: Variables cannot be reassigned +- **Functional**: Functions are first-class values with composition support +- **Pattern-oriented**: Pattern matching as a core feature +- **Recursive**: Support for recursive function calls in function bodies +- **Grouped**: Parentheses support for complex arithmetic expressions +- **Expressive**: Rich set of arithmetic, comparison, and logical operators +- **IO-aware**: Built-in input/output operations with clear impurity markers +- **Educational**: Designed to teach language implementation concepts + +## Limitations + +This is a learning language with intentional limitations: +- Complex nested function calls (e.g., `add double 3 square 2`) are ambiguous without explicit grouping +- There's one known issue with function calls inside assertions (e.g., ..assert add 3 4 = 7), which is a parsing edge case. Workaround: use parentheses around function calls in assertions (e.g., ..assert (add 3 4) = 7) +- Limited to immutable data structures (no mutation operations) +- No error handling beyond basic validation +- No modules or imports +- IO operations are synchronous and block execution + +## Future Enhancements + +Planned features for future versions: +- Ambiguous function call detection and error reporting +- Lists and data structures +- Error handling +- Modules and imports +- Performance optimizations + +## Implementation Details + +The language is implemented in JavaScript with three main components: + +1. **Lexer** (`lexer()`): Converts source code to tokens +2. **Parser** (`parser()`): Builds AST from tokens +3. **Interpreter** (`interpreter()`): Executes AST + +Each component is designed to be modular and extensible for adding new language features. + +## Files + +- `lang.js` - Main language implementation +- `table_basic_test.txt` - Basic table functionality tests +- `table_edge_cases_test.txt` - Advanced table edge cases (some features may not work) +- `README.md` - This documentation +- `NEXT-STEPS.md` - Development roadmap and notes \ No newline at end of file |