diff options
Diffstat (limited to 'js/scripting-lang/WHAT-IS-THIS.md')
| -rw-r--r-- | js/scripting-lang/WHAT-IS-THIS.md | 396 |
1 files changed, 0 insertions, 396 deletions
diff --git a/js/scripting-lang/WHAT-IS-THIS.md b/js/scripting-lang/WHAT-IS-THIS.md deleted file mode 100644 index 8d52481..0000000 --- a/js/scripting-lang/WHAT-IS-THIS.md +++ /dev/null @@ -1,396 +0,0 @@ -# What Is This: Scripting Language Project - -## Project Overview - -This is a **combinator-based functional scripting language** that translates all operations into function calls to standard library combinators. - -## Core Architecture - -### Combinator Foundation -The language eliminates parsing ambiguity by translating every operation into a function call: - -```javascript -// Source code -x + y * z - -// Internally translated to -add(x, multiply(y, z)) -``` - -This approach ensures: -- **Zero ambiguity**: Every expression has exactly one interpretation -- **Functional foundation**: Everything is a function call -- **Extensible design**: Add new operations by adding combinator functions -- **Consistent patterns**: All operations follow the same structure - -### System Components -1. **Lexer** (`lexer.js`): Converts source code into tokens -2. **Parser** (`parser.js`): Translates tokens into AST, converting operators to combinator calls -3. **Interpreter** (`lang.js`): Executes combinator functions from the standard library - -## Language Features - -### Function Application -Functions are applied using juxtaposition (space-separated): -```javascript -f x // Apply function f to argument x -f x y // Apply f to x, then apply result to y -f (g x) // Apply g to x, then apply f to result -``` - -### Function Definitions -Arrow syntax with lexical scoping: -```javascript -factorial : n -> - when n is - 0 then 1 - _ then n * (factorial (n - 1)); -``` - -### Pattern Matching -When expressions with wildcards and nested expressions: -```javascript -classify : x y -> - when x y is - 0 0 then "both zero" - 0 _ then "x is zero" - _ 0 then "y is zero" - _ _ then "neither zero"; -``` - -### Tables -Array-like and key-value entries with boolean keys: -```javascript -// Array-like -numbers : {1, 2, 3, 4, 5}; - -// Key-value pairs -person : {name: "Alice", age: 30, active: true}; - -// Boolean keys -flags : {true: "enabled", false: "disabled"}; - -// Chained access -nested : {user: {profile: {name: "Bob"}}}; -name : nested.user.profile.name; - -// Embedded functions -calculator : { - add: x y -> x + y, - double: x -> x * 2, - classify: x -> when x is 0 then "zero" _ then "non-zero" -}; - -// APL-style element-wise operations -table1 : {a: 1, b: 2, c: 3}; -table2 : {a: 10, b: 20, c: 30}; -sum : each @add table1 table2; -// Result: {a: 11, b: 22, c: 33} -``` - -### Function References -Use `@` to reference functions: -```javascript -double : x -> x * 2; -numbers : {1, 2, 3, 4, 5}; -doubled : map @double numbers; -``` - -### Combinators and Higher-Order Functions - -The language provides a comprehensive set of combinators for functional programming. Understanding when to use each one is key to writing idiomatic code. - -#### Core Combinators - -**`map(f, x)`** - Transform Elements -- **Purpose**: Apply a function to each element of a collection -- **Use when**: You want to transform every element in a table or array -- **Example**: `double : x -> x * 2; doubled : map @double numbers;` - -**`filter(p, x)`** - Select Elements -- **Purpose**: Keep only elements that satisfy a predicate -- **Use when**: You want to remove elements based on a condition -- **Example**: `is_even : x -> x % 2 = 0; evens : filter @is_even numbers;` - -**`reduce(f, init, x)`** - Accumulate Values -- **Purpose**: Combine all elements into a single value -- **Use when**: You want to compute a summary or aggregate -- **Example**: `total : reduce @add 0 numbers;` - -**`each(f, x)`** - Multi-Argument Element-Wise -- **Purpose**: Apply a function to corresponding elements from multiple collections -- **Use when**: You want to combine elements from multiple tables or tables with scalars -- **Example**: `sum : each @add table1 table2;` -- **Important**: Use `map` for single-table operations, `each` for multi-argument operations - -#### Function Composition - -**`compose(f, g)`** - Right-to-Left Composition -- **Purpose**: Combine functions so the output of one becomes the input of another -- **Use when**: You think of transformations as "g then f" (mathematical notation) -- **Example**: `double_then_increment : compose @increment @double;` - -**`pipe(f, g)`** - Left-to-Right Composition -- **Purpose**: Combine functions so the output of one becomes the input of another -- **Use when**: You think of transformations as "f then g" (pipeline notation) -- **Example**: `increment_then_double : pipe @increment @double;` - -#### Table-Specific Operations (`t.` namespace) - -All `t.` namespace operations are immutable and return new tables: - -**`t.map(f, table)`** - Table-Specific Map -- **Purpose**: Apply function to each value in a table -- **Example**: `double : x -> x * 2; doubled_person : t.map @double person;` - -**`t.set(table, key, value)`** - Immutable Set -- **Purpose**: Create a new table with a key set to a value -- **Example**: `updated_person : t.set person "age" 31;` - -**`t.delete(table, key)`** - Immutable Delete -- **Purpose**: Create a new table with a key removed -- **Example**: `person_no_age : t.delete person "age";` - -**`t.merge(table1, table2)`** - Immutable Merge -- **Purpose**: Combine two tables, with table2 values taking precedence -- **Example**: `merged : t.merge base updates;` - -**`t.get(table, key, defaultValue)`** - Safe Access -- **Purpose**: Get a value from a table with a default if key doesn't exist -- **Example**: `name : t.get person "name" "Unknown";` - -#### When to Use Which Combinator - -- **`map` vs `t.map`**: Use `map` for general collections, `t.map` to emphasize table operations -- **`each` vs `map`**: Use `each` for multi-argument operations, `map` for single-table transformations -- **`each` vs `apply`**: Use `each` for element-wise operations across collections, `apply` for single values -- **`compose` vs `pipe`**: Use `compose` for mathematical notation, `pipe` for pipeline notation - -### Standard Library -Comprehensive set of combinator functions: - -**Arithmetic**: `add`, `subtract`, `multiply`, `divide`, `modulo`, `power`, `negate` -**Comparison**: `equals`, `notEquals`, `lessThan`, `greaterThan`, `lessEqual`, `greaterEqual` -**Logical**: `logicalAnd`, `logicalOr`, `logicalXor`, `logicalNot` -**Higher-Order**: `map`, `compose`, `pipe`, `apply`, `filter`, `reduce`, `fold`, `curry`, `each` -**Enhanced**: `identity`, `constant`, `flip`, `on`, `both`, `either` -**Table Operations**: `t.map`, `t.filter`, `t.set`, `t.delete`, `t.merge`, `t.get`, `t.has`, `t.length` - -### IO Operations -Built-in input, output, and assertions: -```javascript -..out "Hello, World!"; -input : ..in; -..assert condition; -``` - -## Key Language Rules - -### Function Application with Negative Arguments -**Requires parentheses** to avoid ambiguity: -```javascript -f (-5) // Correct: applies f to -5 -f -5 // Incorrect: ambiguous syntax -``` - -### Infix Minus Operator -**Always parsed as subtraction**: -```javascript -3 - 4 // Parsed as subtract(3, 4) -(3 - 4) // Parsed as subtract(3, 4) -``` - -### Immutability -**Variables cannot be reassigned** after initial definition: -```javascript -x : 5; -x : 10; // Error: Cannot reassign immutable variable -``` - -**Table operations are immutable** - all `t.` namespace operations return new tables: -```javascript -person : {name: "Alice", age: 30}; -updated : t.set person "age" 31; -// person unchanged, updated is new table -``` - -## Project Structure - -``` -scripting-lang/ -├── lang.js # Main interpreter and standard library -├── lexer.js # Lexical analysis -├── parser.js # Parsing and AST generation -├── tests/ # Test files (.txt format) -│ ├── 01_lexer_basic.txt -│ ├── 02_arithmetic_operations.txt -│ ├── ... -│ └── integration_*.txt -├── design/ # Architecture and design documentation -│ ├── ARCHITECTURE.md # Complete system architecture -│ ├── README.md # Design principles and patterns -│ └── HISTORY/ # Implementation journey -├── docs/ # Generated documentation -└── scratch_tests/ # Temporary debugging tests -``` - -## Usage Instructions - -### Running Scripts -```bash -# Basic execution -node lang.js script.txt - -# With debug output -DEBUG=1 node lang.js script.txt - -# Using Bun -bun lang.js script.txt -``` - -### Testing -```bash -# Run all tests -./run_tests.sh - -# Run individual test -node lang.js tests/01_lexer_basic.txt -``` - -### Debug Mode -Enable detailed output for development: -```bash -DEBUG=1 node lang.js script.txt -``` - -This shows: -- Token stream from lexer -- AST structure from parser -- Function call traces -- Scope information -- Call stack statistics - -## Development Guidelines - -### Adding New Features -1. **Follow the combinator approach**: All operations should translate to function calls -2. **Add tokens** in `lexer.js` for new syntax -3. **Add parsing logic** in `parser.js` for new constructs -4. **Add evaluation logic** in `lang.js` for new operations -5. **Add tests** in `tests/` directory -6. **Update documentation** in `design/` directory - -### Code Style -- **Functional approach**: Prefer pure functions -- **Clear naming**: Descriptive function and variable names -- **Comprehensive testing**: Test edge cases and combinations -- **Documentation**: Comment complex logic and design decisions - -### Testing Strategy -- **Unit tests**: Test individual features in isolation -- **Integration tests**: Test feature combinations -- **Edge cases**: Test boundary conditions and error cases -- **Backward compatibility**: Ensure existing code continues to work - -## Implementation Details - -### Lexer (`lexer.js`) -- Converts source code into tokens -- Handles all language constructs -- Supports comments, whitespace, and error reporting - -### Parser (`parser.js`) -- Translates tokens into Abstract Syntax Tree (AST) -- Converts operators to combinator function calls -- Handles precedence and associativity -- Supports nested expressions and complex constructs - -### Interpreter (`lang.js`) -- Executes AST nodes -- Manages scope and variable binding -- Provides standard library combinator functions -- Handles error reporting and debugging - -### Standard Library -All combinator functions are implemented in `lang.js`: -- **Arithmetic combinators**: Basic math operations -- **Comparison combinators**: Equality and ordering -- **Logical combinators**: Boolean operations -- **Higher-order combinators**: Function manipulation -- **Enhanced combinators**: Utility functions - -## Error Handling - -### Common Errors -- **Undefined variables**: Variables must be defined before use -- **Immutable reassignment**: Variables cannot be reassigned -- **Type errors**: Functions expect specific argument types -- **Parsing errors**: Invalid syntax or unexpected tokens - -### Debug Information -The language provides comprehensive error reporting: -- **Token-level errors**: Invalid syntax -- **AST-level errors**: Invalid expressions -- **Runtime errors**: Execution failures -- **Call stack tracking**: Function call history - -## Future Enhancements - -The language is designed to be extensible. Potential enhancements include: - -### Advanced Table Features -- **Table methods**: Built-in functions for table manipulation -- **Table comprehensions**: Functional table construction -- **Table patterns**: Pattern matching on table structures - -### Language Extensions -- **Modules**: Code organization and reuse -- **Type system**: Optional static typing -- **Macros**: Code generation and metaprogramming -- **Concurrency**: Parallel and asynchronous execution - -### Performance Optimizations -- **Tail call optimization**: Efficient recursive functions -- **Lazy evaluation**: Deferred computation -- **Memoization**: Caching function results -- **Compilation**: Bytecode or native compilation - -## Success Metrics - -### ✅ Achieved Goals -- **Test Coverage**: 100% of test cases passing (23/23) -- **Core Features**: All major language features implemented -- **Table Enhancements**: APL-inspired element-wise operations and immutable table operations -- **Error Handling**: Comprehensive error detection and reporting -- **Documentation**: Complete implementation and usage documentation -- **Architecture**: Clean, extensible combinator-based design -- **Performance**: Efficient parsing and evaluation -- **Reliability**: Robust error handling and edge case coverage - -## Key Takeaways - -### Architecture Understanding -1. **Combinator Foundation**: All operations translate to function calls -2. **Functional Semantics**: Everything is a function or function application -3. **Zero Ambiguity**: Every expression has exactly one interpretation -4. **Extensible Design**: Easy to add new features by adding combinator functions - -### Development Patterns -1. **Test-Driven**: Comprehensive test suite with 20 test files -2. **Debug-First**: Built-in debug mode for development -3. **Documentation-Heavy**: Complete documentation in `design/` directory -4. **Incremental**: Features added incrementally with frequent testing - -### Language Design -1. **Functional Programming**: Immutable variables, lexical scoping, higher-order functions -2. **Pattern Matching**: Natural case expressions with wildcards -3. **Table Support**: Array-like and key-value data structures -4. **Standard Library**: Comprehensive set of combinator functions - -### Implementation Approach -1. **Modular Design**: Clear separation between lexer, parser, and interpreter -2. **Combinator Translation**: Operators automatically converted to function calls -3. **Scope Management**: Proper lexical scoping and variable binding -4. **Error Handling**: Comprehensive error detection and reporting - -This language demonstrates how **functional programming principles** can solve real parsing problems while maintaining intuitive syntax. The combinator foundation provides a solid base for building powerful abstractions, and the implementation is robust and well-tested. \ No newline at end of file |