# Multi-paradigm Programming with Lil - A Guide to Lil's Diverse Styles ## Introduction Lil is a richly multi-paradigm scripting language designed for the Decker creative environment. It seamlessly blends concepts from **imperative**, **functional**, **declarative**, and **vector-oriented** programming languages. This flexibility allows developers to choose the most effective and ergonomic approach for a given task, whether it's managing application state, manipulating complex data structures, or performing efficient bulk computations. Understanding these paradigms is key to writing elegant, efficient, and idiomatic Lil code. ## Core Concepts Lil's power comes from the way it integrates four distinct programming styles. ### Imperative Programming This is the traditional, statement-by-statement style of programming. It involves creating variables, assigning values to them, and using loops and conditionals to control the flow of execution. - **Assignment:** The colon (`:`) is used for assignment. - **Control Flow:** Lil provides `if`/`elseif`/`else` for conditionals and `while` and `each` for loops. - **State Management:** State is typically managed by assigning and re-assigning values to variables, often stored in the properties of Decker widgets between event handlers. ```lil # Imperative approach to summing a list total: 0 numbers: [10, 20, 30] each n in numbers do total: total + n end # total is now 60 ``` ### Functional Programming The functional style emphasizes pure functions, immutability, and the composition of functions without side-effects. - **Immutability:** All core data structures (lists, dictionaries, tables) have copy-on-write semantics. Modifying one does not alter the original value but instead returns a new, amended value. - **First-Class Functions:** Functions are values that can be defined with `on`, assigned to variables, and passed as arguments to other functions. - **Expressions over Statements:** Every statement in Lil is an expression that returns a value. An `if` block returns the value of its executed branch, and an `each` loop returns a new collection containing the result of each iteration. ```lil # Functional approach using a higher-order function on twice f x do f[f[x]] end on double x do x * 2 end result: twice[double 10] # result is 40 ``` ### Declarative (Query-based) Programming For data manipulation, Lil provides a powerful declarative query engine that resembles SQL. Instead of describing *how* to loop through and filter data, you declare *what* data you want. - **Queries:** Use `select`, `update`, and `extract` to query tables (and other collection types). - **Clauses:** Filter, group, and sort data with `where`, `by`, and `orderby` clauses. - **Readability:** Queries often result in more concise and readable code for data transformation tasks compared to imperative loops. ```lil # Declarative query to find developers people: insert name age job with "Alice" 25 "Development" "Sam" 28 "Sales" "Thomas" 40 "Development" end devs: select name from people where job="Development" # devs is now a table with the names "Alice" and "Thomas" ``` ### Vector-Oriented Programming Influenced by languages like APL and K, this paradigm focuses on applying operations to entire arrays or lists (vectors) at once, a concept known as **conforming**. - **Conforming Operators:** Standard arithmetic operators (`+`, `-`, `*`, `/`) work element-wise on lists. - **Efficiency:** Vector operations are significantly more performant than writing equivalent imperative loops. - **The `@` Operator:** The "apply" operator (`@`) can be used to apply a function to each element of a list or to select multiple elements from a list by index. ```lil # Vector-oriented approach to add 10 to each number numbers: [10, 20, 30] result: numbers + 10 # result is [20, 30, 40] ``` ----- ## Key Principles - **Right-to-Left Evaluation:** Expressions are evaluated from right to left unless overridden by parentheses `()`. This is a fundamental rule that affects how all expressions are composed. - **Copy-on-Write Immutability:** Lists, Dictionaries, and Tables are immutable. Operations like `update` or indexed assignments on an expression `(foo)[1]:44` return a new value, leaving the original unchanged. Direct assignment `foo[1]:44` is required to modify the variable `foo` itself. - **Data-Centric Design:** The language provides powerful, built-in tools for data manipulation, especially through its query engine and vector operations. - **Lexical Scoping:** Variables are resolved based on their location in the code's structure. Functions "close over" variables from their containing scope, enabling patterns like counters and encapsulated state. ----- ## Implementation/Usage The true power of Lil emerges when you mix these paradigms to solve problems cleanly and efficiently. ### Basic Example Here, we combine an imperative loop with a vector-oriented operation to process a list of lists. ```lil # Calculate the magnitude of several 2D vectors vectors: [[3,4], [5,12], [8,15]] magnitudes: [] # Imperative loop over the list of vectors each v in vectors do # mag is a vector-oriented unary operator magnitudes: magnitudes & [mag v] end # magnitudes is now [5, 13, 17] ``` ### Advanced Example This example defines a functional-style utility function (`avg`) and uses it within a declarative query to summarize data, an approach common in data analysis. ```lil # Functional helper function on avg x do (sum x) / count x end # A table of sales data sales: insert product category price with "Apple" "Fruit" 0.5 "Banana" "Fruit" 0.4 "Bread" "Grain" 2.5 "Rice" "Grain" 3.0 end # Declarative query that uses the functional helper avgPriceByCategory: select category:first category avg_price:avg[price] by category from sales # avgPriceByCategory is now: # +----------+-----------+ # | category | avg_price | # +----------+-----------+ # | "Fruit" | 0.45 | # | "Grain" | 2.75 | # +----------+-----------+ ``` ----- ## Common Patterns ### Pattern 1: Query over Loop Instead of manually iterating with `each` to filter or transform a collection, use a declarative `select` or `extract` query. This is more concise, often faster, and less error-prone. ```lil # Instead of this imperative loop... high_scores: [] scores: [88, 95, 72, 100, 91] each s in scores do if s > 90 then high_scores: high_scores & [s] end end # ...use a declarative query. high_scores: extract value where value > 90 from scores # high_scores is now [95, 100, 91] ``` ### Pattern 2: Function Application with `@` For simple element-wise transformations on a list, using the `@` operator with a function is cleaner than writing an `each` loop. ```lil # Instead of this... names: ["alice", "bob", "charlie"] capitalized: [] on capitalize s do first s & (1 drop s) end # Simple capitalize, for demo each n in names do capitalized: capitalized & [capitalize n] end # ...use the more functional and concise @ operator. on capitalize s do first s & (1 drop s) end capitalized: capitalize @ names # capitalized is now ["Alice", "Bob", "Charlie"] ``` ----- ## Best Practices - **Embrace Queries:** For any non-trivial data filtering, grouping, or transformation, reach for the query engine first. - **Use Vector Operations:** When performing arithmetic or logical operations on lists, use conforming operators (`+`, `<`, `=`) instead of loops for better performance and clarity. - **Distinguish Equality:** Use the conforming equals `=` within query expressions. Use the non-conforming match `~` in `if` or `while` conditions to avoid accidentally getting a list result. - **Encapsulate with Functions:** Use functions to create reusable components and manage scope, especially for complex logic within Decker event handlers. ----- ## Common Pitfalls - **Right-to-Left Confusion:** Forgetting that `3*2+5` evaluates to `21`, not `11`. Use parentheses `(3*2)+5` to enforce the desired order of operations. - **Expecting Mutation:** Believing that `update ... from my_table` changes `my_table`. It returns a *new* table. You must reassign it: `my_table: update ... from my_table`. - **Comma as Argument Separator:** Writing `myfunc[arg1, arg2]`. This creates a list of two items and passes it as a single argument. The correct syntax is `myfunc[arg1 arg2]`. - **Using `=` in `if`:** Writing `if some_list = some_value` can produce a list of `0`s and `1`s. An empty list `()` is falsey, but a list like `[0,0]` is truthy. Use `~` for a single boolean result in control flow. ----- ## Performance Considerations Vector-oriented algorithms are significantly faster and more memory-efficient than their imperative, element-by-element counterparts. The Lil interpreter is optimized for these bulk operations. For example, replacing values in a list using a calculated mask is preferable to an `each` loop with a conditional inside. ```lil # Slow, iterative approach x: [1, 10, 2, 20, 3, 30] result: each v in x if v < 5 99 else v end end # Fast, vector-oriented approach mask: x < 5 # results in [1,0,1,0,1,0] result: (99 * mask) + (x * !mask) ``` ----- ## Integration Points The primary integration point for Lil is **Decker**. Lil scripts are attached to Decker widgets, cards, and the deck itself to respond to user events (`on click`, `on keydown`, etc.). All paradigms are useful within Decker: - **Imperative:** To sequence actions, like showing a dialog and then navigating to another card. - **Declarative:** To query data stored in a `grid` widget or to find specific cards in the deck, e.g., `extract value where value..widgets.visited.value from deck.cards`. - **Functional/Vector:** To process data before displaying it, without needing slow loops. ----- ## Troubleshooting ### Problem 1: An `if` statement behaves unpredictably with list comparisons. - **Symptoms:** An `if` block either never runs or always runs when comparing a value against a list. - **Solution:** You are likely using the conforming equals operator (`=`), which returns a list of boolean results. In a conditional, you almost always want the non-conforming match operator (`~`), which returns a single `1` or `0`. ### Problem 2: A recursive function crashes with a stack overflow on large inputs. - **Symptoms:** The script terminates unexpectedly when a recursive function is called with a large number or deep data structure. - **Solution:** Lil supports **tail-call elimination**. Ensure your recursive call is the very last operation performed in the function. If it's part of a larger expression (e.g., `1 + my_func[...]`), it is not in a tail position. Rewrite the function to accumulate its result in an argument. ----- ## Examples in Context **Use Case: A Simple To-Do List in Decker** Imagine a Decker card with a `grid` widget named "tasks" (with columns "desc" and "done") and a `field` widget named "summary". ```lil # In the script of the "tasks" grid, to update when it's changed: on change do # Use a DECLARATIVE query to get the done/total counts. # The query source is "me.value", the table in the grid. stats: extract done:sum done total:count done from me.value # Use IMPERATIVE assignment to update the summary field. summary.text: format["%d of %d tasks complete.", stats.done, stats.total] end ``` This tiny script uses a declarative query to read the state and an imperative command to update the UI, demonstrating a practical mix of paradigms.