path: root/js/baba-yaga/docs/08_array-programming.md

# Array Programming

Baba Yaga provides powerful array programming operations inspired by APL, K, and Q languages. These operations enable concise, expressive data transformations and mathematical computations on arrays.

## Philosophy

Array programming treats data as multidimensional arrays and provides operations that work on entire arrays at once, rather than element-by-element processing. This leads to:

- **Concise Code**: Express complex operations in single function calls
- **Mathematical Clarity**: Operations mirror mathematical notation
- **Performance**: Operations are optimized for bulk data processing
- **Composability**: Operations chain together naturally

## Indexing and Selection Operations

### `at` - Select by Indices
Select elements from an array at specific positions:

```baba
data : [10, 20, 30, 40, 50];
indices : [0, 2, 4];
selected : at indices data;        // [10, 30, 50]

// Empty indices return empty array
empty : at [] data;                // []

// Out of bounds indices throw errors
// invalid : at [0, 10] data;      // Error: Index out of bounds
```

### `where` - Find by Predicate
Find indices where a predicate function returns true:

```baba
data : [10, 21, 30, 43, 50];
evenPredicate : x -> x % 2 = 0;
evenIndices : where evenPredicate data;  // [0, 2, 4]

// Find all elements greater than 25
largePredicate : x -> x > 25;
largeIndices : where largePredicate data;  // [2, 3, 4]

// No matches return empty array
neverTrue : x -> false;
empty : where neverTrue data;      // []
```

### `take` - First N Elements
Take the first n elements from an array:

```baba
data : [1, 2, 3, 4, 5, 6];
firstThree : take 3 data;          // [1, 2, 3]
firstZero : take 0 data;           // []
all : take 10 data;                // [1, 2, 3, 4, 5, 6] (all available)

// Negative numbers throw errors
// invalid : take -1 data;         // Error: take expects non-negative number
```

### `drop` - Remove First N Elements
Remove the first n elements from an array:

```baba
data : [1, 2, 3, 4, 5, 6];
lastThree : drop 3 data;           // [4, 5, 6]
none : drop 10 data;               // [] (dropped more than available)
all : drop 0 data;                 // [1, 2, 3, 4, 5, 6] (no change)

// Negative numbers throw errors
// invalid : drop -1 data;         // Error: drop expects non-negative number
```

## Cumulative Operations (Scan)

### `scan` - General Cumulative Operation
Apply a binary function cumulatively across an array:

```baba
// Custom scan with addition
addFunc : acc x -> acc + x;
numbers : [1, 2, 3, 4, 5];
cumulative : scan addFunc 0 numbers;  // [0, 1, 3, 6, 10, 15]

// Scan with multiplication
mulFunc : acc x -> acc * x;
products : scan mulFunc 1 numbers;    // [1, 1, 2, 6, 24, 120]

// Scan with string concatenation
concatFunc : acc x -> acc .. x;
words : ["hello", " ", "world"];
sentence : scan concatFunc "" words;  // ["", "hello", "hello ", "hello world"]
```

### `cumsum` - Cumulative Sum
Specialized scan for addition (most common use case):

```baba
numbers : [1, 2, 3, 4, 5];
cumSums : cumsum numbers;             // [0, 1, 3, 6, 10, 15]

// Equivalent to: scan (acc x -> acc + x) 0 numbers
```

### `cumprod` - Cumulative Product
Specialized scan for multiplication:

```baba
numbers : [1, 2, 3, 4, 5];
cumProducts : cumprod numbers;        // [1, 1, 2, 6, 24, 120]

// Equivalent to: scan (acc x -> acc * x) 1 numbers
```

## Broadcasting Operations

### `broadcast` - Scalar-Array Operations
Apply a binary operation between a scalar and each array element:

```baba
values : [1, 2, 3, 4];
addOp : x y -> x + y;
addTen : broadcast addOp 10 values;   // [11, 12, 13, 14]

// Subtraction
subOp : x y -> x - y;
subtract5 : broadcast subOp 5 values; // [-4, -3, -2, -1] (5 - each element)

// Division
divOp : x y -> x / y;
reciprocals : broadcast divOp 1 values; // [1, 0.5, 0.333..., 0.25]
```

### `zipWith` - Element-wise Binary Operations
Apply a binary operation element-wise to two arrays:

```baba
array1 : [1, 2, 3, 4];
array2 : [10, 20, 30, 40];

// Element-wise addition
addOp : x y -> x + y;
sums : zipWith addOp array1 array2;   // [11, 22, 33, 44]

// Element-wise multiplication
mulOp : x y -> x * y;
products : zipWith mulOp array1 array2; // [10, 40, 90, 160]

// Arrays of different lengths use minimum length
short : [1, 2];
long : [10, 20, 30, 40];
result : zipWith addOp short long;    // [11, 22]
```

### `reshape` - Array Restructuring
Reshape a flat array into a multidimensional structure:

```baba
flatData : [1, 2, 3, 4, 5, 6];

// Reshape into 2x3 matrix
matrix2x3 : reshape [2, 3] flatData;  // 2 rows, 3 columns
// Result: [[1, 2, 3], [4, 5, 6]]

// Reshape into 3x2 matrix  
matrix3x2 : reshape [3, 2] flatData;  // 3 rows, 2 columns
// Result: [[1, 2], [3, 4], [5, 6]]

// Incompatible dimensions throw errors
// invalid : reshape [2, 4] flatData; // Error: Cannot reshape array of length 6 to [2, 4]
```

## Monadic Operations

### `flatMap` - Map and Flatten
Apply a function that returns arrays, then flatten the results:

```baba
// Duplicate each element
duplicator : x -> [x, x];
original : [1, 2, 3];
duplicated : flatMap duplicator original; // [1, 1, 2, 2, 3, 3]

// Generate ranges
rangeFunc : x -> range 1 x;
ranges : flatMap rangeFunc [2, 3];       // [1, 2, 1, 2, 3]

// Filter and transform
evenDoubles : x -> when x % 2 is 0 then [x * 2] _ then [];
numbers : [1, 2, 3, 4, 5];
result : flatMap evenDoubles numbers;    // [4, 8]
```

## Array Programming Patterns

### Data Pipeline Processing
```baba
// Process sales data: filter, transform, aggregate
salesData : [100, 250, 75, 300, 150, 400, 50];

pipeline : data ->
  with (
    // Find high-value sales (>= 200)
    highValueIndices : where (x -> x >= 200) data;
    highValues : at highValueIndices data;
    
    // Apply discount
    discounted : broadcast (x y -> x * y) 0.9 highValues;
    
    // Calculate cumulative revenue
    cumulativeRevenue : cumsum discounted;
  ) -> {
    original: highValues,
    discounted: discounted,
    cumulative: cumulativeRevenue,
    total: (slice cumulativeRevenue (length cumulativeRevenue - 1) (length cumulativeRevenue)).0
  };

result : pipeline salesData;
```

### Matrix Operations
```baba
// Create and manipulate matrices
flatMatrix : [1, 2, 3, 4, 5, 6, 7, 8, 9];
matrix3x3 : reshape [3, 3] flatMatrix;

// Add scalar to all elements
addOp : x y -> x + y;
shifted : broadcast addOp 10 flatMatrix;
shiftedMatrix : reshape [3, 3] shifted;

// Element-wise operations between matrices
matrix2 : [9, 8, 7, 6, 5, 4, 3, 2, 1];
mulOp : x y -> x * y;
elementwiseProduct : zipWith mulOp flatMatrix matrix2;
productMatrix : reshape [3, 3] elementwiseProduct;
```

### Statistical Analysis
```baba
// Statistical operations on datasets
dataset : [23, 45, 67, 12, 89, 34, 56, 78, 90, 11];

analyze : data ->
  with (
    sorted : sort.by data (x -> x);
    n : length data;
    
    // Cumulative statistics
    cumSums : cumsum data;
    runningAverages : broadcast (x y -> x / y) (cumsum data) (range 1 (n + 1));
    
    // Percentile indices
    q1Index : (n + 1) / 4;
    q3Index : 3 * (n + 1) / 4;
  ) -> {
    size: n,
    total: (slice cumSums (n - 1) n).0,
    runningAvgs: runningAverages,
    sorted: sorted
  };

stats : analyze dataset;
```

## Error Handling

Array programming operations include comprehensive error checking:

```baba
// Index out of bounds
data : [1, 2, 3];
// error : at [0, 5] data;           // Error: Index out of bounds

// Invalid reshape dimensions
flatData : [1, 2, 3, 4, 5];
// error : reshape [2, 3] flatData;  // Error: Cannot reshape array of length 5 to [2, 3]

// Type errors
// error : scan "not a function" 0 [1, 2, 3]; // Error: Scan expects a function
// error : broadcast 42 5 [1, 2, 3];          // Error: broadcast expects a function
```

## Performance Considerations

- **Bulk Operations**: Array programming operations are optimized for processing entire arrays
- **Memory Efficiency**: Operations create new arrays (immutable) but reuse underlying data when possible
- **Composition**: Chain operations together for complex transformations without intermediate variables
- **Functional Style**: Pure functions with no side effects enable optimizations

## Integration with Other Features

Array programming operations integrate seamlessly with other Baba Yaga features:

```baba
// With pattern matching
processArray : arr ->
  when (length arr) is
    0 then []
    1 then arr
    n if (n > 10) then take 10 arr  // Limit large arrays
    _ then broadcast (x y -> x + y) 1 arr; // Add 1 to each element

// With error handling using Result types
safeAt : indices data ->
  when (filter (i -> i >= 0 and i < length data) indices) is
    validIndices then Ok (at validIndices data)
    _ then Err "Invalid indices";

// With higher-order functions
applyToColumns : matrix func ->
  with (
    rows : length matrix;
    cols : length matrix.0;
    columnData : i -> map (row -> row.i) matrix;
  ) -> map (i -> func (columnData i)) (range 0 cols);
```

Array programming in Baba Yaga provides a powerful, expressive way to work with collections of data, enabling both mathematical computations and practical data processing tasks.