about summary refs log tree commit diff stats
path: root/js/scripting-lang/TESTING.md
diff options
context:
space:
mode:
Diffstat (limited to 'js/scripting-lang/TESTING.md')
-rw-r--r--js/scripting-lang/TESTING.md246
1 files changed, 246 insertions, 0 deletions
diff --git a/js/scripting-lang/TESTING.md b/js/scripting-lang/TESTING.md
new file mode 100644
index 0000000..f9e874a
--- /dev/null
+++ b/js/scripting-lang/TESTING.md
@@ -0,0 +1,246 @@
+# Baba Yaga Testing Guide
+
+This document explains the comprehensive testing approach for the Baba Yaga scripting language, which includes both JavaScript (reference) and C implementations.
+
+## 🎯 Testing Philosophy
+
+The Baba Yaga project uses a **shared test suite** approach to ensure maximum consistency between implementations:
+
+- **JavaScript Implementation**: Reference implementation and source of truth
+- **C Implementation**: Must match JS implementation behavior exactly
+- **Shared Tests**: Single source of truth for expected behavior
+- **Consistency**: Both implementations must pass identical tests
+
+## 📁 Test Organization
+
+```
+project/
+├── tests/                    # 🎯 Shared test suite (SOURCE OF TRUTH)
+│   ├── unit/                # Unit tests (23 files)
+│   ├── integration/         # Integration tests (4 files)  
+│   ├── turing-completeness/ # Turing completeness proofs (7 files)
+│   ├── run_shared_tests.sh  # Unified test runner
+│   └── README.md            # Detailed test documentation
+├── js/
+│   ├── run_tests.sh         # JS-specific runner (uses shared tests)
+│   └── tests/               # 🚫 Legacy (tests moved to shared)
+└── c/
+    ├── run_tests.sh         # C-specific runner (uses shared tests + C-specific)
+    └── turing_complete_demos/ # 🚫 Legacy (moved to shared)
+```
+
+## 🚀 Quick Start
+
+### Run All Tests on Both Implementations
+
+```bash
+# Test JavaScript implementation
+./tests/run_shared_tests.sh js
+
+# Test C implementation  
+./tests/run_shared_tests.sh c
+
+# Compare implementations (run both and compare results)
+./tests/run_shared_tests.sh js > js_results.txt
+./tests/run_shared_tests.sh c > c_results.txt
+diff js_results.txt c_results.txt
+```
+
+### Run Specific Test Categories
+
+```bash
+# Unit tests only
+./tests/run_shared_tests.sh js unit
+./tests/run_shared_tests.sh c unit
+
+# Integration tests only
+./tests/run_shared_tests.sh js integration  
+./tests/run_shared_tests.sh c integration
+
+# Turing completeness tests only
+./tests/run_shared_tests.sh js turing
+./tests/run_shared_tests.sh c turing
+```
+
+### Legacy Runners (Still Work)
+
+```bash
+# JavaScript (now uses shared tests)
+cd js && ./run_tests.sh
+
+# C (uses shared tests + C-specific tests)
+cd c && ./run_tests.sh
+```
+
+## 📊 Test Categories Explained
+
+### 1. Unit Tests (23 tests)
+
+**Purpose**: Test individual language features in isolation
+
+**Examples**:
+- Lexer functionality
+- Arithmetic operations  
+- Function definitions
+- Pattern matching
+- Data structures
+
+**Coverage**: Every language feature has dedicated unit tests
+
+### 2. Integration Tests (4 tests)
+
+**Purpose**: Test how multiple features work together
+
+**Examples**:
+- Basic features integration
+- Pattern matching with functions
+- Functional programming patterns
+- Multi-parameter expressions
+
+**Coverage**: Common usage patterns and feature combinations
+
+### 3. Turing Completeness Tests (7 tests)
+
+**Purpose**: Formally prove the language is Turing complete
+
+**Examples**:
+- Recursion and infinite computation capability
+- Data structure manipulation  
+- Lambda calculus foundations
+- Complex algorithms (GCD, prime checking, sorting)
+
+**Coverage**: All requirements for Turing completeness
+
+## 🎖️ Test Results Interpretation
+
+### ✅ All Tests Pass
+```
+=== Test Summary ===
+Total tests: 34
+Passed: 34
+Failed: 0
+✅ All tests passed!
+```
+**Meaning**: Implementation is fully consistent with reference
+
+### ⚠️ Some Tests Fail
+```
+=== Test Summary ===  
+Total tests: 34
+Passed: 28
+Failed: 6
+❌ Some tests failed.
+```
+**Meaning**: Implementation has gaps or differences from reference
+
+### 🔍 Debugging Failed Tests
+
+1. **Identify Pattern**: Do failures cluster around specific features?
+2. **Check Reference**: Run same test on JS implementation
+3. **Compare Output**: What's different between implementations?
+4. **File Issue**: Document the discrepancy
+5. **Fix or Document**: Either fix the implementation or document limitation
+
+## 🔄 Development Workflow
+
+### Adding New Language Features
+
+1. **JS First**: Implement feature in JavaScript (reference)
+2. **Write Tests**: Add comprehensive tests to `tests/unit/`
+3. **Verify JS**: Ensure tests pass on JS implementation
+4. **Implement C**: Add feature to C implementation
+5. **Verify Consistency**: Ensure tests pass on both implementations
+
+### Fixing Implementation Gaps
+
+1. **Identify Gap**: Test failure indicates inconsistency
+2. **Reference Check**: JS implementation is source of truth
+3. **Fix C Implementation**: Modify C code to match JS behavior
+4. **Verify Fix**: Ensure test now passes on both implementations
+
+### Regression Testing
+
+Before any release:
+
+```bash
+# Full test suite on both implementations
+./tests/run_shared_tests.sh js && ./tests/run_shared_tests.sh c
+```
+
+If any tests fail, the implementations have diverged and need attention.
+
+## 📈 Test Metrics & Goals
+
+### Current Status
+
+| Metric | Value |
+|--------|-------|
+| Total Test Files | 34 |
+| Unit Tests | 23 |
+| Integration Tests | 4 |
+| Turing Completeness Tests | 7 |
+| Test Assertions | 200+ |
+| Implementation Coverage | JS: 100%, C: ~80% |
+
+### Goals
+
+- **100% Test Pass Rate**: Both implementations pass all tests
+- **Feature Parity**: C implementation matches JS completely
+- **Regression Prevention**: Catch any implementation divergence
+- **Documentation**: Every feature has corresponding tests
+
+## 🛠️ Advanced Testing
+
+### Performance Testing
+
+```bash
+# Stress tests (included in unit tests)
+./tests/run_shared_tests.sh js unit | grep "Performance"
+./tests/run_shared_tests.sh c unit | grep "Performance"
+```
+
+### Specific Feature Testing
+
+```bash
+# Test only arithmetic
+./tests/run_shared_tests.sh js unit | grep -A5 -B5 "Arithmetic"
+
+# Test only functions  
+./tests/run_shared_tests.sh js unit | grep -A5 -B5 "Function"
+```
+
+### Debug Mode
+
+```bash
+# Enable debug output
+DEBUG=1 ./tests/run_shared_tests.sh js unit
+```
+
+## 🤝 Contributing Guidelines
+
+### For New Tests
+
+1. **Location**: Add to appropriate directory in `tests/`
+2. **Format**: Follow established test file format
+3. **Assertions**: Include `..assert` statements for verification
+4. **Both Implementations**: Verify test works on JS and C
+5. **Documentation**: Update test documentation
+
+### For Implementation Changes
+
+1. **Tests First**: Update/add tests before changing implementations
+2. **Reference Implementation**: JS changes require careful consideration
+3. **Consistency Check**: Ensure both implementations still pass tests
+4. **Breaking Changes**: Document any intentional behavior changes
+
+## 🏆 Quality Assurance
+
+The shared test suite ensures:
+
+- ✅ **Behavioral Consistency**: Both implementations behave identically
+- ✅ **Feature Completeness**: All features are thoroughly tested  
+- ✅ **Regression Prevention**: Changes don't break existing functionality
+- ✅ **Turing Completeness**: Language is formally proven complete
+- ✅ **Documentation**: Tests serve as executable specification
+
+**Result**: Confidence that both Baba Yaga implementations are equivalent and reliable.
\ No newline at end of file