Home › Developer Docs › Testing Guide

Aether Testing Guide

This document provides comprehensive guidance on testing the Aether interpreter.

Table of Contents

Test Organization

Directory Structure

Integration tests are one file per feature in tests/, named <feature>_test.rs. Unit tests live beside the code they cover, named <module>_tests.rs (note the plural — that suffix is what distinguishes them):

src/
├── lexer/
│   └── lexer_tests.rs           # Lexer unit tests
├── parser/
│   └── parser_tests.rs          # Parser unit tests
└── interpreter/
    ├── interpreter_tests.rs     # Interpreter unit tests
    └── builtins_tests.rs        # Built-ins unit tests

To list the integration suites and their sizes:

ls tests/
cargo test -- --test-threads=1        # per-suite counts in the output

Test Categories

Unit tests — individual components in isolation, in src/**/*_tests.rs. Fast (< 1 second), no inter-test dependencies. Run as unittests src/lib.rs.

Integration tests — complete programs end-to-end, in tests/. These exercise feature interaction and are where new features get their coverage.

Counts are deliberately not listed here; see ARCHITECTURE.md — Current Status for the last measured total and the command to re-measure.

Running Tests

Basic Commands

# Run all tests
cargo test

# Run specific test
cargo test test_name

# Run tests for specific module
cargo test lexer
cargo test parser
cargo test interpreter

# Show println! output during tests
cargo test -- --nocapture

IMPORTANT: Always use these flags to prevent memory issues:

# Sequential execution (prevents memory pressure)
cargo test -- --test-threads=1

# With output
cargo test -- --nocapture --test-threads=1

# Single test with output
cargo test test_name -- --nocapture --test-threads=1

Why --test-threads=1?

Test Filtering

# Run only integration tests
cargo test --test integration_tests

# Run only unit tests
cargo test --lib

# Run tests matching pattern
cargo test string  # Runs all tests with "string" in name

# Run ignored tests
cargo test -- --ignored

Continuous Testing

# Watch for changes and re-run tests
cargo watch -x test

# Watch with clear screen
cargo watch -c -x "test -- --test-threads=1"

Test-Driven Development

The Red-Green-Refactor Cycle

1. Red: Write a failing test

#[test]
fn test_exponentiation() {
    let result = eval("2 ** 3");
    assert_eq!(result, Value::Int(8));  // FAILS - feature not implemented
}

2. Green: Write minimal code to make it pass

// Add just enough code to pass the test
// Don't over-engineer!

3. Refactor: Improve code while keeping tests green

// Clean up implementation
// Tests should still pass

TDD Workflow Example

Goal: Add string repeat() method

Step 1: Write the test first

#[test]
fn test_string_repeat() {
    let result = eval("\"ha\".repeat(3)");
    assert_eq!(result, Value::string("hahaha".to_string()));
}

Step 2: Run test (it should fail)

cargo test test_string_repeat
# Expected failure: repeat not implemented

Step 3: Implement minimum code to pass

// In stdlib/string.ae
fn repeat(text, n) {
    let result = ""
    let i = 0
    while (i < n) {
        result = result + text
        i = i + 1
    }
    return result
}

Step 4: Run test again (should pass)

cargo test test_string_repeat -- --test-threads=1

Step 5: Add edge case tests

#[test]
fn test_string_repeat_zero() {
    assert_eq!(eval("\"x\".repeat(0)"), Value::string("".to_string()));
}

#[test]
fn test_string_repeat_negative() {
    assert!(eval("\"x\".repeat(-1)").is_err());
}

Step 6: Refactor if needed

Benefits of TDD

✅ Clear requirements - Test defines what “done” means ✅ Confidence - Refactoring doesn’t break functionality ✅ Documentation - Tests show how to use features ✅ Better design - Testable code is usually better code ✅ Regression prevention - Old tests catch new bugs

Writing Tests

Unit Test Structure

#[test]
fn test_<component>_<feature>() {
    // Arrange: Set up test data
    let input = "...";

    // Act: Execute the operation
    let result = operation(input);

    // Assert: Verify the result
    assert_eq!(result, expected);
}

Integration Test Structure

#[test]
fn test_<feature_name>() {
    let source = r#"
        fn main() {
            // Aether code here
        }
    "#;

    let result = run_program(source);
    assert_eq!(result, expected_output);
}

Assertion Helpers

// Equality
assert_eq!(actual, expected);
assert_ne!(actual, unexpected);

// Boolean
assert!(condition);
assert!(!condition);

// Error handling
assert!(result.is_ok());
assert!(result.is_err());

// Pattern matching
match result {
    Ok(Value::Int(n)) => assert_eq!(n, 42),
    _ => panic!("Expected Int"),
}

Test Naming Conventions

Good test names:

Poor test names:

Pattern: test_<what>_<scenario>_<expected_result>

Debugging Test Failures

Step 1: Read the Error Message

---- test_division_by_zero panicked at 'assertion failed: `(left == right)`
  left: `Ok(Int(5))`,
 right: `Err(DivisionByZero)`', tests/integration_test.rs:42:5

Key information:

Step 2: Isolate the Test

# Run only the failing test
cargo test test_division_by_zero -- --nocapture --test-threads=1

Step 3: Add Debug Output

#[test]
fn test_division_by_zero() {
    let source = "10 / 0";
    println!("Input: {}", source);

    let result = eval(source);
    println!("Result: {:?}", result);

    assert!(result.is_err());
}

Step 4: Use Rust Debugger

# Install rust-lldb or rust-gdb
rust-lldb target/debug/deps/aether-<hash>

# Set breakpoint
(lldb) breakpoint set --name test_division_by_zero
(lldb) run

# Step through
(lldb) step
(lldb) print variable_name

Common Test Failures

Memory Issues

error: test failed, to rerun pass '--lib'
signal: 9, SIGKILL: kill

Solution: Use --test-threads=1

Timeout

test hangs indefinitely

Solution:

Floating Point Precision

// Wrong: Exact equality
assert_eq!(result, 3.14159);

// Right: Approximate equality
assert!((result - 3.14159).abs() < 0.00001);

String/Array Comparison

// For Rc-wrapped values, use pattern matching or helper methods
match &value {
    Value::String(s) => assert_eq!(s.as_ref(), "expected"),
    _ => panic!("Expected string"),
}

Test Coverage Goals

Current Coverage

The suite is expected to be fully green — 0 failed, 0 ignored. A new #[ignore] needs a comment saying why and what would un-ignore it.

For the current total, see ARCHITECTURE.md — Current Status.

Coverage by Component

Every shipped feature has a dedicated suite. ls tests/ is the authoritative list; the file name maps to the feature (tcp_test.rs → TCP, enum_test.rs → enums, plugin_v2_test.rs → the V2 FFI protocol).

A feature without a tests/<feature>_test.rs file is not considered shipped — see the Post-Feature Checklist.

What to Test

Always test:

Example - Testing array.push():

// Happy path
test_array_push_adds_element()

// Edge cases
test_array_push_to_empty_array()
test_array_push_multiple_types()

// Integration
test_array_push_in_loop()
test_array_push_with_function_result()

Measuring Coverage

# Install tarpaulin
cargo install cargo-tarpaulin

# Run coverage report
cargo tarpaulin --out Html

# Open report
open tarpaulin-report.html

Coverage Goals:

Best Practices

Do’s ✅

Don’ts ❌

Test Smells

Problem: Tests are slow (> 10 seconds)

Problem: Tests are flaky (pass/fail randomly)

Problem: Tests break on every change

Problem: Can’t understand what test does

Continuous Integration

Future CI Setup

When setting up CI, include:

# .github/workflows/test.yml
name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions-rs/toolchain@v1
        with:
          toolchain: stable
      - name: Run tests
        run: cargo test -- --test-threads=1
      - name: Check formatting
        run: cargo fmt --check
      - name: Run clippy
        run: cargo clippy -- -D warnings

Resources

Internal Documentation

External Resources

Last Updated: July 29, 2026 Phase: 5 Complete

← Development Guide    Lexer →