FerrisScript Documentation

FerrisScript Testing Documentation

Single Source of Truth for Testing Procedures, Patterns, and Tracking

Last Updated: October 10, 2025
Status: Active
Purpose: Central hub for test discoverability and comprehensive testing guidance

🚀 Quick Start

New to testing FerrisScript? Start here:

  1. TESTING_GUIDE.md - Complete guide to all testing patterns
  2. Choose your testing layer:
  3. Run the tests: cargo test --workspace or ferris-test --all

Looking for specific test coverage? → See Test Matrices

📚 Testing Framework Overview

FerrisScript uses a 4-layer testing strategy where each layer validates different concerns:

┌─────────────────────────────────────────────┐
│   Layer 4: Manual Testing (Godot Editor)    │  ← Feature validation
├─────────────────────────────────────────────┤
│   Layer 3: Integration Tests (.ferris)      │  ← End-to-end behavior
├─────────────────────────────────────────────┤
│   Layer 2: GDExtension Tests (GDScript)     │  ← Godot bindings
├─────────────────────────────────────────────┤
│   Layer 1: Unit Tests (Rust)                │  ← Pure logic
└─────────────────────────────────────────────┘

Test Suite Summary (v0.0.4)

Test Type Count Location Run Command
Unit Tests (Compiler) 543 crates/compiler/src/ cargo test -p ferrisscript_compiler
Unit Tests (Runtime) 110 crates/runtime/src/ cargo test -p ferrisscript_runtime
Unit Tests (GDExtension) 11 pass, 10 ignored crates/godot_bind/src/ cargo test -p ferrisscript_godot_bind
Integration Tests 15+ godot_test/scripts/*.ferris ferris-test --all
Benchmark Tests 8 suites crates/*/benches/ cargo bench
Total 843+ All layers cargo test --workspace

Coverage: ~82% (last updated: 2025-10-10)

🔍 Testing Types & Procedures

Unit Testing (Compiler)

What: Tests lexer, parser, type checker, and code generator logic
When: Testing pure compilation logic without runtime execution
Location: crates/compiler/src/ (inline #[cfg(test)] mod tests)

Documentation:

Quick Start:

# Run all compiler tests (543 tests)
cargo test -p ferrisscript_compiler

# Run specific test
cargo test -p ferrisscript_compiler test_parse_assignment

# Run with output
cargo test -p ferrisscript_compiler -- --show-output

Example Test:

#[test]
fn test_parse_assignment() {
    let source = "let x = 42;";
    let result = Parser::parse(source);
    assert!(result.is_ok());
    assert_eq!(result.unwrap().statements.len(), 1);
}

Unit Testing (Runtime)

What: Tests AST interpreter, runtime execution, and node callbacks
When: Testing runtime behavior without Godot GDExtension
Location: crates/runtime/src/ and crates/runtime/tests/

Documentation:

Quick Start:

# Run all runtime tests (110 tests)
cargo test -p ferrisscript_runtime

# Run specific test
cargo test -p ferrisscript_runtime test_call_get_node_function

# Run with output
cargo test -p ferrisscript_runtime -- --show-output

Example Test:

#[test]
fn test_call_get_node_function() {
    let mut runtime = Runtime::new();
    runtime.set_node_callback(|path| Ok(MockNode::new(path)));
    
    let result = runtime.call_function("get_node", vec!["Player"]);
    assert!(result.is_ok());
}

Integration Testing (.ferris Scripts)

What: End-to-end testing of .ferris scripts running in Godot
When: Testing complete feature workflows with real Godot runtime
Location: godot_test/scripts/*.ferris
Tool: ferris-test CLI (from crates/test_harness)

Documentation:

Quick Start:

# Run all integration tests (15+ tests)
ferris-test --all

# Run specific test
ferris-test --script godot_test/scripts/export_properties_test.ferris

# Filter by name
ferris-test --all --filter "signal"

# Verbose output
ferris-test --all --verbose

# JSON format (for CI)
ferris-test --all --format json > results.json

Example Test (godot_test/scripts/signal_test.ferris):

// TEST: signal_emission
// CATEGORY: integration
// EXPECT: success
// ASSERT: Signal emitted correctly

export fn _ready() {
    print("[TEST_START]");
    
    signal health_changed(i32, i32);
    emit_signal("health_changed", 100, 80);
    
    print("[PASS] Signal emitted successfully");
    print("[TEST_END]");
}

Configuration: ferris-test.toml in workspace root

Headless Testing (Godot Runtime)

What: Tests Rust code that requires Godot runtime initialization
When: Testing GDExtension bindings, Godot type construction, PropertyInfo generation
Location: crates/*/tests/headless_integration.rs + godot_test/scripts/*.gd

Documentation:

Quick Start:

# Run headless tests (requires Godot)
cargo test -p ferrisscript_godot_bind test_headless_integration

# Note: Some tests are marked #[ignore] because they require Godot runtime
# These are tested via integration tests with ferris-test instead

Example Test:

#[test]
#[ignore = "requires Godot runtime - tested via ferris-test"]
fn test_export_range_property() {
    godot::init();
    
    let hint = PropertyHint::Range { min: 0, max: 100, step: 1 };
    let result = map_hint(&hint);
    
    assert_eq!(result.hint_string(), "0,100,1");
}

Why Ignored? Many GDExtension tests require godot::init() which can’t run in standard unit tests. These are covered by integration tests instead. See TESTING_GUIDE.md - Why Some Tests Are Ignored for details.

Benchmark Testing

What: Performance benchmarks using Criterion.rs
When: Measuring compiler/runtime performance, regression detection
Location: crates/*/benches/

Documentation:

Quick Start:

# Run all benchmarks
cargo bench

# Run specific benchmark
cargo bench --bench compilation

# Run with baseline comparison
cargo bench -- --save-baseline main
cargo bench -- --baseline main

Benchmark Suites:

Baselines: See BENCHMARK_BASELINE.md for v0.0.4 performance targets

📊 Test Matrices

Test matrices provide systematic tracking of test scenarios across all test types. Use these to:

Node Queries & Signals

Files:

Coverage Summary (as of v0.0.4):

Feature Category Total Scenarios Tested Coverage Priority Gaps
Node Query Functions 45 23 51% Relative/absolute paths, Unicode paths, edge cases
Signal System 30 18 60% Error propagation, signal validation, performance
Overall 75 41 55% Cross-cutting concerns (security, performance)
Status Legend: ✅ PASS ⚠️ PARTIAL ❌ TODO 🚧 IN PROGRESS 💥 FAIL

Quick Links:

Future Test Matrices

Planned (as features are added):

Want to add a test matrix? Follow the pattern in TEST_MATRIX_NODE_QUERIES_SIGNALS.md and update this README.

🛠️ Test Harness & Tooling

ferris-test CLI

Purpose: Run .ferris integration tests headlessly with Godot
Location: crates/test_harness/src/main.rs
Configuration: ferris-test.toml in workspace root

Features:

Documentation:

Usage Examples:

# Basic usage
ferris-test --all

# Filter tests
ferris-test --all --filter "export"

# JSON output for CI
ferris-test --all --format json > results.json

# Verbose debugging
ferris-test --all --verbose

# Single test
ferris-test --script godot_test/scripts/signal_test.ferris

Configuration (ferris-test.toml):

godot_executable = "path/to/godot.exe"
project_path = "./godot_test"
timeout_seconds = 30
output_format = "console"
verbose = false

Environment Overrides:

🔗 Related Documentation

Core Documentation

Testing Strategy & Analysis

Version-Specific Documentation

Performance & Coverage

✅ Testing Checklist for New Features

Use this checklist when adding new functionality:

1. Unit Tests (Required)

2. Integration Tests (Required for User-Facing Features)

3. GDExtension Tests (If Applicable)

4. Benchmarks (If Performance-Critical)

5. Documentation

6. CI/CD

🚨 Troubleshooting

Common Issues

Problem: “Tests pass locally but fail in CI”

Problem: “GDExtension not loaded in tests”

Problem: “ferris-test command not found”

Problem: “Test marked as failed but output looks correct”

Problem: Copilot suggests creating duplicate testing infrastructure

Full Troubleshooting: See TESTING_GUIDE.md - Troubleshooting

📈 Testing Metrics & Goals

Current State (v0.0.4)

Goals (v0.1.0)

Continuous Improvement

💡 Testing Best Practices

  1. Test at the Right Layer: Don’t use integration tests for unit-testable logic
  2. Use Existing Infrastructure: Leverage ferris-test and test_harness instead of creating new tools
  3. Document Test Intent: Every test should clearly state what it validates
  4. Make Tests CI-Friendly: All tests must run headlessly without GUI
  5. Optimize for Fast Feedback: Unit tests <1s, integration tests <30s
  6. Track Coverage Systematically: Use test matrices to identify gaps
  7. Update Documentation: Keep TESTING_GUIDE.md and this README in sync

🔄 Document Maintenance

Update Frequency: After each version release or when testing patterns change

Review Triggers:

Ownership: Project Lead (solo dev) + Community Contributors

Last Review: October 10, 2025 (v0.0.4 completion)

Questions or Issues? See CONTRIBUTING.md or open a GitHub issue.

Want to improve testing? Check TEST_HARNESS_TESTING_STRATEGY.md for enhancement opportunities.

This site is open source. Improve this page.