Skip to content

Conversation

llbbl
Copy link

@llbbl llbbl commented Jun 20, 2025

Add Python Testing Infrastructure

Summary

This PR sets up a comprehensive testing infrastructure for the RedWarden project, transitioning from basic requirements.txt to modern Poetry dependency management and adding pytest with full coverage reporting capabilities.

Changes Made

1. Package Management Migration

  • Migrated to Poetry: Created pyproject.toml with Poetry configuration
  • Preserved dependencies: All existing dependencies from requirements.txt maintained
  • Added test dependencies: pytest, pytest-cov, pytest-mock as development dependencies

2. Testing Configuration

  • pytest configuration: Added comprehensive pytest settings in pyproject.toml
    • Coverage reporting with HTML and XML output
    • Custom test markers (unit, integration, slow)
    • Strict configuration for consistent test behavior
    • Coverage threshold currently set to 0% (should be raised to 80% when adding actual tests)

3. Test Directory Structure

tests/
├── __init__.py
├── conftest.py          # Shared fixtures and configuration
├── test_infrastructure.py  # Validation tests
├── unit/
│   └── __init__.py
└── integration/
    └── __init__.py

4. Shared Test Fixtures

Created comprehensive fixtures in conftest.py:

  • temp_dir: Temporary directory for test files
  • mock_config: Mock configuration dictionary
  • mock_yaml_config: Temporary YAML config file
  • mock_logger: Mock logger object
  • mock_request/mock_response: HTTP request/response mocks
  • mock_plugin: Mock plugin object
  • reset_environment: Environment isolation between tests
  • sample_malleable_profile: Sample C2 profile for testing
  • isolated_imports: Import isolation between tests

5. Development Tooling

  • Updated .gitignore with comprehensive Python/testing/IDE entries
  • Added Poetry script commands: poetry run test and poetry run tests
  • Both commands run pytest with full configuration

Instructions for Running Tests

Initial Setup

# Install Poetry if not already installed
curl -sSL https://install.python-poetry.org | python3 -

# Install dependencies
poetry install

Running Tests

# Run all tests
poetry run test
# or
poetry run tests

# Run with specific markers
poetry run pytest -m unit
poetry run pytest -m integration
poetry run pytest -m "not slow"

# Run specific test file
poetry run pytest tests/test_infrastructure.py

# Run with different output formats
poetry run pytest -v  # Verbose
poetry run pytest -q  # Quiet

Coverage Reports

  • Terminal: Coverage summary shown after each test run
  • HTML Report: Open htmlcov/index.html in a browser
  • XML Report: Available at coverage.xml for CI integration

Next Steps

  1. Raise coverage threshold: Update --cov-fail-under from 0 to 80 in pyproject.toml when writing actual tests
  2. Write unit tests: Add tests for individual modules in tests/unit/
  3. Write integration tests: Add tests for plugin interactions in tests/integration/
  4. CI Integration: Use the generated coverage.xml for CI coverage reporting

Notes

  • The poetry.lock file is gitignored as specified in requirements
  • Coverage is currently at 0% because no actual tests have been written for the codebase
  • All 16 validation tests pass, confirming the infrastructure is working correctly
  • The infrastructure is designed to be immediately usable for test development

- Set up Poetry for modern dependency management
- Configure pytest with coverage reporting and custom markers
- Create organized test directory structure with fixtures
- Add validation tests to verify infrastructure works correctly
- Update .gitignore with testing and development entries
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant