Skip to content

Conversation

fcostaoliveira
Copy link
Collaborator

SPEC Changes Summary

Builder

  • Added cleanup functions: clear/reset pending messages on startup
  • New flag: --skip-clear-pending-on-startup
  • Introduced build signature hashing to reuse artifacts
  • Builds now pushed to arch-specific streams

CLI

  • Renamed --build_arch--arch (default: amd64)
  • Added --command-regex to filter tests by command
  • CLI work also sent to arch-specific streams

Runner

  • Added --commands-regex and --uri options
  • Redis URI parsing (redis://, rediss://)
  • Benchmark metric validation (min QPS, latency > 0)
  • Reduced log verbosity (info → debug in some cases)

Compare

  • New CLI options:
    • --extra-filters
    • --use-test-suites-folder
    • --generate-boxplot, --boxplot-output
    • --command-group-regex, --command-regex
  • Added quartile stats (P25, P50, P75) for regressions & improvements
  • Boxplot generation (via matplotlib)
  • Command/group filtering support

Self-Contained Coordinator

  • New flags:
    • --exclusive-hardware
    • --http-port, --http-auth-username, --http-auth-password
    • --skip-clear-pending-on-startup
  • Embedded HTTP server:
    • GET /ping, GET /containers
    • POST /reset-queue, POST /flush
  • Supports exclusive hardware cleanup
  • Flush timestamp support (skip old work)
  • Container timeout handling

New Test-Suites (Datasets)

  1. memtier_benchmark-1Mkeys-string-setget200c-1KiB-pipeline-10
  2. memtier_benchmark-playbook-leaderboard-top-100
  3. memtier_benchmark-playbook-rate-limiting-lua-100k-sessions
  4. memtier_benchmark-playbook-realtime-analytics-membership
  5. memtier_benchmark-playbook-realtime-analytics-membership-pipeline-10
  6. memtier_benchmark-session-caching-json-100k-sessions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant