AI-Powered Automatic Issue Triage

[yusukebe]

Summary

Implement an AI system to automatically reproduce and classify GitHub issues, reducing maintainer workload and improving response times.

Motivation

• Many issues are "not a bug" but configuration/usage problems
• Maintainers spend significant time on initial triage
• Real bugs get buried in noise
• Current manual process doesn't scale with project growth

Proposed Solution

AI bot that:

1. Parses issue descriptions - Extract reproduction steps, environment details, expected vs actual behavior
2. Generates reproduction test code - Create minimal test cases based on issue description
3. Runs tests in sandbox environment - Execute tests across multiple runtimes (Node.js, Deno, Bun, Cloudflare Workers)
4. Auto-classifies issues - Categorize as: Bug/Not-a-bug/Need-info/Documentation

Benefits

• 80% reduction in manual triage time
• 24h initial response time for all issues
• Better focus on real bugs for maintainers
• Improved documentation derived from common mistakes
• Consistent quality of initial responses

Technical Approach

• GitHub Actions for automation
• Claude Code for AI processing and test generation
• Sandbox environment for safe test execution
• Multi-runtime testing for comprehensive validation

Community Impact

This could become a template for other OSS projects facing similar triage challenges.

Would love to get community feedback on this idea! 🚀

🤖 Generated with Claude Code

[goisaki]

I am interested in this implementation and will make a PoC.
(I may not finish it forever and I don't mind at all what others do)

@yusukebe

1. do you have any plans for an LLM model to use?

   Providers such as OpenAI and Google offer free tier in exchange for learning use of the data. If you don't mind, I think these are the best to use.

2. how much can you afford to pay for API usage per issue/per day?
3. using Github Actions is fine, but Cloudflare Workers (cron) would be a good choice for these reasons. What do you think?
   • Time-limited Context Caching can be used by batching several issues at once
   • Easy to develop locally