Skip to content

Allow for more MSIs for PCI devices #5299

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

Manciukic
Copy link
Contributor

@Manciukic Manciukic commented Jul 8, 2025

Changes

Distinguish between interrupt lines for legacy devices ("irq") and GSIs used by MSIs for PCI devices ("gsi"). This means:

  • creating a new allocator for irq ([5, 23] on x86, [32, 127] on aarch64)
  • changing gsi allocator to wider range, right after irq ([24, 4095] on x86, [128, 4095] on aarch64)
  • allocating legacy devices from irq instead of gsi allocator
  • updating test to ensure we can now create 31 devices on pci enabled, regardless of arch.

TODO: rebase pci branch and this PR on top of ARM max irq number fix (#5283)

Reason

Since we were still only using legacy IRQs for PCI devices, but each used one interrupt per queue plus one for the config, this meant we were limited to a handful of devices and not the expected 32.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • I have read and understand CONTRIBUTING.md.
  • I have run tools/devtool checkstyle to verify that the PR passes the
    automated style checks.
  • I have described what is done in these changes, why they are needed, and
    how they are solving the problem in a clear and encompassing way.
  • I have updated any relevant documentation (both in code and in the docs)
    in the PR.
  • I have mentioned all user-facing changes in CHANGELOG.md.
  • If a specific issue led to this PR, this PR closes the issue.
  • When making API changes, I have followed the
    Runbook for Firecracker API changes.
  • I have tested all new and changed functionalities in unit tests and/or
    integration tests.
  • I have linked an issue to every new TODO.

  • This functionality cannot be added in rust-vmm.

@Manciukic Manciukic changed the base branch from main to feature/pcie July 8, 2025 15:51
Currently, we're limited to 24 GSI lines, which is too little for PCI
devices. Keep the current ranges as "legacy IRQ", and create a new range
for "GSI" that goes up to the kvm theoretical maximum of 4096 lines.

Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Copy link

codecov bot commented Jul 9, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.21%. Comparing base (e8c599c) to head (7f17889).

Additional details and impacted files
@@              Coverage Diff              @@
##           feature/pcie    #5299   +/-   ##
=============================================
  Coverage         80.20%   80.21%           
=============================================
  Files               265      265           
  Lines             30832    30847   +15     
=============================================
+ Hits              24730    24745   +15     
  Misses             6102     6102           
Flag Coverage Δ
5.10-c5n.metal 79.96% <100.00%> (+0.01%) ⬆️
5.10-m5n.metal 79.96% <100.00%> (+<0.01%) ⬆️
5.10-m6a.metal 79.16% <100.00%> (+0.01%) ⬆️
5.10-m6g.metal 76.55% <100.00%> (+0.01%) ⬆️
5.10-m6i.metal 79.95% <100.00%> (+<0.01%) ⬆️
5.10-m7a.metal-48xl 79.14% <100.00%> (+0.01%) ⬆️
5.10-m7g.metal 76.55% <100.00%> (+0.01%) ⬆️
5.10-m7i.metal-24xl 79.93% <100.00%> (+0.01%) ⬆️
5.10-m7i.metal-48xl 79.92% <100.00%> (+0.01%) ⬆️
5.10-m8g.metal-24xl 76.55% <100.00%> (+0.01%) ⬆️
5.10-m8g.metal-48xl 76.55% <100.00%> (+0.01%) ⬆️
6.1-c5n.metal 80.00% <100.00%> (+0.01%) ⬆️
6.1-m5n.metal 80.00% <100.00%> (+<0.01%) ⬆️
6.1-m6a.metal 79.20% <100.00%> (+0.01%) ⬆️
6.1-m6g.metal 76.55% <100.00%> (+0.01%) ⬆️
6.1-m6i.metal 80.00% <100.00%> (+0.01%) ⬆️
6.1-m7a.metal-48xl 79.19% <100.00%> (+0.01%) ⬆️
6.1-m7g.metal 76.55% <100.00%> (+0.01%) ⬆️
6.1-m7i.metal-24xl 80.01% <100.00%> (+0.01%) ⬆️
6.1-m7i.metal-48xl 80.02% <100.00%> (+0.01%) ⬆️
6.1-m8g.metal-24xl 76.55% <100.00%> (+0.01%) ⬆️
6.1-m8g.metal-48xl 76.55% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Manciukic added a commit to Manciukic/firecracker that referenced this pull request Jul 9, 2025
Until firecracker-microvm#5299 is merged, this test will fail. Ignore PCI on this test for
now until we sort out the changes in the other PR.

Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Manciukic added a commit to Manciukic/firecracker that referenced this pull request Jul 9, 2025
Until firecracker-microvm#5299 is merged, this test will fail. Ignore PCI on this test for
now until we sort out the changes in the other PR.

Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Manciukic added a commit to Manciukic/firecracker that referenced this pull request Jul 9, 2025
Until firecracker-microvm#5299 is merged, this test will fail. Ignore PCI on this test for
now until we sort out the changes in the other PR.

Signed-off-by: Riccardo Mancini <mancio@amazon.com>
@Manciukic
Copy link
Contributor Author

Folding this over to #5300

@Manciukic Manciukic closed this Jul 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant