Skip to content

chore(docs): update benchmark results and add throughput tables #2905

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 17, 2025
Merged

chore(docs): update benchmark results and add throughput tables #2905

merged 1 commit into from
Oct 17, 2025

Conversation

soonum
Copy link
Contributor

@soonum soonum commented Oct 15, 2025

This also removes tables in PBS benchmarks for failure probability of 2**-40.

@soonum soonum self-assigned this Oct 15, 2025
@soonum soonum added the documentation Improvements or additions to documentation label Oct 15, 2025
@cla-bot cla-bot bot added the cla-signed label Oct 15, 2025
IceTDrinker
IceTDrinker reviewed Oct 15, 2025
View reviewed changes
Copy link
Member

@IceTDrinker IceTDrinker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CPU looks ok, just one thing on the multi bit PBS seems we gained a lot for the CPU is that expected ?

@soonum soonum force-pushed the dt/doc/update_benchmark_results branch from 1dd7fc3 to 0955fea Compare October 15, 2025 13:08
IceTDrinker
IceTDrinker approved these changes Oct 15, 2025
View reviewed changes
Copy link
Member

@IceTDrinker IceTDrinker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good for CPU, approving here to trigger CI,

I let @agnesLeroy and @pgardratzama check the latency for their devices :)

agnesLeroy
agnesLeroy reviewed Oct 15, 2025
View reviewed changes
Copy link
Contributor

@agnesLeroy agnesLeroy left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, only the GPU division latency needs an update still 🙏

agnesLeroy
agnesLeroy reviewed Oct 15, 2025
View reviewed changes
Copy link
Contributor

@agnesLeroy agnesLeroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait but I only see latency tables, we should also be adding the throughput tables for integer ops @soonum

@IceTDrinker
Copy link
Member

Wait but I only see latency tables, we should also be adding the throughput tables for integer ops @soonum

throughput has been delayed until we have everything for all devices, my understanding is we were missing some

@soonum soonum force-pushed the dt/doc/update_benchmark_results branch from 0955fea to 5b7d35d Compare October 15, 2025 15:34
@zama-bot zama-bot removed the approved label Oct 15, 2025
@soonum soonum requested a review from agnesLeroy October 15, 2025 15:34
@soonum soonum force-pushed the dt/doc/update_benchmark_results branch 2 times, most recently from 6832f86 to ed03dc9 Compare October 16, 2025 09:15
@soonum soonum force-pushed the dt/doc/update_benchmark_results branch from ed03dc9 to 54cb2b3 Compare October 16, 2025 12:43
@soonum soonum requested a review from pgardratzama October 16, 2025 12:44
agnesLeroy
agnesLeroy reviewed Oct 16, 2025
View reviewed changes
Copy link
Contributor

@agnesLeroy agnesLeroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@soonum for the GPU benchmarks I don't see scalar division in the tables, I'm sure we run it though 🤔

@IceTDrinker
Copy link
Member

will need a rebase after the value Agnès mentions is fixed @soonum :)

@soonum soonum force-pushed the dt/doc/update_benchmark_results branch from 54cb2b3 to 8264e56 Compare October 16, 2025 15:34
@zama-bot zama-bot removed the approved label Oct 16, 2025
@soonum
chore(docs): update benchmark results for all backends
de016e2 
This also removes tables in PBS benchmarks for failure probability
of 2**-40.
@soonum soonum force-pushed the dt/doc/update_benchmark_results branch from 8264e56 to de016e2 Compare October 16, 2025 15:37
@soonum soonum requested a review from agnesLeroy October 16, 2025 15:38
@agnesLeroy
Copy link
Contributor

@IceTDrinker I think we're good to merge?

@IceTDrinker
Copy link
Member

@IceTDrinker I think we're good to merge?

Yes

@IceTDrinker IceTDrinker merged commit 0e1082f into main Oct 17, 2025
134 of 135 checks passed
@IceTDrinker IceTDrinker deleted the dt/doc/update_benchmark_results branch October 17, 2025 07:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@agnesLeroy agnesLeroy agnesLeroy approved these changes

@IceTDrinker IceTDrinker IceTDrinker approved these changes

@pgardratzama pgardratzama pgardratzama approved these changes

Assignees

@soonum soonum

Labels

approved cla-signed documentation Improvements or additions to documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@soonum @IceTDrinker @agnesLeroy @pgardratzama @zama-bot