Skip to content

DO NOT MERGE: Performance audit #121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: dev
Choose a base branch
from
Draft

DO NOT MERGE: Performance audit #121

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
be1ec7d
chore: log timing
thelostone-mc Apr 21, 2025
1f51202
chore: fix
thelostone-mc Apr 21, 2025
e66660f
chore: add custom logging just for Allocation
thelostone-mc Apr 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
392 changes: 196 additions & 196 deletions apps/indexer/config.yaml
View file
Open in desktop

Large diffs are not rendered by default.

25 changes: 25 additions & 0 deletions packages/data-flow/src/data-loader/handlers/application.handlers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { ApplicationChangeset, IApplicationRepository } from "@grants-stack-indexer/repository";
import { performanceLogger } from "@grants-stack-indexer/shared";

import { ChangesetHandler } from "../types/index.js";

Expand Down Expand Up @@ -33,11 +34,35 @@ export const createApplicationHandlers = (
}) satisfies ChangesetHandler<"UpdateApplication">,

IncrementApplicationDonationStats: (async (changeset, txConnection): Promise<void> => {
const startTime = performance.now();
const { chainId, roundId, applicationId, amountInUsd } = changeset.args;
await repository.incrementApplicationDonationStats(
{ chainId, roundId, id: applicationId },
amountInUsd,
txConnection,
);
const endTime = performance.now();
const duration = endTime - startTime;

Comment on lines +37 to +46
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Import Node’s high‑resolution timer explicitly

performance.now() is used, but no import { performance } from "node:perf_hooks" (or "perf_hooks") is present.
Relying on the global may break type‑checking in strict projects and older Node versions.

+import { performance } from "node:perf_hooks";
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const startTime = performance.now();
const { chainId, roundId, applicationId, amountInUsd } = changeset.args;
await repository.incrementApplicationDonationStats(
{ chainId, roundId, id: applicationId },
amountInUsd,
txConnection,
);
const endTime = performance.now();
const duration = endTime - startTime;
+import { performance } from "node:perf_hooks";
const startTime = performance.now();
const { chainId, roundId, applicationId, amountInUsd } = changeset.args;
await repository.incrementApplicationDonationStats(
{ chainId, roundId, id: applicationId },
amountInUsd,
txConnection,
);
const endTime = performance.now();
const duration = endTime - startTime;

// Get current application stats for logging
const application = await repository.getApplicationById(applicationId, chainId, roundId);

performanceLogger.logMetric({
timestamp: new Date().toISOString(),
eventType: "Application",
operation: "IncrementApplicationDonationStats",
duration,
totalTime: duration,
chainId,
roundId,
applicationId,
amountInUsd,
uniqueDonorsCount: application?.uniqueDonorsCount,
totalDonationsCount: application?.totalDonationsCount,
details: {
totalAmountDonatedInUsd: application?.totalAmountDonatedInUsd,
status: application?.status,
},
});
Comment on lines +47 to +66
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Extra DB round‑trip could negate the performance gain

Immediately after incrementApplicationDonationStats you fetch the application again to log stats.
That is an additional SELECT per donation:

const application = await repository.getApplicationById(...);

If the repository method that increments stats can return the updated row (e.g. UPDATE … RETURNING * in SQL), you avoid the second query entirely.

}) satisfies ChangesetHandler<"IncrementApplicationDonationStats">,
});
49 changes: 49 additions & 0 deletions packages/data-flow/src/data-loader/handlers/donation.handlers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { DonationChangeset, IDonationRepository } from "@grants-stack-indexer/repository";
import { performanceLogger } from "@grants-stack-indexer/shared";

import { ChangesetHandler } from "../types/index.js";

Expand All @@ -18,10 +19,58 @@ export type DonationHandlers = {
*/
export const createDonationHandlers = (repository: IDonationRepository): DonationHandlers => ({
InsertDonation: (async (changeset, txConnection): Promise<void> => {
const startTime = performance.now();
await repository.insertDonation(changeset.args.donation, txConnection);
const endTime = performance.now();
const duration = endTime - startTime;

performanceLogger.logMetric({
timestamp: new Date().toISOString(),
eventType: "Donation",
operation: "InsertDonation",
duration,
totalTime: duration,
blockNumber: Number(changeset.args.donation.blockNumber),
Comment on lines +22 to +33
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Same timer import issue as above

Add an explicit import of performance from "node:perf_hooks" here as well.

transactionHash: changeset.args.donation.transactionHash,
chainId: changeset.args.donation.chainId,
roundId: changeset.args.donation.roundId,
applicationId: changeset.args.donation.applicationId || undefined,
donorAddress: changeset.args.donation.donorAddress,
recipientAddress: changeset.args.donation.recipientAddress,
amount: changeset.args.donation.amount.toString(),
amountInUsd: changeset.args.donation.amountInUsd,
details: {
tokenAddress: changeset.args.donation.tokenAddress,
amountInRoundMatchToken: changeset.args.donation.amountInRoundMatchToken.toString(),
},
});
Comment on lines +40 to +46
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Potential loss of precision when logging amount

donation.amount is converted with .toString() ✔️,
but amountInUsd is coerced to number in later code. Large values or high‑precision decimals may be truncated:

-amountInUsd: changeset.args.donation.amountInUsd,
+amountInUsd: changeset.args.donation.amountInUsd?.toString(),

The same applies in InsertManyDonations where a Number() cast is used inside reduce.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
amount: changeset.args.donation.amount.toString(),
amountInUsd: changeset.args.donation.amountInUsd,
details: {
tokenAddress: changeset.args.donation.tokenAddress,
amountInRoundMatchToken: changeset.args.donation.amountInRoundMatchToken.toString(),
},
});
// … other code …
someLogger.log({
amount: changeset.args.donation.amount.toString(),
- amountInUsd: changeset.args.donation.amountInUsd,
+ amountInUsd: changeset.args.donation.amountInUsd?.toString(),
details: {
tokenAddress: changeset.args.donation.tokenAddress,
amountInRoundMatchToken: changeset.args.donation.amountInRoundMatchToken.toString(),
},
});
// … other code …

}) satisfies ChangesetHandler<"InsertDonation">,

InsertManyDonations: (async (changeset, txConnection): Promise<void> => {
const startTime = performance.now();
await repository.insertManyDonations(changeset.args.donations, txConnection);
const endTime = performance.now();
const duration = endTime - startTime;

const firstDonation = changeset.args.donations[0];
performanceLogger.logMetric({
timestamp: new Date().toISOString(),
eventType: "Donation",
operation: "InsertManyDonations",
duration,
totalTime: duration,
blockNumber: firstDonation ? Number(firstDonation.blockNumber) : undefined,
transactionHash: firstDonation?.transactionHash,
chainId: firstDonation?.chainId,
roundId: firstDonation?.roundId,
amount: firstDonation ? firstDonation.amount.toString() : undefined,
amountInUsd: firstDonation?.amountInUsd,
details: {
count: changeset.args.donations.length,
totalAmountInUsd: changeset.args.donations
.reduce((sum, d) => sum + Number(d.amountInUsd), 0)
.toString(),
},
});
}) satisfies ChangesetHandler<"InsertManyDonations">,
});
23 changes: 23 additions & 0 deletions packages/data-flow/src/data-loader/handlers/round.handlers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { IRoundRepository, RoundChangeset } from "@grants-stack-indexer/repository";
import { performanceLogger } from "@grants-stack-indexer/shared";

import { ChangesetHandler } from "../types/index.js";

Expand Down Expand Up @@ -52,6 +53,7 @@ export const createRoundHandlers = (repository: IRoundRepository): RoundHandlers
}) satisfies ChangesetHandler<"IncrementRoundFundedAmount">,

IncrementRoundDonationStats: (async (changeset, txConnection): Promise<void> => {
const startTime = performance.now();
const { chainId, roundId, amountInUsd } = changeset.args;
await repository.incrementRoundDonationStats(
{
Expand All @@ -61,6 +63,27 @@ export const createRoundHandlers = (repository: IRoundRepository): RoundHandlers
amountInUsd,
txConnection,
);
const endTime = performance.now();
const duration = endTime - startTime;

// Get current round stats for logging
const round = await repository.getRoundById(chainId, roundId);

performanceLogger.logMetric({
timestamp: new Date().toISOString(),
eventType: "Round",
operation: "IncrementRoundDonationStats",
duration,
totalTime: duration,
chainId,
roundId,
amountInUsd,
uniqueDonorsCount: round?.uniqueDonorsCount,
totalDonationsCount: round?.totalDonationsCount,
details: {
totalAmountDonatedInUsd: round?.totalAmountDonatedInUsd,
},
});
}) satisfies ChangesetHandler<"IncrementRoundDonationStats">,

IncrementRoundTotalDistributed: (async (changeset, txConnection): Promise<void> => {
Expand Down
Loading
Loading