Skip to content

Commit 47a97ec

Browse files
authored
Skip storage scan in dataset upload if disabled for datastore (#8733)
### Steps to test: - For datastore with `reportUsedStorage.enabled = false` (as in sample setup), upload a dataset, no storage scan should be performed for that dataset (check that no values show up in db). ### Issues: - fixes https://scm.slack.com/archives/C02H5T8Q08P/p1751294200593839 ------ - [x] Added changelog entry (create a `$PR_NUMBER.md` file in `unreleased_changes` or use `./tools/create-changelog-entry.py`) - [x] Considered [common edge cases](../blob/master/.github/common_edge_cases.md) - [x] Needs datastore update after deployment
1 parent 2ba04c5 commit 47a97ec

File tree

2 files changed

+11
-5
lines changed

2 files changed

+11
-5
lines changed

app/models/storage/UsedStorageService.scala

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -89,11 +89,15 @@ class UsedStorageService @Inject()(val actorSystem: ActorSystem,
8989
def refreshStorageReportForDataset(dataset: Dataset): Fox[Unit] =
9090
for {
9191
dataStore <- datasetService.dataStoreFor(dataset)
92-
dataStoreClient = new WKRemoteDataStoreClient(dataStore, rpc)
93-
organization <- organizationDAO.findOne(dataset._organization)
94-
report <- dataStoreClient.fetchStorageReport(organization._id, Some(dataset.name))
95-
_ <- organizationDAO.deleteUsedStorageForDataset(dataset._id)
96-
_ <- organizationDAO.upsertUsedStorage(organization._id, dataStore.name, report)
92+
_ <- if (dataStore.reportUsedStorageEnabled) {
93+
val dataStoreClient = new WKRemoteDataStoreClient(dataStore, rpc)
94+
for {
95+
organization <- organizationDAO.findOne(dataset._organization)
96+
report <- dataStoreClient.fetchStorageReport(organization._id, Some(dataset.name))
97+
_ <- organizationDAO.deleteUsedStorageForDataset(dataset._id)
98+
_ <- organizationDAO.upsertUsedStorage(organization._id, dataStore.name, report)
99+
} yield ()
100+
} else Fox.successful(())
97101
} yield ()
98102

99103
}

unreleased_changes/8733.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
### Fixed
2+
- Fixed that on dataset upload the storage usage would get scanned even if it is disabled in config for that particular data store server.

0 commit comments

Comments
 (0)