Skip to content

Commit 98c0fdb

Browse files
marcenacpThe TensorFlow Datasets Authors
authored andcommitted
Increase MAX_SIMULTANEOUS_DOWNLOADS to 100 for segment_anything.
PiperOrigin-RevId: 563683449
1 parent 9d1789c commit 98c0fdb

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

tensorflow_datasets/datasets/segment_anything/segment_anything_dataset_builder.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,9 @@ class Builder(tfds.core.GeneratorBasedBuilder):
7676
f'Download the links file from {_HOMEPAGE_URL}. `manual_dir` should'
7777
f' contain the links file saved as {_LOCAL_LINKS_FILE_NAME}.'
7878
)
79+
# The dataset is ~1000 * 10.5GB = ~10.5TB big, so we try to increase
80+
# parallelism to download. The default is 50.
81+
MAX_SIMULTANEOUS_DOWNLOADS = 100
7982

8083
def _info(self) -> tfds.core.DatasetInfo:
8184
"""Returns the dataset metadata."""

0 commit comments

Comments
 (0)