Skip to content

Commit 4a25ecc

Browse files
iago1460iago1460
committed
Exposing format-pattern and unknown-format-pattern as CLI parameters
1 parent c33100b commit 4a25ecc

File tree

6 files changed

+100
-48
lines changed

6 files changed

+100
-48
lines changed

‎CHANGELOG.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Changelog
2+
3+
## [v2.1] - 2023-03-12
4+
5+
### Features
6+
* Duplication order now uses path length when file names have the same length
7+
* Exposing format-pattern and unknown-format-pattern as CLI parameters
8+
9+
### Bug Fixes
10+
* Fixed report when deleting duplicate files
11+
* Ordering by file length too on the duplicate report given by the inspect command to make behaviour consistent
12+
13+
14+
## [v2.0.1] - 2023-03-04
15+
### Features
16+
17+
* breaking change - Complete rework of the application

‎README.md

Lines changed: 39 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,11 @@ This tool helps you to unify and organise your media files using your own rules.
1212
It also deals with duplicates, so you don't have to.
1313

1414

15+
## Disclaimer
16+
17+
Per design this command line interface tool deletes only duplicate files to potentially avoid any risk of losing data.
18+
19+
1520
## Features
1621

1722
* Move, Copy or delete duplicates operations
@@ -34,26 +39,28 @@ It also deals with duplicates, so you don't have to.
3439

3540
```bash
3641
$ cataloguer --help
37-
38-
Usage: cataloguer [OPTIONS] COMMAND [ARGS]...
39-
40-
Command line interface.
41-
All [OPTIONS] can be passed as environment variables with the "CATALOGUER_" prefix.
42-
file arguments accept file names and a special value "-" to indicate stdin or stdout
43-
44-
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
45-
│ --verbose -v Enables verbose mode. Disabled by default │
46-
│ --interactive/--no-interactive Disables confirmation prompts. Enabled by default │
47-
│ --help Show this message and exit. │
48-
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
49-
╭─ Commands ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
50-
│ copy Copy files. In case of duplicates will take the shortest name. │
51-
│ create-catalogue Creates a new catalogue. │
52-
│ delete-catalogue Deletes a catalogue. No files are affected. │
53-
│ delete-duplicates Delete duplicates. │
54-
│ inspect Inspects a path or a catalogue │
55-
│ move Move files. In case of duplicates will take the shortest name. │
56-
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
42+
43+
Usage: cataloguer [OPTIONS] COMMAND [ARGS]...
44+
45+
Command line interface.
46+
All [OPTIONS] can be passed as environment variables with the "CATALOGUER_" prefix.
47+
file arguments accept file names and a special value "-" to indicate stdin or stdout
48+
49+
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
50+
│ --verbose -v Enables verbose mode. Disabled by default │
51+
│ --format-pattern TEXT Pattern template. e.g. %Y/%m/{file} │
52+
│ --unknown-format-pattern TEXT Pattern template fallback when date cannot get extracted │
53+
│ --interactive/--no-interactive Disables confirmation prompts. Enabled by default │
54+
│ --help Show this message and exit. │
55+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
56+
╭─ Commands ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
57+
│ copy Copy files. In case of duplicates will take the shortest name. │
58+
│ create-catalogue Creates a new catalogue. │
59+
│ delete-catalogue Deletes a catalogue. No files are affected. │
60+
│ delete-duplicates Delete duplicates. │
61+
│ inspect Inspects a path or a catalogue │
62+
│ move Move files. In case of duplicates will take the shortest name. │
63+
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
5764
```
5865
5966
@@ -66,11 +73,10 @@ We are going to start creating a new directory `media`:
6673
We are going to create a new catalogue named `local_photos` which is going to get store on the `media` directory.
6774
We specify our format pattern so photos are group by `year` and a subgroup of `month`:
6875
69-
export CATALOGUER_FORMAT_PATTERN=%Y/%m/{file}
70-
cataloguer create-catalogue local_media --path ./media
76+
cataloguer create-catalogue local_media ./media --format-pattern %Y/%m/{file}
7177
7278
73-
Now, we add some photos from an old storage driver:
79+
Now, we add some photos from an old storage drive:
7480
7581
cataloguer copy /mnt/hdd/old_photos local_media
7682
@@ -92,7 +98,7 @@ To get a summary of our catalogue we run:
9298
9399
## Options
94100
95-
`CATALOGUER_FORMAT_PATTERN` accepts the following patterns
101+
`format-pattern` accepts the following patterns
96102
* Common date codes:
97103
* `%d`: Day of the month as number
98104
* `%m`: Month as number
@@ -103,26 +109,29 @@ To get a summary of our catalogue we run:
103109
* `/` Specifies a new folder
104110
* `{media_type}`: File type (`image`, `video`)
105111
* `{media_format}`: File format (`jpeg`, `tiff`, `png`, `gif`, `mp4` ...)
106-
* `{file}` Original filename
107-
* `{file_extension}` Original filename extension
108-
* `{file_name}` Original filename without the extension
112+
* `{file}` Original filename (`photo.jpg`)
113+
* `{file_extension}` filename extension (`photo`)
114+
* `{file_name}` filename without the extension (`jpg`)
109115
* `{relative_path}` Relative path to the source directory
110116
111117
112118
### Advance usage:
113-
114-
`CATALOGUER_UNKNOWN_PATH_FORMAT` Accepts the same variables as `CATALOGUER_FORMAT_PATTERN` but date patterns
119+
`unknown-format-pattern` Accepts the same variables as `format-pattern` but date patterns
115120
are resolved using the current date since it was not possible to recover the creation date of the file.
116121
This can be useful to not leave files behind.
117122
123+
Variables can also be specified as environment variables, using a `CATALOGUER_` prefix. e.g:
124+
125+
export CATALOGUER_FORMAT_PATTERN=%Y/%m/{file}
126+
118127
`CATALOGUER_STORAGE_LOCATION` Accepts any path. That location will store cataloguer metadata.
119128
By default, it will create a `.catalogues` in the user's home directory.
120129
121130
#### Examples:
122131
123132
Pattern to fix file extensions keeping the folder structure:
124133
125-
CATALOGUER_FORMAT_PATTERN={relative_path}/{basename}.{media_format} cataloguer ./input ./output
134+
cataloguer --format-pattern {relative_path}/{basename}.{media_format} move ./input ./output
126135
127136
128137
## TODO list

‎cataloguer/cli.py

Lines changed: 41 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -47,13 +47,15 @@ def __str__(self):
4747
help="Enables verbose mode. Disabled by default",
4848
default=False,
4949
)
50+
@click.option("--format-pattern", help='Pattern template. e.g. %Y/%m/{file}', required=False)
51+
@click.option("--unknown-format-pattern", help='Pattern template fallback when date cannot get extracted', required=False)
5052
@click.option(
5153
"--interactive/--no-interactive",
5254
help="Disables confirmation prompts. Enabled by default",
5355
default=True,
5456
)
5557
@click.pass_context
56-
def cli(ctx, verbose, interactive):
58+
def cli(ctx, verbose, interactive, format_pattern, unknown_format_pattern):
5759
"""
5860
Command line interface.
5961
@@ -62,7 +64,7 @@ def cli(ctx, verbose, interactive):
6264
file arguments accept file names and a special value "-" to indicate stdin or stdout
6365
"""
6466
if not ctx.obj:
65-
global_settings = GlobalSettings()
67+
global_settings = GlobalSettings(format_pattern=format_pattern, unknown_format_pattern=unknown_format_pattern)
6668
ctx.obj = Context(
6769
global_settings=global_settings,
6870
storage=Storage(path=global_settings.storage_location),
@@ -110,8 +112,12 @@ def inspect(ctx: Context, src, media_only):
110112
print_table_summary(name=name, files=files, duplicated_files=duplicated_files)
111113

112114
if duplicated_files:
115+
duplicated_list_of_files_sorted_by_name_length = list(
116+
sorted(duplicated_list, key=lambda file: (len(file.path.name), len(str(file.path))))
117+
for duplicated_list in duplicated_files
118+
)
113119
print_duplicate_files(
114-
duplicated_files=duplicated_files, from_path=directory.path
120+
duplicated_files=duplicated_list_of_files_sorted_by_name_length, from_path=directory.path
115121
)
116122

117123
if isinstance(directory, Catalogue):
@@ -143,8 +149,10 @@ def delete_catalogue(ctx: Context, name):
143149
@cli.command()
144150
@click.argument("name")
145151
@click.argument("src", required=False)
152+
@click.option("--format-pattern", help='Pattern template. e.g. %Y/%m/{file}', required=False)
153+
@click.option("--unknown-format-pattern", help='Pattern template fallback when date cannot get extracted', required=False)
146154
@click.pass_obj
147-
def create_catalogue(ctx: Context, name, src):
155+
def create_catalogue(ctx: Context, name, src, format_pattern, unknown_format_pattern):
148156
"""
149157
Creates a new catalogue.
150158
"""
@@ -155,6 +163,13 @@ def create_catalogue(ctx: Context, name, src):
155163
if not src_path or not src_path.is_dir():
156164
raise click.BadParameter(f'Error "{src}" is not an existing path')
157165

166+
format_pattern = format_pattern or ctx.global_settings.format_pattern
167+
unknown_format_pattern = unknown_format_pattern or ctx.global_settings.unknown_format_pattern
168+
if not format_pattern:
169+
raise click.BadParameter(
170+
'Error there is no format pattern specified'
171+
)
172+
158173
catalogue_path = src_path or ctx.workdir
159174
existing_catalogue = ctx.storage.load_catalogue(name)
160175

@@ -165,8 +180,8 @@ def create_catalogue(ctx: Context, name, src):
165180

166181
new_catalogue = Catalogue(
167182
name=name,
168-
format_pattern=ctx.global_settings.format_pattern,
169-
unknown_format_pattern=ctx.global_settings.unknown_format_pattern,
183+
format_pattern=format_pattern,
184+
unknown_format_pattern=unknown_format_pattern,
170185
path=catalogue_path,
171186
)
172187
new_catalogue.explore()
@@ -261,7 +276,7 @@ def extract_files(src_data):
261276

262277
if duplicated_list_of_files:
263278
duplicated_list_of_files_sorted_by_name_length = list(
264-
sorted(duplicated_list, key=lambda file: len(file.path.name))
279+
sorted(duplicated_list, key=lambda file: (len(file.path.name), len(str(file.path))))
265280
for duplicated_list in duplicated_list_of_files
266281
)
267282

@@ -325,16 +340,29 @@ def operate(ctx, src, dst, operation_mode, dry_run=False):
325340
f'Error "{dst}" is neither a catalogue or an existing directory'
326341
)
327342

328-
if isinstance(src_data, File) and (
329-
not dst_data or not isinstance(dst_data, (Catalogue, Directory))
330-
):
343+
if isinstance(src_data, (Catalogue, Directory)) and dst_data:
344+
if src_data.path.is_relative_to(dst_data.path):
345+
raise click.BadParameter(
346+
f'Error "{dst}" cannot be a subdirectory of {src}'
347+
)
348+
349+
if operation_mode in (Operation.MOVE, Operation.COPY):
350+
format_pattern = ctx.global_settings.format_pattern
351+
if dst_data and isinstance(dst_data, Catalogue):
352+
format_pattern = format_pattern or dst_data.format_pattern
353+
if not format_pattern:
354+
raise click.BadParameter(
355+
'Error there is no format pattern specified'
356+
)
357+
358+
if isinstance(src_data, File) and not dst_data:
331359
raise click.BadParameter(
332360
f'Error "{src}" is a file but no valid destination was provided'
333361
)
334362

335363
with console.status(
336364
f"[green]Inspecting files...",
337-
) as status:
365+
):
338366
(
339367
duplicated_list_of_files_sorted_by_name_length,
340368
duplicated_discarded_files,
@@ -447,8 +475,8 @@ def process_files(
447475
path_format = ctx.global_settings.format_pattern
448476
unknown_format_pattern = ctx.global_settings.unknown_format_pattern
449477
if isinstance(dst_data, Catalogue):
450-
path_format = dst_data.format_pattern
451-
unknown_format_pattern = dst_data.unknown_format_pattern
478+
path_format = path_format or dst_data.format_pattern
479+
unknown_format_pattern = unknown_format_pattern or dst_data.unknown_format_pattern
452480

453481
tree = DirectoryTree()
454482
skipped_tree = DirectoryTree()

0 commit comments

Comments
 (0)