Skip to content

Commit a84fabe

Browse files
simsekhalitsimsekhalit
committed
Release v1.0.0
0 parents  commit a84fabe

File tree

9 files changed

+1135
-0
lines changed

9 files changed

+1135
-0
lines changed

‎.gitignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
*.egg-info/
2+
.idea
3+
__pycache__
4+
build/
5+
dist/
6+
venv

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2022 Halit Şimşek
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md‎

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
# Concurrent Sync
2+
A concurrent sync tool which works similar to `rsync`. It supports syncing given sources with multiple targets
3+
concurrently.
4+
5+
## Requirements
6+
Python >= 3.8 is required. (CPython and PyPy are both supported)
7+
8+
## Installation
9+
Concurrent Sync can be either installed directly via pip:
10+
```shell
11+
pip install concurrent-sync
12+
```
13+
Or it can be installed from the source:
14+
```shell
15+
git clone https://github.com/simsekhalit/concurrent-sync.git
16+
python3 -m pip install ./concurrent-sync
17+
```
18+
19+
## Manual
20+
```
21+
$ python3 -m csync --help
22+
usage: csync [-h] [--max-memory MAX_MEMORY] [--target TARGET] SOURCE [SOURCE ...] TARGET
23+
24+
A concurrent sync tool which works similar to rsync. It supports syncing given sources with multiple targets concurrently.
25+
26+
positional arguments:
27+
SOURCE specify source directories/files
28+
TARGET specify target directory
29+
30+
optional arguments:
31+
-h, --help show this help message and exit
32+
--max-memory MAX_MEMORY
33+
specify allowed max memory usage as percent
34+
--target TARGET specify additional target directories
35+
36+
For more information: https://github.com/simsekhalit/concurrent-sync
37+
```
38+
Concurrent Sync takes one or more source paths and one or more target paths as arguments.
39+
All the given source paths are synced with each given target path concurrently.
40+
* Only the missing or changed files are copied from source to target.
41+
* While checking if a file is changed, its modification time and size are used similar to `rsync`.
42+
* Trailing slash at the end of the source is interpreted in a similar way to `rsync`.
43+
44+
<br>Following **Examples** section clarifies the working mechanics in a more clear way.
45+
46+
## Examples
47+
### 1. One source path and one target path are given
48+
49+
#### Source
50+
```
51+
/mnt/Source
52+
/mnt/Source/File1
53+
/mnt/Source/File2
54+
/mnt/Source/Folder1
55+
/mnt/Source/Folder2
56+
/mnt/Source/Folder2/File3
57+
```
58+
59+
#### Target
60+
```
61+
/mnt/Target
62+
```
63+
64+
Following command is executed:
65+
```shell
66+
python3 -m csync /mnt/Source /mnt/Target
67+
```
68+
69+
After the sync is completed, target becomes:
70+
```
71+
/mnt/Target
72+
/mnt/Target/Source
73+
/mnt/Target/Source/File1
74+
/mnt/Target/Source/File2
75+
/mnt/Target/Source/Folder1
76+
/mnt/Target/Source/Folder2
77+
/mnt/Target/Source/Folder2/File3
78+
```
79+
80+
### 2. Two source paths and one target are given
81+
82+
#### Source 1
83+
```
84+
/mnt/Source1
85+
/mnt/Source1/File1
86+
/mnt/Source1/File2
87+
```
88+
89+
#### Source 2
90+
```
91+
/mnt/Source2
92+
/mnt/Source2/File3
93+
/mnt/Source2/File4
94+
```
95+
96+
#### Target
97+
```
98+
/mnt/Target
99+
```
100+
101+
Following command is executed:
102+
```shell
103+
python3 -m csync /mnt/Source1 /mnt/Source2 /mnt/Target
104+
```
105+
106+
After the sync is completed, target becomes:
107+
```
108+
/mnt/Target
109+
/mnt/Target/Source1
110+
/mnt/Target/Source1/File1
111+
/mnt/Target/Source1/File2
112+
/mnt/Target/Source2
113+
/mnt/Target/Source2/File3
114+
/mnt/Target/Source2/File4
115+
```
116+
117+
### 3. Source with trailing slash and target are given
118+
119+
#### Source
120+
```
121+
/mnt/Source
122+
/mnt/Source/File1
123+
/mnt/Source/File2
124+
```
125+
126+
#### Target
127+
```
128+
/mnt/Target
129+
```
130+
131+
Following command is executed:
132+
```shell
133+
python3 -m csync /mnt/Source/ /mnt/Target
134+
```
135+
136+
After the sync is completed, target becomes:
137+
```
138+
/mnt/Target
139+
/mnt/Target/File1
140+
/mnt/Target/File2
141+
```
142+
143+
### 4. Source and target with common paths
144+
While syncing subdirectories of source paths with target paths, redundant files/folders are removed.
145+
146+
#### Source
147+
```
148+
/mnt/Source
149+
/mnt/Source/Folder
150+
/mnt/Source/Folder/File1
151+
/mnt/Source/Folder/File2
152+
```
153+
154+
#### Target
155+
```
156+
/mnt/Target
157+
/mnt/Target/Source
158+
/mnt/Target/Source/Folder
159+
/mnt/Target/Source/Folder/File3
160+
```
161+
162+
Following command is executed:
163+
```shell
164+
python3 -m csync /mnt/Source /mnt/Target
165+
```
166+
167+
After the sync is completed, target becomes:
168+
```
169+
/mnt/Target
170+
/mnt/Target/Source
171+
/mnt/Target/Source/Folder
172+
/mnt/Target/Source/Folder/File1
173+
/mnt/Target/Source/Folder/File2
174+
```
175+
176+
Since `File3` is no longer in the source path it's deleted from the target as well.
177+
178+
### 5. One source path and two target paths are given
179+
180+
#### Source
181+
```
182+
/mnt/Source
183+
/mnt/Source/File1
184+
/mnt/Source/File2
185+
```
186+
187+
#### Target 1
188+
```
189+
/mnt/Target1
190+
```
191+
192+
#### Target 2
193+
```
194+
/mnt/Target2
195+
```
196+
197+
Following command is executed:
198+
```shell
199+
python3 -m csync /mnt/Source /mnt/Target1 --target /mnt/Target2
200+
```
201+
202+
After the sync is completed, targets become:
203+
204+
#### Target 1
205+
```
206+
/mnt/Target1
207+
/mnt/Target1/Source
208+
/mnt/Target1/Source/File1
209+
/mnt/Target1/Source/File2
210+
```
211+
212+
#### Target 2
213+
```
214+
/mnt/Target2
215+
/mnt/Target2/Source
216+
/mnt/Target2/Source/File1
217+
/mnt/Target2/Source/File2
218+
```

‎csync/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/usr/bin/env python3
2+
from .core import *
3+
4+
__version__ = "1.0.0"
5+
__version_info__ = (1, 0, 0)

‎csync/__main__.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
#!/usr/bin/env python3
2+
import sys
3+
4+
from .cli import main
5+
6+
main(sys.argv[1:])

‎csync/cli.py‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
#!/usr/bin/env python3
2+
import argparse
3+
import os
4+
import signal
5+
import sys
6+
from typing import List, Sequence
7+
8+
from .core import ConcurrentSync
9+
10+
__all__ = [
11+
"main",
12+
"parse_args",
13+
"setup_signals",
14+
]
15+
16+
17+
def setup_signals(concurrent_sync: ConcurrentSync) -> None:
18+
def handler(_, __):
19+
concurrent_sync.terminate()
20+
21+
for s in (signal.SIGHUP, signal.SIGINT, signal.SIGTERM):
22+
signal.signal(s, handler)
23+
24+
25+
def _validate_targets(targets: List[str]) -> None:
26+
for target in targets:
27+
abs_target = os.path.abspath(target)
28+
if not os.path.exists(abs_target):
29+
print(f"'{target}' does not exist!", file=sys.stderr)
30+
sys.exit(2)
31+
32+
33+
def _prepare_sources(sources: List[str]) -> List[str]:
34+
result = []
35+
for src in sources:
36+
abs_src = os.path.abspath(src)
37+
if not os.path.exists(abs_src):
38+
print(f"'{src}' does not exist!", file=sys.stderr)
39+
sys.exit(2)
40+
41+
if src.endswith(os.sep):
42+
if os.path.isdir(abs_src):
43+
children = os.listdir(abs_src)
44+
result += (os.path.join(abs_src, c) for c in children)
45+
else:
46+
print(f"'{src}' is not a directory!", file=sys.stderr)
47+
sys.exit(2)
48+
else:
49+
result.append(abs_src)
50+
51+
return result
52+
53+
54+
def parse_args(args: Sequence[str]) -> argparse.Namespace:
55+
description = ("A concurrent sync tool which works similar to rsync. "
56+
"It supports syncing given sources with multiple targets concurrently.")
57+
epilog = "For more information: https://github.com/simsekhalit/concurrent-sync"
58+
parser = argparse.ArgumentParser("csync", description=description, epilog=epilog)
59+
parser.add_argument("--max-memory", type=int, help="specify allowed max memory usage as percent")
60+
parser.add_argument("sources", nargs="+", help="specify source directories/files", metavar="SOURCE")
61+
parser.add_argument("targets", action="append", help="specify target directory", metavar="TARGET")
62+
parser.add_argument("--target", action="append", help="specify additional target directories", metavar="TARGET",
63+
dest="targets")
64+
args = parser.parse_args(args)
65+
66+
args.sources = _prepare_sources(args.sources)
67+
_validate_targets(args.targets)
68+
69+
return args
70+
71+
72+
def main(args: Sequence[str]) -> None:
73+
args = parse_args(args)
74+
concurrent_sync = ConcurrentSync(args.sources, args.targets, max_memory=args.max_memory)
75+
setup_signals(concurrent_sync)
76+
concurrent_sync.run()
77+
78+
79+
if __name__ == "__main__":
80+
main(sys.argv[1:])

0 commit comments

Comments
 (0)