Skip to content

Commit bf8dad3

Browse files
atollkatollk
authored
Merge branch 'master' into issue_459_371
2 parents 9e628ee + f6a6195 commit bf8dad3

36 files changed

+1360
-338
lines changed

‎.github/workflows/package.yml

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
name: Package
2+
3+
on:
4+
push:
5+
tags:
6+
- 'v2.*'
7+
8+
jobs:
9+
10+
build-wheel:
11+
runs-on: ubuntu-latest
12+
name: Build wheel distribution
13+
steps:
14+
- name: Checkout code
15+
uses: actions/checkout@v2
16+
with:
17+
submodules: true
18+
- name: Setup Python ${{ matrix.python-version }}
19+
uses: actions/setup-python@v2
20+
with:
21+
python-version: ${{ matrix.python-version }}
22+
- name: Update build dependencies
23+
run: python -m pip install -U pip wheel setuptools
24+
- name: Build wheel distribution
25+
run: python setup.py bdist_wheel
26+
- name: Store built wheel
27+
uses: actions/upload-artifact@v2
28+
with:
29+
name: dist
30+
path: dist/*
31+
32+
build-sdist:
33+
runs-on: ubuntu-latest
34+
name: Build source distribution
35+
steps:
36+
- name: Checkout code
37+
uses: actions/checkout@v2
38+
with:
39+
submodules: true
40+
- name: Set up Python 3.9
41+
uses: actions/setup-python@v2
42+
with:
43+
python-version: 3.9
44+
- name: Update build dependencies
45+
run: python -m pip install -U pip wheel setuptools
46+
- name: Build source distribution
47+
run: python setup.py sdist
48+
- name: Store source distribution
49+
uses: actions/upload-artifact@v2
50+
with:
51+
name: dist
52+
path: dist/*
53+
54+
test-sdist:
55+
runs-on: ubuntu-latest
56+
name: Test source distribution
57+
needs:
58+
- build-sdist
59+
steps:
60+
- name: Checkout code
61+
uses: actions/checkout@v2
62+
with:
63+
submodules: true
64+
- name: Setup Python 3.9
65+
uses: actions/setup-python@v2
66+
with:
67+
python-version: 3.9
68+
- name: Download source distribution
69+
uses: actions/download-artifact@v2
70+
with:
71+
name: dist
72+
path: dist
73+
- name: Install source distribution
74+
run: python -m pip install dist/fs-*.tar.gz
75+
- name: Remove source code
76+
run: rm -rvd fs
77+
- name: Install test requirements
78+
run: python -m pip install -r tests/requirements.txt
79+
- name: Test installed package
80+
run: python -m unittest discover -vv
81+
82+
test-wheel:
83+
runs-on: ubuntu-latest
84+
name: Test wheel distribution
85+
needs:
86+
- build-wheel
87+
steps:
88+
- name: Checkout code
89+
uses: actions/checkout@v2
90+
with:
91+
submodules: true
92+
- name: Setup Python 3.9
93+
uses: actions/setup-python@v2
94+
with:
95+
python-version: 3.9
96+
- name: Download wheel distribution
97+
uses: actions/download-artifact@v2
98+
with:
99+
name: dist
100+
path: dist
101+
- name: Install wheel distribution
102+
run: python -m pip install dist/fs-*.whl
103+
- name: Remove source code
104+
run: rm -rvd fs
105+
- name: Install test requirements
106+
run: python -m pip install -r tests/requirements.txt
107+
- name: Test installed package
108+
run: python -m unittest discover -vv
109+
110+
upload:
111+
environment: PyPI
112+
runs-on: ubuntu-latest
113+
name: Upload
114+
needs:
115+
- build-sdist
116+
- build-wheel
117+
- test-sdist
118+
- test-wheel
119+
steps:
120+
- name: Download built distributions
121+
uses: actions/download-artifact@v2
122+
with:
123+
name: dist
124+
path: dist
125+
- name: Publish distributions to PyPI
126+
if: startsWith(github.ref, 'refs/tags/v')
127+
uses: pypa/gh-action-pypi-publish@master
128+
with:
129+
user: __token__
130+
password: ${{ secrets.PYPI_API_TOKEN }}
131+
skip_existing: false

‎.github/workflows/test.yml

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -32,20 +32,36 @@ jobs:
3232
run: python -m pip install tox tox-gh-actions
3333
- name: Test with tox
3434
run: python -m tox
35-
- name: Collect coverage results
36-
uses: AndreMiras/coveralls-python-action@develop
35+
- name: Store partial coverage reports
36+
uses: actions/upload-artifact@v2
3737
with:
38-
parallel: true
39-
flag-name: test (${{ matrix.python-version}})
38+
name: coverage
39+
path: .coverage.*
4040

4141
coveralls:
4242
needs: test
4343
runs-on: ubuntu-latest
4444
steps:
45+
- name: Checkout code
46+
uses: actions/checkout@v1
47+
- name: Setup Python 3.9
48+
uses: actions/setup-python@v2
49+
with:
50+
python-version: 3.9
51+
- name: Install coverage package
52+
run: python -m pip install -U coverage
53+
- name: Download partial coverage reports
54+
uses: actions/download-artifact@v2
55+
with:
56+
name: coverage
57+
- name: Combine coverage
58+
run: python -m coverage combine
59+
- name: Report coverage
60+
run: python -m coverage report
61+
- name: Export coverage to XML
62+
run: python -m coverage xml
4563
- name: Upload coverage statistics to Coveralls
4664
uses: AndreMiras/coveralls-python-action@develop
47-
with:
48-
parallel-finished: true
4965

5066
lint:
5167
runs-on: ubuntu-latest

‎CHANGELOG.md

Lines changed: 36 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,31 +10,62 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
1010

1111
### Added
1212

13+
- To `fs.walk.Walker`, added parameters `filter_glob` and `exclude_glob`.
14+
Closes [#459](https://github.com/PyFilesystem/pyfilesystem2/issues/459).
15+
16+
### Fixed
17+
18+
- Elaborated documentation of `filter_dirs` and `exclude_dirs` in `fs.walk.Walker`.
19+
Closes [#371](https://github.com/PyFilesystem/pyfilesystem2/issues/371).
20+
21+
22+
## [2.4.13] - 2021-03-27
23+
24+
### Added
25+
1326
- Added FTP over TLS (FTPS) support to FTPFS.
1427
Closes [#437](https://github.com/PyFilesystem/pyfilesystem2/issues/437),
1528
[#449](https://github.com/PyFilesystem/pyfilesystem2/pull/449).
16-
- To `fs.walk.Walker`, added parameters `filter_glob` and `exclude_glob`.
17-
Closes [#459](https://github.com/PyFilesystem/pyfilesystem2/issues/459).
29+
- `PathError` now supports wrapping an exception using the `exc` argument.
30+
Closes [#453](https://github.com/PyFilesystem/pyfilesystem2/issues/453).
31+
- Better documentation of the `writable` parameter of `fs.open_fs`, and
32+
hint about using `fs.wrap.read_only` when a read-only filesystem is
33+
required. Closes [#441](https://github.com/PyFilesystem/pyfilesystem2/issues/441).
1834

1935
### Changed
2036

2137
- Make `FS.upload` explicit about the expected error when the parent directory of the destination does not exist.
2238
Closes [#445](https://github.com/PyFilesystem/pyfilesystem2/pull/445).
2339
- Migrate continuous integration from Travis-CI to GitHub Actions and introduce several linters
2440
again in the build steps ([#448](https://github.com/PyFilesystem/pyfilesystem2/pull/448)).
25-
Closes [#446](https://github.com/PyFilesystem/pyfilesystem2/pull/446).
41+
Closes [#446](https://github.com/PyFilesystem/pyfilesystem2/issues/446).
2642
- Stop requiring `pytest` to run tests, allowing any test runner supporting `unittest`-style
2743
test suites.
2844
- `FSTestCases` now builds the large data required for `upload` and `download` tests only
2945
once in order to reduce the total testing time.
46+
- `MemoryFS.move` and `MemoryFS.movedir` will now avoid copying data.
47+
Closes [#452](https://github.com/PyFilesystem/pyfilesystem2/issues/452).
48+
- `FS.removetree("/")` behaviour has been standardized in all filesystems, and
49+
is expected to clear the contents of the root folder without deleting it.
50+
Closes [#471](https://github.com/PyFilesystem/pyfilesystem2/issues/471).
51+
- `FS.getbasic` is now deprecated, as it is redundant with `FS.getinfo`,
52+
and `FS.getinfo` is now explicitly expected to return the *basic* info
53+
namespace unconditionally. Closes [#469](https://github.com/PyFilesystem/pyfilesystem2/issues/469).
3054

3155
### Fixed
3256

3357
- Make `FTPFile`, `MemoryFile` and `RawWrapper` accept [`array.array`](https://docs.python.org/3/library/array.html)
3458
arguments for the `write` and `writelines` methods, as expected by their base class [`io.RawIOBase`](https://docs.python.org/3/library/io.html#io.RawIOBase).
3559
- Various documentation issues, including `MemoryFS` docstring not rendering properly.
36-
- Elaborated documentation of `filter_dirs` and `exclude_dirs` in `fs.walk.Walker`.
37-
Closes [#371](https://github.com/PyFilesystem/pyfilesystem2/issues/371).
60+
- Avoid creating a new connection on every call of `FTPFS.upload`. Closes [#455](https://github.com/PyFilesystem/pyfilesystem2/issues/455).
61+
- `WrapReadOnly.removetree` not raising a `ResourceReadOnly` when called. Closes [#468](https://github.com/PyFilesystem/pyfilesystem2/issues/468).
62+
- `WrapCachedDir.isdir` and `WrapCachedDir.isfile` raising a `ResourceNotFound` error on non-existing path ([#470](https://github.com/PyFilesystem/pyfilesystem2/pull/470)).
63+
- `FTPFS` not listing certain entries with sticky/SUID/SGID permissions set by Linux server ([#473](https://github.com/PyFilesystem/pyfilesystem2/pull/473)).
64+
Closes [#451](https://github.com/PyFilesystem/pyfilesystem2/issues/451).
65+
- `scandir` iterator not being closed explicitly in `OSFS.scandir`, occasionally causing a `ResourceWarning`
66+
to be thrown. Closes [#311](https://github.com/PyFilesystem/pyfilesystem2/issues/311).
67+
- Incomplete type annotations for the `temp_fs` parameter of `WriteTarFS` and `WriteZipFS`.
68+
Closes [#410](https://github.com/PyFilesystem/pyfilesystem2/issues/410).
3869

3970

4071
## [2.4.12] - 2021-01-14

‎appveyor.yml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,12 @@ environment:
2020

2121
install:
2222
# We need wheel installed to build wheels
23-
- "%PYTHON%\\python.exe -m pip install pytest pytest-randomly pytest-cov psutil pyftpdlib mock"
23+
- "%PYTHON%\\python.exe -m pip install -U pip wheel setuptools"
24+
- "%PYTHON%\\python.exe -m pip install pytest"
25+
- "%PYTHON%\\python.exe -m pip install -r tests/requirements.txt"
2426
- "%PYTHON%\\python.exe setup.py install"
2527

2628
build: off
2729

2830
test_script:
29-
- "%PYTHON%\\python.exe -m pytest -v tests"
31+
- "%PYTHON%\\python.exe -m pytest"

‎fs/_ftp_parse.py

Lines changed: 33 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,8 @@
1919
RE_LINUX = re.compile(
2020
r"""
2121
^
22-
([ldrwx-]{10})
22+
([-dlpscbD])
23+
([r-][w-][xsS-][r-][w-][xsS-][r-][w-][xtT-][\.\+]?)
2324
\s+?
2425
(\d+)
2526
\s+?
@@ -55,9 +56,7 @@
5556

5657

5758
def get_decoders():
58-
"""
59-
Returns all available FTP LIST line decoders with their matching regexes.
60-
"""
59+
"""Return all available FTP LIST line decoders with their matching regexes."""
6160
decoders = [
6261
(RE_LINUX, decode_linux),
6362
(RE_WINDOWSNT, decode_windowsnt),
@@ -110,14 +109,14 @@ def _decode_linux_time(mtime):
110109

111110

112111
def decode_linux(line, match):
113-
perms, links, uid, gid, size, mtime, name = match.groups()
114-
is_link = perms.startswith("l")
115-
is_dir = perms.startswith("d") or is_link
112+
ty, perms, links, uid, gid, size, mtime, name = match.groups()
113+
is_link = ty == "l"
114+
is_dir = ty == "d" or is_link
116115
if is_link:
117116
name, _, _link_name = name.partition("->")
118117
name = name.strip()
119118
_link_name = _link_name.strip()
120-
permissions = Permissions.parse(perms[1:])
119+
permissions = Permissions.parse(perms)
121120

122121
mtime_epoch = _decode_linux_time(mtime)
123122

@@ -148,13 +147,34 @@ def _decode_windowsnt_time(mtime):
148147

149148

150149
def decode_windowsnt(line, match):
151-
"""
152-
Decodes a Windows NT FTP LIST line like one of these:
150+
"""Decode a Windows NT FTP LIST line.
151+
152+
Examples:
153+
Decode a directory line::
154+
155+
>>> line = "11-02-18 02:12PM <DIR> images"
156+
>>> match = RE_WINDOWSNT.match(line)
157+
>>> pprint(decode_windowsnt(line, match))
158+
{'basic': {'is_dir': True, 'name': 'images'},
159+
'details': {'modified': 1518358320.0, 'type': 1},
160+
'ftp': {'ls': '11-02-18 02:12PM <DIR> images'}}
161+
162+
Decode a file line::
163+
164+
>>> line = "11-02-18 03:33PM 9276 logo.gif"
165+
>>> match = RE_WINDOWSNT.match(line)
166+
>>> pprint(decode_windowsnt(line, match))
167+
{'basic': {'is_dir': False, 'name': 'logo.gif'},
168+
'details': {'modified': 1518363180.0, 'size': 9276, 'type': 2},
169+
'ftp': {'ls': '11-02-18 03:33PM 9276 logo.gif'}}
170+
171+
Alternatively, the time might also be present in 24-hour format::
153172
154-
`11-02-18 02:12PM <DIR> images`
155-
`11-02-18 03:33PM 9276 logo.gif`
173+
>>> line = "11-02-18 15:33 9276 logo.gif"
174+
>>> match = RE_WINDOWSNT.match(line)
175+
>>> decode_windowsnt(line, match)["details"]["modified"]
176+
1518363180.0
156177
157-
Alternatively, the time (02:12PM) might also be present in 24-hour format (14:12).
158178
"""
159179
is_dir = match.group("size") == "<DIR>"
160180

‎fs/_repr.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ def make_repr(class_name, *args, **kwargs):
2727
>>> MyClass('Will')
2828
MyClass('foo', name='Will')
2929
>>> MyClass(None)
30-
MyClass()
30+
MyClass('foo')
3131
3232
"""
3333
arguments = [repr(arg) for arg in args]

‎fs/_url_tools.py

Lines changed: 17 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,15 @@
1111

1212
def url_quote(path_snippet):
1313
# type: (Text) -> Text
14-
"""
15-
On Windows, it will separate drive letter and quote windows
16-
path alone. No magic on Unix-alie path, just pythonic
17-
`pathname2url`
14+
"""Quote a URL without quoting the Windows drive letter, if any.
15+
16+
On Windows, it will separate drive letter and quote Windows
17+
path alone. No magic on Unix-like path, just pythonic
18+
`~urllib.request.pathname2url`.
1819
1920
Arguments:
20-
path_snippet: a file path, relative or absolute.
21+
path_snippet (str): a file path, relative or absolute.
22+
2123
"""
2224
if _WINDOWS_PLATFORM and _has_drive_letter(path_snippet):
2325
drive_letter, path = path_snippet.split(":", 1)
@@ -34,17 +36,19 @@ def url_quote(path_snippet):
3436

3537
def _has_drive_letter(path_snippet):
3638
# type: (Text) -> bool
37-
"""
38-
The following path will get True
39-
D:/Data
40-
C:\\My Dcouments\\ test
39+
"""Check whether a path contains a drive letter.
4140
42-
And will get False
41+
Arguments:
42+
path_snippet (str): a file path, relative or absolute.
4343
44-
/tmp/abc:test
44+
Example:
45+
>>> _has_drive_letter("D:/Data")
46+
True
47+
>>> _has_drive_letter(r"C:\\System32\\ test")
48+
True
49+
>>> _has_drive_letter("/tmp/abc:test")
50+
False
4551
46-
Arguments:
47-
path_snippet: a file path, relative or absolute.
4852
"""
4953
windows_drive_pattern = ".:[/\\\\].*$"
5054
return re.match(windows_drive_pattern, path_snippet) is not None

0 commit comments

Comments
 (0)