Skip to content

Commit 682746f

Browse files
authored
Merge pull request #11 from remigermain/feat--optimize-parser
Feat optimize parser
2 parents c287a30 + 000a696 commit 682746f

File tree

8 files changed

+308
-230
lines changed

8 files changed

+308
-230
lines changed

.github/workflows/main.yml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,25 +8,25 @@ on:
88
jobs:
99
build:
1010
name: Python ${{ matrix.python-version }}
11-
runs-on: ubuntu-20.04
11+
# https://stackoverflow.com/questions/70959954/error-waiting-for-a-runner-to-pick-up-this-job-using-github-actions
12+
runs-on: ubuntu-latest
1213

1314
strategy:
1415
matrix:
1516
python-version:
16-
- "3.6"
17-
- "3.9"
1817
- "3.9"
1918
- "3.11"
2019
- "3.12"
20+
- "3.13"
2121

2222
steps:
23-
- uses: actions/checkout@v2
23+
- uses: actions/checkout@v5
2424

25-
- uses: actions/setup-python@v2
25+
- uses: actions/setup-python@v6
2626
with:
2727
python-version: ${{ matrix.python-version }}
2828

29-
- uses: actions/cache@v2
29+
- uses: actions/cache@v4
3030
with:
3131
path: ~/.cache/pip
3232
key: ${{ runner.os }}-pip-${{ hashFiles('requirements/dev.txt') }}

README.md

Lines changed: 100 additions & 111 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,9 @@
33
<a href="https://u8views.com/github/remigermain"><img src="https://u8views.com/api/v1/github/profiles/66946113/views/day-week-month-total-count.svg" width="1px" height="1px"></a>
44
[![CI](https://github.com/remigermain/nested-multipart-parser/actions/workflows/main.yml/badge.svg)](https://github.com/remigermain/nested-multipart-parser/actions/workflows/main.yml)
55
[![pypi](https://img.shields.io/pypi/v/nested-multipart-parser)](https://pypi.org/project/nested-multipart-parser/)
6-
![PyPI - Downloads](https://img.shields.io/pypi/dm/Nested-multipart-parser)
7-
8-
Parser for nested data for '*multipart/form*', you can use it in any python project, or use the Django Rest Framework integration.
6+
[![PyPI - Downloads](https://img.shields.io/pypi/dm/Nested-multipart-parser)](https://pypistats.org/packages/nested-multipart-parser)
97

8+
Parser for nested data for *multipart/form*, usable in any Python project or via the [Django Rest Framework integration](https://www.django-rest-framework.org/community/third-party-packages/#parsers)..
109
# Installation:
1110

1211
```bash
@@ -35,6 +34,16 @@ def my_view():
3534

3635
### Django Rest Framework
3736

37+
you can define parser for all view in settings.py
38+
```python
39+
REST_FRAMEWORK = {
40+
"DEFAULT_PARSER_CLASSES": [
41+
"nested_multipart_parser.drf.DrfNestedParser",
42+
]
43+
}
44+
```
45+
or directly in your view
46+
3847
```python
3948
from nested_multipart_parser.drf import DrfNestedParser
4049
...
@@ -46,7 +55,7 @@ class YourViewSet(viewsets.ViewSet):
4655

4756
## What it does:
4857

49-
The parser take the request data and transform it to a Python dictionary:
58+
The parser takes the request data and transforms it into a Python dictionary.
5059

5160
example:
5261

@@ -94,126 +103,106 @@ example:
94103
}
95104
```
96105

97-
## How it works:
98-
99-
Attributes where sub keys are full numbers only are automatically converted into lists:
106+
## How it works
107+
### Lists
100108

109+
Attributes whose sub‑keys are *only numbers* become Python lists:
101110
```python
102-
data = {
103-
'title[0]': 'my-value',
104-
'title[1]': 'my-second-value'
105-
}
106-
output = {
107-
'title': [
108-
'my-value',
109-
'my-second-value'
110-
]
111-
}
112-
113-
# Be aware of the fact that you have to respect the order of the indices for arrays, thus
114-
'title[2]': 'my-value' # Invalid (you have to set title[0] and title[1] before)
115-
116-
# Also, you can't create an array on a key already set as a prinitive value (int, boolean or string):
117-
'title': 42,
118-
'title[object]': 42 # Invalid
111+
data = {
112+
'title[0]': 'my-value',
113+
'title[1]': 'my-second-value'
114+
}
115+
output = {
116+
'title': [
117+
'my-value',
118+
'my-second-value'
119+
]
120+
}
119121
```
122+
> Important notes
120123
121-
122-
123-
Attributes where sub keys are other than full numbers are converted into Python dictionary:
124-
124+
- Indices must be contiguous and start at 0.
125+
- You cannot turn a primitive (int, bool, str) into a list later, e.g.
125126
```python
126-
data = {
127-
'title.key0': 'my-value',
128-
'title.key7': 'my-second-value'
129-
}
130-
output = {
131-
'title': {
132-
'key0': 'my-value',
133-
'key7': 'my-second-value'
134-
}
135-
}
136-
137-
138-
# You have no limit for chained key:
139-
# with "mixed-dot" separator option (same as 'mixed' but with dot after list to object):
140-
data = {
141-
'the[0].chained.key[0].are.awesome[0][0]': 'im here !!'
142-
}
143-
# with "mixed" separator option:
144-
data = {
145-
'the[0]chained.key[0]are.awesome[0][0]': 'im here !!'
146-
}
147-
# With "bracket" separator option:
148-
data = {
149-
'the[0][chained][key][0][are][awesome][0][0]': 'im here !!'
150-
}
151-
# With "dot" separator option:
152-
data = {
153-
'the.0.chained.key.0.are.awesome.0.0': 'im here !!'
154-
}
127+
'title': 42,
128+
'title[object]': 42 # ❌ invalid
155129
```
156130

131+
### Dictionaries
157132

133+
Attributes whose sub‑keys are *not pure numbers* become nested dictionaries:
134+
```python
135+
data = {
136+
'title.key0': 'my-value',
137+
'title.key7': 'my-second-value'
138+
}
139+
output = {
140+
'title': {
141+
'key0': 'my-value',
142+
'key7': 'my-second-value'
143+
}
144+
}
145+
```
158146

159-
For this to work perfectly, you must follow the following rules:
160-
161-
- A first key always need to be set. ex: `title[0]` or `title`. In both cases the first key is `title`
162-
163-
- For `mixed` or `mixed-dot` options, brackets `[]` is for list, and dot `.` is for object
147+
### Chaining keys
164148

165-
- For `mixed-dot` options is look like `mixed` but with dot when object follow list
149+
>Keys can be chained arbitrarily. Below are examples for each separator option:
166150
167-
- For `bracket` each sub key need to be separate by brackets `[ ]` or with `dot` options `.`
151+
|Separator| Example key | Meaning|
152+
|-|-|-|
153+
|mixed‑dot| the[0].chained.key[0].are.awesome[0][0] |List → object → list → object …|
154+
|mixed| the[0]chained.key[0]are.awesome[0][0] | Same as mixed‑dot but without the dot after a list|
155+
|bracket| the[0][chained][key][0][are][awesome][0][0] | Every sub‑key is wrapped in brackets|
156+
|dot |the.0.chained.key.0.are.awesome.0.0 | Dots separate every level; numeric parts become lists|
168157

169-
- For `bracket` or `dot`options, if a key is number is convert to list else a object
170158

171-
- Don't put spaces between separators.
159+
Rules to keep in mind
160+
- First key must exist – e.g. title[0] or just title.
161+
- For mixed / mixed‑dot, [] denotes a list and . denotes an object.
162+
- mixed‑dot behaves like mixed but inserts a dot when an object follows a list.
163+
- For bracket, each sub‑key must be surrounded by brackets ([ ]).
164+
- For bracket or dot, numeric sub‑keys become list elements; non‑numeric become objects.
165+
- No spaces between separators.
166+
- By default, duplicate keys are disallowed (see options).
167+
- Empty structures are supported:
168+
Empty list → "article.authors[]": None → {"article": {"authors": []}}
169+
Empty dict → "article.": None → {"article": {}} (available with dot, mixed, mixed‑dot)
172170

173-
- By default, you can't set set duplicates keys (see options)
174171

175-
- You can set empty dict/list:
176-
for empty list: `"article.authors[]": None` -> `{"article": {"authors": [] }}`
177-
for empty dict: `"article.": None` -> `{"article": {} }`
178-
`.` last dot for empty dict (availables in `dot`, `mixed` and `mixed-dot` options)
179-
`[]` brackets empty for empty list (availables in `brackets`, `mixed` and `mixed-dot` options)
180-
181172

182173

183174
## Options
184175

185176
```python
186177
{
187-
# Separators:
188-
# with mixed-dot: article[0].title.authors[0]: "jhon doe"
189-
# with mixed: article[0]title.authors[0]: "jhon doe"
190-
# with bracket: article[0][title][authors][0]: "jhon doe"
191-
# with dot: article.0.title.authors.0: "jhon doe"
192-
'separator': 'bracket' or 'dot' or 'mixed' or 'mixed-dot', # default is `mixed-dot`
193-
194-
195-
# raise a expections when you have duplicate keys
196-
# ex :
197-
# {
198-
# "article": 42,
199-
# "article[title]": 42,
200-
# }
201-
'raise_duplicate': True, # default is True
202-
203-
# override the duplicate keys, you need to set "raise_duplicate" to False
204-
# ex :
205-
# {
206-
# "article": 42,
207-
# "article[title]": 42,
208-
# }
209-
# the out is
210-
# ex :
211-
# {
212-
# "article"{
213-
# "title": 42,
214-
# }
215-
# }
216-
'assign_duplicate': False # default is False
178+
# Separator (default: 'mixed‑dot')
179+
# mixed‑dot : article[0].title.authors[0] -> "john doe"
180+
# mixed : article[0]title.authors[0] -> "john doe"
181+
# bracket : article[0][title][authors][0] -> "john doe"
182+
# dot : article.0.title.authors.0 -> "john doe"
183+
'separator': 'bracket' | 'dot' | 'mixed' | 'mixed‑dot',
184+
185+
# Raise an exception when duplicate keys are encountered
186+
# Example:
187+
# {
188+
# "article": 42,
189+
# "article[title]": 42,
190+
# }
191+
'raise_duplicate': True, # default: True
192+
193+
# Override duplicate keys (requires raise_duplicate=False)
194+
# Example:
195+
# {
196+
# "article": 42,
197+
# "article[title]": 42,
198+
# }
199+
# Result:
200+
# {
201+
# "article": {
202+
# "title": 42
203+
# }
204+
# }
205+
'assign_duplicate': False, # default: False
217206
}
218207
```
219208

@@ -223,20 +212,20 @@ For this to work perfectly, you must follow the following rules:
223212
# settings.py
224213
...
225214

215+
# settings.py
226216
DRF_NESTED_MULTIPART_PARSER = {
227-
"separator": "mixed-dot",
228-
"raise_duplicate": True,
229-
"assign_duplicate": False,
217+
"separator": "mixeddot",
218+
"raise_duplicate": True,
219+
"assign_duplicate": False,
230220

231-
# output of parser is converted to querydict
232-
# if is set to False, dict python is returned
233-
"querydict": True,
221+
# If True, the parser’s output is converted to a QueryDict;
222+
# if False, a plain Python dict is returned.
223+
"querydict": True,
234224
}
235225
```
236226

237227
## JavaScript integration:
238-
239-
You can use this [multipart-object](https://github.com/remigermain/multipart-object) library to easy convert object to flat nested object formatted for this library
228+
A companion [multipart-object](https://github.com/remigermain/multipart-object) library exists to convert a JavaScript object into the flat, nested format expected by this parser.
240229

241230
## License
242231

bench/bench.py

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
import time
2+
3+
from nested_multipart_parser import NestedParser
4+
5+
6+
def bench(data, count):
7+
v = []
8+
for _ in range(count):
9+
start = time.perf_counter()
10+
parser = NestedParser(data)
11+
parser.is_valid()
12+
validate_data = parser.validate_data
13+
end = time.perf_counter()
14+
v.append(end - start)
15+
16+
return sum(v) / len(v)
17+
18+
19+
def big(count):
20+
data = {
21+
"title": "title",
22+
"date": "time",
23+
"langs[0].id": "id",
24+
"langs[0].title": "title",
25+
"langs[0].description": "description",
26+
"langs[0].language": "language",
27+
"langs[1].id": "id1",
28+
"langs[1].title": "title1",
29+
"langs[1].description": "description1",
30+
"langs[1].language": "language1",
31+
"test.langs[0].id": "id",
32+
"test.langs[0].title": "title",
33+
"test.langs[0].description": "description",
34+
"test.langs[0].language": "language",
35+
"test.langs[1].id": "id1",
36+
"test.langs[1].title": "title1",
37+
"test.langs[1].description": "description1",
38+
"test.langs[1].language": "language1",
39+
"deep.nested.dict.test.langs[0].id": "id",
40+
"deep.nested.dict.test.langs[0].title": "title",
41+
"deep.nested.dict.test.langs[0].description": "description",
42+
"deep.nested.dict.test.langs[0].language": "language",
43+
"deep.nested.dict.test.langs[1].id": "id1",
44+
"deep.nested.dict.test.langs[1].title": "title1",
45+
"deep.nested.dict.test.langs[1].description": "description1",
46+
"deep.nested.dict.test.langs[1].language": "language1",
47+
"deep.nested.dict.with.list[0].test.langs[0].id": "id",
48+
"deep.nested.dict.with.list[0].test.langs[0].title": "title",
49+
"deep.nested.dict.with.list[1].test.langs[0].description": "description",
50+
"deep.nested.dict.with.list[1].test.langs[0].language": "language",
51+
"deep.nested.dict.with.list[1].test.langs[1].id": "id1",
52+
"deep.nested.dict.with.list[1].test.langs[1].title": "title1",
53+
"deep.nested.dict.with.list[0].test.langs[1].description": "description1",
54+
"deep.nested.dict.with.list[0].test.langs[1].language": "language1",
55+
}
56+
return bench(data, count)
57+
58+
59+
def small(count):
60+
data = {
61+
"title": "title",
62+
"date": "time",
63+
"langs[0].id": "id",
64+
"langs[0].title": "title",
65+
"langs[0].description": "description",
66+
"langs[0].language": "language",
67+
"langs[1].id": "id1",
68+
"langs[1].title": "title1",
69+
"langs[1].description": "description1",
70+
"langs[1].language": "language1",
71+
}
72+
return bench(data, count)
73+
74+
75+
count = 10_000
76+
print(f"{small(count)=}")
77+
print(f"{big(count)=}")

0 commit comments

Comments
 (0)