Skip to content

Commit 014f55d

Browse files
manu-sjjavierdlrm
andauthored
[FSTORE-1777] Support aysnc predict functions inside a deployment's predict function. (#485)
* adding docs for async predictor * Update docs/user_guides/mlops/serving/predictor.md Co-authored-by: Javier de la Rúa Martínez <javierdlrm@outlook.com> --------- Co-authored-by: Javier de la Rúa Martínez <javierdlrm@outlook.com>
1 parent 9483d29 commit 014f55d

File tree

1 file changed

+17
-0
lines changed

1 file changed

+17
-0
lines changed

docs/user_guides/mlops/serving/predictor.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -171,6 +171,23 @@ Once you are done with the changes, click on `Create new deployment` at the bott
171171
# Use the model to make predictions
172172
# return self.model.predict(inputs)
173173
```
174+
=== "Async Predictor"
175+
``` python
176+
class Predictor():
177+
178+
def __init__(self):
179+
""" Initialization code goes here"""
180+
# Model files can be found at os.environ["MODEL_FILES_PATH"]
181+
# self.model = ... # load your model
182+
183+
async def predict(self, inputs):
184+
""" Asynchronously serve predictions using the trained model"""
185+
# Preform async operations that required
186+
# result = await some_async_preprocessing(inputs)
187+
188+
# Use the model to make predictions
189+
# return self.model.predict(result)
190+
```
174191
=== "Predictor (vLLM deployments only)"
175192
``` python
176193
import os

0 commit comments

Comments
 (0)