Merge pull request #4 from mongodb-labs/PYTHON-4089

Jibola · web-flow · commit c6659cdc41a4 · 2023-12-15T11:24:27.000-05:00
PYTHON-4089: Switch to using ODMantic
diff --git a/README.md b/README.md
@@ -4,7 +4,7 @@ Accelerate your next web development project with this FastAPI/React/MongoDB bas
 
 This project is for developers looking to build and maintain full-feature progressive web applications using Python on the backend / Typescript on the frontend, and want the complex-but-routine aspects of auth 'n auth, and component and deployment configuration, taken care of, including interactive API documentation. 
 
-This is an **experimental** fork of [Sebastián Ramírez's](https://github.com/tiangolo) [Full Stack FastAPI and PostgreSQL Base Project Generator](https://github.com/tiangolo/full-stack-fastapi-postgresql) and [Whythawk's](https://github.com/whythawk) [Full Stack FastAPI and PostgreSQL Base Project Generator](https://github.com/whythawk/full-stack-fastapi-postgresql). FastAPI is updated to version 0.103.2, MongoDB Motor 3.4, Beanie ODM 1.23, and the frontend to React.
+This is an **experimental** fork of [Sebastián Ramírez's](https://github.com/tiangolo) [Full Stack FastAPI and PostgreSQL Base Project Generator](https://github.com/tiangolo/full-stack-fastapi-postgresql) and [Whythawk's](https://github.com/whythawk) [Full Stack FastAPI and PostgreSQL Base Project Generator](https://github.com/whythawk/full-stack-fastapi-postgresql). FastAPI is updated to version 0.103.2, MongoDB Motor 3.4, ODMantic ODM 1.0.0, and the frontend to React.
 
 
 - [Screenshots](#screenshots)
@@ -50,7 +50,7 @@ This FastAPI, React, MongoDB repo will generate a complete web application stack
 - **Authentication** user management schemas, models, crud and apis already built, with OAuth2 JWT token support & default hashing. Offers _magic link_ authentication, with password fallback, with cookie management, including `access` and `refresh` tokens.
 - [**FastAPI**](https://github.com/tiangolo/fastapi) backend with [Inboard](https://inboard.bws.bio/) one-repo Docker images:
   - **MongoDB Motor** https://motor.readthedocs.io/en/stable/
-  - **MongoDB Beanie** for handling ODM creation https://beanie-odm.dev/
+  - **MongoDB ODMantic** for handling ODM creation https://art049.github.io/odmantic/
   - **Common CRUD** support via generic inheritance.
   - **Standards-based**: Based on (and fully compatible with) the open standards for APIs: [OpenAPI](https://github.com/OAI/OpenAPI-Specification) and [JSON Schema](http://json-schema.org/).
   - [**Many other features**]("https://fastapi.tiangolo.com/features/"): including automatic validation, serialization, interactive documentation, etc.
@@ -90,6 +90,9 @@ This stack is in an experimental state, so there is no guarantee for bugs or iss
 
 See notes: 
 
+## CalVer 2023.12.XX
+- Replaced Beanie usage with ODMantic
+
 ## CalVer 2023.11.10
 
 - Replaced Next/Vue.js frontend framework with entirely React/Redux
diff --git a/docs/getting-started.md b/docs/getting-started.md
@@ -25,7 +25,7 @@ This FastAPI, React, MongoDB repo will generate a complete web application stack
 - **Authentication** user management schemas, models, crud and apis already built, with OAuth2 JWT token support & default hashing. Offers _magic link_ authentication, with password fallback, with cookie management, including `access` and `refresh` tokens.
 - [**FastAPI**](https://github.com/tiangolo/fastapi) backend with [Inboard](https://inboard.bws.bio/) one-repo Docker images:
   - **Mongo Motor** https://motor.readthedocs.io/en/stable/
-  - **Mongo Beanie** for handling ODM creation https://beanie-odm.dev/
+  - **MongoDB ODMantic** for handling ODM creation https://art049.github.io/odmantic/
   - **Common CRUD** support via generic inheritance.
   - **Standards-based**: Based on (and fully compatible with) the open standards for APIs: [OpenAPI](https://github.com/OAI/OpenAPI-Specification) and [JSON Schema](http://json-schema.org/).
   - [**Many other features**]("https://fastapi.tiangolo.com/features/"): including automatic validation, serialization, interactive documentation, etc.
diff --git a/{{cookiecutter.project_slug}}/backend/app/.DS_Store b/{{cookiecutter.project_slug}}/backend/app/.DS_Store
diff --git a/{{cookiecutter.project_slug}}/backend/app/README.md b/{{cookiecutter.project_slug}}/backend/app/README.md
@@ -92,7 +92,7 @@ Next, open your editor at `./backend/app/` (instead of the project root: `./`),
 $ code .
 ```
 
-Modify or add beanie models in `./backend/app/app/models/` (make sure to include them in `MODELS` within `.backend/app/app/models/__init__.py`), Pydantic schemas in `./backend/app/app/schemas/`, API endpoints in `./backend/app/app/api/`, CRUD (Create, Read, Update, Delete) utils in `./backend/app/app/crud/`. The easiest might be to copy the ones for Items (models, endpoints, and CRUD utils) and update them to your needs.
+Modify or add odmantic models in `./backend/app/app/models/` (make sure to include them in `MODELS` within `.backend/app/app/models/__init__.py`), Pydantic schemas in `./backend/app/app/schemas/`, API endpoints in `./backend/app/app/api/`, CRUD (Create, Read, Update, Delete) utils in `./backend/app/app/crud/`. The easiest might be to copy the ones for Items (models, endpoints, and CRUD utils) and update them to your needs.
 
 Add and modify tasks to the Celery worker in `./backend/app/app/worker.py`.
 
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/crud/base.py b/{{cookiecutter.project_slug}}/backend/app/app/crud/base.py
@@ -3,9 +3,11 @@
 from fastapi.encoders import jsonable_encoder
 from pydantic import BaseModel
 from motor.core import AgnosticDatabase
+from odmantic import AIOEngine
 
 from app.db.base_class import Base
 from app.core.config import settings
+from app.db.session import get_engine
 
 ModelType = TypeVar("ModelType", bound=Base)
 CreateSchemaType = TypeVar("CreateSchemaType", bound=BaseModel)
@@ -23,18 +25,19 @@ def __init__(self, model: Type[ModelType]):
         * `schema`: A Pydantic model (schema) class
         """
         self.model = model
+        self.engine: AIOEngine = get_engine()
 
     async def get(self, db: AgnosticDatabase, id: Any) -> Optional[ModelType]:
-        return await self.model.get(id)
+        return await self.engine.find_one(self.model, self.model.id == id)
 
     async def get_multi(self, db: AgnosticDatabase, *, page: int = 0, page_break: bool = False) -> list[ModelType]:
         offset = {"skip": page * settings.MULTI_MAX, "limit": settings.MULTI_MAX} if page_break else {}
-        return await self.model.find_all(**offset).to_list()
+        return await self.engine.find(self.model, **offset)
 
     async def create(self, db: AgnosticDatabase, *, obj_in: CreateSchemaType) -> ModelType:
         obj_in_data = jsonable_encoder(obj_in)
         db_obj = self.model(**obj_in_data)  # type: ignore
-        return await db_obj.create()
+        return await self.engine.save(db_obj)
 
     async def update(
         self, db: AgnosticDatabase, *, db_obj: ModelType, obj_in: Union[UpdateSchemaType, Dict[str, Any]]
@@ -43,16 +46,16 @@ async def update(
         if isinstance(obj_in, dict):
             update_data = obj_in
         else:
-            update_data = obj_in.dict(exclude_unset=True)
+            update_data = obj_in.model_dump(exclude_unset=True)
         for field in obj_data:
             if field in update_data:
                 setattr(db_obj, field, update_data[field])
         # TODO: Check if this saves changes with the setattr calls
-        await db_obj.save()
+        await self.engine.save(db_obj)
         return db_obj
 
     async def remove(self, db: AgnosticDatabase, *, id: int) -> ModelType:
         obj = await self.model.get(id)
         if obj:
-            await obj.delete()
+            await self.engine.delete(obj)
         return obj
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/crud/crud_token.py b/{{cookiecutter.project_slug}}/backend/app/app/crud/crud_token.py
@@ -1,7 +1,5 @@
 from __future__ import annotations
 from motor.core import AgnosticDatabase
-from beanie import WriteRules
-from beanie.operators import And, In, PullAll
 
 from app.crud.base import CRUDBase
 from app.models import User, Token
@@ -12,27 +10,31 @@
 class CRUDToken(CRUDBase[Token, RefreshTokenCreate, RefreshTokenUpdate]):
     # Everything is user-dependent
     async def create(self, db: AgnosticDatabase, *, obj_in: str, user_obj: User) -> Token:
-        db_obj = await self.model.find_one(self.model.token == obj_in)
+        db_obj = await self.engine.find_one(self.model, self.model.token == obj_in)
         if db_obj:
             if db_obj.authenticates_id != user_obj.id:
                 raise ValueError("Token mismatch between key and user.")
             return db_obj
         else:
-            new_token = self.model(token=obj_in, authenticates_id=user_obj.id)
-            user_obj.refresh_tokens.append(new_token)
-            await user_obj.save(link_rule=WriteRules.WRITE)
+            new_token = self.model(token=obj_in, authenticates_id=user_obj)
+            user_obj.refresh_tokens.append(new_token.id)
+            await self.engine.save_all([new_token, user_obj])
             return new_token
 
     async def get(self, *, user: User, token: str) -> Token:
-        return await user.find_one(And(User.id == user.id, User.refresh_tokens.token == token), fetch_links=True)
+        return await self.engine.find_one(User, ((User.id == user.id) & (User.refresh_tokens.token == token)))
 
     async def get_multi(self, *, user: User, page: int = 0, page_break: bool = False) -> list[Token]:
         offset = {"skip": page * settings.MULTI_MAX, "limit": settings.MULTI_MAX} if page_break else {}
-        return await User.find(In(User.refresh_tokens, user.refresh_tokens), **offset).to_list()
+        return await self.engine.find(User, (User.refresh_tokens.in_([user.refresh_tokens])), **offset)
 
     async def remove(self, db: AgnosticDatabase, *, db_obj: Token) -> None:
-        await User.update_all(PullAll({User.refresh_tokens: [db_obj.to_ref()]}))
-        await db_obj.delete()
+        users = []
+        async for user in self.engine.find(User, User.refresh_tokens.in_([db_obj.id])):
+            user.refresh_tokens.remove(db_obj.id)
+            users.append(user)
+        await self.engine.save(users)
+        await self.engine.delete(db_obj)
 
 
 token = CRUDToken(Token)
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/crud/crud_user.py b/{{cookiecutter.project_slug}}/backend/app/app/crud/crud_user.py
@@ -12,7 +12,7 @@
 # ODM, Schema, Schema
 class CRUDUser(CRUDBase[User, UserCreate, UserUpdate]):
     async def get_by_email(self, db: AgnosticDatabase, *, email: str) -> Optional[User]:
-        return await User.find_one(User.email == email)
+        return await self.engine.find_one(User, User.email == email)
 
     async def create(self, db: AgnosticDatabase, *, obj_in: UserCreate) -> User:
         # TODO: Figure out what happens when you have a unique key like 'email'
@@ -24,7 +24,7 @@ async def create(self, db: AgnosticDatabase, *, obj_in: UserCreate) -> User:
             "is_superuser": obj_in.is_superuser,
         }
 
-        return await User(**user).create()
+        return await self.engine.save(User(**user))
 
     async def update(self, db: AgnosticDatabase, *, db_obj: User, obj_in: Union[UserUpdate, Dict[str, Any]]) -> User:
         if isinstance(obj_in, dict):
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/db/base_class.py b/{{cookiecutter.project_slug}}/backend/app/app/db/base_class.py
@@ -1,3 +1,3 @@
-from beanie import Document
+from odmantic import Model
 
-Base = Document
+Base = Model
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/db/init_db.py b/{{cookiecutter.project_slug}}/backend/app/app/db/init_db.py
@@ -1,13 +1,10 @@
 from pymongo.database import Database
-from beanie import init_beanie
 
 from app import crud, schemas
 from app.core.config import settings
-from app.models import MODELS
 
 
 async def init_db(db: Database) -> None:
-    await init_beanie(db, document_models=MODELS)
     user = await crud.user.get_by_email(db, email=settings.FIRST_SUPERUSER)
     if not user:
         # Create user auth
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/db/session.py b/{{cookiecutter.project_slug}}/backend/app/app/db/session.py
@@ -1,27 +1,34 @@
 from app.core.config import settings
 from app.__version__ import __version__
 from motor import motor_asyncio, core
+from odmantic import AIOEngine
 from pymongo.driver_info import DriverInfo
 
-DRIVER_INFO=DriverInfo(name="full-stack-fastapi-mongodb", version=__version__)
+DRIVER_INFO = DriverInfo(name="full-stack-fastapi-mongodb", version=__version__)
 
 
 class _MongoClientSingleton:
     mongo_client: motor_asyncio.AsyncIOMotorClient | None
+    engine: AIOEngine
 
     def __new__(cls):
         if not hasattr(cls, "instance"):
             cls.instance = super(_MongoClientSingleton, cls).__new__(cls)
             cls.instance.mongo_client = motor_asyncio.AsyncIOMotorClient(
-                settings.MONGO_DATABASE_URI,
-                driver=DRIVER_INFO
+                settings.MONGO_DATABASE_URI, driver=DRIVER_INFO
+            )
+            cls.instance.engine = AIOEngine(
+                client=cls.instance.mongo_client,
+                database=settings.MONGO_DATABASE
             )
         return cls.instance
 
 
 def MongoDatabase() -> core.AgnosticDatabase:
     return _MongoClientSingleton().mongo_client[settings.MONGO_DATABASE]
 
+def get_engine() -> AIOEngine:
+    return _MongoClientSingleton().engine
 
 async def ping():
     await MongoDatabase().command("ping")
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/main.py b/{{cookiecutter.project_slug}}/backend/app/app/main.py
@@ -1,17 +1,13 @@
 from fastapi import FastAPI
 from starlette.middleware.cors import CORSMiddleware
-from beanie import init_beanie
 from contextlib import asynccontextmanager
 
 from app.api.api_v1.api import api_router
 from app.core.config import settings
-from app.db.session import MongoDatabase
-from app.models import MODELS
 
 
 @asynccontextmanager
 async def app_init(app: FastAPI):
-    await init_beanie(MongoDatabase(), document_models=MODELS)
     app.include_router(api_router, prefix=settings.API_V1_STR)
     yield
 
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/models/__init__.py b/{{cookiecutter.project_slug}}/backend/app/app/models/__init__.py
@@ -1,8 +1,2 @@
-from beanie import Document
 from .user import User
 from .token import Token
-
-MODELS: list[Document] = [User, Token]
-
-for model in MODELS:
-    model.model_rebuild()
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/models/token.py b/{{cookiecutter.project_slug}}/backend/app/app/models/token.py
@@ -1,15 +1,13 @@
 from __future__ import annotations
 
-from typing import TYPE_CHECKING
-from beanie.odm.fields import PydanticObjectId
+from odmantic import Reference
 
 from app.db.base_class import Base
 
-if TYPE_CHECKING:
-    from .user import User  # noqa: F401
+from .user import User  # noqa: F401
 
 
 # Consider reworking to consolidate information to a userId. This may not work well
 class Token(Base):
     token: str
-    authenticates_id: PydanticObjectId
+    authenticates_id: User = Reference()
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/models/user.py b/{{cookiecutter.project_slug}}/backend/app/app/models/user.py
@@ -1,8 +1,8 @@
 from __future__ import annotations
-from typing import TYPE_CHECKING, Optional
+from typing import TYPE_CHECKING, Any, Optional
 from datetime import datetime
-from pydantic import Field, EmailStr
-from beanie.odm.fields import Link
+from pydantic import EmailStr
+from odmantic import ObjectId, Field
 
 from app.db.base_class import Base
 
@@ -19,10 +19,10 @@ class User(Base):
     modified: datetime = Field(default_factory=datetime_now_sec)
     full_name: str = Field(default="")
     email: EmailStr
-    hashed_password: Optional[str]
-    totp_secret: Optional[str] = Field(default=None)
+    hashed_password: Any = Field(default=None)
+    totp_secret: Any = Field(default=None)
     totp_counter: Optional[int] = Field(default=None)
     email_validated: bool = Field(default=False)
     is_active: bool = Field(default=False)
     is_superuser: bool = Field(default=False)
-    refresh_tokens: list[Link["Token"]] = Field(default_factory=list)
+    refresh_tokens: list[ObjectId] = Field(default_factory=list)
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/schemas/token.py b/{{cookiecutter.project_slug}}/backend/app/app/schemas/token.py
@@ -1,16 +1,15 @@
 from typing import Optional
 from pydantic import BaseModel
-from beanie.odm.fields import PydanticObjectId
-from beanie import Document
+from odmantic import Model, ObjectId
 
 
 class RefreshTokenBase(BaseModel):
     token: str
-    authenticates: Optional[Document] = None
+    authenticates: Optional[Model] = None
 
 
 class RefreshTokenCreate(RefreshTokenBase):
-    authenticates: Document
+    authenticates: Model
 
 
 class RefreshTokenUpdate(RefreshTokenBase):
@@ -29,14 +28,14 @@ class Token(BaseModel):
 
 
 class TokenPayload(BaseModel):
-    sub: Optional[PydanticObjectId] = None
+    sub: Optional[ObjectId] = None
     refresh: Optional[bool] = False
     totp: Optional[bool] = False
 
 
 class MagicTokenPayload(BaseModel):
-    sub: Optional[PydanticObjectId] = None
-    fingerprint: Optional[PydanticObjectId] = None
+    sub: Optional[ObjectId] = None
+    fingerprint: Optional[ObjectId] = None
 
 
 class WebToken(BaseModel):
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/schemas/user.py b/{{cookiecutter.project_slug}}/backend/app/app/schemas/user.py
@@ -1,6 +1,7 @@
 from typing import Optional
-from pydantic import BaseModel, Field, EmailStr, constr, field_validator
-from beanie import PydanticObjectId
+from typing_extensions import Annotated
+from pydantic import BaseModel, Field, EmailStr, StringConstraints, field_validator
+from odmantic import ObjectId
 
 
 class UserLogin(BaseModel):
@@ -20,17 +21,17 @@ class UserBase(BaseModel):
 # Properties to receive via API on creation
 class UserCreate(UserBase):
     email: EmailStr
-    password: Optional[constr(min_length=8, max_length=64)] = None
+    password: Optional[Annotated[str, StringConstraints(min_length=8, max_length=64)]] = None
 
 
 # Properties to receive via API on update
 class UserUpdate(UserBase):
-    original: Optional[constr(min_length=8, max_length=64)] = None
-    password: Optional[constr(min_length=8, max_length=64)] = None
+    original: Optional[Annotated[str, StringConstraints(min_length=8, max_length=64)]] = None
+    password: Optional[Annotated[str, StringConstraints(min_length=8, max_length=64)]] = None
 
 
 class UserInDBBase(UserBase):
-    id: Optional[PydanticObjectId] = None
+    id: Optional[ObjectId] = None
 
     class Config:
         from_attributes = True
diff --git a/{{cookiecutter.project_slug}}/backend/app/app/scratchpad.py b/{{cookiecutter.project_slug}}/backend/app/app/scratchpad.py
diff --git a/{{cookiecutter.project_slug}}/backend/app/pyproject.toml b/{{cookiecutter.project_slug}}/backend/app/pyproject.toml
diff --git a/{{cookiecutter.project_slug}}/frontend/app/lib/api/auth.ts b/{{cookiecutter.project_slug}}/frontend/app/lib/api/auth.ts
