Skip to content

Commit 9761d08

Browse files
authored
Merge pull request #194 from IBM/test-coverage
Test coverage and integration tests
2 parents 4573f6e + 9bb17c4 commit 9761d08

File tree

3 files changed

+514
-163
lines changed

3 files changed

+514
-163
lines changed

tests/README.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# 🧪 MCP Gateway Testing Guide
2+
3+
This repository comes with a **three-tier test-suite**:
4+
5+
| layer | location | what it covers |
6+
| -------------------- | --------------------- | ------------------------------------------------------------------------------------- |
7+
| **Unit** | `tests/unit/…` | Fast, isolated tests for individual functions, services, models and handlers. |
8+
| **Integration** | `tests/integration/…` | Happy-path flows that stitch several endpoints together with `TestClient`. |
9+
| **End-to-End (E2E)** | `tests/e2e/…` | Full, high-level workflows that drive the running server (admin, federation, client). |
10+
11+
```
12+
tests/
13+
├── conftest.py # shared fixtures
14+
├── e2e/ … 3 files
15+
├── integration/ … 8 files
16+
└── unit/ … 60+ files
17+
```
18+
19+
---
20+
21+
## Quick commands
22+
23+
| purpose | command |
24+
| ------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
25+
| **Run the full suite (default):** | `pytest -q` |
26+
| Unit tests only | `pytest tests/unit` |
27+
| A single module (verbose) | `pytest -v tests/unit/mcpgateway/test_main.py` |
28+
| **Coverage for `mcpgateway/main.py` from *unit* suite** | <br>`pytest tests/unit/mcpgateway/test_main.py \`<br>` --cov=mcpgateway.main \`<br>` --cov-report=term-missing` |
29+
| **HTML coverage report (all code)** | `make htmlcov` → open `htmlcov/index.html` |
30+
| Project-wide tests + coverage (CI default) | `make test` |
31+
32+
---
33+
34+
## Coverage workflow
35+
36+
1. **Spot the gaps**
37+
38+
```bash
39+
pytest tests/unit/mcpgateway/test_main.py \
40+
--cov=mcpgateway.main \
41+
--cov-report=term-missing
42+
```
43+
44+
Lines listed under *Missing* are un-executed.
45+
46+
2. **Write focused tests**
47+
48+
Add/extend tests in the relevant sub-folder (unit ➜ fine-grained; integration ➜ flows).
49+
50+
3. **Iterate** until the target percentage (or 100 %) is reached.
51+
52+
---
53+
54+
## Test layout & naming conventions
55+
56+
* Each top-level domain inside `mcpgateway/` has a mirrored **unit-test
57+
package**: `tests/unit/mcpgateway/<domain>/`.
58+
*Example*: `mcpgateway/services/tool_service.py`
59+
`tests/unit/mcpgateway/services/test_tool_service.py`.
60+
61+
* **Integration tests** live in `tests/integration/` and use
62+
`TestClient`, but patch actual DB/network calls with `AsyncMock`.
63+
64+
* **E2E tests** (optional) assume a running server and may involve
65+
HTTP requests, WebSockets, SSE streams, etc.
66+
67+
* Log-replay / load-test artefacts are parked in `tests/hey/` (ignored by CI).
68+
69+
---
70+
71+
## Fixtures cheat-sheet
72+
73+
| fixture | scope | description |
74+
| -------------- | -------------------------------------------- | ------------------------------------------------ |
75+
| `test_client` | function | A FastAPI `TestClient` with JWT auth overridden. |
76+
| `auth_headers` | function | A ready-made `Authorization: Bearer …` header. |
77+
| Extra helpers | module-level `conftest.py` files per folder. | |
78+
79+
---
80+
81+
## Makefile targets
82+
83+
| target | does… |
84+
| -------------- | ---------------------------------------------------------------------- |
85+
| `make test` | Runs `pytest --cov --cov-report=term` across the whole repo. |
86+
| `make htmlcov` | Re-runs tests and generates an **HTML coverage report** in `htmlcov/`. |
87+
| `make lint` | Static analysis (ruff, mypy, etc.) — optional in CI. |
88+
89+
---

tests/integration/test_integration.py

Lines changed: 266 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,266 @@
1+
# -*- coding: utf-8 -*-
2+
# tests/integration/test_integration.py
3+
"""End-to-end happy-path integration tests for the MCP Gateway API.
4+
5+
Copyright 2025
6+
SPDX-License-Identifier: Apache-2.0
7+
Authors: Mihai Criveti
8+
9+
These tests exercise several endpoints together instead of in isolation:
10+
11+
1. Create a tool ➜ create a server that references that tool.
12+
2. MCP protocol handshake: /protocol/initialize ➜ /protocol/ping.
13+
3. Full resource life-cycle: register ➜ read content.
14+
4. Invoke a tool via JSON-RPC.
15+
5. Aggregate metrics endpoint.
16+
17+
All external service calls are patched out with AsyncMocks so the FastAPI app
18+
under test never touches a real database or network.
19+
"""
20+
21+
from __future__ import annotations
22+
23+
import urllib.parse
24+
from datetime import datetime
25+
from unittest.mock import ANY, AsyncMock, patch
26+
27+
import pytest
28+
from fastapi.testclient import TestClient
29+
30+
from mcpgateway.main import app, require_auth
31+
from mcpgateway.schemas import ResourceRead, ServerRead, ToolMetrics, ToolRead
32+
from mcpgateway.types import InitializeResult, ResourceContent, ServerCapabilities
33+
34+
35+
# -----------------------------------------------------------------------------
36+
# Test fixtures (local to this file; move to conftest.py to share project-wide)
37+
# -----------------------------------------------------------------------------
38+
@pytest.fixture
39+
def test_client() -> TestClient:
40+
"""FastAPI TestClient with auth dependency overridden."""
41+
app.dependency_overrides[require_auth] = lambda: "integration-test-user"
42+
client = TestClient(app)
43+
yield client
44+
app.dependency_overrides.pop(require_auth, None)
45+
46+
47+
@pytest.fixture
48+
def auth_headers() -> dict[str, str]:
49+
"""Dummy Bearer token accepted by the overridden dependency."""
50+
return {"Authorization": "Bearer 123.123.integration"}
51+
52+
53+
# -----------------------------------------------------------------------------
54+
# Shared mock objects
55+
# -----------------------------------------------------------------------------
56+
MOCK_METRICS = {
57+
"total_executions": 1,
58+
"successful_executions": 1,
59+
"failed_executions": 0,
60+
"failure_rate": 0.0,
61+
"min_response_time": 0.01,
62+
"max_response_time": 0.01,
63+
"avg_response_time": 0.01,
64+
"last_execution_time": "2025-01-01T00:00:00",
65+
}
66+
67+
MOCK_TOOL = ToolRead(
68+
id="tool-1",
69+
name="test_tool",
70+
original_name="test_tool",
71+
url="http://example.com/tools/test",
72+
description="demo",
73+
request_type="POST",
74+
integration_type="MCP",
75+
headers={"Content-Type": "application/json"},
76+
input_schema={"type": "object", "properties": {"foo": {"type": "string"}}},
77+
annotations={},
78+
jsonpath_filter=None,
79+
auth=None,
80+
created_at=datetime(2025, 1, 1),
81+
updated_at=datetime(2025, 1, 1),
82+
is_active=True,
83+
gateway_id=None,
84+
execution_count=0,
85+
metrics=ToolMetrics(**MOCK_METRICS),
86+
gateway_slug="default",
87+
original_name_slug="test-tool",
88+
)
89+
90+
MOCK_SERVER = ServerRead(
91+
id="srv-1",
92+
name="test_server",
93+
description="integration server",
94+
icon=None,
95+
created_at=datetime(2025, 1, 1),
96+
updated_at=datetime(2025, 1, 1),
97+
is_active=True,
98+
associated_tools=[MOCK_TOOL.id],
99+
associated_resources=[],
100+
associated_prompts=[],
101+
metrics=MOCK_METRICS,
102+
)
103+
104+
MOCK_RESOURCE = ResourceRead(
105+
id=1,
106+
uri="file:///tmp/hello.txt",
107+
name="Hello",
108+
description="demo text",
109+
mime_type="text/plain",
110+
size=5,
111+
created_at=datetime(2025, 1, 1),
112+
updated_at=datetime(2025, 1, 1),
113+
is_active=True,
114+
metrics=MOCK_METRICS,
115+
)
116+
117+
# URL-escaped version of the resource URI (used in path parameters)
118+
RESOURCE_URI_ESC = urllib.parse.quote(MOCK_RESOURCE.uri, safe="")
119+
120+
121+
# -----------------------------------------------------------------------------
122+
# Integration test class
123+
# -----------------------------------------------------------------------------
124+
class TestIntegrationScenarios:
125+
"""Happy-path flows that stitch several endpoints together."""
126+
127+
# --------------------------------------------------------------------- #
128+
# 1. Create a tool ➜ create a server that references that tool #
129+
# --------------------------------------------------------------------- #
130+
@patch("mcpgateway.main.server_service.register_server", new_callable=AsyncMock)
131+
@patch("mcpgateway.main.tool_service.register_tool", new_callable=AsyncMock)
132+
def test_server_with_tools_workflow(
133+
self,
134+
mock_register_tool: AsyncMock,
135+
mock_register_server: AsyncMock,
136+
test_client: TestClient,
137+
auth_headers,
138+
):
139+
mock_register_tool.return_value = MOCK_TOOL
140+
mock_register_server.return_value = MOCK_SERVER
141+
142+
# 1a. register a tool
143+
tool_req = {"name": "test_tool", "url": "http://example.com"}
144+
resp_tool = test_client.post("/tools/", json=tool_req, headers=auth_headers)
145+
assert resp_tool.status_code == 200
146+
mock_register_tool.assert_awaited_once()
147+
148+
# 1b. register a server that references that tool
149+
srv_req = {
150+
"name": "test_server",
151+
"description": "integration server",
152+
"associated_tools": [MOCK_TOOL.id],
153+
}
154+
resp_srv = test_client.post("/servers/", json=srv_req, headers=auth_headers)
155+
assert resp_srv.status_code == 201
156+
assert resp_srv.json()["associatedTools"] == [MOCK_TOOL.id]
157+
mock_register_server.assert_awaited_once()
158+
159+
# --------------------------------------------------------------------- #
160+
# 2. MCP protocol: initialize ➜ ping #
161+
# --------------------------------------------------------------------- #
162+
@patch("mcpgateway.main.session_registry.handle_initialize_logic", new_callable=AsyncMock)
163+
def test_initialize_and_ping_workflow(
164+
self,
165+
mock_init: AsyncMock,
166+
test_client: TestClient,
167+
auth_headers,
168+
):
169+
mock_init.return_value = InitializeResult(
170+
protocolVersion="2025-03-26",
171+
capabilities=ServerCapabilities(prompts={}, resources={}, tools={}, logging={}, roots={}, sampling={}),
172+
serverInfo={"name": "gw", "version": "1.0"},
173+
instructions="hello",
174+
)
175+
176+
init_body = {
177+
"protocol_version": "2025-03-26",
178+
"capabilities": {},
179+
"client_info": {"name": "pytest", "version": "0.0.0"},
180+
}
181+
resp_init = test_client.post("/protocol/initialize", json=init_body, headers=auth_headers)
182+
assert resp_init.status_code == 200
183+
mock_init.assert_awaited_once()
184+
185+
resp_ping = test_client.post(
186+
"/protocol/ping",
187+
json={"jsonrpc": "2.0", "method": "ping", "id": "X"},
188+
headers=auth_headers,
189+
)
190+
assert resp_ping.status_code == 200
191+
assert resp_ping.json() == {"jsonrpc": "2.0", "id": "X", "result": {}}
192+
193+
# --------------------------------------------------------------------- #
194+
# 3. Resource life-cycle #
195+
# --------------------------------------------------------------------- #
196+
@patch("mcpgateway.main.resource_service.register_resource", new_callable=AsyncMock)
197+
@patch("mcpgateway.main.resource_service.read_resource", new_callable=AsyncMock)
198+
def test_resource_lifecycle(
199+
self,
200+
mock_read: AsyncMock,
201+
mock_register: AsyncMock,
202+
test_client: TestClient,
203+
auth_headers,
204+
):
205+
mock_register.return_value = MOCK_RESOURCE
206+
207+
create_body = {
208+
"uri": MOCK_RESOURCE.uri,
209+
"name": MOCK_RESOURCE.name,
210+
"description": "demo text",
211+
"content": "Hello", # required by ResourceCreate
212+
}
213+
resp_create = test_client.post("/resources/", json=create_body, headers=auth_headers)
214+
assert resp_create.status_code == 200
215+
mock_register.assert_awaited_once()
216+
217+
# read content
218+
mock_read.return_value = ResourceContent(type="resource", uri=MOCK_RESOURCE.uri, mime_type="text/plain", text="Hello")
219+
resp_read = test_client.get(f"/resources/{RESOURCE_URI_ESC}", headers=auth_headers)
220+
assert resp_read.status_code == 200
221+
assert resp_read.json()["text"] == "Hello"
222+
mock_read.assert_awaited_once()
223+
224+
# --------------------------------------------------------------------- #
225+
# 4. Invoke a tool via JSON-RPC #
226+
# --------------------------------------------------------------------- #
227+
@patch("mcpgateway.main.tool_service.invoke_tool", new_callable=AsyncMock)
228+
def test_rpc_tool_invocation_flow(
229+
self,
230+
mock_invoke: AsyncMock,
231+
test_client: TestClient,
232+
auth_headers,
233+
):
234+
mock_invoke.return_value = {
235+
"content": [{"type": "text", "text": "ok"}],
236+
"is_error": False,
237+
}
238+
239+
rpc_body = {"jsonrpc": "2.0", "id": 7, "method": "test_tool", "params": {"foo": "bar"}}
240+
resp = test_client.post("/rpc/", json=rpc_body, headers=auth_headers)
241+
assert resp.status_code == 200
242+
assert resp.json()["content"][0]["text"] == "ok"
243+
mock_invoke.assert_awaited_once_with(db=ANY, name="test_tool", arguments={"foo": "bar"})
244+
245+
# --------------------------------------------------------------------- #
246+
# 5. Metrics aggregation endpoint #
247+
# --------------------------------------------------------------------- #
248+
@patch("mcpgateway.main.prompt_service.aggregate_metrics", new_callable=AsyncMock, return_value={"p": 1})
249+
@patch("mcpgateway.main.server_service.aggregate_metrics", new_callable=AsyncMock, return_value={"s": 1})
250+
@patch("mcpgateway.main.resource_service.aggregate_metrics", new_callable=AsyncMock, return_value={"r": 1})
251+
@patch("mcpgateway.main.tool_service.aggregate_metrics", new_callable=AsyncMock, return_value={"t": 1})
252+
def test_metrics_happy_path(
253+
self,
254+
_tm,
255+
_rm,
256+
_sm,
257+
_pm,
258+
test_client: TestClient,
259+
auth_headers,
260+
):
261+
resp = test_client.get("/metrics", headers=auth_headers)
262+
assert resp.status_code == 200
263+
payload = resp.json()
264+
# Make sure all four keys are present regardless of exact values.
265+
for key in ("tools", "resources", "servers", "prompts"):
266+
assert key in payload

0 commit comments

Comments
 (0)