Add example for running a local/on-prem LLM with MCP

[stex2005]

It would be really nice to have an example that shows how to connect MCP to a locally hosted LLM (e.g. vLLM, Ollama, LM Studio, custom API).

Right now, the workflow is clear for using existing MCP-enabled clients (like Claude Desktop), but some users are asking how to:

• Point MCP to a self-hosted LLM endpoint
• Run the LLM service alongside ROS and the MCP server

Suggestion:

• Provide a sample config or script for connecting to a local LLM
• Document the minimum requirements (e.g., JSON tool/function call support in the LLM API)

[GSNCodes]

Hey @stex2005 / @rjohn-v
I'd like to work on this. I'm thinking of adding an example using Ollama :)

[Usimian]

I've used this with LMStudio and it does work although not perfectly (LLM related issues).
Still testing it with turtlesim before trying real hardware.
Thanks for the repo!

[mokcontoro]

@Usimian Wow great to see that you already tried this. If you can share some examples or tutorial, it would be fantastic. We would be able to host the example in the example folder if it looks good! Even if it is not very functional, we can share it in Discussion, then people would know the current status of on-prem LLMs with the integration of ROS MCP. It would be helpful for other engineers who want to try similar task.

[stex2005]

I’m currently working on a simplified version of a Gemini client, which will be added under a new client folder.

The idea is to keep each LLM client self-contained with its own README and setup instructions, making it easier to maintain and extend in the future. This will include an on-prem implementation as part of the broader client architecture discussed in this issue.