Skip to content

POC for AI UI generation by syed. #769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

SyedAbdullah58-dev
Copy link
Contributor

@SyedAbdullah58-dev SyedAbdullah58-dev commented Apr 4, 2025

PR Description

Hey @ashitaprasad @animator - This PR has only POC changes, The Idea is explained more in depth in the proposal in another PR. Currenlty I did not wrote the test cases neither I am executing it because for POC I used the main.dart file of the project and that is how I am running it. As I replaced the original main.dart code with mine.

Related Issues

#617

  • Closes #

Checklist

  • I have gone through the contributing guide
  • I have updated my branch and synced it with project main branch before making this PR
  • I am using the latest Flutter stable branch (run flutter upgrade and verify)
  • I have run the tests (flutter test) and all tests are passing

Added/updated tests?

We encourage you to add relevant test cases.

  • Yes
  • No, and this is why: please replace this line with details on why tests have not been included

OS on which you have developed and tested the feature?

  • Windows
  • macOS
  • Linux

@SyedAbdullah58-dev
Copy link
Contributor Author

In this PoC, AI is currently implemented as a rule-based decision engine (via the Python ai_ui_schema_generator.py script) that analyzes API response structures (e.g., JSON arrays, field types, value distributions) and intelligently maps them to UI components like tables, charts, dropdowns, toggles, etc.

In the current state of the PoC, the AI agent (ai_ui_schema_generator.py) is not yet integrated into the Flutter app. Instead, we're using a manually created mockSchema in Flutter that reflects the kind of UI schema the AI agent would generate.
However, the AI logic has already been implemented in Python. It:

Parses the API response (e.g., list of objects)
Inspects value types, uniqueness, and structure
Assigns suitable UI components like chart, toggle, dropdown, text
This Python module serves as the foundation of the AI agent, and the next step is to expose it as an API (e.g., via FastAPI) and connect it to the Flutter frontend. That integration will demonstrate the full AI-driven dynamic UI pipeline.

Architecture Diagram:

            +-------------------------+
            |     API Response       |
            |   (JSON or XML data)   |
            +-----------+-------------+
                        |
                        v
     +------------------+------------------+
     | AI Schema Generator (Python logic) |
     |  - Rule-based analysis             |
     |  - Data profiling / heuristics     |
     |  - Future LLM integration          |
     +------------------+------------------+
                        |
                        v
                +-------+--------+
                | UI Schema JSON |
                +-------+--------+
                        |
                        v
           +------------+-------------+
           | Flutter Dynamic Renderer |
           |  - Tables, Charts, Cards |
           |  - Toggles, Sliders, etc |
           +--------------------------+

The current PoC validates the rendering side while keeping the AI engine ready for connection in the next phase.
Generated_UI

While it's not an ML model yet, this component mimics an early expert system — using heuristics based on data types and patterns (e.g., numeric + low variance → dropdown, numeric + high variance → chart, booleans → toggle).

  • This architecture sets up the foundation for future integration with:
  • LLMs for semantic inference (e.g., label-based UI detection)
  • AutoML/ML models to learn from schema-to-UI mappings

So in summary, the “AI” is being used to analyze, reason, and generate UI layout decisions, replacing manual UI design steps — and is positioned to evolve into a smarter agent during GSoC.

@animator
Copy link
Member

animator commented Apr 4, 2025

Thank you for sending the PoC.
We have reviewed your approach.

@animator animator closed this Apr 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants