Skip to content

feat(friendly-meals-flutter): add ingredients OCR #127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat(friendly-meals-flutter): add ingredients OCR #127

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 62 additions & 2 deletions firebase-ai-friendly-meals/flutter/lib/recipe_generator_screen.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import 'dart:typed_data';

import 'package:firebase_ai/firebase_ai.dart';
import 'package:flutter/material.dart';
import 'package:flutter_markdown/flutter_markdown.dart';
import 'package:image_picker/image_picker.dart';

class RecipeGeneratorScreen extends StatefulWidget {
const RecipeGeneratorScreen({super.key});
Expand All @@ -15,6 +18,54 @@ class _RecipeGeneratorScreenState extends State<RecipeGeneratorScreen> {
bool _showRecipeCard = false;
String _generatedRecipe = "";
bool _isLoading = false;
bool _isExtractingIngredients = false;

void _pickImageAndExtractIngredients() async {
setState(() {
_isExtractingIngredients = true;
});

try {
final imagePicker = ImagePicker();
final pickedFile = await imagePicker.pickImage(
source: ImageSource.gallery,
);

if (pickedFile != null) {
final imageBytes = await pickedFile.readAsBytes();
final model = FirebaseAI.googleAI().generativeModel(
model: 'gemini-2.5-flash-lite',
);

final prompt = Content.multi([
TextPart("""
Please analyze this image and list all visible food ingredients.
Format the response as a comma-separated list of ingredients.
Be specific with measurements where possible,
but focus on identifying the ingredients accurately.
"""),
InlineDataPart('image/jpeg', imageBytes),
]);
Comment on lines +36 to +48
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For better maintainability and robustness, it's recommended to extract hardcoded values into constants and dynamically determine the image MIME type.

  1. Model Name & Prompt: The model name 'gemini-2.5-flash-lite' and the prompt text are hardcoded. Extracting them into const variables improves readability and makes them easier to update.
  2. MIME Type: The MIME type is hardcoded as 'image/jpeg'. The user might select a different image format (e.g., PNG), which could cause processing to fail. Using the mime package to determine the MIME type from the file path makes this more robust.

To apply this suggestion, you'll need to add import 'package:mime/mime.dart'; at the top of your file. The mime package is already available as a transitive dependency.

        const modelName = 'gemini-2.5-flash-lite';
        const ocrPrompt = """
            Please analyze this image and list all visible food ingredients.
            Format the response as a comma-separated list of ingredients.
            Be specific with measurements where possible, 
            but focus on identifying the ingredients accurately.
            """;
        final mimeType = lookupMimeType(pickedFile.path) ?? 'image/jpeg';
        final model = FirebaseAI.googleAI().generativeModel(
          model: modelName,
        );

        final prompt = Content.multi([
          TextPart(ocrPrompt),
          InlineDataPart(mimeType, imageBytes),
        ]);


final response = await model.generateContent([prompt]);

if (response.text != null) {
_ingredientsController.text = response.text!;
}
}
} catch (e) {
ScaffoldMessenger.of(context).showSnackBar(
SnackBar(
content: Text('Error processing image: $e'),
backgroundColor: Colors.red,
),
);
Comment on lines +57 to +62
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using BuildContext after an await call can lead to runtime errors if the widget is removed from the tree while the async operation is running (e.g., the user navigates to another screen). This violates the use_build_context_synchronously lint rule.

To prevent potential crashes, you should always check if the widget is still mounted before interacting with its BuildContext in an async method.

      if (!mounted) return;
      ScaffoldMessenger.of(context).showSnackBar(
        SnackBar(
          content: Text('Error processing image: $e'),
          backgroundColor: Colors.red,
        ),
      );

} finally {
setState(() {
_isExtractingIngredients = false;
});
}
}

void _handleGenerateRecipe() async {
if (_ingredientsController.text.isEmpty) {
Expand Down Expand Up @@ -74,9 +125,18 @@ class _RecipeGeneratorScreenState extends State<RecipeGeneratorScreen> {
const SizedBox(height: 16.0),
TextField(
controller: _ingredientsController,
decoration: const InputDecoration(
decoration: InputDecoration(
labelText: 'Enter ingredients (comma separated)',
border: OutlineInputBorder(),
border: const OutlineInputBorder(),
suffixIcon: _isExtractingIngredients
? const Padding(
padding: EdgeInsets.all(8.0),
child: CircularProgressIndicator(),
)
: IconButton(
icon: const Icon(Icons.camera_alt),
onPressed: _pickImageAndExtractIngredients,
),
),
),
const SizedBox(height: 16.0),
Expand Down
122 changes: 121 additions & 1 deletion firebase-ai-friendly-meals/flutter/pubspec.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,14 @@ packages:
url: "https://pub.dev"
source: hosted
version: "1.19.1"
cross_file:
dependency: transitive
description:
name: cross_file
sha256: "7caf6a750a0c04effbb52a676dce9a4a592e10ad35c34d6d2d0e4811160d5670"
url: "https://pub.dev"
source: hosted
version: "0.3.4+2"
crypto:
dependency: transitive
description:
Expand All @@ -81,6 +89,38 @@ packages:
url: "https://pub.dev"
source: hosted
version: "1.3.3"
file_selector_linux:
dependency: transitive
description:
name: file_selector_linux
sha256: "54cbbd957e1156d29548c7d9b9ec0c0ebb6de0a90452198683a7d23aed617a33"
url: "https://pub.dev"
source: hosted
version: "0.9.3+2"
file_selector_macos:
dependency: transitive
description:
name: file_selector_macos
sha256: "8c9250b2bd2d8d4268e39c82543bacbaca0fda7d29e0728c3c4bbb7c820fd711"
url: "https://pub.dev"
source: hosted
version: "0.9.4+3"
file_selector_platform_interface:
dependency: transitive
description:
name: file_selector_platform_interface
sha256: a3994c26f10378a039faa11de174d7b78eb8f79e4dd0af2a451410c1a5c3f66b
url: "https://pub.dev"
source: hosted
version: "2.6.2"
file_selector_windows:
dependency: transitive
description:
name: file_selector_windows
sha256: "320fcfb6f33caa90f0b58380489fc5ac05d99ee94b61aa96ec2bff0ba81d3c2b"
url: "https://pub.dev"
source: hosted
version: "0.9.3+4"
firebase_ai:
dependency: "direct main"
description:
Expand Down Expand Up @@ -182,6 +222,14 @@ packages:
url: "https://pub.dev"
source: hosted
version: "0.7.7+1"
flutter_plugin_android_lifecycle:
dependency: transitive
description:
name: flutter_plugin_android_lifecycle
sha256: "6382ce712ff69b0f719640ce957559dde459e55ecd433c767e06d139ddf16cab"
url: "https://pub.dev"
source: hosted
version: "2.0.29"
flutter_test:
dependency: "direct dev"
description: flutter
Expand All @@ -208,6 +256,70 @@ packages:
url: "https://pub.dev"
source: hosted
version: "4.1.2"
image_picker:
dependency: "direct main"
description:
name: image_picker
sha256: "021834d9c0c3de46bf0fe40341fa07168407f694d9b2bb18d532dc1261867f7a"
url: "https://pub.dev"
source: hosted
version: "1.1.2"
image_picker_android:
dependency: transitive
description:
name: image_picker_android
sha256: b08e9a04d0f8d91f4a6e767a745b9871bfbc585410205c311d0492de20a7ccd6
url: "https://pub.dev"
source: hosted
version: "0.8.12+25"
image_picker_for_web:
dependency: transitive
description:
name: image_picker_for_web
sha256: "717eb042ab08c40767684327be06a5d8dbb341fe791d514e4b92c7bbe1b7bb83"
url: "https://pub.dev"
source: hosted
version: "3.0.6"
image_picker_ios:
dependency: transitive
description:
name: image_picker_ios
sha256: "05da758e67bc7839e886b3959848aa6b44ff123ab4b28f67891008afe8ef9100"
url: "https://pub.dev"
source: hosted
version: "0.8.12+2"
image_picker_linux:
dependency: transitive
description:
name: image_picker_linux
sha256: "34a65f6740df08bbbeb0a1abd8e6d32107941fd4868f67a507b25601651022c9"
url: "https://pub.dev"
source: hosted
version: "0.2.1+2"
image_picker_macos:
dependency: transitive
description:
name: image_picker_macos
sha256: "1b90ebbd9dcf98fb6c1d01427e49a55bd96b5d67b8c67cf955d60a5de74207c1"
url: "https://pub.dev"
source: hosted
version: "0.2.1+2"
image_picker_platform_interface:
dependency: transitive
description:
name: image_picker_platform_interface
sha256: "886d57f0be73c4b140004e78b9f28a8914a09e50c2d816bdd0520051a71236a0"
url: "https://pub.dev"
source: hosted
version: "2.10.1"
image_picker_windows:
dependency: transitive
description:
name: image_picker_windows
sha256: "6ad07afc4eb1bc25f3a01084d28520496c4a3bb0cb13685435838167c9dcedeb"
url: "https://pub.dev"
source: hosted
version: "0.2.1+1"
leak_tracker:
dependency: transitive
description:
Expand Down Expand Up @@ -272,6 +384,14 @@ packages:
url: "https://pub.dev"
source: hosted
version: "1.16.0"
mime:
dependency: transitive
description:
name: mime
sha256: "41a20518f0cb1256669420fdba0cd90d21561e560ac240f26ef8322e45bb7ed6"
url: "https://pub.dev"
source: hosted
version: "2.0.0"
path:
dependency: transitive
description:
Expand Down Expand Up @@ -391,4 +511,4 @@ packages:
version: "3.0.3"
sdks:
dart: ">=3.8.0-278.1.beta <4.0.0"
flutter: ">=3.22.0"
flutter: ">=3.27.0"
1 change: 1 addition & 0 deletions firebase-ai-friendly-meals/flutter/pubspec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ dependencies:
flutter_markdown: ^0.7.7+1
firebase_core: ^4.0.0
firebase_ai: ^3.1.0
image_picker: ^1.1.2

dev_dependencies:
flutter_test:
Expand Down