Skip to content

[Feature Request] pass addition inputs to target function when using the evaluate method #3526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yanggaome opened this issue Jul 9, 2024 · 2 comments
Assignees
Labels
enhancement New feature or request long-term

Comments

@yanggaome
Copy link

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is.

Not sure if this is already supported, but I couldn't find one example

from promptflow.evals.evaluate import evaluate

def user_call(*, question: str, **kwargs):
    # question is a column from data
    # how can I pass in additional inputs to this target user function?

user_call_config = get_config()
evaluate(
data = input_data,
target = user_call # how can i pass in user_call_config variable into user_call?
)

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Currently I am using global variables so the user_call can access, but would like to understand how I can pass in variables that not from data to that user_call

Additional context
Add any other context or screenshots about the feature request here.

@luigiw
Copy link
Member

luigiw commented Jul 18, 2024

This is blocked by pf core/devkit.

@yanggaome
Copy link
Author

@luigiw , i talked to Ankit and was able to have a workaround by creating the target as a class instead of function, i can store the information in the class.

but another thing is, instead of passing available columns where we need to explicitly specify which columns to be used, can we just get the entire row of data? that will help us to generalize the api signature

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request long-term
Projects
None yet
Development

No branches or pull requests

3 participants