Skip to content

[kubectl-plugin] Use dashboard API instead of the stdout of the ray job submit CLI to get Job submission ID #3556

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
MortalHappiness opened this issue May 7, 2025 · 1 comment · May be fixed by #3569
Open

[kubectl-plugin] Use dashboard API instead of the stdout of the ray job submit CLI to get Job submission ID #3556

MortalHappiness opened this issue May 7, 2025 · 1 comment · May be fixed by #3569
Assignees
@LeoLiao123
Labels
cli kubectl plugin

Comments

@MortalHappiness
Copy link
Member

MortalHappiness commented May 7, 2025 •
edited
Loading

See ray-project/ray#52780 (comment) and ray-project/ray#52780 (comment)

It's fine to wrap the ray job submit command, but we should not rely on its stdout. We need to call the dashboard REST API instead.

Since we'll only submit one job to one RayJob CR, we can use the list job API. It should return only one job. Then we can get the job submission ID from that. Make requests to http://localhost:8265/api/jobs/ in the head pod.

Note that calling dashboard API should be in a dedicated go routine so that it won't be blocked by the ray job submit CLI.

Related code snippets:
@MortalHappiness MortalHappiness mentioned this issue May 7, 2025
Merged
8 tasks
@MortalHappiness MortalHappiness added the cli kubectl plugin label May 7, 2025
@MortalHappiness MortalHappiness changed the title [kubectl-plugin] Use dashboard API instead of ray job submit CLI for kubectl ray job submit [kubectl-plugin] Use dashboard API instead of the stdout of the ray job submit CLI to get Job submission ID May 7, 2025
@LeoLiao123
Copy link
Contributor

Hi @MortalHappiness, I would like to take on this!

@LeoLiao123 LeoLiao123 linked a pull request May 9, 2025 that will close this issue
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@LeoLiao123 LeoLiao123

Labels
cli kubectl plugin
Projects
None yet
Milestone
No milestone
2 participants
@MortalHappiness @LeoLiao123