Cover job submission API

[gingergeeks]

[laparn]

I would like to insist on this bug. It would be very interesting to have it. Obviously, drmaa (v1) is there and may be used. However, there is one drawback to drmaa : you can only run it as being the user using the program. What would be nice would be to have a launch(user="toto",...) function, where you could choose which user starts the job. It is a key function for any web interface (well there are ways to overcome that, but it will be not very smart).

[gingergeeks]

Once all the new API code is in for 14.11 plus the additional contributions from others, test examples and any major outstanding issues that may affect 14.11 are resolved then this will shortcoming will be addressed. I hope to start coding up a basic submit class over this christmas break.

[ajmazurie]

I second this feature request. After spending a while playing with the API to gather information about running jobs (I couldn't get information about the nodes yet, see #52) I was wondering about how to launch one. I was surprised not to find any reference to it. Is that a planned feature?

Best,
Aurélien

[gingergeeks]

Yep, I know it's been open for a long time and I agree but I need to resolve the outstanding issues first.

[giovtorres]

Hi everyone,

I know it has been a while but I wanted to let you know that we are working on wrapping job submit functions and hope to have a decent amount of it wrapped by the time 17.02 is released. For example:

>>> import pyslurm
>>> pyslurm.job.allocate_resources_blocking()
python: job 27205708 queued and waiting for resources
python: job 27205708 has been allocated resources
[debug] job_id:  27205708
[debug] node list:  cn0014
[debug] cancelled job id:  27205708

This is in the dev-16.05-props branch and will be eventually merged into master.

Giovanni

[leezu]

17.02 has been released by now. Is there any plan to go forward with merging the job submit wrapper functions in master? Thanks for your efforts in creating this software!

[giovtorres]

Hi @leezu, I've made some progress with this, but it's been a while. I need to create a branch based on 17.02. I'll try to get to that this week, then you will be able to test out the code. Thanks!

[leezu]

Thanks for the heads up @giovtorres. My university is using version 14.11.11, I'm not sure if your code relies on any APIs released in more recent versions. If not I'll be happy to help testing.

[giovtorres]

That version is a bit older. We've mostly been doing development on 16.05 and 17.02, focusing more on the newer versions of Slurm as they come out. Unfortunately, I don't think we will be backporting any new code into 14.11 versions.

[xuhang57]

@giovtorres Wondering how it goes with the job submission :)

[giovtorres]

Hi, sorry for the delay on this. I did manage to update the development code to 17.02. However, I haven't done much with the job submission portion just yet. I would have preferred to switch over to the newer API before and put efforts there when making newer changes, especially a big change such as job submission, in order to avoid duplication.

I'm currently setting up some automated testing, which should help move things along a bit.

[xuhang57]

Thanks, @giovtorres , just curious, have you got a chance to try to test this PR: #65?

Although it is based on slurm 2.6, it should be compatible (or minor modifications needed) with slurm 16.05.5 since no much code change on job submission between these two versions.