Pick up a completed ensemble and run for longer (in model time)

[CRosieWilliams]

A useful feature would be to be able to take a complete (or partially complete) ensemble and send it off again but with a longer model end time (i.e. with all the same parameters but, say, if each run was run for 50 years, and we now want to run another 50 years). It would need to pick up the last checkpoint from the original run and run again with everything the same but with a modified end time.

Current fix (by James) is:

Say you have run-1, run-3, run-6 and run-7

Copy these so that they're number 0-3
Copy the original template to a new folder
Edit the driver.j2 for new end time
Edit each LAST_EXIT to 124
Rerun as new ensemble

Copy run to be named new-run and template to be new_template

[JimCircadian]

This might be easily possible once the WAVIhpc additions are in place to sort out not rerunning whole ensembles, plus we could potentially rewrite the driver file dynamically (well in fact you can do that already, by supplying a CLI argument with the length of simulation run and then templating that into the driver file...)

Sooooo, really this is a WAVIhpc change, but I'll leave it here for the moment 😄