intended to work with python 3+
Based on pyNVML
- Allows you to get
nvidia-smioutput for multiple connected computers at once, and display it on a nice appindicator for Unity/Gnome users.
- Allows you to get a notification every time a new process is launched or finished. A minimum of 200MB memory use is needed for the notification to appear.
- This tool is aimed at small research teams, with multiple GPU-equipped computers, which you can manually ssh to. At a glance you can see every usage of your computer stock, and where you can launch your computation. It also provides some basis if you want to develop a tool to automatically launch your computation on the least busy computer of your network.
sudo python3 setup.py install
To allow clients to access your computer's smi stats, simply run
server_smi.py
But you can also enable it as a service that will be launched at boot.
- Ubuntu 15+ : the service file allow the server_smi to run automatically during boot
sudo systemctl daemon-reload
sudo systemctl enable server_smi.service
to uninstall:
sudo systemctl disable server_smi.service
- Ubuntu 14 : you have to daemonize the script and put it in init.d
sudo cp server_smi_daemon.sh /etc/init.d/.
sudo chmod 0755 /etc/init.d/server_smi_daemon.sh
sudo update-rc.d server_smi_daemon.sh defaults
to uninstall:
sudo update-rc.d -f service_smi_daemon.sh remove
(gnome and unity environment only for the moment):
client_smi_appindicator.py
client_smi.py
To add your own hosts, simply run a smi_client (CLI or gnome) once and add your entries in the json file that should be here:
~/.client_smi/hosts_to_smi.json
Default ones are ml1 and ml2 with associated colors
Server-side, gpu usage history is stored in ~/.server_smi/history.csv , usage is written on it every ~30 sec, feel free to make some data science with it.
To disable it, you can comment Line before installing