A collection of scripts and procedures for operating a Stake Pool, DRep or a Cardano node. Developed by Upstream Stake Pool UPSTR.
File tree
tree --filesfirst -L 3
.
├── LICENSE
├── README.md
├── env.example
├── metadata
│ ├── drep.example.json
│ └── metadata.example.json
├── scripts
│ ├── address.sh
│ ├── common.sh
│ ├── govern.sh
│ ├── network.sh
│ ├── node.sh
│ ├── pool.sh
│ ├── query.sh
│ ├── tx.sh
│ └── node
│ ├── build.sh
│ ├── download.sh
│ ├── install.sh
│ ├── mithril.sh
│ └── update.sh
└── services
├── cardano-node.service
├── grafana-mithril-dashboard.json
├── grafana-node-dashboard.json
├── mithril.service
├── ngrok.service
└── prometheus.yml
Assumptions
- Your OS, LAN network, ports and user are already configured.
- The Ngrok script requires you to know how to set up your own ngrok account and endpoints.
- You are comfortable with cardano-node / cardano-cli and SPO requirements
- You are comfortable with Linux and managing networks and servers
- You are able to setup your cold node by copying the binaries, scripts and keys securely as required.
We use a docker container to run a local node simulation on testnets. This should not be used for mainnet.
# Build and start the docker containers
./docker/run.sh up -d --build
# OPTIONAL: run fixtures to generate address credentials
./docker/fixture.sh addresses
Once your containers are running you can run node operation scripts as usual:
# Run scripts in the container, e.g.
./docker/script.sh node.sh view
./docker/exec.sh node scripts/query.sh uxto
# OR Connect to the cardano node container
docker exec -it node bash
# Restart a container e.g. prometheus
./docker/run.sh restart prometheus
# Rebuld containers f changes have been made to compose OR .env file
./docker/run.sh up -d --build
This table describes the env variables you most likely need to adjust to suit your system and their available options. Read through these options before proceeding to the installation.
env variables
NODE_NETWORK
|
sanchonet preview preprod mainnet
|
One of the supported Cardano networks |
NODE_VERSION
|
10.1.4 |
The current node version. Must be > the version defined here. |
NODE_HOME
|
"/home/upstream/Cardano"
|
The home folder for your node. |
NODE_TYPE
|
relay producer cold
|
The type of node you are running. |
NODE_USER
|
upstream
|
The user running the node. |
NODE_BUILD
|
0 1 2
|
The build type. |
NODE_PORT
|
6000
|
The nodes local port number. |
NODE_HOSTADDR
|
0.0.0.0
|
The nodes local host address. |
NODE_CARDANOSCAN_API
|
API key
|
A Cardanoscan.io API key used to fetch pool data displayed in Grafana. |
MITHRIL_RELAY_HOST
|
http:192.168.X.X
|
Your mithril relay host address excluding port. |
MITHRIL_RELAY_PORT
|
1234
|
Your mithril relay port. |
BIN_PATH
|
$HOME/local/bin
|
Your users local bin path. |
PACKAGER
|
apt-get
|
System package manager. |
SERVICE_PATH
|
/etc/systemd/system
|
System service path. |
IMPORTANT - Skip the node install steps for docker environments
Get started by creating a directory and pulling this repo, and edit the env file (see table below for env descriptions and configure).
mkdir Cardano && cd Cardano
git clone https://github.com/devhalls/spo-operational-scripts.git .
Create and edit your env file:
cp -p env.example env && nano env
When your env is configured, run the installation.
scripts/node.sh install
IMPORTANT - Mithril snapshots are downloaded during docker installation and can be skipped for docker environments.
Once installation is complete, download the mithril binaries and run mithril sync.
scripts/node.sh mithril download
scripts/node.sh mithril sync
IMPORTANT - Skip this step for docker environments and use docker scripts to start and stop the node.
After installation is complete you can start, stop or restart the node service.
scripts/node.sh start
scripts/node.sh stop
scripts/node.sh restart
When you would like to update the node, edit the env with your new target NODE_VERSION and run the node update script.
nano env
scripts/node.sh update
This is an example of allowing the node port through a firewall, its expected you will secure your node as appropriate for mainnet releases.
# Allow SSH
sudo ufw allow OpenSSH
# Allow node traffic
sudo ufw allow $NODE_PORT/tcp
# Restart any apply rule
sudo ufw disable
sudo ufw enable
To register a stake pool you must have a running fully synced node. We can then generate the following assets:
pool assets
|
payment verification key |
|
payment signing key |
|
funded address linked to stake |
|
staking verification key |
|
staking signing key |
|
registered stake address |
|
cold signing key |
|
cold verification key |
|
KES signing key |
|
KES verification key |
|
VRF signing key |
|
VRF verification key |
|
operational certificate |
|
issue counter |
metadata url |
Public URL for metadata file |
metadata hash |
Hash of the json file |
Start your pool registration by generating node keys and a node operational certificate, along with your KES keys and VRF keys.
# PRODUCER: Query network params and take note of the 'KES period'
scripts/query.sh params
scripts/query.sh kes_period
# COLD: Genreate node keys and operational certificate
scripts/pool.sh generate_kes_keys
scripts/pool.sh generate_node_keys
scripts/pool.sh generate_node_op_cert <KES period>
# COPY: node.cert to your producer node
# PRODUCER: Generate your node vrf key
scripts/pool.sh generate_vrf_keys
Create payment keys, stake keys and generate addresses from the keys. Ensure you fund your payment addres and query the chain to confirm your have UXTOs.
# COLD: Generate payment and stake keys
scripts/address.sh generate_payment_keys
scripts/address.sh generate_stake_keys
scripts/address.sh generate_payment_address
scripts/address.sh generate_stake_address
# COPY: The payment.addr and stake.addr to your producer node
# EXTERNAL: Fund your payment address (see faucet link at the bottom of this readme)
# PRODUCER: Query the address uxto to ensure funds arrived in your payment.addr
scripts/query.sh uxto
Create a stake address certificate and submit the transaction to complete the registration.
# COLD: Get the stakeAddressDeposit value then generate a stake certificate
scripts/query.sh params stakeAddressDeposit
scripts/address.sh generate_stake_reg_cert <lovelace>
# COPY: stake.cert to your producer node
# PRODUCER: build stake registration tx
scripts/tx.sh stake_reg_raw
# COPY: tx.raw to your cold node
# COLD: Sign the stage registration transaction tx.raw
scripts/tx.sh stake_reg_sign
# COPY: tx.signed to your producer node
# PRODUCER: Submit the signed transaction
scripts/tx.sh submit
Create a pool registration certificates and submit the transaction to complete the registration.
# PRODUCER: Generate your metadata hash and take note along with the min pool cost
scripts/pool.sh generate_pool_meta_hash
scripts/query.sh params minPoolCost
# COLD: Gerenate pool registration certificate and pool delegate certificate
scripts/pool.sh generate_pool_reg_cert <pledge>, <cost>, <margin>, <relayAddr>, <relayPort>, <metaUrl>, <metaHash>
scripts/address.sh generate_stake_del_cert
# COPY: pool.cert and deleg.cert to your producer node
# PRODUCER: build you pool cert raw transaction
scripts/tx.sh pool_reg_raw
# COPY: tx.raw to your cold node
# COLD: Sign the pool certificate transaction tx.raw
scripts/tx.sh pool_reg_sign
# COPY: tx.signed to your producer node
# PRODUCER: Submit the signed transaction
scripts/tx.sh submit
# COLD: Now you have registered your pool, get your pool.id
scripts/pool.sh get_pool_id
# COPY: pool.id to your producer node, and to your replay node
To complete pool registration edit your topology to suit your replay configuration and restart your producer node.
# PRODUCER: Edit your typology and add your relay configuration
nano cardano-node/topology.json
# PRODUCER: Update your env NODE_TYPE=producer
nano env
# PRODUCER: Then restart the producer
scripts/node.sh restart
As an SPO there are a few things you must do to keep a producing block producing pool.
Knowing what's going on under the hood is essential to running a node.
# Display the node service status
scripts/node.sh status
# Run the gLiveView script
scripts/node.sh view
# Watch the node service logs
scripts/node.sh watch
# Read file contents from the node directories
scripts/query.sh config topology.json
scripts/query.sh key stake.addr
# Query your KES period and state
scripts/query.sh kes_period
scripts/query.sh kes_state
# Query the tip or chain params, with an optional param name
scripts/query.sh tip
scripts/query.sh tip epoch
scripts/query.sh params
scripts/query.sh params treasuryCut
# Query node prometheus metrics
scripts/query.sh metrics
scripts/query.sh metrics cardano_node_metrics_peerSelection_warm
View your node state via Grafana dashboards makes it easy to manage your nodes. Once you have installed the necessary packages and configs restart your nodes and you can visit the dashboard. Dashboard: MONITOR_NODE_IP:3000 Username: admin Password: admin (change your password after first login)
# ALL NODES: Install prometheus explorer on all nodes
scripts/node.sh install prometheus_explorer
# MONITOR: Install grafana on the monitoring node only
scripts/node.sh install grafana
# ALL NODES: Check the service status
scripts/node.sh watch_prom_ex
scripts/node.sh status_prom_ex
# MONITOR: Check the service status
scripts/node.sh watch_prom
scripts/node.sh watch_grafana
scripts/node.sh status_prom
scripts/node.sh status_grafana
# ALL NODES: Restart the prometheus services
scripts/node.sh restart_prom
# MONITOR: Restart the grafana services
scripts/node.sh restart_grafana
# MONITOR: Edit your prometheus config to collect data from all your replays, then restart
sudo nano /etc/prometheus/prometheus.yml
scripts/node.sh restart_prometheus
# MONITOR: You may need to add the prometheus user to the folders group to avoid permission issues
sudo usermod -a -G upstream prometheus
# MONITOR: You may also need to change the user in the prometheus.yml if you still experience permissions issues
sudo nano /lib/systemd/system/prometheus-node-exporter.service
To enable metrics from Cardanoscan API, set the env API key in NODE_CARDANOSCAN_API, then run the following commands:
# MONITOR: create the pool.id file and paste in your 'Pool ID' which you can get from https://cardanoscan.io (or generate it on your cold device)
nano cardano-node/keys/pool.id
# MONITOR: Check you can retrieve stats, and check if theres no error in the response
scripts/pool.sh get_stats
# If successful (you see stats output) setup a crontab to fetch data periodically
crontab -e
# Get data from Cardanoscan every hour at 5 past the hour
5 * * * * /home/upstream/Cardano/scripts/pool.sh get_stats >> /home/upstream/Cardano/cardano-node/logs/crontab.log 2>&1
You must rotate your KES keys every 90 days or you will not be able to produce blocks.
# PRODUCER: Check the current status and take note of the 'kesPeriod'
scripts/query.sh kes
scripts/query.sh kes_period
# COLD: Rotate the node
scripts/pool.sh rotate_kes <kesPeriod>
# COPY: node.cert and kes.skey to producer node
# PRODUCER: Restart the node
scripts/node.sh restart
# PRODUCER: Check the updates have applied
scripts/query.sh kes
Checking when you are due to mint blocks is essential to running your stake pool.
# PRODUER: Check the next epoch leader schedule
scripts/query.sh leader next
# PRODUER: OR you can check the current epoch if needed
scripts/query.sh leader current
# COPY: Copy the out put ready to past to your monitor nodes grafana csv file
# MONITOR: Paste in the below file (if your runnong a testnet node with only a producer this is done automatically)
sudo nano /usr/share/grafana/slots.csv
It's vitally import you make multiple backups of your node cold keys, this is what gives you control over your node. You can also backup your producer and relays to make redeployment simpler.
# COLD: Backup the keys.
.
├── $NETWORK_PATH/keys
# PRODUCER: Backup the below directories and env configuration, EXCLUDING the $NETWORK_PATH/db folder which contains the blockchain database.
.
├── env
├── metadata
├── $NETWORK_PATH
When you need to update your pool metadata, min cost or other pool params you must regenerate your pool.cert and deleg.cert using the same steps as when you first created these.
# PRODUCER: Generate your metadata hash and take note along with the min pool cost
scripts/pool.sh generate_pool_meta_hash
scripts/query.sh params minPoolCost
# COLD: Gerenate pool registration certificate and pool delegate certificate
scripts/pool.sh generate_pool_reg_cert <pledge>, <cost>, <margin>, <relayAddr>, <relayPort>, <metaUrl>, <metaHash>
scripts/address.sh generate_stake_del_cert
# COPY: pool.cert and deleg.cert to your producer node
# PRODUCER: build you pool cert raw transaction passing in 0 as a deposit for renewals
scripts/tx.sh pool_reg_raw 0
# COPY: tx.raw to your cold node
# COLD: Sign the pool certificate transaction tx.raw
scripts/tx.sh pool_reg_sign
# COPY: tx.signed to your producer node
# PRODUCER: Submit the signed transaction
scripts/tx.sh submit
In order to withdraw your SPO rewards you will need to participate in Cardano Governance by delegating your voting power. You have 4 possibilities when choosing how you wish to participate, listed below. Along with these you can also register your stake address as a DRep and participate in Governance directly as your own representative.
- delegate to a DRep who can vote on your behalf
- delegate to a DRep script who can vote on your behalf
- delegate your voting power to auto abstain
- delegate your voting power to a vote of on-confidence
# COLD: Generate your vote delegation certificate using one of the 4 options:
scripts/address.sh generate_stake_vote_cert drep <drepId>
scripts/address.sh generate_stake_vote_cert script <scriptHash>
scripts/address.sh generate_stake_vote_cert abstain
scripts/address.sh generate_stake_vote_cert no-confidence
# PRODUCER: Build the tx.raw with the $DELE_VOTE_CERT
scripts/tx.sh stake_vote_reg_raw
# COPY: tx.raw to your cold node
# COLD: Sign the withdraw transaction tx.raw
scripts/tx.sh stake_reg_sign
# COPY: tx.signed to your producer node
# PRODUCER: Submit the signed transaction
scripts/tx.sh submit
# PRODUCER: build you pool cert raw transaction
scripts/tx.sh pool_withdraw_raw
# COPY: tx.raw to your cold node
# COLD: Sign the withdraw transaction tx.raw
scripts/tx.sh stake_reg_sign
# COPY: tx.signed to your producer node
# PRODUCER: Submit the signed transaction
scripts/tx.sh submit
Running a Stake Pool requires participation in Cardano governance. From time to time you will need to cast your SPO vote for various governance actions.
# PRODUCER: Query the govern action id then build the vote
scripts/govern.sh action <govActionId>
# COLD: cast your vote on your cold machine
scripts/govern.sh vote <govActionId> <govActionIndex> <'yes' | 'no' | 'abstain'>
# COPY: vote.raw to your producer node
# PRODUCER: build the raw transaction with vote.raw as input
scripts/tx.sh vote_raw
# COPY: tx.raw to your cold node
# COLD: Sign the vote transaction tx.raw
scripts/tx.sh vote_sign
# COPY: tx.signed to your producer node
# PRODUCER: Submit the signed transaction
scripts/tx.sh submit
# PRODUCER: Get the retirement epoch window
poolRetireMaxEpoch=$(scripts/query.sh params poolRetireMaxEpoch)
epoch=$(scripts/query.sh tip epoch)
minRetirementEpoch=$(( ${epoch} + 1 ))
maxRetirementEpoch=$(( ${epoch} + ${poolRetireMaxEpoch} ))
echo earliest epoch for retirement is: ${minRetirementEpoch}
echo latest epoch for retirement is: ${maxRetirementEpoch}
# COLD: generate deregistration certificate ($POOL_DREG_CERT)
scripts/pool.sh generate_pool_dreg_cert <epoch>
# COPY: copy pool.dereg to producer
# PRODUCER: bbuild a tx with the dregistration certificate
scripts/tx.sh build 0 --certificate-file cardano-node/keys/pool.dereg
# COPY: copy temp/tx.raw to cold
# COLD: sign the transaction with your payment and node keys
scripts/tx.sh sign --signing-key-file cardano-node/keys/payment.skey --signing-key-file cardano-node/keys/node.skey
# COPY: copy temp/tx.signed to cold
To register as a DRep you must have a running fully synced node. We can then generate the following assets:
DRep assets
|
DRep verification key |
|
DRep signing key |
|
DRep certificate |
|
DRep ID |
Start your DRep registration by generating keys and a DRep certificate.
# PRODUCER: Generate DRep keys and ID
scripts/govern.sh drep_keys
scripts/govern.sh drep_id
# PRODUCER: Generate the DRep registration certificate assuming oyu have a public metadata URL
scripts/govern.sh drep_cert <url>
# PRODUCER: Build a transaction with the drep certificate
scripts/tx.sh drep_reg_raw
# COPY: tx.raw to your cold node
# COLD: Sign the transaction
scripts/tx.sh drep_reg_sign
# COPY: tx.signed to your producer node
# PRODUCER: Submit the transaction
scripts/tx.sh submit
Being a DRep requires participation in Cardano governance. From time to time you will need to cast your DRep vote for various governance actions.
# PRODUCER: Query the govern action id then build the vote
scripts/govern.sh action <govActionId>
# COLD: cast your vote on your cold machine - to vote as a DRep ensure you pass the last param: 'drep'
scripts/govern.sh vote <govActionId> <govActionIndex> <'yes' | 'no' | 'abstain'> drep
# COPY: vote.raw to your producer node
# PRODUCER: build the raw transaction with vote.raw as input
scripts/tx.sh vote_raw
# COPY: tx.raw to your cold node
# COLD: Sign the vote transaction tx.raw
scripts/tx.sh vote_sign /home/upstream/Cardano/cardano-node/keys/keys/drep.skey
# COPY: tx.signed to your producer node
# PRODUCER: Submit the signed transaction
scripts/tx.sh submit
To register as a DRep you must have a running fully synced node. We can then generate the following assets:
Committee member assets
|
Committee hot verification key |
|
Committee hot signing key |
|
Committee cold verification key |
|
Committee cold signing key |
|
Hashed cold verification key |
|
Committee hot > cold certificate |
Start your Committee registration by generating keys and a certificate.
# COLD: Generate Committee keys and certificate
scripts/govern.sh cc_cold_keys
scripts/govern.sh cc_cold_hash
scripts/govern.sh cc_hot_keys
scripts/govern.sh cc_cert
# COPY: copy the cc.cert to the producer
# PRODUCER: Build a transaction with the Committee certificate
scripts/tx.sh build 0 2 --certificate-file "/home/upstream/Cardano/cardano-node/keys/cc.cert"
# COPY: tx.raw to your cold node
# COLD: Sign the transaction
scripts/tx.sh sign --signing-key-file "/home/upstream/Cardano/cardano-node/keys/payment.skey" --signing-key-file "/home/upstream/Cardano/cardano-node/keys/cc-cold.skey"
# COPY: tx.signed to your producer node
# PRODUCER: Submit the transaction
scripts/tx.sh submit
Although voting methods are detailed above this updated process makes it easier to build transactions:
# COPY: Create you public rational named rationale-GOV_ACTION_ID-GOV_ACTION_INDEX.jsonld and publish to a public location
# COLD: Then create the vote.json file as a DRep or SPO:
scripts/govern.sh vote_json $GOV_ACTION_HEX $GOV_ACTION_INDEX abstain $RATIONALE_URL.jsonld
# COPY: vote.json to producer
# PRODUCER: build your raw transaction
scripts/tx.sh build 0 2 --metadata-json-file ~/Cardano/cardano-node/temp/vote.json
# COLD: sign the transaction
scripts/tx.sh sign --signing-key-file ~/Cardano/cardano-node/keys/node.skey --signing-key-file ~/Cardano/cardano-node/keys/payment.skey
# PRODUCER: send the transaction
scripts/tx.sh submit
Installed on a Relay connected to your block producing SPOs topology.
# RELAY: Download blockfrost and init
scripts/node/icebreaker.sh download
# RELAY: Install blockfrost service and start
scripts/node/icebreaker.sh install
# Check installed version
blockfrost-platform --version
You can review icebreaker status using the BlockFrost UI:
- https://blockfrost.grafana.net/public-dashboards/8d618eda298d472a996ca3473ab36177
- https://platform.blockfrost.io/verification
If your running a block producing Stake Pool on the Preview network, you can opt to run a Midnight block producer. Following the current guides from Midnight testnet guides to deploy alongside the Cardano Preview network.
- Midnight Docs - How to become a Midnight Validator
- Midnight GitHub - Docker compose
- Midnight Monitoring - LiveView
For installation within our toolchain, install the above dependencies, clone the Midnight repository and follow the setup docs.
# Setup directory and clone the repo
source script/common.sh
cd $NODE_HOME && mkdir cardano-midnight && cd cardano-midnight
git clone https://github.com/midnightntwrk/midnight-node-docker.git .
# Follow the midnight docs for full setup instructions
# Launch wizard used for configurations once all partner services are up and running
./midnight-node.sh wizards --help
# Then you can start and restart containers
docker compose -f ./compose-partner-chains.yml -f ./compose.yml -f ./proof-server.yml up -d
docker compose -f ./compose-partner-chains.yml -f ./compose.yml -f ./proof-server.yml restart
Once you have completed the registration steps and all services are operational, you can validate your node operations and registration by querying the local rpc.
Note you MUST modify the APPEND_ARGS in .envrc
or author RPC calls will fail:
export APPEND_ARGS="--validator --allow-private-ip --pool-limit 10 --trie-cache-size 0 --prometheus-external --unsafe-rpc-external --rpc-methods=Unsafe --rpc-cors all"
Example registration at epoch 997 will return the validator list with your sidechain_pub_key present. Search the results to ensure you are present and the 'isValid' parameter is true.
# Query sidechain status
curl -L -X POST -H "Content-Type: application/json" -d '{
"jsonrpc": "2.0",
"method": "sidechain_getStatus",
"params": [997],
"id": 1
}' http://127.0.0.1:9944 | jq
# Query the validators and find your sidechain_pub_key
curl -L -X POST -H "Content-Type: application/json" -d '{
"jsonrpc": "2.0",
"method": "sidechain_getAriadneParameters",
"params": [999],
"id": 1
}' http://127.0.0.1:9944 | jq
To confirm your Midnight Validator keys are configure correctly query the author_hasKey for each key:
# Validate the sidechain_pub_key
curl -L -X POST -H "Content-Type: application/json" -d '{
"jsonrpc": "2.0",
"method": "author_hasKey",
"params": ["<KEY>", "crch"],
"id": 1
}' http://127.0.0.1:9944 | jq
# Validate the aura_pub_key
curl -L -X POST -H "Content-Type: application/json" -d '{
"jsonrpc": "2.0",
"method": "author_hasKey",
"params": ["<KEY>", "aura"],
"id": 1
}' http://127.0.0.1:9944 | jq
# Validate the grandpa_pub_key
curl -L -X POST -H "Content-Type: application/json" -d '{
"jsonrpc": "2.0",
"method": "author_hasKey",
"params": ["<KEY>", "gran"],
"id": 1
}' http://127.0.0.1:9944 | jq
Once running you can monitor you midnight node using the docker logs and the community tool ./LiveView.sh linked above.
# Enter the node shell
docker exec -it <CONTAINER_ID> bash
# Watch logs for each midnight service
docker logs -f --tail 100 cardano-ogmios
docker logs -f --tail 100 cardano-db-sync
docker logs -f --tail 100 db-sync-postgres
docker logs -f --tail 100 cardano-node
docker logs -f --tail 100 midnight-node
# LiveView tool is our recommended way to monitor your Midnight producer
./LiveView.sh
( )
Parenthesis = mandatory parameters.[ ]
Square brackets = optional parameters.< >
Angle brackets = parameter types.|
Bar = Choice between several options.
Usage: query.sh [
tip [name <STRING>] |
params [name <STRING>] |
metrics [name <STRING>] |
config (name <STRING>) |
key [name <STRING>] |
kes |
kes_period |
uxto [address <ADDRESS>] |
leader [period <INT>] |
rewards [name <STRING>] |
help [-h]
]
- Upstream SPO - @upstream_ada
- Devhalls - @devhalls
- Grafana dashboard and leader slot query initially used from https://github.com/sanskys/SNSKY
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this plugin better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (git checkout -b feature/AmazingFeature)
- Commit your Changes (git commit -m 'Add some AmazingFeature')
- Push to the Branch (git push origin feature/AmazingFeature)
- Open a Pull Request
Distributed under the GPL-3.0 License. See LICENSE.txt for more information.