Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
VERSION = "Electrum Protocol 1.4.x"
VERSION = "Electrum Protocol 1.6.x"

# -- Project information -----------------------------------------------------

Expand Down
38 changes: 22 additions & 16 deletions docs/protocol-basics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -68,14 +68,12 @@ revision number.

A party to a connection will speak all protocol versions in a range,
say from `protocol_min` to `protocol_max`, which may be the same.
When a connection is made, both client and server must initially
assume the protocol to use is their own `protocol_min`.

The client should send a :func:`server.version` RPC call as early as
possible in order to negotiate the precise protocol version; see its
description for more detail. All responses received in the stream
from and including the server's response to this call will use its
negotiated protocol version.
The client must send a :func:`server.version` RPC call as the first
message on the wire, in order to negotiate the precise protocol
version; see its description for more detail.
All responses received in the stream from and including the server's
response to this call will use its negotiated protocol version.


.. _script hashes:
Expand Down Expand Up @@ -144,23 +142,31 @@ Status
To calculate the `status` of a :ref:`script hash <script hashes>` (or
address):

1. order confirmed transactions to the script hash by increasing
height (and position in the block if there are more than one in a
block)
1. Consider all transactions touching the script hash (both those spending
from it, and those funding it), both confirmed and unconfirmed (in mempool).

2. form a string that is the concatenation of strings
2. Order confirmed transactions by increasing height (and position in the
block if there are more than one in a block).

3. form a string that is the concatenation of strings
``"tx_hash:height:"`` for each transaction in order, where:

* ``tx_hash`` is the transaction hash in hexadecimal

* ``height`` is the height of the block it is in.

3. Next, with mempool transactions in any order, append a similar
string for those transactions, but where **height** is ``-1`` if the
transaction has at least one unconfirmed input, and ``0`` if all
inputs are confirmed.
4. For mempool transactions, we define **height** to be ``-1`` if the
transaction has at least one unconfirmed input, and ``0`` if all inputs are
confirmed.

5. Order mempool transactions by ``(-height, tx_hash)``, that is,
``0`` height txs come before ``-1`` height txs, and secondarily the
txid (in network byteorder) is used to arrive at a canonical ordering.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that the byteorder used for the sorting is not the human-readable txid byteorder but the network one.
I chose this because it fits the existing internals of electrumx much nicer.

However in cculianu/electrum-cash-protocol#5 (comment) @cculianu expressed frustration about this choice.

I guess I don't mind either way though.
Any strong opinions?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate your asking others for feedback on this.

the network one.

Just to be annoying -- I'll smartly say that it can be argued the "network" order is only useful if you speak the p2p protocol. Otherwise it can be argued the true network order is the RPC order. :P

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any strong opinions?

Network byteorder makes sense to me for such an internal behavior, IMO. But, the most important thing is that it's well defined.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a detailed example in 919df4f, to try to avoid any misunderstanding.

Copy link

@cculianu cculianu Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You guys are seriously muddled in your thinking:

  • First off, referring to it as "network" order is misleading. WHICH network are we referring to here? The p2p protocol? Nobody involved in Electrum-style wallets speaks that protocol.
  • For the purposes of this discussion, let's be more precise:
    • Electrum protocol uses "big endian" hash order (that is, reversed from the in-memory order one gets from sha256d) -- it uses this EVERYWHERE in the protocol.
    • Bitcoin p2p network, for its messages, etc, uses "little endian" hash order (that is, exactly what you get from sha256d).

This latter thing you are calling "network" order. However the term is misleading because for our network, the network order is actually the other order! The big endian order!

Are we clear on that?

So what you are proposing is that this will be the only place anywhere within this protocol where we are to look at txn hashes as being in the opposite order from every other place.

Are we clear on what is happening here?

I strongly disagree with this approach since:

  • It's unexpected
  • It's ugly
  • There is no justification for it other than "well Electrum does it this way..."

Please change this to what Fulcrum was already doing since 2020.

I won't comply with this in Fulcrum because it's dumb. Sorry. I already solved this problem in 2020 and it's doing the "natural" thing -- using the hash order for the hashes as they appear everywhere else in the protocol. That's what most people expect.

What you want to do here is just unexpected and creates more work for me. And all for no good reason. No can do. Sorry.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a strong opinion, but Cormorant (Sparrow's EPS clone) is using big endian order as well. And I agree stating 'big endian' or 'little endian' is clearer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First off, referring to it as "network" order is misleading. WHICH network are we referring to here? The p2p protocol? Nobody involved in Electrum-style wallets speaks that protocol.

Fair enough, I meant the bitcoin p2p network.

  • For the purposes of this discussion, let's be more precise:
    • Electrum protocol uses "big endian" hash order (that is, reversed from the in-memory order one gets from sha256d) -- it uses this EVERYWHERE in the protocol.
    • Bitcoin p2p network, for its messages, etc, uses "little endian" hash order (that is, exactly what you get from sha256d).

Is that really more precise? Unlike what you say, the output of sha256 is big-endian. :D

see page 12 (of 36) of https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.180-4.pdf :

Throughout this specification, the “big-endian” convention is used

See also, https://en.wikipedia.org/wiki/SHA-256, where at least for the pseudo-code it states:

Big-endian convention is used [...]

Or see e.g. https://learnmeabitcoin.com/technical/general/little-endian/ , which says little-endian is used for integers but big-endian is used for hashes in the bitcoin p2p protocol.

Even if you still disagree, at least please accept that it is not intuitive or clear what big-endian or little-endian is when thinking of the byte-vector output of a hash function. The output is not an integer.
Clearly it is not intuitive if what you thought is actually the opposite of what the standard says.

I won't comply with this in Fulcrum because it's dumb. [...] No can do. Sorry.

Ok, I changed the order to what you suggested.
I wish you had been clearer from the start that this is a complete blocker for you.

See 394a8fd

And I agree stating 'big endian' or 'little endian' is clearer.

I disagree. I hope I illustrated how that would not be clear at all. :P
The updated text now says:

the txid (endianness same as human-readable hex string used for display) is used to arrive at a canonical ordering.

I think that should be clear. And in any case, there is an example.

and creates more work for me.

I just spent an hour updating the server code, the client code, and writing a new example for the spec.
It was writing the new example that took most of that time.

Is it really that much work to add a reverse iterator or a minus sign to your comparator?
The diff was 6 characters in e-x:
spesmilo/electrumx@0f3e261


6. Next, with mempool transactions in the specified order, append a similar
string.

4. The :dfn:`status` of the script hash is the :func:`sha256` hash of the
7. The :dfn:`status` of the script hash is the :func:`sha256` hash of the
full string expressed as a hexadecimal string, or :const:`null` if the
string is empty because there are no transactions.

Expand Down
43 changes: 42 additions & 1 deletion docs/protocol-changes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Protocol Changes
================

This documents lists changes made by protocol version.
This document lists changes made by protocol version.

Version 1.0
===========
Expand Down Expand Up @@ -175,3 +175,44 @@ New methods
-----------

* :func:`blockchain.name.get_value_proof` to resolve a name (with proof). Name index coins (e.g. Namecoin) only.


Version 1.5
===========

(this version number was skipped, no corresponding protocol is defined)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that I skipped 1.5.x as Fulcrum and https://github.com/cculianu/electrum-cash-protocol uses these version numbers.
However I now realise this is nevertheless somewhat problematic:
if a client wants to support protocol version 1.4 and 1.6 but not 1.5.x, they are in quite a pickle.
If the client sends server.version("electrum/4.6.2", ["1.4", "1.6"]) to Fulcrum 2.0, the server will select protocol version 1.5.x.

By design of server.version, clients must support a continuous range of versions.
OTOH, servers are free to pick-and-choose any particular protocol version they want and can skip interim versions.

From a quick read of "electrum-cash-protocol", I think all the changes in the interim versions are backwards compatible. So maybe this is all ~fine... Nevertheless, the situation is unfortunate.



.. _version 1.6:

Version 1.6
===========

Changes
-------

* Breaking change for the version negotiation: we now mandate that
the :func:`server.version` message must be the first message sent.
That is, version negotiation must happen before any other messages.
* Also for :func:`server.version`, the server must tolerate and ignore
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is stupid. Again, this is what the features map is for. But whatever.

extraneous arguments, to allow for extensions in future protocol versions.
* The status of a scripthash has its definition tightened in a
backwards-compatible way: mempool txs now have a canonical ordering
defined for the calculation (previously their order was undefined).
* :func:`blockchain.scripthash.get_mempool` previously did not define
an order for mempool transactions. We now mandate a specific ordering.
* Optional *mode* argument added to :func:`blockchain.estimatefee`.
* :func:`blockchain.block.headers` now returns headers as a list,
instead of a single concatenated hex string.

New methods
-----------

* :func:`blockchain.transaction.broadcast_package` to broadcast a package of transactions (submitpackage).
* :func:`mempool.get_info` to get more detailed and general relayfee info.

Removed methods
---------------

* :func:`blockchain.relayfee` is removed. The `minrelaytxfee` field
of the new :func:`mempool.get_info` RPC is a direct replacement.
Loading