Skip to content

Commit 4aa8f84

Browse files
cculianu=fyookballblockparty-sh
authored
Add RPA Support (#234)
* Start adding RPA files. * Update Servers.h -- add batchid for rpc methods * Update Servers.cpp -- add batchId to RPA methods * Update Servers.cpp - add batchId params to generic async * Add key 'rpa' to features map to quell client-side warnings * Code quality fixups and make it compile on latest clang It wasn't compiling at all on latest clang. Also in this commit some code quality fixups and nits, and avoid some double-copies. Also added additional unit testing of prefixSearch & remove functionality. * fix bug * add some sloppy testing code for debug of client Also in this commit: Add files missed by previous merge * Optimize ReusableBlock::serializeInput to be faster This should help reduce CPU usage on initial synch and in general. We added a facility to hash bitcoin objects "in-place", rather than what we were doing before which was serializing them then hashing the serialized bytes. * Refactor - Move the serialization stuff into the .cpp file to avoid header noise and speed up compilation. - Add the trie map thingie into the headers for Fulcrum.pro - Misc. other small nits * Added utility class PackedNumView We will need this later for our new rpa data storage technique. * Added the `Rpa` module This will replace the facilities in `ReusableBlock.cpp` & `.h`. Also ported over the unit tests from `ReusableBlock` to this `Rpa` module. * Tweak to support PackedNumView of 32-bits * Made Rpa::PrefixTable support a read-only "view" into serialized data We will need this in order to quickly be able to read from the DB without too much allocation or other processing to service requests. Also in this commit: - Updated unit tests - Modified GenericVectorReader: added GetPos() and seek() methods * Rpa::PrefixTable ser/deser error path tweak Improved exeption messages and added paranoia check(s) * Some tweaks and additional in-code comments Small refactoring tweaks to the Rpa namespace classes and some small amounts of comments added to document the intention behind the code better. * Small perf. tweak for BTC::Hash2ByteArrayRev And also added some unit tests for various functions we touched/added recently. Also a small nit/refactor in Rpa.h * Removed Jt's Trie-based implementation, swapped in my own Also added some tests and other refactorings. Still TODO: - Mempool handling - Options handling to enable/disable this index - Finish TODOs in comments - Lots of other stuff like maybe an asynch indexing of RPA in the background for servers that are already "up" * Made Rpa logging less verbose by default * Allocate DB memory property for RPA (don't exceed db_mem) Also in this commit, some nits. TODO: If RPA index is disabled, give the memory back to scripthash_unspent and utxoset (which is where we took it from). * Fixed hex parsing bug for blockchain.reusable.* RPCs Turns out our Prefix(uint16, uint8_t) c'tor was buggy due to misplaced parens, so RPC was broken. Fixed. Also added unit tests to test this case as well as others. Also added some perf logging for dev (to be removed later) to the guts function that does the work for blockchain.reusable.get_history. * Nit * Tweaks to unit tests * Added better profile printing for debug, plus 1 nit * Fixed arg parsing for blockchain.reusable.get_history * Added come conf file args for RPA, renamed RPC methods, raised min prefix to 8 bits Conf file args to control various RPA aspects (min prefix, max history, etc) were added. Also, renamed blockchain.reusable.* -> blockchain.rpa.*. The old blockchain.reusable names are still supported but are deprecated. We raised the min prefix to 8 bits because 4 is too small and leads to heavy-ish server load on some queries. We also set the number of blocks one can scan with blockchain.rpa.get_history to a limit of 60 by default (configurable), to make for small and light queries to the server. * Removed unused #include * Added MempoolPrefixTable Will be used by the mempool. Still needs tests. * Simplified MempoolPrefixTable (it doesn't need 2 associative containers) * Hooked RPA into Mempool; works. Also added "tests" in the mempool bench to use it. * Added some more MempoolPrefixTable unit tests * Added more logic to Storage and Controller to handle RPA - added an "auto" mode that is auto-on for BCH, off for every other coin - user can override this auto mode (which is the default) with a cli or conf file arg - misc nits and fixups Still more to do in this regard. * Added rpa_start_height conf option Suppress indexing until this height. Defaults to -1 which means "Automatic" and is height 825,000 for mainnet, 0 for all other nets. * Tweaks to RPA max history code - Re-use the history-too-large lambda mechanism we use in getHistory() - Have rpa_max_history inherit max_history if max_history was specified and rpa_max_history was not (since this is what users might expect). * Refactor and fixups to getRpaHistory() Made the RPC to blockchain.rpa.get_history take params in the same from,to way as blockchain.scripthash.get_history. blockchain.reusable.get_history still works like the old way. Neither of them return mempool (unlike *.scripthash.get_history). Also switched the getRpaHistory() function to use a rocksdb iterator to scan records in sequence, since this should in theory be faster than individual O(log N) db gets. Also other minor fixes. * Tweak to getRpaHistory() Just forward the iterator 1 item at a time since it should be faster. Also refine the logic to not append mempool unconditionally if we didn't hit tipHeight in the confirmed scan (branch not currently used). * Optimized PackedNumView deserialization Use built-in byteswap functions rather than looping and doing it ourselves. Should be faster. * Optimized PackedNumView::Make Leverage byteswap calls that are possibly-no-ops is host and destination byte order match, and even if they don't, should be faster anyway than our hand-crafted loops that achieve same. * Added some Rpa stats tracking in Storage.cpp And also loading the DB now does faster checks. Still todo: use firstHeight and lastHeight from DB to decide if/how to (re)synch the index on app startup. * Fleshed out the initial check of the RPA db more, still more to do. We need to now have a way to synch the index separately in Controller.. and handle all corner cases that may arise. * Fixes and nits, mainly in loadCheckRpaDB * Small nits and header cleanup * Added method getRpaDBHeightRange to Storage May be useful later for the Controller. * wip * Refactored code that puts RPA data into DB into a function It's now in Storage::addRpaDataForHeight_nolock, since it does some defensive sanity checking. * Added 2 fields to RpaOnlyModeData * Got RPA index sync independent of block sync working It needs work in recovering from DL failure and other corner cases but it basically works. * Solved the last of the consistency corner cases on RPA index synch I'm pretty sure we are solid now and the RPA index eventally synchs separate of the general block download on config change. Meaning users get a decent experience with the index if they play with enabled/disabled toggling. * Bumped version to 1.10.0 This is due to the addition of the RPA index facility. Also bumped protocol version to 1.5.3 due to addition of new RPA RPCs. * Fixed percent display for RPA Index synch It really should be a percentage of the current download progress and not a full blockchain percentage as the normal blocks synch is. Fixed. * Corrected a debug string message * Took the bitcoin byte swap functions out of the `bitcoin` namespace This is because on some platforms they are actually #defines to some global thing, so eg `bitcoin::htole16` was failing to compile on such platforms. * Fixed some compile issue on Ubuntu 22 GCC-11 + Qt5 didn't like some of the stuff we did in recent commits. Fixed. * Fixed a failing test: `rpcmsgid` for Linux * Follow-up * Disabled the rpa subscribe/unsubscribe RPC methods (for now) They are unimplemented anyway and no clients use them (for now). * 2 nits * Fixed a potential bug * Renamed a /debug endpoint key * Some rename rpa_history_blocks_limit -> rpa_history_blocks And also some other minor tweaks. Mostly a renaming/nit commit. * A small refactoring of some boilerplate * Added docs for RPA options to example conf file in docs/ dir. * Made the rpa.get_history call use [from, to) (exclusive) range This is more akin to how existing calls operate. Also updated the electrum-cash-protocol submodule pointer to latest. * Updated electrum-cash-protocol submodule pointer * Update to electurm-cash-protocol module copyright * Got rid of some dead code and updated some comments * Corrected a comment --------- Co-authored-by: = <=jonaldfyookball@outlook.com> Co-authored-by: fyookball <jonaldfyookball@outlook.com> Co-authored-by: blockparty <hello@blockparty.sh>
1 parent 17dca71 commit 4aa8f84

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+3448
-168
lines changed

Fulcrum.pro

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -323,9 +323,11 @@ SOURCES += \
323323
Mixins.cpp \
324324
Mgr.cpp \
325325
Options.cpp \
326+
PackedNumView.cpp \
326327
PeerMgr.cpp \
327328
RecordFile.cpp \
328329
RollingBloomFilter.cpp \
330+
Rpa.cpp \
329331
RPC.cpp \
330332
RPCMsgId.cpp \
331333
ServerMisc.cpp \
@@ -369,9 +371,11 @@ HEADERS += \
369371
Mgr.h \
370372
Mixins.h \
371373
Options.h \
374+
PackedNumView.h \
372375
PeerMgr.h \
373376
RecordFile.h \
374377
RollingBloomFilter.h \
378+
Rpa.h \
375379
RPC.h \
376380
RPCMsgId.h \
377381
ServerMisc.h \
@@ -398,6 +402,10 @@ HEADERS += robin_hood/robin_hood.h
398402
RESOURCES += \
399403
resources.qrc
400404

405+
contains(DEFINES, ENABLE_TESTS) {
406+
RESOURCES += resources/testdata/testdata.qrc
407+
}
408+
401409
# Bitcoin related sources & headers
402410
SOURCES += \
403411
bitcoin/amount.cpp \

contrib/rpm/fulcrum.spec

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
Name: {{{ git_repo_name name="fulcrum" }}}
2-
Version: 1.9.8
2+
Version: 1.10.0
33
Release: {{{ git_repo_version }}}%{?dist}
44
Summary: A fast & nimble SPV server for Bitcoin Cash & Bitcoin BTC
55

doc/fulcrum-example-config.conf

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1013,3 +1013,76 @@ rpcpassword = hunter1
10131013
# useful for admins wishing to integrate Fulcrum with monitoring software.
10141014
#
10151015
#pidfile = /path/to/fulcrum.pid
1016+
1017+
1018+
1019+
#-------------------------------------------------------------------------------
1020+
# Reusable Payment Address (RPA) Options
1021+
#-------------------------------------------------------------------------------
1022+
1023+
# Enable RPA indexing - `rpa` - DEFAULT: 1 for BCH, 0 for all other coins
1024+
#
1025+
# Whether or not to enable the BCH-specific "RPA" index.
1026+
#
1027+
# See: https://github.com/imaginaryusername/Reusable_specs/blob/master/reusable_addresses.md
1028+
#
1029+
# This index takes ~42M of space currently on mainnet (but may grow to >3GB as
1030+
# the blockchain advances over time). If this index is enabled, the
1031+
# `blockchain.rpa.*` and/or the `blockchain.reusable.*` RPC methods will be
1032+
# available to clients, and the `server.features` map will contain an "rpa"
1033+
# key to indicate that the server supports RPA.
1034+
#
1035+
# If unspecified, then the RPA index and associated RPCs will only be enabled
1036+
# for BCH, and will be disabled for all coins.
1037+
#
1038+
#rpa = 1
1039+
1040+
1041+
# RPA starting block height - `rpa_start_height` - DEFAULT: 825000 for mainnet
1042+
# 0 all other nets
1043+
#
1044+
# Limit the RPA index to start at this block height. Blocks before this height
1045+
# will not have their data indexed by the RPA index. The default for mainnet is
1046+
# to save space and cycles since before a certain block height, no RPA wallets
1047+
# were in existence anyway since RPA had not yet been invented.
1048+
#
1049+
#rpa_start_height = 825000
1050+
1051+
1052+
# RPA history scan block limit - `rpa_history_blocks` - DEFAULT: 60
1053+
#
1054+
# For the `blockchain.rpa.get_history` and/or `blockchain.reusable.get_history`
1055+
# RPC methods, limit the number of blocks that client can request to scan for RPA
1056+
# transactions in a single RPC call to this number of blocks. In other words,
1057+
# results will be truncated if the client requests a wider height range in their
1058+
# request than this number. The reason for this limit is that clients should be
1059+
# making many frequent fast calls to the server so as to maximize the server's
1060+
# ability to multiplex requests (many small requests is better than a few larger
1061+
# ones when it comes to perceived server responsiveness). Specifying this to be
1062+
# a large value (say, >1000) is a potential DoS vector.
1063+
#
1064+
#rpa_history_blocks = 60
1065+
1066+
1067+
# RPA maximum history results limit - `rpa_max_history` - DEFAULT: `max_history`
1068+
#
1069+
# This is similar to the configuration option `max_history` (search for it above),
1070+
# but it can be independently specified for the RPA subsystem to be larger or
1071+
# smaller than the app-level `max_history`. If unspecified, this option will
1072+
# inherit whatever the app-level `max_history` setting is.
1073+
#
1074+
#rpa_max_history = 125000
1075+
1076+
1077+
# RPA prefix bits minimum - `rpa_prefix_bits_min` - DEFAULT: 8
1078+
#
1079+
# Affects the minimum "prefix" value that is accepted by the
1080+
# `blockchain.rpa.get_history` RPC method, in terms of number of bits. Specify
1081+
# a value in the range: [4, 16]. This is a low-level configuration variable and
1082+
# the default of 8 should be good for all extant RPA clients. 4 offers a larger
1083+
# anonymity set to clients when they perform queries (as they will get more
1084+
# haystack to their 1 needled they are looking for), but it comes with a
1085+
# performance penalty on the server-side, which is why we set the default
1086+
# minimum to 8 in Fulcrum.
1087+
#
1088+
#rpa_prefix_bits_min = 8

doc/unix-man-page.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
% FULCRUM(1) Version 1.9.8 | Fulcrum Manual
1+
% FULCRUM(1) Version 1.10.0 | Fulcrum Manual
22
% Fulcrum is written by Calin Culianu (cculianu)
3-
% January 13, 2024
3+
% March 01, 2024
44

55
# NAME
66

201 KB
Binary file not shown.

resources/testdata/testdata.qrc

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
<RCC>
2+
<qresource prefix="/testdata">
3+
<file>bch_block_833705.bin</file>
4+
</qresource>
5+
</RCC>

src/App.cpp

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
#include "Controller.h"
2323
#include "Json/Json.h"
2424
#include "Logger.h"
25+
#include "Rpa.h"
2526
#include "ServerMisc.h"
2627
#include "Servers.h"
2728
#include "Storage.h"
@@ -521,6 +522,15 @@ void App::parseArgs()
521522
" option only takes effect on initial sync, otherwise this option has no effect.\n"),
522523
QString("MB"),
523524
},
525+
{
526+
"rpa",
527+
QString("Explicitly enable the Reusable Payment Address index and offer the associated \"blockchain.rpa.*\" RPC"
528+
" methods to clients. To explicitly disable this facility, use the CLI arg --no-rpa. Default is: %1.\n")
529+
.arg(options->rpa.enabledSpecToString())
530+
},
531+
{
532+
"no-rpa", QString("<hidden>")
533+
},
524534
{
525535
"dump-sh",
526536
QString("*** This is an advanced debugging option *** Dump script hashes. If specified, after the database"
@@ -554,6 +564,13 @@ void App::parseArgs()
554564
});
555565
}
556566

567+
// Hide options that we marked above as hidden by setting the description to: "<hidden>"
568+
for (auto & opt : allOptions) {
569+
if (opt.description() == "<hidden>") {
570+
opt.setFlags(opt.flags() | QCommandLineOption::HiddenFromHelp);
571+
}
572+
}
573+
557574
parser.addOptions(allOptions);
558575
QString configArgDesc = "Configuration file (optional). To read configuration variables from the environment instead, ";
559576
#ifdef Q_OS_LINUX
@@ -1379,6 +1396,90 @@ void App::parseArgs()
13791396
DebugM("config: pidfile = ", options->pidFileAbsPath, " (size: ", QFileInfo(options->pidFileAbsPath).size(), " bytes)");
13801397
});
13811398
}
1399+
1400+
// CLI: --rpa
1401+
// conf: rpa
1402+
if (const bool psetYes = parser.isSet("rpa"), psetNo = parser.isSet("no-rpa"); psetYes || psetNo || conf.hasValue("rpa")) {
1403+
bool val{};
1404+
if (!psetYes && !psetNo) {
1405+
bool ok{};
1406+
val = conf.boolValue("rpa", false, &ok);
1407+
if (!ok) throw BadArgs("rpa: bad value. Specify a boolean value such as 0, 1, true, false, yes, no");
1408+
}
1409+
else if (psetYes && psetNo) throw BadArgs("Cannot specify --rpa and --no-rpa at the same time!");
1410+
else val = psetYes; // will be false if psetNo here
1411+
options->rpa.enabledSpec = val ? Options::Rpa::Enabled : Options::Rpa::Disabled;
1412+
Util::AsyncOnObject(this, [val] { DebugM("config: rpa = ", val); });
1413+
}
1414+
1415+
// conf: rpa_max_history
1416+
if (conf.hasValue("rpa_max_history")) {
1417+
bool ok;
1418+
int mh = conf.intValue("rpa_max_history", -1, &ok);
1419+
if (!ok || mh < options->maxHistoryMin || mh > options->maxHistoryMax)
1420+
throw BadArgs(QString("rpa_max_history: bad value. Specify a value in the range [%1, %2]")
1421+
.arg(options->maxHistoryMin).arg(options->maxHistoryMax));
1422+
options->rpa.maxHistory = mh;
1423+
// log this later in case we are in syslog mode
1424+
Util::AsyncOnObject(this, [mh]{ Debug() << "config: rpa_max_history = " << mh; });
1425+
} else {
1426+
// Otherwise, if nothing specified, we have special logic here:
1427+
// We inherit whatever the user specified for max_history, if anything (may be default)
1428+
options->rpa.maxHistory = options->maxHistory;
1429+
if (conf.hasValue("max_history")) {
1430+
Util::AsyncOnObject(this, [mh = options->maxHistory]{
1431+
Debug() << "config: rpa_max_history = " << mh << " (inherited from max_history)";
1432+
});
1433+
}
1434+
}
1435+
1436+
// conf: rpa_history_block_limit / rpa_history_blocks
1437+
if (const bool b1 = conf.hasValue("rpa_history_blocks"), b2 = conf.hasValue("rpa_history_block_limit"); b1 || b2) {
1438+
// support either: "rpa_history_block_limit" or "rpa_history_blocks", but not both
1439+
if (b1 && b2) throw BadArgs("Both `rpa_history_blocks` and `rpa_history_block_limit` were found in the config file; this looks like a typo.");
1440+
const QString confKey(b1 ? "rpa_history_blocks" : "rpa_history_block_limit");
1441+
bool ok;
1442+
const int limit = conf.intValue(confKey, -1, &ok);
1443+
if (!ok || limit < 0 || unsigned(limit) < options->rpa.historyBlockLimitMin || unsigned(limit) > options->rpa.historyBlockLimitMax)
1444+
throw BadArgs(QString("%1: bad value. Specify a value in the range [%2, %3]")
1445+
.arg(confKey).arg(options->rpa.historyBlockLimitMin).arg(options->rpa.historyBlockLimitMax));
1446+
options->rpa.historyBlockLimit = unsigned(limit);
1447+
// log this later in case we are in syslog mode
1448+
Util::AsyncOnObject(this, [limit, confKey]{ Debug() << "config: " << confKey << " = " << limit; });
1449+
}
1450+
1451+
// conf: rpa_prefix_bits_min
1452+
static_assert(Options::Rpa::defaultPrefixBitsMin >= Rpa::PrefixBitsMin && Options::Rpa::defaultPrefixBitsMin <= Rpa::PrefixBits
1453+
&& !(Options::Rpa::defaultPrefixBitsMin & 0b11));
1454+
if (conf.hasValue("rpa_prefix_bits_min")) {
1455+
bool ok;
1456+
int pbm = conf.intValue("rpa_prefix_bits_min", -1, &ok);
1457+
if (!ok || pbm < int(Rpa::PrefixBitsMin) || pbm > int(Rpa::PrefixBits) || pbm & 0b11 /* fancy way to check if multiple of 4 */) {
1458+
throw BadArgs(QString("rpa_prefix_bits_min: bad value. Specify a number that is a multiple of 4 and that is in the range [%1, %2].")
1459+
.arg(Rpa::PrefixBitsMin).arg(Rpa::PrefixBits));
1460+
}
1461+
options->rpa.prefixBitsMin = pbm;
1462+
// log this later in case we are in syslog mode
1463+
Util::AsyncOnObject(this, [pbm]{ Debug() << "config: rpa_prefix_bits_min = " << pbm; });
1464+
}
1465+
1466+
// conf: rpa_start_height
1467+
if (const auto b1 = conf.hasValue("rpa_start_height"), b2 = conf.hasValue("rpa_starting_height"); b1 || b2) {
1468+
// support either: "rpa_start_height" or "rpa_starting_height", but not both
1469+
if (b1 && b2) throw BadArgs("Both `rpa_start_height` and `rpa_starting_height` were found in the config file; this looks like a typo.");
1470+
const QString confKey(b1 ? "rpa_start_height" : "rpa_starting_height");
1471+
bool ok;
1472+
int ht = conf.intValue(confKey, -1, &ok);
1473+
if (!ok || ht < -1 /* -1 ok, -2 not, etc*/ || (ht >= 0 && ht > int(Storage::MAX_HEADERS)))
1474+
throw BadArgs(QString("%1: bad value. Specify a block height between [0, %2], or use -1 to"
1475+
" auto-configure this setting with a chain-specific default (%3 for mainnet, %4 for"
1476+
" all other nets).")
1477+
.arg(confKey).arg(Storage::MAX_HEADERS).arg(Options::Rpa::defaultStartHeightForMainnet)
1478+
.arg(Options::Rpa::defaultStartHeightOtherNets));
1479+
options->rpa.requestedStartHeight = ht;
1480+
// log this later in case we are in syslog mode
1481+
Util::AsyncOnObject(this, [ht, confKey]{ Debug() << "config: " << confKey << " = " << ht; });
1482+
}
13821483
}
13831484

13841485
namespace {

src/BTC.cpp

Lines changed: 41 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,15 +23,10 @@
2323
#include "bitcoin/crypto/endian.h"
2424
#include "bitcoin/crypto/sha256.h"
2525
#include "bitcoin/hash.h"
26-
#include "bitcoin/pubkey.h"
27-
#include "bitcoin/streams.h"
28-
#include "bitcoin/utilstrencodings.h"
29-
#include "bitcoin/version.h"
3026

3127
#include <QMap>
3228

3329
#include <algorithm>
34-
#include <atomic>
3530
#include <utility>
3631

3732
namespace bitcoin
@@ -236,3 +231,44 @@ namespace BTC
236231

237232

238233
} // end namespace BTC
234+
235+
236+
#ifdef ENABLE_TESTS
237+
#include "App.h"
238+
239+
#include "bitcoin/transaction.h"
240+
#include "bitcoin/uint256.h"
241+
242+
namespace {
243+
void test()
244+
{
245+
// Misc. unit tests for BTC namespace utility functions
246+
Log() << "Testing Hash2ByteArrayRev ...";
247+
bitcoin::uint256 hash = bitcoin::uint256S("080bb1010c4d32f3cb16c6a7f1ac2a949d0b5b0f0396f183870be7032cfc4da9");
248+
if (hash.ToString() != "080bb1010c4d32f3cb16c6a7f1ac2a949d0b5b0f0396f183870be7032cfc4da9") throw Exception("Hash parse fail");
249+
const QByteArray qba(reinterpret_cast<const char *>(std::as_const(hash).data()), hash.size());
250+
if (ByteView{hash} != ByteView{qba}) throw Exception("2");
251+
if (qba.toHex() != "a94dfc2c03e70b8783f196030f5b0b9d942aacf1a7c616cbf3324d0c01b10b08") throw Exception("Hash parse did not yield expected result");
252+
auto rhash = BTC::Hash2ByteArrayRev(hash);
253+
Debug() << "Expected hash: " << rhash.toHex();
254+
if (rhash.toHex() != "080bb1010c4d32f3cb16c6a7f1ac2a949d0b5b0f0396f183870be7032cfc4da9") throw Exception("BTC::Hash2ByteArrayRev is broken");
255+
256+
Log() << "Testing Deserialize ...";
257+
const auto txnhex = "0100000001e7b81293c58fa088412949e485f7a7310c386a267a1825284e79c083d26b55670000000084410b00"
258+
"086668d9c26c3bf44b4f136512d7edae0f01ddd66844e312fa00f54250e93457b5e2c823ca31ab452d22f27181"
259+
"b13ce3560b974130b5e8a9e1b3ab820d0d414104e8806002111e3dfb6944e63a42461832437f2bbd616facc269"
260+
"10becfa388642972aaf555ffcdc2cdc07a248e7881efa7f456634e1bdb11485dbbc9db20cb669dfeffffff01fb"
261+
"0cfe00000000001976a914590888ac04b1f1cf01f08110cca83dd3e3da7f7388accbb90c00";
262+
auto tx = BTC::Deserialize<bitcoin::CTransaction>(Util::ParseHexFast(txnhex));
263+
if (hash != tx.GetHash()) throw Exception("Txn did not deserialize ok");
264+
265+
Log() << "Testing HashInPlace ...";
266+
if (BTC::HashInPlace(tx) != qba) throw Exception("Txn hash in place failed");
267+
if (BTC::HashInPlace(tx, false, /* reversed = */true) != rhash) throw Exception("Txn hash in place reversed failed");
268+
269+
Log(Log::BrightWhite) << "All btcmisc unit tests passed!";
270+
}
271+
272+
auto t1 = App::registerTest("btcmisc", test);
273+
} // namespace
274+
#endif

src/BTC.h

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
//
22
// Fulcrum - A fast & nimble SPV Server for Bitcoin Cash
3-
// Copyright (C) 2019-2023 Calin A. Culianu <calin.culianu@gmail.com>
3+
// Copyright (C) 2019-2024 Calin A. Culianu <calin.culianu@gmail.com>
44
//
55
// This program is free software: you can redistribute it and/or modify
66
// it under the terms of the GNU General Public License as published by
@@ -21,6 +21,7 @@
2121
#include "Util.h"
2222

2323
#include "bitcoin/block.h"
24+
#include "bitcoin/hash.h"
2425
#include "bitcoin/script.h"
2526
#include "bitcoin/streams.h"
2627
#include "bitcoin/transaction.h"
@@ -31,9 +32,11 @@
3132
#include <QMetaType>
3233
#include <QString>
3334

35+
#include <algorithm>
3436
#include <cstddef> // for std::byte, etc
3537
#include <cstring> // for memcpy
3638
#include <ios>
39+
#include <iterator>
3740
#include <type_traits>
3841
#include <utility> // for pair, etc
3942

@@ -170,17 +173,27 @@ namespace BTC
170173
inline QByteArray HashOnce(const QByteArray &b) { return Hash(b, true); }
171174
/// Like the Hash() function above, except does hash160 once. (not reversed).
172175
extern QByteArray Hash160(const QByteArray &);
176+
/// Hash any Bitcoin object in-place and return the hash. If `once` == true, we do single-sha256 hashing. If
177+
/// `reversed` == true, we reverse the result (making it big-endian ready for JSON).
178+
template <typename BitcoinObject>
179+
QByteArray HashInPlace(const BitcoinObject &bo, bool once = false, bool reversed = false) {
180+
QByteArray ret(bitcoin::CHash256::OUTPUT_SIZE, Qt::Uninitialized); // allocate without initializing
181+
bitcoin::SerializeHashInPlace(ret.data(), bo, bitcoin::SER_GETHASH, bitcoin::PROTOCOL_VERSION, once);
182+
if (reversed) std::reverse(ret.begin(), ret.end());
183+
return ret;
184+
}
173185

174186
/// Takes a hash in bitcoin memory order and returns a deep copy QByteArray of the data, reversed
175187
/// (this is intended to keep our representation of bitcoin data closer to how we will send it to clients down
176188
/// the wire -- we send all hex encoded hashes in reverse order as is customary when representing bitcoin
177189
/// hashes in hex). See BlockProc.cpp for an example of where this is used.
178190
template <class BitcoinHashT>
179191
QByteArray Hash2ByteArrayRev(const BitcoinHashT &hash) {
180-
QByteArray ret(reinterpret_cast<const char *>(hash.begin()), hash.width()); // deep copy
181-
std::reverse(ret.begin(), ret.end()); // reverse it
192+
QByteArray ret(hash.width(), Qt::Uninitialized);
193+
std::copy(std::reverse_iterator(hash.end()), std::reverse_iterator(hash.begin()),
194+
reinterpret_cast<uint8_t *>(ret.data())); // reversed copy
182195
return ret;
183-
};
196+
}
184197

185198
/// returns true iff cscript is OP_RETURN, false otherwise
186199
inline bool IsOpReturn(const bitcoin::CScript &cs) {

0 commit comments

Comments
 (0)