Skip to content

Optimize the memory allocation in trie operation #31832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rjl493456442 opened this issue May 15, 2025 · 1 comment
Open

Optimize the memory allocation in trie operation #31832

rjl493456442 opened this issue May 15, 2025 · 1 comment

Comments

@rjl493456442
Copy link
Member

Build ID: 15e9e5882df65907942fa3b5d1ec150f3a7d3b90
Type: inuse_space
Time: 2025-05-11 15:03:46 CST
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) alloc_space
(pprof) top
Showing nodes accounting for 75.21TB, 54.11% of 138.99TB total
Dropped 2219 nodes (cum <= 0.69TB)
Showing top 10 nodes out of 231
      flat  flat%   sum%        cum   cum%
   25.46TB 18.32% 18.32%    29.25TB 21.05%  github.com/ethereum/go-ethereum/trie.(*hasher).hashFullNodeChildren
   12.96TB  9.32% 27.64%    25.60TB 18.42%  github.com/ethereum/go-ethereum/trie.decodeFull
   12.65TB  9.10% 36.74%    12.65TB  9.10%  github.com/ethereum/go-ethereum/trie.decodeRef
    7.37TB  5.30% 42.05%     7.37TB  5.30%  github.com/ethereum/go-ethereum/rlp.(*encBuffer).makeBytes
    6.18TB  4.45% 46.49%     6.18TB  4.45%  github.com/ethereum/go-ethereum/trie.(*tracer).onRead
    2.42TB  1.74% 48.24%     3.96TB  2.85%  github.com/ethereum/go-ethereum/core/state.newObject
    2.42TB  1.74% 49.97%     2.42TB  1.74%  golang.org/x/crypto/sha3.NewLegacyKeccak256 (inline)
    2.01TB  1.44% 51.42%     2.51TB  1.81%  github.com/ethereum/go-ethereum/core/state.(*stateObject).finalise
    1.96TB  1.41% 52.83%     4.30TB  3.10%  github.com/ethereum/go-ethereum/core/state.(*stateObject).GetCommittedState
    1.78TB  1.28% 54.11%     1.78TB  1.28%  github.com/ethereum/go-ethereum/core/state.(*journal).append

The memory profile shows that most of memory allocation is spent on several functions,
such as trie node hashing. The current speed for memory allocation is about 5-600 MB/s
which is insane. These part should be investigated for a memory allocation friendly change.

@nadtech-hub
Copy link

May I take on it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants