Skip to content

simplicity-sys: separate out linker symbols in C library by version name #297

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 23, 2025

Conversation

apoelstra
Copy link
Collaborator

First several commits (all the ones that don't update libsimplicity) from #296.

There are actually only two changes here:

* In vendor-simplicity.sh I added a couple `sed` lines to search-and-replace
  simplicity_ symbols with rustsimplicity_X_y_ ones, then re-ran the script.
* In src/tests/ffi.rs I added a bunch of #[link_name] attributes to compensate.

One way to test this is to rename all the 0_4s to 0_3s e.g. with

    find . -exec sed -i s/_0_4_/_0_3_/g {} \;

Then re-run vendor-simplicity.sh and see that your `git diff` has become empty.
@apoelstra
Copy link
Collaborator Author

cc @canndrew can you review this? Probably fine to just check that it seems half-sensible. I'm not sure it's worth your time to understand the vagaries of C/Rust FFI.

@apoelstra
Copy link
Collaborator Author

Then if you're on a roll you can keep right on to #296. Feel free to ping me about how to use these vendoring scripts.

@uncomputable
Copy link
Collaborator

Does this renaming resolve linker conflicts when running multiple versions of rust-simplicity in parallel? I am surprised, because we did a similar fix in the past.

@apoelstra
Copy link
Collaborator Author

@uncomputable yes.

And maybe you are thinking of:

  • when we added versions to the rust_{malloc,calloc,free} methods in update libsimplicity to c54a8db0e4c69a571aa67c3bc3188eb33ca67dcb #270. This was not sufficient and I'm not sure why we thought it was. (Clearly there was some issue in our own CI that it fixed, but like, we were exposing methods like c_full_add_16 from every version of simplicity-sys up to that point, so it was not sufficient)
  • when we added the simplictiy_ prefixes to everything upstream. This was necessary, in that it gave us something we could apply sed to, but it wasn't sufficent because we didn't do the sed thing til now.

@apoelstra
Copy link
Collaborator Author

apoelstra commented Jul 15, 2025

For future reference, this set of fixes are there because otherwise when we update libsimplicity, our fuzztest (which attempts to compile both the local version of simplicity and the 0.3 release on crates.io) fails to build due to symbol conflicts. This failure manifests in CI and you can reproduce it locally (with some difficulty, since you need to rebase #296 on the commit before this one, or just obtain commit
ad7a60b from somewhere) by going into the fuzz directory and running

RUSTFLAGS=--cfg=fuzzing cargo build --bin construct_type

You can see these failures (for now) at https://github.com/BlockstreamResearch/rust-simplicity/actions/runs/16205734263/job/45755435923?pr=296 and also I have uploaded the logs here so they won't be deleted

The specific failures are

  = note: rust-lld: error: duplicate symbol: c_set_txEnv
          >>> defined at env.c:65 (depend/env.c:65)
          >>>            env.o:(c_set_txEnv) in archive /home/apoelstra/code/BlockstreamResearch/rust-simplicity/master/target/debug/deps/libsimplicity_sys-327598d5c3f11e50.rlib
          >>> defined at env.c:64 (depend/env.c:64)
          >>>            env.o:(.text.c_set_txEnv+0x0) in archive /home/apoelstra/code/BlockstreamResearch/rust-simplicity/master/target/debug/deps/libsimplicity_sys-4173a70ba17044c7.rlib

          rust-lld: error: duplicate symbol: c_sizeof_txEnv
          >>> defined at env.c:12 (depend/env.c:12)
          >>>            env.o:(c_sizeof_txEnv) in archive /home/apoelstra/code/BlockstreamResearch/rust-simplicity/master/target/debug/deps/libsimplicity_sys-327598d5c3f11e50.rlib
          >>> defined at env.c:11 (depend/env.c:11)
          >>>            env.o:(.rodata.c_sizeof_txEnv+0x0) in archive /home/apoelstra/code/BlockstreamResearch/rust-simplicity/master/target/debug/deps/libsimplicity_sys-4173a70ba17044c7.rlib

          rust-lld: error: duplicate symbol: c_alignof_txEnv
          >>> defined at env.c:19 (depend/env.c:19)
          >>>            env.o:(c_alignof_txEnv) in archive /home/apoelstra/code/BlockstreamResearch/rust-simplicity/master/target/debug/deps/libsimplicity_sys-327598d5c3f11e50.rlib
          >>> defined at env.c:18 (depend/env.c:18)
          >>>            env.o:(.rodata.c_alignof_txEnv+0x0) in archive /home/apoelstra/code/BlockstreamResearch/rust-simplicity/master/target/debug/deps/libsimplicity_sys-4173a70ba17044c7.rlib
          clang-15: error: linker command failed with exit code 1 (use -v to see invocation)

which suggests that we don't notice conflicts except when we actually change methods, and maybe we hadn't changed any methods (at least, none that the fuzzer was trying to use) until now.

The process here was:

* Manually replace c_X with c_v0_3_X everywhere (actually only in two files,
  wrapper.h and jets_ffi.rs)
* Run update_jets.sh and make sure it changes all the 0_3s to 0_4s.

Note that now both update_jets.sh and vendor-simplicity.sh update wrapper.h.
This is fine since the change is idempotent. But we really should clean up
and combine these two scripts one of these days..

The 0_3 test is easy to do during review, though if you are on NixOS then replace
the cabal build/exec lines by first nix-building the `haskell` target in Simplicity
and then directly calling result/bin/GenRustJets in there.
As before, I first manually added _0_3_ version numbers, modified the vendoring
script, ran it, and made sure that the resulting diff added 0_4s and worked.
This field tells Cargo what library we're linking in, and helps it
provide sane error messages (rather than linker spew) when somebody
tries to compile multiple versions of the library that link the
same thing.

When we provide a links field we should also provide a build field.
This is set to build.rs by default, but crate2nix seems to get confused
when one is present but not the other.
@apoelstra apoelstra force-pushed the 2025-07/domainsep branch from 382e327 to 901127b Compare July 21, 2025 03:17
@apoelstra apoelstra marked this pull request as draft July 21, 2025 12:34
@apoelstra apoelstra marked this pull request as ready for review July 21, 2025 12:35
@apoelstra apoelstra marked this pull request as draft July 21, 2025 12:35
@apoelstra apoelstra marked this pull request as ready for review July 21, 2025 12:35
Copy link
Collaborator Author

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On 901127b successfully ran local tests

@apoelstra
Copy link
Collaborator Author

cc @canndrew can you give this a cursory review and an ack?

The other one is not ready because upstream is generating code with broken paths, but this one is good to go.

@canndrew
Copy link
Collaborator

ACK 901127b

@apoelstra apoelstra merged commit f154a83 into master Jul 23, 2025
39 checks passed
@apoelstra apoelstra deleted the 2025-07/domainsep branch July 23, 2025 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants