openrsync: New port

[artkiver]

• closes https://trac.macports.org/ticket/72311

Description

Type(s)

• bugfix
• enhancement
• security fix

Tested on

macOS 15.4 24E248 arm64
Command Line Tools 16.3.0.0.1.1742442376

Verification

Have you

• followed our Commit Message Guidelines?
• squashed and minimized your commits?
• checked that there aren't other open pull requests for the same change?
• referenced existing tickets on Trac with full URL in commit message?
• checked your Portfile with port lint?
• tried existing tests with sudo port test?
• tried a full install with sudo port -vst install?
• tested basic functionality of all binary files?
• checked that the Portfile's most important variants haven't been broken?

[artkiver]

I am not sure if the warnings generated by %port test are concerning, but I thought it best to include them here just in case others might have some salient perspective and improvements:

port test ---> Fetching distfiles for openrsync ---> Verifying checksums for openrsync ---> Extracting openrsync ---> Configuring openrsync Warning: Configuration logfiles contain indications of -Wimplicit-function-declaration; check that features were not accidentally disabled: crypt_newhash: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log crypt_checkpass: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log explicit_bzero: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log getexecname: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log memrchr: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log pledge: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log reallocarray: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log recallocarray: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log scan_scaled: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log setresgid: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log setresuid: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log timingsafe_memcmp: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log unveil: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log ---> Building openrsync ---> Staging openrsync into destroot Warning: violation by /opt/local/man Warning: openrsync violates the layout of the ports-filesystems! Warning: Please fix or indicate this misbehavior (if it is intended), it will be an error in future releases! ---> Testing openrsync

[herbygillot]

Warning: violation by /opt/local/man Warning: openrsync violates the layout of the ports-filesystems!

Try also setting MANDIR in both the configure and build steps:

configure.pre_args PREFIX=${prefix} \
                   MANDIR=${prefix}/share/man

build.pre_args     MANDIR=${prefix}/share/man

You can try only setting it in the configure phase only.

But if that doesn't work, it might also be that you'll need to set it in the destroot phase as well, in which case you'd also add destroot.pre_args MANDIR=...

[artkiver]

MANDIR=${prefix}/share/man

build.pre_args MANDIR=${prefix}/share/man

Groovy!

Adding:

` MANDIR=${prefix}/share/man

build.pre_args MANDIR=${prefix}/share/man`

At least got rid of the man page warning during %port test.

I still see these warnings:

Warning: Configuration logfiles contain indications of -Wimplicit-function-declaration; check that features were not accidentally disabled: crypt_newhash: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log crypt_checkpass: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log explicit_bzero: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log getexecname: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log memrchr: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log pledge: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log reallocarray: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log recallocarray: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log scan_scaled: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log setresgid: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log setresuid: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log timingsafe_memcmp: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log unveil: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log

But some of those (e.g. pledge and unveil) are OpenBSD specific functions, so I understand if macOS doesn't know how to handle them.

[herbygillot]

MANDIR=${prefix}/share/man
build.pre_args MANDIR=${prefix}/share/man

Groovy!

Adding:

` MANDIR=${prefix}/share/man

build.pre_args MANDIR=${prefix}/share/man`

At least got rid of the man page warning during %port test.

I still see these warnings:

Warning: Configuration logfiles contain indications of -Wimplicit-function-declaration; check that features were not accidentally disabled: crypt_newhash: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log crypt_checkpass: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log explicit_bzero: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log getexecname: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log memrchr: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log pledge: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log reallocarray: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log recallocarray: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log scan_scaled: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log setresgid: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log setresuid: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log timingsafe_memcmp: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log unveil: found in openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed/config.log

But some of those (e.g. pledge and unveil) are OpenBSD specific functions, so I understand if macOS doesn't know how to handle them.

It seems like on the face of it, openrsync is just referencing a bunch of OpenBSD-native functions that aren't being declared in macOS. There does seem to be a way to whitelist implicitly declared functions ( Example:

macports-ports/net/wget/Portfile

Line 89 in 4911f2e

configure.checks.implicit_function_declaration.whitelist-append strchr

), but I don't know if that's the right thing to do in this situation. Might need to check with @ryandesign.

[herbygillot]

Also the warning earlier about the man pages was because they were being installed to a path in the MacPorts root directory hierarchy that MacPorts isn't expecting.

Like MacPorts doesn't want to see a port just stow something into /opt/local/some_random_dir. It will consider this a violation of the expected directory layout.

/opt/local/man is not where MacPorts expects man pages to be installed to, hence why we set MANDIR to /opt/local/share/man, which is the expected man page location.

[jmroot]

It seems like on the face of it, openrsync is just referencing a bunch of OpenBSD-native functions that aren't being declared in macOS. There does seem to be a way to whitelist implicitly declared functions ( Example:

macports-ports/net/wget/Portfile

Line 89 in 4911f2e

configure.checks.implicit_function_declaration.whitelist-append strchr

), but I don't know if that's the right thing to do in this situation. Might need to check with @ryandesign.

Functions that are not present in existing macOS versions should be added to the global lists in _resources/port1.0/checks/implicit_function_declaration. Adding exceptions to individual Portfiles should only be necessary for cases where the function does exist but the build system is deliberately doing something that triggers an implicit declaration error. A common example is autoconf using strchr to check how the compiler responds to implicit declaration.

[artkiver]

_resources/port1.0/checks/implicit_function_declaration

So, if I understood you correctly, basically to every file in:

https://github.com/macports/macports-ports/tree/master/_resources/port1.0/checks/implicit_function_declaration

append something like:

crypt_newhash crypt_checkpass explicit_bzero getexecname memrchr pledge reallocarray recallocarray scan_scaled setresgid setresuid timingsafe_memcmp unveil

And that will suppress the warnings?

I've seen explicit_bzero in the past too (and searching Trac see it pop up in some other places) but at least one of those (e.g. in rpki-client) is during a ./configure check so autoconf I guess looks to see if such things are available first; whereas if I had to guess, oconfigure (referenced in the Trac issue 72311) which is utilized by openrsync doesn't bother with that?

Since these warnings only come up while running % port test

I guess I am wondering how serious they are and what the best approach is if contending with them or whether they can be safely ignored?

Thanks for all the perspective though, I feel as if I am always learning something new in this kind of discourse.

[aeiouaeiouaeiouaeiouaeiouaeiou]

I'm quite curious to know when the Apple team will post the source code of their fork and how they solved the problem with these implicit declarations (or broke it altogether like they did with xar).

[artkiver]

I'm quite curious to know when the Apple team will post the source code of their fork and how they solved the problem with these implicit declarations (or broke it altogether like they did with xar).

I think some of the OpenBSD developers would probably appreciate similar insights!

Also see this thread on the FediVerse starting here for additional discourse from some BSD devs (including the primary author of openrsync):

https://mastodon.sdf.org/@ParadeGrotesque/114298852894809118

[jmroot]

_resources/port1.0/checks/implicit_function_declaration

So, if I understood you correctly, basically to every file in:

https://github.com/macports/macports-ports/tree/master/_resources/port1.0/checks/implicit_function_declaration

append something like:

crypt_newhash crypt_checkpass explicit_bzero getexecname memrchr pledge reallocarray recallocarray scan_scaled setresgid setresuid timingsafe_memcmp unveil

And that will suppress the warnings?

One per line and in alphabetical order, but basically yes.

I've seen explicit_bzero in the past too (and searching Trac see it pop up in some other places) but at least one of those (e.g. in rpki-client) is during a ./configure check so autoconf I guess looks to see if such things are available first; whereas if I had to guess, oconfigure (referenced in the Trac issue 72311) which is utilized by openrsync doesn't bother with that?

Since these warnings only come up while running % port test

I guess I am wondering how serious they are and what the best approach is if contending with them or whether they can be safely ignored?

Based on the output you showed, the warnings are being printed in the configure phase (as expected). It is normal for configure scripts to generate these kinds of errors when checking for functions that are not provided by the current platform, which is why we have the whitelists to filter out this case. After that, the only warnings printed should be for the actual problems where they forgot to include a header, causing a configure check to produce the wrong result.

[jmroot]

I'm quite curious to know when the Apple team will post the source code of their fork and how they solved the problem with these implicit declarations (or broke it altogether like they did with xar).

There is openrsync code published at https://github.com/apple-oss-distributions/rsync though I can't say how well it corresponds to the binary in the current macOS release.

[artkiver]

_resources/port1.0/checks/implicit_function_declaration

So, if I understood you correctly, basically to every file in:
https://github.com/macports/macports-ports/tree/master/_resources/port1.0/checks/implicit_function_declaration
append something like:
crypt_newhash crypt_checkpass explicit_bzero getexecname memrchr pledge reallocarray recallocarray scan_scaled setresgid setresuid timingsafe_memcmp unveil
And that will suppress the warnings?

One per line and in alphabetical order, but basically yes.

OK, there was probably a smarter way to do that (there were 26 .list files; but some of them were also symlinks) but I updated the PR with modifications to those, one per line and in alphabetical order as specified!

[artkiver]

Ooph, I don't think I realized what the re-request review option did. Sorry if that caused any additional notifications or extra work.

[aeiouaeiouaeiouaeiouaeiouaeiou]

Functions that are not present in existing macOS versions should be added to the global lists in _resources/port1.0/checks/implicit_function_declaration.

Despite the warnings configure apparently only checks if the required functions are present, while they are defined and always used from the non-system compats.c file (Apple's fork also includes this slightly outdated file).
I should test the build on my legacy system.

[aeiouaeiouaeiouaeiouaeiouaeiou]

Build on 10.6 fails with this error:

--->  Configuring openrsync
Executing:  cd "/opt/local/var/macports/build/_Users_strega_macports-ports_net_openrsync/openrsync/work/openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed" && ./configure PREFIX=/opt/local MANDIR=/opt/local/share/man
config.log: writing...
usage: mktemp [-d] [-q] [-t prefix] [-u] template ...
       mktemp [-d] [-q] [-u] -t prefix
mktemp: failed
Command failed:  cd "/opt/local/var/macports/build/_Users_strega_macports-ports_net_openrsync/openrsync/work/openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed" && ./configure PREFIX=/opt/local MANDIR=/opt/local/share/man
Exit code: 1

[artkiver]

Build on 10.6 fails with this error:

--->  Configuring openrsync
Executing:  cd "/opt/local/var/macports/build/_Users_strega_macports-ports_net_openrsync/openrsync/work/openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed" && ./configure PREFIX=/opt/local MANDIR=/opt/local/share/man
config.log: writing...
usage: mktemp [-d] [-q] [-t prefix] [-u] template ...
       mktemp [-d] [-q] [-u] -t prefix
mktemp: failed
Command failed:  cd "/opt/local/var/macports/build/_Users_strega_macports-ports_net_openrsync/openrsync/work/openrsync-a257c0f495af2b5ee6b41efc6724850a445f87ed" && ./configure PREFIX=/opt/local MANDIR=/opt/local/share/man
Exit code: 1

Thanks for testing on 10.6! I don't have any systems running that easily accessible at the moment.

[aeiouaeiouaeiouaeiouaeiouaeiou]

@artkiver I don't require testing on it, just posted for information.
The point of making a fix for older systems will only be when this PR is merged (since I want to see on which buildbots this also happens).

@jmroot @reneeotten

[barracuda156]

@aeiouaeiouaeiouaeiouaeiouaeiou @artkiver I can confirm it fails due to illegal syntax for mktemp. If it is problematic to fix the syntax, it should work to use gmktemp form coreutils instead.

[barracuda156]

Also, the build does not respect MacPorts’ ldflags, so legacysupport is not linked to. That has to be fixed.

Once that is fixed (using coreutils and legacysupport with ensuring linking is actually done), the build completes successfully.

[artkiver]

@barracuda156 suggestions LGTM! Thanks!