MacOSX.sdk: add SDK subports for 10.3-10.5 & 13-14 #23312

MarcusCalhoun-Lopez · 2024-04-01T14:19:05Z

Description

Type(s)

bugfix
enhancement
security fix

Tested on

macOS x.y
Xcode x.y / Command Line Tools x.y.z

Verification

Have you

followed our Commit Message Guidelines?
squashed and minimized your commits?
checked that there aren't other open pull requests for the same change?
referenced existing tickets on Trac with full URL in commit message?
checked your Portfile with port lint --nitpick?
tried existing tests with sudo port test?
tried a full install with sudo port -vst install?
tested basic functionality of all binary files?
checked that the Portfile's most important variants haven't been broken?

macportsbot · 2024-04-01T14:19:08Z

Notifying maintainers:
@ryandesign for port MacOSX.sdk.

ryandesign

When I first saw the email notification of this PR I thought you were adding the missing current SDKs.

ryandesign · 2024-04-02T06:20:05Z

devel/MacOSX.sdk/Portfile

@@ -4,7 +4,7 @@ PortSystem              1.0

 name                    MacOSX.sdk
 categories              devel
-platforms               macosx
+platforms               {macosx any}


This is a good idea.

It will trigger rebuilds of every subport so I will need to load the distfiles that can't be downloaded automatically onto one of the build machines manually before merging this PR.

Unfortunately it looks like this port has bitrotted. For newer SDKs it downloaded them direct from Apple's CDN but now Apple is delivering a 403 response, e.g.:

% curl -I https://swcdn.apple.com/content/downloads/28/19/071-82714-A_HUFJRGID6Y/nnmlg28zs93jdbrsqtk6krr4mmaize40hb/CLTools_macOSLMOS_SDK.pkg HTTP/1.1 403 Forbidden …

so we either need a new URL from which to download the packages that used to be available via Apple's CDN or we need to revert to the old strategy of manually downloading the CLT dmgs from the Apple Developer web site and adjust the port's programming accordingly.

How does one determine the proper URL from swcdn.apple.com?
There are various posts with URLs from swcdn.apple.com, but none of them seem to mention how to find them.

I probably had just found one of those posts at the time. I don't remember.

It would seem that someone has taken the time to search for this information.
I have updated the pull request to remove nonexistent files.
However, I believe the fix (e48c336) constitutes a stealth update, but fixing it would disrupt the existing distfile naming convention.

I also added SDKs for macOS 13-14 (fe83f2f), but I do not know how to find the release dates of the distfiles.

I believe the fix (e48c336) constitutes a stealth update, but fixing it would disrupt the existing distfile naming convention.

And the SDK version is still exactly the same? Can you compare the old and new distfiles to see what the difference is? If you don't have the old distfiles, I can check if I saved them, which I was in the habit of doing for this port.

This will fix https://trac.macports.org/ticket/69952 by the way.

I also added SDKs for macOS 13-14 (fe83f2f), but I do not know how to find the release dates of the distfiles.

You could check if there is an HTTP Last-Modified header. If not, you could check the Internet; if not Apple, then surely somebody has posted something to announce these releases. You could also look at the modification date of the dmg or the files in it.

And the SDK version is still exactly the same? Can you compare the old and new distfiles to see what the difference is? If you don't have the old distfiles, I can check if I saved them, which I was in the habit of doing for this port.

As far as I can tell, they have the same SDK version.
Unfortunately, I do not have access to the old distfiles, so I do not know what the differences are.

This will fix https://trac.macports.org/ticket/69952 by the way.

Thank you for pointing this out.
I have updated the commit message.

You could check if there is an HTTP Last-Modified header. If not, you could check the Internet; if not Apple, then surely somebody has posted something to announce these releases. You could also look at the modification date of the dmg or the files in it.

I have updated the dates using the HTTP Last-Modified header.

ryandesign · 2024-04-02T06:23:15Z

devel/MacOSX.sdk/Portfile

+    #    variant_set universal
+    #}
+    #
+    supported_archs     noarch


This is certainly inaccurate but this port is a very special case so maybe this is an acceptable simplification.

Do you know if this would do anything to fix https://trac.macports.org/ticket/66918? Does rev-upgrade skip scanning ports marked noarch? If so, that would be an argument in favor of this change.

This is certainly inaccurate but this port is a very special case so maybe this is an acceptable simplification.

I would be happy to revert the change.
Setting configure.cxx_stdlib to empty is similarly inaccurate.
Should configure.cxx_stdlib also be set to either libc++ or libstdc++ depending on the SDK version?

Do you know if this would do anything to fix https://trac.macports.org/ticket/66918? Does rev-upgrade skip scanning ports marked noarch? If so, that would be an argument in favor of this change.

I am afraid the rev-upgrade issue persists even with the change.

I would be happy to revert the change.

I'm not saying yet that we should revert it. It certainly would reduce the number of systems on which the port needs to build, and the archives that get produced would indeed be the same.

Setting configure.cxx_stdlib to empty is similarly inaccurate.

That is true… the comment in the port explains that this was done to eliminate an unnecessary clang dependency. If there is a better solution to that, that's fine.

Should configure.cxx_stdlib also be set to either libc++ or libstdc++ depending on the SDK version?

I'm not sure if that would have the same effect.

Does rev-upgrade skip scanning ports marked noarch?

No, and that's probably because it also scans for missing files. This issue will require a change in "base", I'm afraid. But maybe that can be as simple as making noarch skip the registration of binary files. IIRC this is basically what I did to have a semi-functional rev-upgrade feature in my LinuxPorts adaptation (I made isBinary() a stub returning false unconditionally so).

This would make some sort of sense ... treating all files installed by a port claiming that it has only "noarch" files as if they were not "binary" even if some of them would actually fit that description. I think it can be argued that no port setting this expects or should expect to have its files checked for shared library dependencies.

The safer alternative would be a new rev_upgrade_skip_archceck option.

This issue will require a change in "base", I'm afraid.

Or an unorthodox approach: create a nicely compressed DMG of the SDK directory and install that, plus a script that handles the mounting so the user doesn't have to remember where the DMG is installed. Ports that get their content off a DMG know how to mount them on custom mountpoint not under /Volumes so that script could make the DMG content appear under the same path as where it's found now.

I think there are additional advantages to this; these SDKs install a really large number of (small) files which make them potentially SSD-unfriendly and costly for both the registry and the rev-upgrade scan. Is there any indication that people who install these ports actually use them more than occasionally? I suppose a startupitem could be provided for those who do, so the DMG gets mounted when they log in?

Fixing the rev-upgrade issue would definitely be nice.
However, as you point out, it must either be fixed in the base or in the Portfile with an "unorthodox approach."
The ideas are interesting, but I think they are a bit outside the scope of this PR.

create a nicely compressed DMG of the SDK directory and install that

Yuck!

I suppose a startupitem could be provided for those who do, so the DMG gets mounted when they log in?

Eww!

devel/MacOSX.sdk/Portfile

MarcusCalhoun-Lopez · 2024-04-25T12:55:40Z

Of course, there is no rush on this pull request.
However, I am a little unclear about what issues still need to be addressed before merging it.

ryandesign · 2024-05-13T06:50:46Z

I am a little unclear about what issues still need to be addressed before merging it.

Sorry, I haven't really looked at this again since my April comments.

The files installed by the MacOSX.sdk subports should be exactly the same regardless of the system, so add `any` to `platforms`. There might be use in installing an older SDK on a newer system, so do not limit `supported_archs`.

Fixes https://trac.macports.org/ticket/69952

MarcusCalhoun-Lopez · 2024-05-14T05:42:59Z

Sorry, I haven't really looked at this again since my April comments.

Thank you for taking the time to review the changes.
There is certainly no rush.
This port is extremely useful, but I suspect to only to a very small population.

RJVB · 2024-05-14T11:02:04Z

On Monday May 13 2024 22:40:51 MarcusCalhoun-Lopez wrote: However, as you point out, it must either be fixed in the base or in the Portfile with an "unorthodox approach." The ideas are interesting, but I think they are a bit outside the scope of this PR.

Unorthodox but AFAICT not in violation of anything. In the end it isn't that different from ports that install files in the post-activation phase, or (perhaps a better analogy) ports that contain software that installs its own runtime files when used. Add a startup item and the user will notice very little of the change This PR already fixes the fetch issue in addition to adding subports so adding another fix doesn't really change its nature. But I'm not planning to get involved any further :) OTOH, pushing the fix for the fetch issue ASAP would be courteous ...

This port is extremely useful, but I suspect to only to a very small population.

My own interest was limited to being able to generate a fat binary with Apple ARM components, which I could do without even installing the port (= by pointing `isysroot` into the port's destroot.) Having an idea how and what for people the port would be probably be helpful in deciding how to approach the rev-upgrade issue. It seems obvious that this port can't be used as any kind of dependency of another port. So if everyone already deactivates it when not in use then there really wouldn't be much hurry to fix the rev-upgrade issue if it weren't for the fact that frequent de- and re-activation of a port with this many (small) files really isn't very kind on SSDs (from what I've understood).

ryandesign · 2024-05-22T19:07:02Z

It seems obvious that this port can't be used as any kind of dependency of another port.

I don't know why you say that. I created this port for ports that need older or newer SDKs. Ideally, the newer/older SDKs would only need to be installed on the buildbot workers and then the binary that gets produced would work without it. I didn't particularly intend users to use the older/newer SDKs directly, though there's no reason they can't do that if they want.

Regarding the rev-upgrade issue, I simply didn't notice when I created the ports since I had rev-upgrade turned off in my MacPorts installation.

RJVB · 2024-05-22T20:20:24Z

Yuck! Eww!

I do know why you say that.

I don't know why you say that.

I should probably have said "any kind of runtime dependency". What I fail to see is why any port would need an older SDK, much less a newer one than the host one (except outliers like the half-baked 10.10 SDK on 10.9). I don't think I've never encounter APIs that have been deprecated away from the system headers but not from the system frameworks (which would be a case where you'd need the old SDK). And building against a newer SDK isn't going to provide the implementations for all those new functions it declares... That the buildbots could use older SDKs to build for older OS versions is a different matter: I don't suppose they will ever run rev-upgrade (if they work in any way like the Ubuntu/Launchpad build bots).

Gcenx · 2024-05-25T06:54:07Z

Any chance this can get merged?

Gcenx · 2024-05-25T07:00:11Z

devel/MacOSX.sdk/Portfile

+    homepage            https://developer.apple.com/documentation/macos-release-notes/macos-14-release-notes
+}
+
+subport MacOSX13.sdk {


I’d ideally want this for game-porting-toolkit to avoid even more patches.

reneeotten · 2024-08-16T16:53:39Z

is there a path forward here? If so, let's make some progress, otherwise let's close the PR

RJVB · 2024-08-16T18:03:26Z

What is this even blocking on? Just the fact that the CI/testfarm/whatever can't find a logfile when it wants to?

MarcusCalhoun-Lopez · 2024-08-16T18:09:32Z

I think I have addressed all the specific concerns expressed.
My best guess as to why the pull request has not been merged is that there are higher priorities.

reneeotten · 2024-08-16T18:23:39Z

@MarcusCalhoun-Lopez then just merge it yourself.

I can't follow all the discussion here and @ryandesign hasn't been back to leave a response so this will remain in limbo. If you think it's all done then merge it. Of someone wants to make changes that can be done later.

Gcenx · 2024-12-26T05:55:09Z

Can we finally get this merged?

cooljeanius · 2025-05-27T17:42:57Z

Can we finally get this merged?

Seconded!

RJVB · 2025-05-30T16:25:25Z

I just tried to use the (patched) version to fetch the 10.7 SDK, and apparently I am not entitled to get the required Xcode installer with the developer account I've been using since forever.

Why's that? Do you need to be a paying developer for that, nowadays?

EDIT: to the rescue O:^)

Gcenx · 2025-05-30T16:46:11Z

I just tried to use the (patched) version to fetch the 10.7 SDK, and apparently I am not entitled to get the required Xcode installer with the developer account I've been using since forever.

Why's that? Do you need to be a paying developer for that, nowadays?

EDIT: to the rescue O:^)

You can download Xcode 2.3 & Xcode Tools v1.0 from https://developer.apple.com/download/all/ without even having a paid developers account.

RJVB · 2025-05-30T20:34:05Z

On Friday May 30 2025 09:46:33 Dean M Greer wrote: You can download Xcode 2.3 & Xcode Tools v1.0 from https://developer.apple.com/download/all/ without even having a paid developers account.

And they contain the 10.7 SDK?

macportsbot assigned ryandesign Apr 1, 2024

macportsbot added type: enhancement maintainer: requires approval by: member Created by a member with commit rights labels Apr 1, 2024

ryandesign reviewed Apr 2, 2024

View reviewed changes

MarcusCalhoun-Lopez force-pushed the more-sdks branch 2 times, most recently from fbceab4 to fe83f2f Compare April 9, 2024 11:25

MarcusCalhoun-Lopez changed the title ~~MacOSX.sdk: add SDK subports for 10.3-10.5~~ MacOSX.sdk: add SDK subports for 10.3-10.5 & 13-14 Apr 9, 2024

MarcusCalhoun-Lopez added 4 commits May 13, 2024 22:01

MacOSX.sdk: prefer new version of vercmp

10421e5

MacOSX.sdk: use configure prefix on build_arch

3797a41

MacOSX.sdk: add SDK subports for 10.3-10.5

f78b896

MacOSX.sdk: mark as system agnostic

a91ce50

The files installed by the MacOSX.sdk subports should be exactly the same regardless of the system, so add `any` to `platforms`. There might be use in installing an older SDK on a newer system, so do not limit `supported_archs`.

MarcusCalhoun-Lopez force-pushed the more-sdks branch from fe83f2f to 93d0ac4 Compare May 14, 2024 05:04

MarcusCalhoun-Lopez added 2 commits May 13, 2024 22:33

MacOSX.sdk: remove nonexistent distfiles

72c9555

Fixes https://trac.macports.org/ticket/69952

MacOSX13.sdk, MacOSX14.sdk: new subports

9c80a61

MarcusCalhoun-Lopez force-pushed the more-sdks branch from 93d0ac4 to 9c80a61 Compare May 14, 2024 05:35

Gcenx reviewed May 25, 2024

View reviewed changes

cooljeanius approved these changes Nov 21, 2024

View reviewed changes

Uh oh!

MacOSX.sdk: add SDK subports for 10.3-10.5 & 13-14 #23312

Are you sure you want to change the base?

MacOSX.sdk: add SDK subports for 10.3-10.5 & 13-14 #23312

Uh oh!

Conversation

MarcusCalhoun-Lopez commented Apr 1, 2024

Description

Type(s)

Tested on

Verification

Uh oh!

macportsbot commented Apr 1, 2024

Uh oh!

ryandesign left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ryandesign Apr 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RJVB May 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RJVB May 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

MarcusCalhoun-Lopez commented Apr 25, 2024

Uh oh!

ryandesign commented May 13, 2024

Uh oh!

MarcusCalhoun-Lopez commented May 14, 2024

Uh oh!

RJVB commented May 14, 2024 via email

Uh oh!

ryandesign commented May 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RJVB commented May 22, 2024 via email

Uh oh!

Gcenx commented May 25, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

reneeotten commented Aug 16, 2024

Uh oh!

RJVB commented Aug 16, 2024

Uh oh!

MarcusCalhoun-Lopez commented Aug 16, 2024

Uh oh!

reneeotten commented Aug 16, 2024

Uh oh!

Gcenx commented Dec 26, 2024

Uh oh!

ryandesign Apr 2, 2024 •

edited

Loading

RJVB May 13, 2024 •

edited

Loading

RJVB May 13, 2024 •

edited

Loading

ryandesign commented May 22, 2024 •

edited

Loading

RJVB commented May 30, 2025 •

edited

Loading