Out-of-tree clang and flang can't find compiler-rt

[DavidTruby]

If either clang or flang are built out-of-tree against an already compiled LLVM, they are unable to find compiler-rt when linking. This is due to the path to compiler-rt being hard-coded to the same install tree as the clang driver, meaning that if compiler-rt is installed elsewhere it doesn't get picked up.

[llvmbot]

@llvm/issue-subscribers-flang-driver

Author: David Truby (DavidTruby)

If either clang or flang are built out-of-tree against an already compiled LLVM, they are unable to find compiler-rt when linking. This is due to the path to compiler-rt being hard-coded to the same install tree as the clang driver, meaning that if compiler-rt is installed elsewhere it doesn't get picked up.

[llvmbot]

@llvm/issue-subscribers-clang-driver

Author: David Truby (DavidTruby)

If either clang or flang are built out-of-tree against an already compiled LLVM, they are unable to find compiler-rt when linking. This is due to the path to compiler-rt being hard-coded to the same install tree as the clang driver, meaning that if compiler-rt is installed elsewhere it doesn't get picked up.

[DavidTruby]

This is causing the flang-aarch64-libcxx and flang-aarch64-out-of-tree bots to fail as flang on AArch64 always links to compiler-rt for a couple of symbols missing in libgcc, however this is not a flang specific issue and an out-of-tree clang configured with -DCLANG_DEFAULT_RTLIB=compiler-rt or using -rtlib=compiler-rt will also fail. We had some discussion in llvm/llvm-zorg#482 but I wanted to open a ticket here for more visibility in case anyone has any ideas what we can do about this.

[luporl]

Some ideas of how this could be fixed:

1. Add a new driver flag, such as --compiler-rt-libdir, that would override its default path.
2. Add a cmake rule to copy compiler-rt libraries to clang/flang build directory, when built out-of-tree.
3. Instead of hard-coding compiler-rt's path, look for it in library search path.

Option 1 seems to be the most flexible and with low risk, with the main disadvantages being a new driver flag added and a slightly more complex way to find compiler-rt.

Option 2 looks safe too. LLVM's directory location should already be available when building out-of-tree. The main disadvantage would be the extra complexity in cmake files.

Option 3 seems to be the most risky, especially for clang, as it would change its default behavior. But if adopted only for flang, it would be consistent with flang-rt, that already uses this approach.

[DavidTruby]

I'd rather not have the lookup behaviour for compiler-rt be different for clang and flang, at the moment sharing all that logic makes things much smoother I think.
Option 2 could be a quick and dirty workaround but doesn't feel like the "right" fix. Probably the extra flag is the better way to do it. Out-of-tree users would still then need a config file to specify that flag though I guess?

[luporl]

Out-of-tree users would still then need a config file to specify that flag though I guess?

Right, for the tests to work there needs to be some way to tell them that an additional flag must be used.

I think the best option would be to check if we are building out-of-tree and if so add the new flag when linking, as the user already has to specify the LLVM directory.
We already do something similar on MacOS, where the proper -isysroot flag is added:
https://github.com/llvm/llvm-project/pull/77365/files#diff-2f6a629120184fc3102f82f1937bbe82bcd7ff8015e314f8b2d14922452ce7ad