Merge bitcoin/bitcoin#27676: macOS: Bump minimum required runtime version and prepare for building with upstream LLVM

3df6070466 contrib: remove macOS lazy_bind check (fanquake)
9bc357e205 build: explicitly opt-in to new fixup_chains functionality for darwin (Cory Fields)
fb61bc0c02 depends: Bump MacOS minimum runtime requirement to 11.0 (Cory Fields)
c2cd47280c depends: bump darwin clang to 11.1 (Cory Fields)

Pull request description:

  This (I believe) resolves the last of the blockers for [switching us away from cctools and instead using out-of-the-box llvm and lld](https://github.com/bitcoin/bitcoin/pull/21778) for building Darwin binaries.

  For now, we continue building with a pre-packaged llvm and cctools, but after this PR the clang+lld combo should just work for anyone trying it. Additionally after this PR, the new runtime `fixup_chains` behavior will be in-use, as ld64 uses it as well.

  The commits may seem unrelated, so in detail:

  lld (llvm's linker) has been a work-in-progress for Darwin for years. Recently though, it has gained nearly all of the features we require. The last missing feature from ld64, `-Wl,-bind_at_load`, is not implemented in lld; as far as I can tell [lazy loading has conceptually been replaced by fixup chains](https://www.emergetools.com/blog/posts/iOS15LaunchTime).

  So that means we don't need ld64's `bind_at_load` as long as lld can handle `-Wl,-fixup_chains` (which it can). I've added it to our configure as a linker option mostly so that we can see it in the logs; it's default-on as long as the minimum version is >11.0.

  About that: the runtime functionality required for `-Wl,-fixup_chains` [requires macOS >=11.0](https://github.com/llvm/llvm-project/blob/release/16.x/lld/MachO/Driver.cpp#L1021). Hence the commit that bumps the minimum version. Our current min runtime of `10.15` has been unsupported since September 2022, so I don't expect this bump to be controversial.

  Lastly, with the minimum runtime version bumped to 11.0, our current version of pre-compiled clang we use for macOS is too old to understand `-mmacosx-version-min=11.0` because it expects `=10.x`. So I've made the smallest possible bump (from 10.0.1 to 11.1.0) to a version that understands. This bump is arbitrary and unfortunate, but likely to be short-lived as we may end up replacing it with llvm+lld for v26 anyway. I've held off on bumping the SDK as I think that makes sense to do as part of the lld switch instead.

ACKs for top commit:
  hebasto:
    ACK 3df6070466
  gruve-p:
    ACK https://github.com/bitcoin/bitcoin/pull/27676/commits/3df60704661cdb5e61ea2b999f468f3a1d16105f
  fanquake:
    ACK 3df6070466
  TheCharlatan:
    ACK 3df6070466

Tree-SHA512: 0200ec4a3b88df33877ae82c15b5c04d745852550923f491a354b391cac65f88e4df116a40055c23a8cbcfcdfb9a376c6ae8fdd0e898e7b966bc213dcb5857cf
This commit is contained in:
fanquake committed 2023-06-27 09:57:21 +01:00
commit 35b01e8504
11 files changed
+17 -26

No files matched your search

+1 -1
View File
@@ -314,7 +314,7 @@ task:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV << : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
task: task:
name: 'macOS 10.15 [gui, no tests] [jammy]' name: 'macOS 11.0 [gui, no tests] [jammy]'
<< : *CONTAINER_DEPENDS_TEMPLATE << : *CONTAINER_DEPENDS_TEMPLATE
container: container:
docker_arguments: docker_arguments:
+1
View File
@@ -1008,6 +1008,7 @@ if test "$TARGET_OS" = "darwin"; then
AX_CHECK_LINK_FLAG([-Wl,-dead_strip], [CORE_LDFLAGS="$CORE_LDFLAGS -Wl,-dead_strip"], [], [$LDFLAG_WERROR]) AX_CHECK_LINK_FLAG([-Wl,-dead_strip], [CORE_LDFLAGS="$CORE_LDFLAGS -Wl,-dead_strip"], [], [$LDFLAG_WERROR])
AX_CHECK_LINK_FLAG([-Wl,-dead_strip_dylibs], [CORE_LDFLAGS="$CORE_LDFLAGS -Wl,-dead_strip_dylibs"], [], [$LDFLAG_WERROR]) AX_CHECK_LINK_FLAG([-Wl,-dead_strip_dylibs], [CORE_LDFLAGS="$CORE_LDFLAGS -Wl,-dead_strip_dylibs"], [], [$LDFLAG_WERROR])
AX_CHECK_LINK_FLAG([-Wl,-bind_at_load], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,-bind_at_load"], [], [$LDFLAG_WERROR]) AX_CHECK_LINK_FLAG([-Wl,-bind_at_load], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,-bind_at_load"], [], [$LDFLAG_WERROR])
AX_CHECK_LINK_FLAG([-Wl,-fixup_chains], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,-fixup_chains"], [], [$LDFLAG_WERROR])
fi fi
AC_CHECK_HEADERS([endian.h sys/endian.h byteswap.h sys/select.h sys/prctl.h sys/sysctl.h vm/vm_param.h sys/vmmeter.h sys/resources.h]) AC_CHECK_HEADERS([endian.h sys/endian.h byteswap.h sys/select.h sys/prctl.h sys/sysctl.h vm/vm_param.h sys/vmmeter.h sys/resources.h])
-8
View File
@@ -158,13 +158,6 @@ def check_MACHO_NOUNDEFS(binary) -> bool:
''' '''
return binary.header.has(lief.MachO.HEADER_FLAGS.NOUNDEFS) return binary.header.has(lief.MachO.HEADER_FLAGS.NOUNDEFS)
def check_MACHO_LAZY_BINDINGS(binary) -> bool:
'''
Check for no lazy bindings.
We don't use or check for MH_BINDATLOAD. See #18295.
'''
return binary.dyld_info.lazy_bind == (0,0)
def check_MACHO_Canary(binary) -> bool: def check_MACHO_Canary(binary) -> bool:
''' '''
Check for use of stack canary Check for use of stack canary
@@ -214,7 +207,6 @@ BASE_PE = [
BASE_MACHO = [ BASE_MACHO = [
('NOUNDEFS', check_MACHO_NOUNDEFS), ('NOUNDEFS', check_MACHO_NOUNDEFS),
('LAZY_BINDINGS', check_MACHO_LAZY_BINDINGS),
('Canary', check_MACHO_Canary), ('Canary', check_MACHO_Canary),
] ]
+1 -1
View File
@@ -232,7 +232,7 @@ def check_MACHO_libraries(binary) -> bool:
return ok return ok
def check_MACHO_min_os(binary) -> bool: def check_MACHO_min_os(binary) -> bool:
if binary.build_version.minos == [10,15,0]: if binary.build_version.minos == [11,0,0]:
return True return True
return False return False
+6 -8
View File
@@ -120,13 +120,13 @@ class TestSecurityChecks(unittest.TestCase):
if arch == lief.ARCHITECTURES.X86: if arch == lief.ARCHITECTURES.X86:
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-flat_namespace','-Wl,-allow_stack_execute','-fno-stack-protector']), self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-flat_namespace','-Wl,-allow_stack_execute','-fno-stack-protector']),
(1, executable+': failed NOUNDEFS LAZY_BINDINGS Canary PIE NX CONTROL_FLOW')) (1, executable+': failed NOUNDEFS Canary PIE NX CONTROL_FLOW'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-flat_namespace','-Wl,-allow_stack_execute','-fstack-protector-all']), self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-flat_namespace','-Wl,-allow_stack_execute','-fstack-protector-all']),
(1, executable+': failed NOUNDEFS LAZY_BINDINGS PIE NX CONTROL_FLOW')) (1, executable+': failed NOUNDEFS PIE NX CONTROL_FLOW'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-flat_namespace','-fstack-protector-all']), self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-flat_namespace','-fstack-protector-all']),
(1, executable+': failed NOUNDEFS LAZY_BINDINGS PIE CONTROL_FLOW')) (1, executable+': failed NOUNDEFS PIE CONTROL_FLOW'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-fstack-protector-all']), self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-fstack-protector-all']),
(1, executable+': failed LAZY_BINDINGS PIE CONTROL_FLOW')) (1, executable+': failed PIE CONTROL_FLOW'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-bind_at_load','-fstack-protector-all']), self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-bind_at_load','-fstack-protector-all']),
(1, executable+': failed PIE CONTROL_FLOW')) (1, executable+': failed PIE CONTROL_FLOW'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-bind_at_load','-fstack-protector-all', '-fcf-protection=full']), self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-bind_at_load','-fstack-protector-all', '-fcf-protection=full']),
@@ -136,11 +136,9 @@ class TestSecurityChecks(unittest.TestCase):
else: else:
# arm64 darwin doesn't support non-PIE binaries, control flow or executable stacks # arm64 darwin doesn't support non-PIE binaries, control flow or executable stacks
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-flat_namespace','-fno-stack-protector']), self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-flat_namespace','-fno-stack-protector']),
(1, executable+': failed NOUNDEFS LAZY_BINDINGS Canary')) (1, executable+': failed NOUNDEFS Canary'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-flat_namespace','-fstack-protector-all']), self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-flat_namespace','-fstack-protector-all']),
(1, executable+': failed NOUNDEFS LAZY_BINDINGS')) (1, executable+': failed NOUNDEFS'))
self.assertEqual(call_security_check(cc, source, executable, ['-fstack-protector-all']),
(1, executable+': failed LAZY_BINDINGS'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-bind_at_load','-fstack-protector-all']), self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-bind_at_load','-fstack-protector-all']),
(0, '')) (0, ''))
+1 -1
View File
@@ -121,7 +121,7 @@ class TestSymbolChecks(unittest.TestCase):
} }
''') ''')
self.assertEqual(call_symbol_check(cc, source, executable, ['-Wl,-platform_version','-Wl,macos', '-Wl,10.15', '-Wl,11.4']), self.assertEqual(call_symbol_check(cc, source, executable, ['-Wl,-platform_version','-Wl,macos', '-Wl,11.0', '-Wl,11.4']),
(1, f'{executable}: failed SDK')) (1, f'{executable}: failed SDK'))
def test_PE(self): def test_PE(self):
+1 -1
View File
@@ -608,5 +608,5 @@ inspecting signatures in Mach-O binaries.")
((string-contains target "-linux-") ((string-contains target "-linux-")
(list (make-bitcoin-cross-toolchain target))) (list (make-bitcoin-cross-toolchain target)))
((string-contains target "darwin") ((string-contains target "darwin")
(list clang-toolchain-10 binutils cmake-minimal xorriso python-signapple)) (list clang-toolchain-11 binutils cmake-minimal xorriso python-signapple))
(else '()))))) (else '())))))
+1 -1
View File
@@ -1,4 +1,4 @@
OSX_MIN_VERSION=10.15 OSX_MIN_VERSION=11.0
OSX_SDK_VERSION=11.0 OSX_SDK_VERSION=11.0
XCODE_VERSION=12.2 XCODE_VERSION=12.2
XCODE_BUILD_ID=12B45b XCODE_BUILD_ID=12B45b
+3 -3
View File
@@ -1,12 +1,12 @@
package=native_clang package=native_clang
$(package)_version=10.0.1 $(package)_version=11.1.0
$(package)_download_path=https://github.com/llvm/llvm-project/releases/download/llvmorg-$($(package)_version) $(package)_download_path=https://github.com/llvm/llvm-project/releases/download/llvmorg-$($(package)_version)
ifneq (,$(findstring aarch64,$(BUILD))) ifneq (,$(findstring aarch64,$(BUILD)))
$(package)_file_name=clang+llvm-$($(package)_version)-aarch64-linux-gnu.tar.xz $(package)_file_name=clang+llvm-$($(package)_version)-aarch64-linux-gnu.tar.xz
$(package)_sha256_hash=90dc69a4758ca15cd0ffa45d07fbf5bf4309d47d2c7745a9f0735ecffde9c31f $(package)_sha256_hash=18df38247af3fba0e0e2991fb00d7e3cf3560b4d3509233a14af699ef0039e1c
else else
$(package)_file_name=clang+llvm-$($(package)_version)-x86_64-linux-gnu-ubuntu-16.04.tar.xz $(package)_file_name=clang+llvm-$($(package)_version)-x86_64-linux-gnu-ubuntu-16.04.tar.xz
$(package)_sha256_hash=48b83ef827ac2c213d5b64f5ad7ed082c8bcb712b46644e0dc5045c6f462c231 $(package)_sha256_hash=c691a558967fb7709fb81e0ed80d1f775f4502810236aa968b4406526b43bee1
endif endif
define $(package)_stage_cmds define $(package)_stage_cmds
+1 -1
View File
@@ -36,7 +36,7 @@ Compatibility
============== ==============
Bitcoin Core is supported and extensively tested on operating systems Bitcoin Core is supported and extensively tested on operating systems
using the Linux kernel, macOS 10.15+, and Windows 7 and newer. Bitcoin using the Linux kernel, macOS 11.0+, and Windows 7 and newer. Bitcoin
Core should also work on most other Unix-like systems but is not as Core should also work on most other Unix-like systems but is not as
frequently tested on them. It is not recommended to use Bitcoin Core on frequently tested on them. It is not recommended to use Bitcoin Core on
unsupported systems. unsupported systems.
+1 -1
View File
@@ -3,7 +3,7 @@
<plist version="0.9"> <plist version="0.9">
<dict> <dict>
<key>LSMinimumSystemVersion</key> <key>LSMinimumSystemVersion</key>
<string>10.15.0</string> <string>11</string>
<key>LSArchitecturePriority</key> <key>LSArchitecturePriority</key>
<array> <array>