guix: Use --cores instead of --max-jobs
In Guix, there are two flags for controlling parallelism:
Note: When I say "derivation," think "package"
--cores=n
- controls the number of CPU cores to build each derivation. This is
the value passed to `make`'s `--jobs=` flag.
- defaults to 0: as many cores as is available
--max-jobs=n
- controls how many derivations can be built in parallel
- defaults to 1
Therefore, if set --max-jobs=$MAX_JOBS and don't set --cores, Guix could
theoretically spin up $MAX_JOBS * $(nproc) number of threads, and that's
no good.
So we could either default to --cores=1, --max-jobs=$MAX_JOBS
- Pro: --cores=1 means that `make` will be invoked with `-j1`,
avoiding problems with package whose build systems and test
suites break when running multi-threaded.
- Con: There will be times when only 1 or 2 derivations can be built
at a time, because the rest of the dependency graph all depend
on those 1 or 2 derivations. During these times, the machine
will be severely under-utilized.
or --cores=$MAX_JOBS, --max-jobs=1
- Pro: We don't encounter prolonged periods of
severe under-utilization mentioned above.
- Con: Many packages' build systems and test suites break when running
multi-threaded.
or --cores=1, --max-jobs=1 and let the user override with
$ADDITIONAL_GUIX_COMMON_FLAGS
This commit is contained in:
1 parent
66daf4cb3b
commit
d5a71e9785
3 files changed
+63
-15
No files matched your search
@@ -26,7 +26,7 @@ cat << EOF
|
||||
Required environment variables as seen inside the container:
|
||||
HOST: ${HOST:?not set}
|
||||
SOURCE_DATE_EPOCH: ${SOURCE_DATE_EPOCH:?not set}
|
||||
MAX_JOBS: ${MAX_JOBS:?not set}
|
||||
JOBS: ${JOBS:?not set}
|
||||
DISTSRC: ${DISTSRC:?not set}
|
||||
OUTDIR: ${OUTDIR:?not set}
|
||||
EOF
|
||||
@@ -173,7 +173,7 @@ esac
|
||||
####################
|
||||
|
||||
# Build the depends tree, overriding variables that assume multilib gcc
|
||||
make -C depends --jobs="$MAX_JOBS" HOST="$HOST" \
|
||||
make -C depends --jobs="$JOBS" HOST="$HOST" \
|
||||
${V:+V=1} \
|
||||
${SOURCES_PATH+SOURCES_PATH="$SOURCES_PATH"} \
|
||||
${BASE_CACHE+BASE_CACHE="$BASE_CACHE"} \
|
||||
@@ -267,7 +267,7 @@ mkdir -p "$DISTSRC"
|
||||
sed -i.old 's/-lstdc++ //g' config.status libtool src/univalue/config.status src/univalue/libtool
|
||||
|
||||
# Build Bitcoin Core
|
||||
make --jobs="$MAX_JOBS" ${V:+V=1}
|
||||
make --jobs="$JOBS" ${V:+V=1}
|
||||
|
||||
# Perform basic ELF security checks on a series of executables.
|
||||
make -C src --jobs=1 check-security ${V:+V=1}
|
||||
@@ -344,7 +344,7 @@ mkdir -p "$DISTSRC"
|
||||
{
|
||||
find "${DISTNAME}/bin" -type f -executable -print0
|
||||
find "${DISTNAME}/lib" -type f -print0
|
||||
} | xargs -0 -n1 -P"$MAX_JOBS" -I{} "${DISTSRC}/contrib/devtools/split-debug.sh" {} {} {}.dbg
|
||||
} | xargs -0 -n1 -P"$JOBS" -I{} "${DISTSRC}/contrib/devtools/split-debug.sh" {} {} {}.dbg
|
||||
;;
|
||||
esac
|
||||
|
||||
|
||||
Reference in new issue
Block a user