use yet even more bin/jval-wrapper.sh instead of direct grep of JSON files
We use the `bin/jval-wrapper.sh` tool to obtain information from JSON files instead of attempting using a "GROSS HACK" that involved using `grep(1)` (and friends) as those "GROSS HACK"s assumed that the layout of the JSON was canonical. The `bin/jval-wrapper.sh` tool is able to deal with various layouts of JSON including non-canonical layouts. Moreover, by using the `bin/jval-wrapper.sh` tool we greatly reduce the places in bin tools that contain "GROSS HACK"s. Adjusted `bin/gen-status.sh` to use `bin/jval-wrapper.sh`. Changed `bin/gen-status.sh` VERSION from "1.5.8 2024-08-05" to "1.6 2024-08-17". Performed `make gen_status` to test the above under macOS. Adjusted `bin/gen-years.sh` to use `bin/jval-wrapper.sh`. Changed `bin/gen-years.sh` VERSION from "1.3.7 2024-08-05" to "1.4 2024-08-17". Performed `make gen_years` to test the above under macOS. Adjusted `bin/output-index-author.sh` to use `bin/jval-wrapper.sh`. Changed `bin/output-index-author.sh` VERSION from "1.7.3 2024-08-05" to "1.8 2024-08-17". Performed `make entry_index` to test the above under macOS. Added `make quick_year_index` to generate year level index.html files when year level `YYYY/README.md` is newer. The `make quick_www` calls `quick_year_index` instead of `make gen_year_index`. The `bin` directory is now sat as `BIN_PATH=$TOPDIR/bin`, where `$TOPDIR` is determined by top of the git repo (as determined by the command: `git rev-parse --is-inside-work-tree`) and potentially modified by `-d topdir`. After `cd $TOPDIR`, we set `BIN_DIR=bin`. All tools from this repo are now assumed to under "$BIN_DIR". Removed command like options such as `-T md2html.sh`, and `-p tool` that used to be able to change the location of a `bin` tool. Changed `bin/gen-authors.sh` VERSION from "1.11 2024-08-13" to "1.12 2024-08-18". Changed `bin/gen-location.sh` VERSION from "1.5 2024-08-17" to "1.6 2024-08-18". Changed `bin/readme2index.sh` VERSION from "1.2.7 2024-08-05" to "1.3 2024-08-17". Modified `bin/all-run.sh` to use the `bin` directory mechanism. Modified `bin/all-years.sh` to use the `bin` directory mechanism. Modified `bin/gen-other-html.sh` to use the `bin` directory mechanism. Modified `bin/gen-year-index.sh` to use the `bin` directory mechanism. Modified `bin/md2html.sh` to use the `bin` directory mechanism. Modified `bin/quick-readme2index.sh` to use the `bin` directory mechanism. Modified `bin/readme2index.sh` to use the `bin` directory mechanism. Changed `bin/all-run.sh` VERSION from "1.3.6 2024-08-05" to "1.4 2024-08-17". Changed `bin/all-years.sh` VERSION from "1.0.5 2024-08-05" to "1.1 2024-08-17". Changed `bin/gen-other-html.sh` VERSION from "1.3 2024-08-11" to "1.4 2024-08-17". Changed `bin/gen-year-index.sh` VERSION from "1.1.2 2024-08-05" to "1.2 2024-08-17". Changed `bin/quick-readme2index.sh` VERSION from "1.1.7 2024-08-05" to "1.2 2024-08-17". Changed `bin/md2html.sh` VERSION from "1.5.5 2024-08-05" to "1.6 2024-08-17". Added some whitespace in the form of empty lines to tools. Made checks for tools under `bin` directory more consistent. When using `-Q` (quick mode) and the prerequisite files are newer, we force the target to be touched so that a later run will not have to build the target HTML file. Performed `make quick_www` to test the above under macOS. Performed `make www` to test the above under macOS.
This commit is contained in:
Landon Curt Noll committed
1 parent
f159369a0a
commit
0e5864b8b0
39 files changed
+1694
-795
No files matched your search
+52
-26
@@ -26,6 +26,7 @@
|
||||
#
|
||||
# Share and enjoy! :-)
|
||||
|
||||
|
||||
# firewall - run only with a bash that is version 5.1.8 or later
|
||||
#
|
||||
# The "/usr/bin/env bash" command must result in using a bash that
|
||||
@@ -69,6 +70,7 @@ if [[ -z ${BASH_VERSINFO[0]} ||
|
||||
exit 4
|
||||
fi
|
||||
|
||||
|
||||
# setup bash file matching
|
||||
#
|
||||
# We must declare arrays with -ag or -Ag, and we need loops to "export" modified variables.
|
||||
@@ -81,6 +83,7 @@ shopt -u nocaseglob # disable strict case matching
|
||||
shopt -u extglob # enable extended globbing patterns
|
||||
shopt -s globstar # enable ** to match all files and zero or more directories and subdirectories
|
||||
|
||||
|
||||
# set variables referenced in the usage message
|
||||
#
|
||||
export VERSION="1.3.10 2024-08-05"
|
||||
@@ -110,10 +113,11 @@ if [[ -z "$GIT_TOOL" ]]; then
|
||||
exit 8
|
||||
fi
|
||||
|
||||
|
||||
# set usage message
|
||||
#
|
||||
export USAGE="usage: $0 [-h] [-v level] [-V] [-d topdir] [-D docroot/] [-n] [-N]
|
||||
[-p tool] [-w site_url] [-W]
|
||||
[-w site_url] [-W]
|
||||
YYYY/dir
|
||||
|
||||
-h print help message and exit
|
||||
@@ -126,8 +130,6 @@ export USAGE="usage: $0 [-h] [-v level] [-V] [-d topdir] [-D docroot/] [-n] [-N]
|
||||
-n go thru the actions, but do not update any files (def: do the action)
|
||||
-N do not process file, just parse arguments and ignore the file (def: process the file)
|
||||
|
||||
-p tool This option is ignored
|
||||
|
||||
-w site_url This option is ignored
|
||||
|
||||
-W Warn if a file in the manifest is missing: (def: error if a file is missing)
|
||||
@@ -148,15 +150,17 @@ Exit codes:
|
||||
|
||||
$NAME version: $VERSION"
|
||||
|
||||
|
||||
# setup
|
||||
#
|
||||
export NOOP=
|
||||
export DO_NOT_PROCESS=
|
||||
export EXIT_CODE="0"
|
||||
|
||||
|
||||
# parse command line
|
||||
#
|
||||
while getopts :hv:Vd:D:nNp:U:w:W flag; do
|
||||
while getopts :hv:Vd:D:nNU:w:W flag; do
|
||||
case "$flag" in
|
||||
h) echo "$USAGE" 1>&2
|
||||
exit 2
|
||||
@@ -173,7 +177,6 @@ while getopts :hv:Vd:D:nNp:U:w:W flag; do
|
||||
;;
|
||||
N) DO_NOT_PROCESS="-N"
|
||||
;;
|
||||
p) ;;
|
||||
U) ;;
|
||||
w) ;;
|
||||
W) CAP_W_FLAG_FOUND="true"
|
||||
@@ -195,7 +198,7 @@ while getopts :hv:Vd:D:nNp:U:w:W flag; do
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
#
|
||||
# parse the command line arguments
|
||||
#
|
||||
if [[ $V_FLAG -ge 3 ]]; then
|
||||
@@ -214,6 +217,7 @@ fi
|
||||
#
|
||||
export ENTRY_PATH="$1"
|
||||
|
||||
|
||||
# verify that we have a topdir directory
|
||||
#
|
||||
REPO_NAME=$(basename "$REPO_TOP_URL")
|
||||
@@ -234,6 +238,7 @@ if [[ ! -d $TOPDIR ]]; then
|
||||
exit 6
|
||||
fi
|
||||
|
||||
|
||||
# cd to topdir
|
||||
#
|
||||
if [[ ! -e $TOPDIR ]]; then
|
||||
@@ -257,6 +262,7 @@ if [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: now in directory: $(/bin/pwd)" 1>&2
|
||||
fi
|
||||
|
||||
|
||||
# verify that we have a bin subdirectory
|
||||
#
|
||||
export BIN_PATH="$TOPDIR/bin"
|
||||
@@ -266,6 +272,28 @@ if [[ ! -d $BIN_PATH ]]; then
|
||||
fi
|
||||
export BIN_DIR="bin"
|
||||
|
||||
|
||||
# verify we have our awk tool
|
||||
#
|
||||
export FILELIST_ENTRY_JSON_AWK="$BIN_DIR/filelist.entry.json.awk"
|
||||
if [[ ! -e $FILELIST_ENTRY_JSON_AWK ]]; then
|
||||
echo "$0: ERROR: filelist.entry.json.awk does not exist: $FILELIST_ENTRY_JSON_AWK" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ ! -f $FILELIST_ENTRY_JSON_AWK ]]; then
|
||||
echo "$0: ERROR: filelist.entry.json.awk is not a file: $FILELIST_ENTRY_JSON_AWK" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ ! -r $FILELIST_ENTRY_JSON_AWK ]]; then
|
||||
echo "$0: ERROR: filelist.entry.json.awk is not a readable file: $FILELIST_ENTRY_JSON_AWK" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ ! -s $FILELIST_ENTRY_JSON_AWK ]]; then
|
||||
echo "$0: ERROR: filelist.entry.json.awk is not a not a non-empty readable file: $FILELIST_ENTRY_JSON_AWK" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
|
||||
|
||||
# verify that ENTRY_PATH is a entry directory
|
||||
#
|
||||
if [[ ! -d $ENTRY_PATH ]]; then
|
||||
@@ -339,25 +367,6 @@ if [[ ! -r $ENTRY_JSON ]]; then
|
||||
exit 5
|
||||
fi
|
||||
|
||||
# verify we have our awk tool
|
||||
#
|
||||
export FILELIST_ENTRY_JSON_AWK="$BIN_DIR/filelist.entry.json.awk"
|
||||
if [[ ! -e $FILELIST_ENTRY_JSON_AWK ]]; then
|
||||
echo "$0: ERROR: filelist.entry.json.awk does not exist: $FILELIST_ENTRY_JSON_AWK" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ ! -f $FILELIST_ENTRY_JSON_AWK ]]; then
|
||||
echo "$0: ERROR: filelist.entry.json.awk is not a file: $FILELIST_ENTRY_JSON_AWK" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ ! -r $FILELIST_ENTRY_JSON_AWK ]]; then
|
||||
echo "$0: ERROR: filelist.entry.json.awk is not a readable file: $FILELIST_ENTRY_JSON_AWK" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ ! -s $FILELIST_ENTRY_JSON_AWK ]]; then
|
||||
echo "$0: ERROR: filelist.entry.json.awk is not a not a non-empty readable file: $FILELIST_ENTRY_JSON_AWK" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
|
||||
# verify we have non-empty readable ioccc.css file
|
||||
#
|
||||
@@ -379,6 +388,7 @@ if [[ ! -s $IOCCC_CSS ]]; then
|
||||
exit 6
|
||||
fi
|
||||
|
||||
|
||||
# verify we have non-empty readable var.mk file
|
||||
#
|
||||
export VAR_MK="var.mk"
|
||||
@@ -399,6 +409,7 @@ if [[ ! -s $VAR_MK ]]; then
|
||||
exit 6
|
||||
fi
|
||||
|
||||
|
||||
# verify we have non-empty readable 1337.mk
|
||||
#
|
||||
export LEET_MK="1337.mk"
|
||||
@@ -419,6 +430,7 @@ if [[ ! -s $LEET_MK ]]; then
|
||||
exit 6
|
||||
fi
|
||||
|
||||
|
||||
# determine the name of our tarball
|
||||
#
|
||||
# TARBALL - the same of the compressed tarball to verify and if needed re-build
|
||||
@@ -427,6 +439,7 @@ fi
|
||||
export TARBALL="$YYYY_DIR/$ENTRY_ID.tar.bz2"
|
||||
export REBUILD_TARBALL=
|
||||
|
||||
|
||||
# parameter debugging
|
||||
#
|
||||
if [[ $V_FLAG -ge 3 ]]; then
|
||||
@@ -447,6 +460,7 @@ if [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: CD_FAILED=$CD_FAILED" 1>&2
|
||||
echo "$0: debug[3]: BIN_PATH=$BIN_PATH" 1>&2
|
||||
echo "$0: debug[3]: BIN_DIR=$BIN_DIR" 1>&2
|
||||
echo "$0: debug[3]: FILELIST_ENTRY_JSON_AWK=$FILELIST_ENTRY_JSON_AWK" 1>&2
|
||||
echo "$0: debug[3]: YEAR_DIR=$YEAR_DIR" 1>&2
|
||||
echo "$0: debug[3]: ENTRY_DIR=$ENTRY_DIR" 1>&2
|
||||
echo "$0: debug[3]: ENTRY_ID=$ENTRY_ID" 1>&2
|
||||
@@ -455,7 +469,6 @@ if [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: DOT_PATH=$DOT_PATH" 1>&2
|
||||
echo "$0: debug[3]: DOT_PATH_CONTENT=$DOT_PATH_CONTENT" 1>&2
|
||||
echo "$0: debug[3]: ENTRY_JSON=$ENTRY_JSON" 1>&2
|
||||
echo "$0: debug[3]: FILELIST_ENTRY_JSON_AWK=$FILELIST_ENTRY_JSON_AWK" 1>&2
|
||||
echo "$0: debug[3]: IOCCC_CSS=$IOCCC_CSS" 1>&2
|
||||
echo "$0: debug[3]: VAR_MK=$VAR_MK" 1>&2
|
||||
echo "$0: debug[3]: LEET_MK=$LEET_MK" 1>&2
|
||||
@@ -463,6 +476,7 @@ if [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: REBUILD_TARBALL=$REBUILD_TARBALL" 1>&2
|
||||
fi
|
||||
|
||||
|
||||
# If -N, time to exit
|
||||
#
|
||||
if [[ -n $DO_NOT_PROCESS ]]; then
|
||||
@@ -472,6 +486,7 @@ if [[ -n $DO_NOT_PROCESS ]]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
|
||||
# test - if needed - if the tarball is a non-empty writable file
|
||||
#
|
||||
if [[ -z $REBUILD_TARBALL && ! -e $TARBALL ]]; then
|
||||
@@ -503,6 +518,7 @@ if [[ -z $REBUILD_TARBALL && ! -s $TARBALL ]]; then
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
# create a temporary file manifest list
|
||||
#
|
||||
export TMP_MANIFEST_LIST=".tmp.$NAME.MANIFEST_LIST.$$.tmp"
|
||||
@@ -525,6 +541,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: because of -n, temporary file manifest list is not used: $TMP_MANIFEST_LIST" 1>&2
|
||||
fi
|
||||
|
||||
|
||||
# create a temporary sort list of files to tar
|
||||
#
|
||||
export TMP_FILES_TO_TAR=".tmp.$NAME.FILES_TO_TAR.$$.tmp"
|
||||
@@ -547,6 +564,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: because of -n, temporary list of files to tar is not used: $TMP_FILES_TO_TAR" 1>&2
|
||||
fi
|
||||
|
||||
|
||||
# create a temporary exit code
|
||||
#
|
||||
# It is a pain to set the EXIT_CODE deep inside a loop, so we write the EXIT_CODE into a file
|
||||
@@ -572,6 +590,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: because of -n, temporary exit code is not used: $TMP_EXIT_CODE" 1>&2
|
||||
fi
|
||||
|
||||
|
||||
# generate sorted list of entry files from the entry's manifest
|
||||
#
|
||||
# We also add ioccc.css and var.mk from the top level.
|
||||
@@ -598,6 +617,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: because of -n, temporary file manifest list was not formed: $TMP_MANIFEST_LIST" 1>&2
|
||||
fi
|
||||
|
||||
|
||||
# generate sorted list of found entry files
|
||||
#
|
||||
# We exclude the compressed tarball itself because we are writing it.
|
||||
@@ -638,6 +658,7 @@ if [[ -z $EXIT_CODE ]]; then
|
||||
exit 16
|
||||
fi
|
||||
|
||||
|
||||
# create a temporary list of files found in the tarball
|
||||
#
|
||||
export TMP_TARBALL_LIST=".tmp.$NAME.TARBALL_LIST.$$.tmp"
|
||||
@@ -660,6 +681,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: because of -n, temporary of files in the tarball: $TARBALL is not used: $TMP_TARBALL_LIST" 1>&2
|
||||
fi
|
||||
|
||||
|
||||
# create a temporary tarball
|
||||
#
|
||||
export TMP_TARBALL=".tmp.$NAME.TARBALL.$$.tmp"
|
||||
@@ -682,6 +704,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: because of -n, temporary tarball is not used: $TMP_TARBALL" 1>&2
|
||||
fi
|
||||
|
||||
|
||||
# test - if needed - if the sorted list of files from the tarball matches the sorted list of found entry files
|
||||
#
|
||||
if [[ -z $REBUILD_TARBALL ]]; then
|
||||
@@ -722,6 +745,7 @@ if [[ -z $REBUILD_TARBALL ]]; then
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
# test - if needed - if the files in the tarball match the contents of the entry's files
|
||||
#
|
||||
if [[ -z $REBUILD_TARBALL ]]; then
|
||||
@@ -746,6 +770,7 @@ if [[ -z $REBUILD_TARBALL ]]; then
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
# test - if needed - update the tarball
|
||||
#
|
||||
# We use gnu tar to form the bzip2 compressed tarball with careful arguments
|
||||
@@ -865,6 +890,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[1]: no need to update $TARBALL" 1>&2
|
||||
fi
|
||||
|
||||
|
||||
# All Done!!! All Done!!! -- Jessica Noll, Age 2
|
||||
#
|
||||
if [[ -z $NOOP ]]; then
|
||||
|
||||
Reference in new issue
Block a user