use yet even more bin/jval-wrapper.sh instead of direct grep of JSON files

We use the `bin/jval-wrapper.sh` tool to obtain information from JSON
files instead of attempting using a "GROSS HACK" that involved using
`grep(1)` (and friends) as those "GROSS HACK"s assumed that the layout of
the JSON was canonical.  The `bin/jval-wrapper.sh` tool is able to deal
with various layouts of JSON including non-canonical layouts.  Moreover,
by using the `bin/jval-wrapper.sh` tool we greatly reduce the places in
bin tools that contain "GROSS HACK"s.

Adjusted `bin/gen-status.sh` to use `bin/jval-wrapper.sh`.

Changed `bin/gen-status.sh` VERSION from "1.5.8 2024-08-05"
to "1.6 2024-08-17".

Performed `make gen_status` to test the above under macOS.

Adjusted `bin/gen-years.sh` to use `bin/jval-wrapper.sh`.

Changed `bin/gen-years.sh` VERSION from "1.3.7 2024-08-05"
to "1.4 2024-08-17".

Performed `make gen_years` to test the above under macOS.

Adjusted `bin/output-index-author.sh` to use `bin/jval-wrapper.sh`.

Changed `bin/output-index-author.sh` VERSION from "1.7.3 2024-08-05"
to "1.8 2024-08-17".

Performed `make entry_index` to test the above under macOS.

Added `make quick_year_index` to generate year level index.html files
when year level `YYYY/README.md` is newer.

The `make quick_www` calls `quick_year_index` instead of `make gen_year_index`.

The `bin` directory is now sat as `BIN_PATH=$TOPDIR/bin`, where
`$TOPDIR` is determined by top of the git repo (as determined by
the command: `git rev-parse --is-inside-work-tree`) and potentially
modified by `-d topdir`.  After `cd $TOPDIR`, we set `BIN_DIR=bin`.

All tools from this repo are now assumed to under "$BIN_DIR".  Removed
command like options such as `-T md2html.sh`, and `-p tool` that used
to be able to change the location of a `bin` tool.

Changed `bin/gen-authors.sh` VERSION from "1.11 2024-08-13"
to "1.12 2024-08-18".

Changed `bin/gen-location.sh` VERSION from "1.5 2024-08-17"
to "1.6 2024-08-18".

Changed `bin/readme2index.sh` VERSION from "1.2.7 2024-08-05"
to "1.3 2024-08-17".

Modified `bin/all-run.sh` to use the `bin` directory mechanism.
Modified `bin/all-years.sh` to use the `bin` directory mechanism.
Modified `bin/gen-other-html.sh` to use the `bin` directory mechanism.
Modified `bin/gen-year-index.sh` to use the `bin` directory mechanism.
Modified `bin/md2html.sh` to use the `bin` directory mechanism.
Modified `bin/quick-readme2index.sh` to use the `bin` directory mechanism.
Modified `bin/readme2index.sh` to use the `bin` directory mechanism.

Changed `bin/all-run.sh` VERSION from "1.3.6 2024-08-05"
to "1.4 2024-08-17".

Changed `bin/all-years.sh` VERSION from "1.0.5 2024-08-05"
to "1.1 2024-08-17".

Changed `bin/gen-other-html.sh` VERSION from "1.3 2024-08-11"
to "1.4 2024-08-17".

Changed `bin/gen-year-index.sh` VERSION from "1.1.2 2024-08-05"
to "1.2 2024-08-17".

Changed `bin/quick-readme2index.sh` VERSION from "1.1.7 2024-08-05"
to "1.2 2024-08-17".

Changed `bin/md2html.sh` VERSION from "1.5.5 2024-08-05"
to "1.6 2024-08-17".

Added some whitespace in the form of empty lines to tools.

Made checks for tools under `bin` directory more consistent.

When using `-Q` (quick mode) and the prerequisite files are newer,
we force the target to be touched so that a later run will not
have to build the target HTML file.

Performed `make quick_www` to test the above under macOS.

Performed `make www` to test the above under macOS.
This commit is contained in:
Landon Curt Noll committed 2024-08-18 02:10:25 -07:00
1 parent f159369a0a
commit 0e5864b8b0
39 files changed
+1694 -795

No files matched your search

+90 -36
View File
@@ -26,6 +26,7 @@
#
# Share and enjoy! :-)
# firewall - run only with a bash that is version 5.1.8 or later
#
# The "/usr/bin/env bash" command must result in using a bash that
@@ -82,9 +83,10 @@ shopt -u nocaseglob # disable strict case matching
shopt -u extglob # enable extended globbing patterns
shopt -s globstar # enable ** to match all files and zero or more directories and subdirectories
# set variables referenced in the usage message
#
export VERSION="1.5.8 2024-08-05"
export VERSION="1.6 2024-08-17"
NAME=$(basename "$0")
export NAME
export V_FLAG=0
@@ -102,7 +104,6 @@ fi
export TOPDIR
export DOCROOT_SLASH="./"
export TAGLINE="bin/$NAME"
export MD2HTML_SH="bin/md2html.sh"
export REPO_TOP_URL="https://github.com/ioccc-src/temp-test-ioccc"
# GitHub puts individual files under the "blob/master" sub-directory.
export REPO_URL="$REPO_TOP_URL/blob/master"
@@ -124,19 +125,21 @@ if [[ -z "$LS_TOOL" ]]; then
echo "$0: FATAL: ls tool is not installed or not in \$PATH" 1>&2
exit 5
fi
#
export IOCCC_STATUS_VERSION="1.0 2024-03-09"
#
export NOOP=
export DO_NOT_PROCESS=
export EXIT_CODE="0"
# clear options we will add to tools
#
unset TOOL_OPTION
declare -ag TOOL_OPTION
export MODTIME_METHOD=""
# output_modtime - file modification time in W3C Datetime format:
#
# https://www.w3.org/TR/NOTE-datetime
@@ -212,6 +215,7 @@ function output_modtime
return 0
}
# output_status_json
#
# write a status.json file to standard output (stdout)
@@ -290,13 +294,11 @@ function output_status_json
# determine the "contest_status" that was in status.json
#
OLD_CONTEST_STATUS=$(grep '"contest_status"[[:space:]]*:[[:space:]]*"' "$STATUS_PATH" |
sed -e 's/^.*:[[:space:]]*"//' -e 's/",*//')
OLD_CONTEST_STATUS=$("$JVAL_WRAPPER" -w -b "$STATUS_PATH" '$..contest_status')
# determine the "status_update" that was in status.json
#
OLD_STATUS_UPDATE=$(grep '"status_update"[[:space:]]*:[[:space:]]*"' "$STATUS_PATH" |
sed -e 's/^.*:[[:space:]]*"//' -e 's/",*//')
OLD_STATUS_UPDATE=$("$JVAL_WRAPPER" -w -b "$STATUS_PATH" '$..status_update')
# update the status.json file modification date if the "contest_status" changed
#
@@ -333,11 +335,11 @@ function output_status_json
return 0
}
# usage
#
export USAGE="usage: $0 [-h] [-v level] [-V] [-d topdir] [-n] [-N]
[-t tagline] [-T md2html.sh]
[p | pending | o | open | j | judging | c | closed]
[-t tagline] [p | pending | o | open | j | judging | c | closed]
-h print help message and exit
-v level set verbosity level (def level: 0)
@@ -355,7 +357,6 @@ export USAGE="usage: $0 [-h] [-v level] [-V] [-d topdir] [-n] [-N]
-t tagline string to write about the tool that formed the markdown content (def: $TAGLINE)
NOTE: 'tagline' may be enclosed within, but may NOT contain an internal single-quote, or double-quote.
-T md2html.sh run 'markdown to html tool' to convert markdown into HTML (def: $MD2HTML_SH)
-w site_url Base URL of the website (def: $SITE_URL)
NOTE: The '-w site_url' is passed as leading options on tool command lines.
@@ -368,7 +369,7 @@ export USAGE="usage: $0 [-h] [-v level] [-V] [-d topdir] [-n] [-N]
NOTE: The '-v level' is passed as initial command line options to the 'markdown to html tool' (md2html.sh).
The 'tagline' is passed as '-t tagline' to the 'markdown to html tool' (md2html.sh), after the '-v level'.
Any '-T md2html.sh', '-p tool', '-P pandoc_opts', '-U top_url'
Any '-P pandoc_opts', '-U top_url'
are passed to the 'markdown to html tool' (md2html.sh), and will be before any command line arguments.
Exit codes:
@@ -383,9 +384,10 @@ Exit codes:
$NAME version: $VERSION"
# parse command line
#
while getopts :hv:Vd:D:nNt:T:w: flag; do
while getopts :hv:Vd:D:nNt:w: flag; do
case "$flag" in
h) echo "$USAGE" 1>&2
exit 2
@@ -434,10 +436,6 @@ while getopts :hv:Vd:D:nNt:T:w: flag; do
TAGLINE="$OPTARG"
# -t tagline always added after arg parsing
;;
T) MD2HTML_SH="$OPTARG"
TOOL_OPTION+=("-T")
TOOL_OPTION+=("$MD2HTML_SH")
;;
w) SITE_URL="$OPTARG"
TOOL_OPTION+=("-w")
TOOL_OPTION+=("$SITE_URL")
@@ -459,7 +457,7 @@ while getopts :hv:Vd:D:nNt:T:w: flag; do
;;
esac
done
#
# remove the options
#
shift $(( OPTIND - 1 ));
@@ -493,6 +491,7 @@ if [[ $# -eq 1 ]]; then
esac
fi
# always add the '-v level' option, unless level is empty, to the set of options passed to the md2html.sh tool
#
if [[ -n $V_FLAG ]]; then
@@ -500,6 +499,7 @@ if [[ -n $V_FLAG ]]; then
TOOL_OPTION+=("$V_FLAG")
fi
# always add the '-t tagline' option, unless tagline is empty, to the set of options passed to the md2html.sh tool
#
if [[ -n $TAGLINE ]]; then
@@ -507,16 +507,19 @@ if [[ -n $TAGLINE ]]; then
TOOL_OPTION+=("$TAGLINE")
fi
# always add the '-U URL' for the top level status.html file
#
TOOL_OPTION+=("-U")
TOOL_OPTION+=("$SITE_URL/status.html")
# always add the '-D docroot/' for the top level status.html file
#
TOOL_OPTION+=("-D")
TOOL_OPTION+=("$DOCROOT_SLASH")
# verify that we have a topdir directory
#
REPO_NAME=$(basename "$REPO_TOP_URL")
@@ -537,6 +540,7 @@ if [[ ! -d $TOPDIR ]]; then
exit 6
fi
# cd to topdir
#
if [[ ! -e $TOPDIR ]]; then
@@ -560,21 +564,51 @@ if [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: now in directory: $(/bin/pwd)" 1>&2
fi
# verify that the md2html tool is executable
# verify that we have a bin subdirectory
#
export BIN_PATH="$TOPDIR/bin"
if [[ ! -d $BIN_PATH ]]; then
echo "$0: ERROR: bin is not a directory under topdir: $BIN_PATH" 1>&2
exit 6
fi
export BIN_DIR="bin"
# verify that the bin/md2html.sh tool is executable
#
export MD2HTML_SH="$BIN_DIR/md2html.sh"
if [[ ! -e $MD2HTML_SH ]]; then
echo "$0: ERROR: md2html.sh does not exist: $MD2HTML_SH" 1>&2
echo "$0: ERROR: bin/md2html.sh does not exist: $MD2HTML_SH" 1>&2
exit 5
fi
if [[ ! -f $MD2HTML_SH ]]; then
echo "$0: ERROR: md2html.sh is not a regular file: $MD2HTML_SH" 1>&2
echo "$0: ERROR: bin/md2html.sh is not a regular file: $MD2HTML_SH" 1>&2
exit 5
fi
if [[ ! -x $MD2HTML_SH ]]; then
echo "$0: ERROR: md2html.sh is not an executable file: $MD2HTML_SH" 1>&2
echo "$0: ERROR: bin/md2html.sh is not an executable file: $MD2HTML_SH" 1>&2
exit 5
fi
# verify that the bin/jval-wrapper.sh tool is executable
#
JVAL_WRAPPER="$BIN_DIR/jval-wrapper.sh"
if [[ ! -e $JVAL_WRAPPER ]]; then
echo "$0: ERROR: bin/jval-wrapper.sh does not exist: $JVAL_WRAPPER" 1>&2
exit 5
fi
if [[ ! -f $JVAL_WRAPPER ]]; then
echo "$0: ERROR: bin/jval-wrapper.sh is not a regular file: $JVAL_WRAPPER" 1>&2
exit 5
fi
if [[ ! -x $JVAL_WRAPPER ]]; then
echo "$0: ERROR: bin/jval-wrapper.sh is not an executable file: $JVAL_WRAPPER" 1>&2
exit 5
fi
# verify readable non-empty news.md file
#
export NEWS_MD="news.md"
@@ -595,6 +629,7 @@ if [[ ! -s $NEWS_MD ]]; then
exit 1
fi
# verify readable non-empty status.json file
#
export STATUS_JSON="status.json"
@@ -615,6 +650,7 @@ if [[ ! -s $STATUS_JSON ]]; then
exit 1
fi
# verify readable non-empty status.md file
#
export STATUS_MD="status.md"
@@ -635,10 +671,12 @@ if [[ ! -s $STATUS_MD ]]; then
exit 1
fi
# note status.html file
#
export STATUS_HTML="status.html"
# verify we have a non-empty readable .top file
#
export TOP_FILE=".top"
@@ -659,6 +697,7 @@ if [[ ! -s $TOP_FILE ]]; then
exit 6
fi
# verify that we have an inc subdirectory
#
export INC_PATH="$TOPDIR/inc"
@@ -668,6 +707,7 @@ if [[ ! -d $INC_PATH ]]; then
fi
export INC_DIR="inc"
# verify that we have an next subdirectory
#
export NEXT_PATH="$TOPDIR/next"
@@ -677,6 +717,7 @@ if [[ ! -d $NEXT_PATH ]]; then
fi
export NEXT_DIR="next"
# determine how we can determine the file modification time in W3C Datetime format:
#
# https://www.w3.org/TR/NOTE-datetime
@@ -734,6 +775,7 @@ else
fi
fi
# print running info if verbose
#
# If -v 3 or higher, print exported variables in order that they were exported.
@@ -746,13 +788,10 @@ if [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: TOPDIR=$TOPDIR" 1>&2
echo "$0: debug[3]: DOCROOT_SLASH=$DOCROOT_SLASH" 1>&2
echo "$0: debug[3]: TAGLINE=$TAGLINE" 1>&2
echo "$0: debug[3]: MD2HTML_SH=$MD2HTML_SH" 1>&2
echo "$0: debug[3]: REPO_TOP_URL=$REPO_TOP_URL" 1>&2
echo "$0: debug[3]: REPO_URL=$REPO_URL" 1>&2
echo "$0: debug[3]: SITE_URL=$SITE_URL" 1>&2
echo "$0: debug[3]: JPARSE_TOOL=$JPARSE_TOOL" 1>&2
echo "$0: debug[3]: STAT_TOOL=$STAT_TOOL" 1>&2
echo "$0: debug[3]: LS_TOOL=$LS_TOOL" 1>&2
echo "$0: debug[3]: IOCCC_STATUS_VERSION=$IOCCC_STATUS_VERSION" 1>&2
echo "$0: debug[3]: NOOP=$NOOP" 1>&2
echo "$0: debug[3]: DO_NOT_PROCESS=$DO_NOT_PROCESS" 1>&2
@@ -761,9 +800,16 @@ if [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: TOOL_OPTION[$index]=${TOOL_OPTION[$index]}" 1>&2
done
echo "$0: debug[3]: MODTIME_METHOD=$MODTIME_METHOD" 1>&2
echo "$0: debug[3]: STAT_TOOL=$STAT_TOOL" 1>&2
echo "$0: debug[3]: LS_TOOL=$LS_TOOL" 1>&2
echo "$0: debug[3]: IOCCC_STATUS_VERSION=$IOCCC_STATUS_VERSION" 1>&2
echo "$0: debug[3]: CONTEST_STATUS=$CONTEST_STATUS" 1>&2
echo "$0: debug[3]: REPO_NAME=$REPO_NAME" 1>&2
echo "$0: debug[3]: CD_FAILED=$CD_FAILED" 1>&2
echo "$0: debug[3]: BIN_DIR=$BIN_DIR" 1>&2
echo "$0: debug[3]: BIN_PATH=$BIN_PATH" 1>&2
echo "$0: debug[3]: MD2HTML_SH=$MD2HTML_SH" 1>&2
echo "$0: debug[3]: JVAL_WRAPPER=$JVAL_WRAPPER" 1>&2
echo "$0: debug[3]: NEWS_MD=$NEWS_MD" 1>&2
echo "$0: debug[3]: STATUS_JSON=$STATUS_JSON" 1>&2
echo "$0: debug[3]: STATUS_MD=$STATUS_MD" 1>&2
@@ -775,6 +821,7 @@ if [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: NEXT_DIR=$NEXT_DIR" 1>&2
fi
# validate JSON in status.json
#
"$JPARSE_TOOL" -q "$STATUS_JSON"
@@ -786,18 +833,11 @@ fi
# obtain contest_status if CONTEST_STATUS was not set in the arg
#
# XXX - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - XXX
# XXX - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - XXX
# XXX - until we have the jnamval command, we must FAKE PARSE the author/author_handle.json file - XXX
# XXX - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - XXX
# XXX - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - XXX
#
if [[ -z $CONTEST_STATUS ]]; then
# obtain the contest_status from status.json
#
CONTEST_STATUS=$(grep '"contest_status"[[:space:]]*:[[:space:]]*"' "$STATUS_JSON" |
sed -e 's/^.*:[[:space:]]*"//' -e 's/",*//')
CONTEST_STATUS=$("$JVAL_WRAPPER" -w -b "$STATUS_JSON" '$..contest_status')
if [[ -z $CONTEST_STATUS ]]; then
echo "$0: ERROR: cannot determine contest_status from status.json: $STATUS_JSON" 1>&2
exit 1
@@ -806,13 +846,13 @@ if [[ -z $CONTEST_STATUS ]]; then
# validate and normalize the contest_status from status.json
#
case "$CONTEST_STATUS" in
p|pending) CONTEST_STATUS="pending"
pending) CONTEST_STATUS="pending"
;;
o|open) CONTEST_STATUS="open"
open) CONTEST_STATUS="open"
;;
j|judging) CONTEST_STATUS="judging"
judging) CONTEST_STATUS="judging"
;;
c|closed) CONTEST_STATUS="closed"
closed) CONTEST_STATUS="closed"
;;
*) echo "$0: ERROR: unexpected status from $status.json: $CONTEST_STATUS" 1>&2
echo 1>&2
@@ -822,6 +862,7 @@ if [[ -z $CONTEST_STATUS ]]; then
esac
fi
# determine the rules.md markdown file
#
export RULES_MD="$NEXT_DIR/rules.md"
@@ -843,6 +884,7 @@ if [[ ! -s $RULES_MD ]]; then
fi
export RULES_HTML="$NEXT_DIR/rules.html"
# determine the rules.md markdown file
#
export GUIDELINES_MD="$NEXT_DIR/guidelines.md"
@@ -864,6 +906,7 @@ if [[ ! -s $GUIDELINES_MD ]]; then
fi
export GUIDELINES_HTML="$NEXT_DIR/guidelines.html"
# determine the rules hdr file
#
export RULES_HDR="$INC_PATH/rules.$CONTEST_STATUS.hdr"
@@ -884,6 +927,7 @@ if [[ ! -s $RULES_HDR ]]; then
exit 1
fi
# determine the guidelines hdr file
#
export GUIDELINES_HDR="$INC_PATH/guidelines.$CONTEST_STATUS.hdr"
@@ -904,6 +948,7 @@ if [[ ! -s $GUIDELINES_HDR ]]; then
exit 1
fi
# -N stops early before any processing is performed
#
if [[ -n $DO_NOT_PROCESS ]]; then
@@ -913,6 +958,7 @@ if [[ -n $DO_NOT_PROCESS ]]; then
exit 0
fi
# create a temporary status.json file
#
export TMP_STATUS_JSON=".tmp.$NAME.STATUS_JSON.$$.tmp"
@@ -935,6 +981,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: because of -n, temporary status.json file is not used: $TMP_STATUS_JSON" 1>&2
fi
# create a temporary rules.md file
#
export TMP_RULES_MD=".tmp.$NAME.RULES_MD.$$.tmp"
@@ -957,6 +1004,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: because of -n, temporary rules.md file is not used: $TMP_RULES_MD" 1>&2
fi
# create a temporary guidelines.md file
#
export TMP_GUIDELINES_MD=".tmp.$NAME.GUIDELINES_MD.$$.tmp"
@@ -979,6 +1027,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: because of -n, temporary guidelines.md file is not used: $TMP_GUIDELINES_MD" 1>&2
fi
# generate the temporary status.json file
#
# In this stage, we use the date of the actual status.json file.
@@ -1069,6 +1118,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: because of -n, temporary status.json file is NOT written into: $TMP_STATUS_JSON" 1>&2
fi
# use the md2html.sh tool to form the status.html file, unless -n
#
if [[ -z $NOOP ]]; then
@@ -1095,6 +1145,7 @@ elif [[ $V_FLAG -ge 5 ]]; then
"status.md $STATUS_HTML" 1>&2
fi
# update rules.md
#
if [[ -z $NOOP ]]; then
@@ -1179,6 +1230,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: because of -n, rules.md was not updated: $RULES_MD" 1>&2
fi
# update guidelines.md
#
if [[ -z $NOOP ]]; then
@@ -1263,6 +1315,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: because of -n, guidelines.md was not updated: $GUIDELINES_MD" 1>&2
fi
# file cleanup
#
if [[ -z $NOOP ]]; then
@@ -1271,6 +1324,7 @@ elif [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: because of -n, disabled: rm -f -- $TMP_STATUS_JSON" 1>&2
fi
# All Done!!! All Done!!! -- Jessica Noll, Age 2
#
exit 0