use yet even more bin/jval-wrapper.sh instead of direct grep of JSON files

We use the `bin/jval-wrapper.sh` tool to obtain information from JSON
files instead of attempting using a "GROSS HACK" that involved using
`grep(1)` (and friends) as those "GROSS HACK"s assumed that the layout of
the JSON was canonical.  The `bin/jval-wrapper.sh` tool is able to deal
with various layouts of JSON including non-canonical layouts.  Moreover,
by using the `bin/jval-wrapper.sh` tool we greatly reduce the places in
bin tools that contain "GROSS HACK"s.

Adjusted `bin/gen-status.sh` to use `bin/jval-wrapper.sh`.

Changed `bin/gen-status.sh` VERSION from "1.5.8 2024-08-05"
to "1.6 2024-08-17".

Performed `make gen_status` to test the above under macOS.

Adjusted `bin/gen-years.sh` to use `bin/jval-wrapper.sh`.

Changed `bin/gen-years.sh` VERSION from "1.3.7 2024-08-05"
to "1.4 2024-08-17".

Performed `make gen_years` to test the above under macOS.

Adjusted `bin/output-index-author.sh` to use `bin/jval-wrapper.sh`.

Changed `bin/output-index-author.sh` VERSION from "1.7.3 2024-08-05"
to "1.8 2024-08-17".

Performed `make entry_index` to test the above under macOS.

Added `make quick_year_index` to generate year level index.html files
when year level `YYYY/README.md` is newer.

The `make quick_www` calls `quick_year_index` instead of `make gen_year_index`.

The `bin` directory is now sat as `BIN_PATH=$TOPDIR/bin`, where
`$TOPDIR` is determined by top of the git repo (as determined by
the command: `git rev-parse --is-inside-work-tree`) and potentially
modified by `-d topdir`.  After `cd $TOPDIR`, we set `BIN_DIR=bin`.

All tools from this repo are now assumed to under "$BIN_DIR".  Removed
command like options such as `-T md2html.sh`, and `-p tool` that used
to be able to change the location of a `bin` tool.

Changed `bin/gen-authors.sh` VERSION from "1.11 2024-08-13"
to "1.12 2024-08-18".

Changed `bin/gen-location.sh` VERSION from "1.5 2024-08-17"
to "1.6 2024-08-18".

Changed `bin/readme2index.sh` VERSION from "1.2.7 2024-08-05"
to "1.3 2024-08-17".

Modified `bin/all-run.sh` to use the `bin` directory mechanism.
Modified `bin/all-years.sh` to use the `bin` directory mechanism.
Modified `bin/gen-other-html.sh` to use the `bin` directory mechanism.
Modified `bin/gen-year-index.sh` to use the `bin` directory mechanism.
Modified `bin/md2html.sh` to use the `bin` directory mechanism.
Modified `bin/quick-readme2index.sh` to use the `bin` directory mechanism.
Modified `bin/readme2index.sh` to use the `bin` directory mechanism.

Changed `bin/all-run.sh` VERSION from "1.3.6 2024-08-05"
to "1.4 2024-08-17".

Changed `bin/all-years.sh` VERSION from "1.0.5 2024-08-05"
to "1.1 2024-08-17".

Changed `bin/gen-other-html.sh` VERSION from "1.3 2024-08-11"
to "1.4 2024-08-17".

Changed `bin/gen-year-index.sh` VERSION from "1.1.2 2024-08-05"
to "1.2 2024-08-17".

Changed `bin/quick-readme2index.sh` VERSION from "1.1.7 2024-08-05"
to "1.2 2024-08-17".

Changed `bin/md2html.sh` VERSION from "1.5.5 2024-08-05"
to "1.6 2024-08-17".

Added some whitespace in the form of empty lines to tools.

Made checks for tools under `bin` directory more consistent.

When using `-Q` (quick mode) and the prerequisite files are newer,
we force the target to be touched so that a later run will not
have to build the target HTML file.

Performed `make quick_www` to test the above under macOS.

Performed `make www` to test the above under macOS.
This commit is contained in:
Landon Curt Noll committed 2024-08-18 02:10:25 -07:00
1 parent f159369a0a
commit 0e5864b8b0
39 files changed
+1694 -795

No files matched your search

+65 -22
View File
@@ -17,7 +17,7 @@
#
# Each tool run with command line of the form:
#
# tool [any -D, -t tagline, -T md2html.sh, -p tool, -w site_url] [more_options] YYYY/dir
# tool [any -D, -t tagline, -w site_url] [more_options] YYYY/dir
#
# For example:
#
@@ -55,6 +55,7 @@
#
# Share and enjoy! :-)
# firewall - run only with a bash that is version 5.1.8 or later
#
# The "/usr/bin/env bash" command must result in using a bash that
@@ -111,9 +112,10 @@ shopt -u nocaseglob # disable strict case matching
shopt -u extglob # enable extended globbing patterns
shopt -s globstar # enable ** to match all files and zero or more directories and subdirectories
# set variables referenced in the usage message
#
export VERSION="1.3.6 2024-08-05"
export VERSION="1.4 2024-08-17"
NAME=$(basename "$0")
export NAME
export V_FLAG=0
@@ -131,8 +133,6 @@ fi
export TOPDIR
export DOCROOT_SLASH="../../"
export TAGLINE="unspecified"
export MD2HTML_SH="bin/md2html.sh"
export PANDOC_WRAPPER="bin/pandoc-wrapper.sh"
export REPO_TOP_URL="https://github.com/ioccc-src/temp-test-ioccc"
# GitHub puts individual files under the "blob/master" sub-directory.
export REPO_URL="$REPO_TOP_URL/blob/master"
@@ -142,15 +142,17 @@ export NOOP=
export DO_NOT_PROCESS=
export EXIT_CODE="0"
# clear options we will add to tools
#
unset TOOL_OPTION
declare -ag TOOL_OPTION
# usage
#
export USAGE="usage: $0 [-h] [-v level] [-V] [-d topdir] [-D docroot/] [-n] [-N]
[-t tagline] [-T md2html.sh] [-p tool] [-w site_url]
[-t tagline] [-w site_url]
tool [more_options]
-h print help message and exit
@@ -167,10 +169,6 @@ export USAGE="usage: $0 [-h] [-v level] [-V] [-d topdir] [-D docroot/] [-n] [-N]
-t tagline string to write about the tool that formed the markdown content (def: $TAGLINE)
NOTE: The '-t tagline' is passed as leading options on tool command lines.
-T md2html.sh run 'markdown to html tool' to convert markdown into HTML (def: $MD2HTML_SH)
NOTE: The '-T md2html.sh' is passed as leading options on tool command lines.
-p tool run 'pandoc wrapper tool' (not pandoc path) during HTML phase number 21 (def: use $PANDOC_WRAPPER)
-w site_url Base URL of the website (def: $SITE_URL)
NOTE: The '-w site_url' is passed as leading options on tool command lines.
@@ -178,7 +176,7 @@ export USAGE="usage: $0 [-h] [-v level] [-V] [-d topdir] [-D docroot/] [-n] [-N]
tool the tool to run over all entries
[more_options] additional tool command line options to use before the YYYY/dir argument
NOTE: Any '-D docroot/', '-t tagline', '-T md2html.sh', '-p tool', '-w site_url'
NOTE: Any '-D docroot/', '-t tagline', '-w site_url'
are passed to the 'tool' at the beginning of the command line, and
before any optional 'more_options' and before the final YYYY/dir argument.
@@ -195,9 +193,10 @@ Exit codes:
$NAME version: $VERSION"
# parse command line
#
while getopts :hv:Vd:D:nNt:T:p:w: flag; do
while getopts :hv:Vd:D:nNt:w: flag; do
case "$flag" in
h) echo "$USAGE" 1>&2
exit 2
@@ -246,14 +245,6 @@ while getopts :hv:Vd:D:nNt:T:p:w: flag; do
TOOL_OPTION+=("-t")
TOOL_OPTION+=("$TAGLINE")
;;
T) MD2HTML_SH="$OPTARG"
TOOL_OPTION+=("-T")
TOOL_OPTION+=("$MD2HTML_SH")
;;
p) PANDOC_WRAPPER="$OPTARG"
TOOL_OPTION+=("-p")
TOOL_OPTION+=("$PANDOC_WRAPPER")
;;
w) SITE_URL="$OPTARG"
TOOL_OPTION+=("-w")
TOOL_OPTION+=("$SITE_URL")
@@ -275,7 +266,7 @@ while getopts :hv:Vd:D:nNt:T:p:w: flag; do
;;
esac
done
#
# remove the options
#
shift $(( OPTIND - 1 ));
@@ -303,6 +294,7 @@ case "$#" in
;;
esac
# verify that we have a topdir directory
#
REPO_NAME=$(basename "$REPO_TOP_URL")
@@ -323,6 +315,7 @@ if [[ ! -d $TOPDIR ]]; then
exit 6
fi
# cd to topdir
#
if [[ ! -e $TOPDIR ]]; then
@@ -346,6 +339,51 @@ if [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: now in directory: $(/bin/pwd)" 1>&2
fi
# verify that we have a bin subdirectory
#
export BIN_PATH="$TOPDIR/bin"
if [[ ! -d $BIN_PATH ]]; then
echo "$0: ERROR: bin is not a directory under topdir: $BIN_PATH" 1>&2
exit 6
fi
export BIN_DIR="bin"
# verify that the bin/md2html.sh tool is executable
#
export MD2HTML_SH="$BIN_DIR/md2html.sh"
if [[ ! -e $MD2HTML_SH ]]; then
echo "$0: ERROR: bin/md2html.sh does not exist: $MD2HTML_SH" 1>&2
exit 5
fi
if [[ ! -f $MD2HTML_SH ]]; then
echo "$0: ERROR: bin/md2html.sh is not a regular file: $MD2HTML_SH" 1>&2
exit 5
fi
if [[ ! -x $MD2HTML_SH ]]; then
echo "$0: ERROR: bin/md2html.sh is not an executable file: $MD2HTML_SH" 1>&2
exit 5
fi
# verify that the bin/pandoc-wrapper.sh tool is executable
#
export PANDOC_WRAPPER="$BIN_DIR/pandoc-wrapper.sh"
if [[ ! -e $PANDOC_WRAPPER ]]; then
echo "$0: ERROR: bin/md2html.sh does not exist: $PANDOC_WRAPPER" 1>&2
exit 5
fi
if [[ ! -f $PANDOC_WRAPPER ]]; then
echo "$0: ERROR: bin/md2html.sh is not a regular file: $PANDOC_WRAPPER" 1>&2
exit 5
fi
if [[ ! -x $PANDOC_WRAPPER ]]; then
echo "$0: ERROR: bin/md2html.sh is not an executable file: $PANDOC_WRAPPER" 1>&2
exit 5
fi
# verify we have a non-empty readable .top file
#
export TOP_FILE=".top"
@@ -408,8 +446,6 @@ if [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: TOPDIR=$TOPDIR" 1>&2
echo "$0: debug[3]: DOCROOT_SLASH=$DOCROOT_SLASH" 1>&2
echo "$0: debug[3]: TAGLINE=$TAGLINE" 1>&2
echo "$0: debug[3]: MD2HTML_SH=$MD2HTML_SH" 1>&2
echo "$0: debug[3]: PANDOC_WRAPPER=$PANDOC_WRAPPER" 1>&2
echo "$0: debug[3]: REPO_TOP_URL=$REPO_TOP_URL" 1>&2
echo "$0: debug[3]: REPO_URL=$REPO_URL" 1>&2
echo "$0: debug[3]: SITE_URL=$SITE_URL" 1>&2
@@ -422,9 +458,14 @@ if [[ $V_FLAG -ge 3 ]]; then
echo "$0: debug[3]: TOOL=$TOOL" 1>&2
echo "$0: debug[3]: REPO_NAME=$REPO_NAME" 1>&2
echo "$0: debug[3]: CD_FAILED=$CD_FAILED" 1>&2
echo "$0: debug[3]: BIN_PATH=$BIN_PATH" 1>&2
echo "$0: debug[3]: BIN_DIR=$BIN_DIR" 1>&2
echo "$0: debug[3]: MD2HTML_SH=$MD2HTML_SH" 1>&2
echo "$0: debug[3]: PANDOC_WRAPPER=$PANDOC_WRAPPER" 1>&2
echo "$0: debug[3]: TOP_FILE=$TOP_FILE" 1>&2
fi
# -N stops early before any processing is performed
#
if [[ -n $DO_NOT_PROCESS ]]; then
@@ -434,6 +475,7 @@ if [[ -n $DO_NOT_PROCESS ]]; then
exit 0
fi
# process each year
#
export YYYY
@@ -620,6 +662,7 @@ for YYYY in $(< "$TOP_FILE"); do
done
done
# All Done!!! All Done!!! -- Jessica Noll, Age 2
#
if [[ $EXIT_CODE -ne 0 ]]; then