use jsp tool, workaround JSON Unicode bugs, misc fixes
If the [jsp tool](https://github.com/kjozsa/jsp) is installed, it will be used in favor of the [JSONPath.sh tool](https://github.com/lcn2/JSONPath.sh). Updated `bin/jfmt-wrapper.sh`, `bin/jprint-wrapper.sh`, and `bin/jval-wrapper.sh` to use jsp if found. These 3 tools may be given `-t tool` where tool is either "_jsp_" or "_JSONPath.sh_" for force the use of the tool. They also use `-Q` to perform a quick check tool using trivial input. The `bin/jval-wrapper.sh` tool had been simplified. The `-w`, `-i` and `-u` options are gone. Instead it has: ``` -b print only values (def: print both JSON member and JSON value) -q remove enclosing double quotes (def: keep any enclosing double quotes) ``` Add `make all_jfmt` to Makefile to canonically format all entry and author handle JSON files. A sed script `bin/unicode-fix.sed` has been added that converts JSON string "_\u[0-9A-Fa-f]{4}_" encoding that may arise from known JSON data. This is done as a work-a-round for [Unicode / UTF-8 issues](https://github.com/xexyl/jparse/issues/13) with `jstrdecode(1)`. The `bin/unicode-chk.sh` tool, if given on file arguments, scans all `.entry.json` and `author/author_handle.json` JSON files to be sure that, after `bin/unicode-fix.sed` processing, no JSON string "_\u[0-9A-Fa-f]{4}_" encodings remain. Removed all `-j jparse` args from commands. Instead the `jparse(1)` tool is looked for along $PATH. Added `combine_author_handle.sh` tool to transform all of the `author/author_handle.json` files into a temporary as a single JSON file so that `bin/jval-wrapper.sh` tool may be used to extract all "_sort_word_" or all "_entry_id_" values. Changed `bin/all-jfmt.sh` VERSION from "1.0 2024-08-31" to "1.1 2024-09-24". Changed `bin/csv2entry.sh` VERSION from "1.2.3 2024-08-30" to "1.3 2024-09-23". Changed `bin/cvt-submission.sh` VERSION from "1.1 2024-08-31" to "1.2 2024-09-23". Changed `bin/entry2csv.sh` VERSION from "1.2.1 2024-08-17" to "1.3 2024-09-23". Changed `bin/gen-authors.sh` VERSION from "1.12 2024-08-18" to "1.13 2024-09-23". Changed `bin/gen-location.sh` VERSION from "1.6 2024-08-18" to "1.7 2024-09-23". Changed `bin/gen-status.sh` VERSION from "1.6 2024-08-17" to "1.7 2024-09-23". Changed `bin/gen-years.sh` VERSION from "1.4 2024-08-17" to "1.5 2024-09-23". Changed `bin/jfmt-wrapper.sh` VERSION from "1.1 2024-08-20" to "1.2 2024-09-23". Changed `bin/jprint-wrapper.sh` VERSION from "1.1 2024-08-20" to "1.3 2024-09-23". Changed `bin/jval-wrapper.sh` VERSION from "1.0.4 2024-08-20" to "1.1 2024-09-23". Changed `bin/manifest.entry.json.awk` VERSION from "1.1.2 2024-07-28" to "1.2 2024-09-17". Changed `bin/output-index-author.sh` VERSION from "1.8 2024-08-17" to "1.9 2024-09-23". Changed `bin/output-year-index.sh` VERSION from "1.4.4 2024-08-05" to "1.5 2024-09-23". Changed `bin/subst.entry-index.sh` VERSION from "1.5 2024-08-13" to "1.6 2024-09-23". Added `bin/combine_author_handle.sh` VERSION "1.0 2024-09-23". Added `bin/unicode-chk.sh` VERSION "1.0 2024-09-23". Added `bin/unicode-fix.sed` VERSION "1.0 2024-09-23". Perform `make www` to perform the above and test under macOS. FYI: On our MacBook Pro, `make www` took: ``` make www 631.79s user 264.95s system 94% cpu 15:53.71 total ``` FYI: On our MacBook Pro, `make quick_www` took: ``` make quick_www 81.43s user 58.26s system 102% cpu 2:16.02 total ```
This commit is contained in:
Landon Curt Noll committed
1 parent
e84d6eb095
commit
499be5983d
19 files changed
+3044
-1213
No files matched your search
+21
-23
@@ -86,7 +86,7 @@ shopt -s globstar # enable ** to match all files and zero or more directories an
|
||||
|
||||
# set variables referenced in the usage message
|
||||
#
|
||||
export VERSION="1.6 2024-08-17"
|
||||
export VERSION="1.7 2024-09-23"
|
||||
NAME=$(basename "$0")
|
||||
export NAME
|
||||
export V_FLAG=0
|
||||
@@ -108,12 +108,6 @@ export REPO_TOP_URL="https://github.com/ioccc-src/temp-test-ioccc"
|
||||
# GitHub puts individual files under the "blob/master" sub-directory.
|
||||
export REPO_URL="$REPO_TOP_URL/blob/master"
|
||||
export SITE_URL="https://ioccc-src.github.io/temp-test-ioccc"
|
||||
JPARSE_TOOL=$(type -P jparse)
|
||||
export JPARSE_TOOL
|
||||
if [[ -z "$JPARSE_TOOL" ]]; then
|
||||
echo "$0: FATAL: jparse tool is not installed or not in \$PATH" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
#
|
||||
STAT_TOOL=$(type -P stat)
|
||||
if [[ -z "$STAT_TOOL" ]]; then
|
||||
@@ -294,11 +288,23 @@ function output_status_json
|
||||
|
||||
# determine the "contest_status" that was in status.json
|
||||
#
|
||||
OLD_CONTEST_STATUS=$("$JVAL_WRAPPER" -w -b "$STATUS_PATH" '$..contest_status')
|
||||
PATTERN='$..contest_status'
|
||||
OLD_CONTEST_STATUS=$("$JVAL_WRAPPER" -b -q "$STATUS_PATH" "$PATTERN")
|
||||
status="$?"
|
||||
if [[ $status -ne 0 || -z $OLD_CONTEST_STATUS ]]; then
|
||||
echo "$0: ERROR: cannot determine old contest_status from status.json: $STATUS_JSON" 1>&2
|
||||
return 11
|
||||
fi
|
||||
|
||||
# determine the "status_update" that was in status.json
|
||||
#
|
||||
OLD_STATUS_UPDATE=$("$JVAL_WRAPPER" -w -b "$STATUS_PATH" '$..status_update')
|
||||
PATTERN='$..status_update'
|
||||
OLD_STATUS_UPDATE=$("$JVAL_WRAPPER" -b -q "$STATUS_PATH" "$PATTERN")
|
||||
status="$?"
|
||||
if [[ $status -ne 0 || -z $OLD_STATUS_UPDATE ]]; then
|
||||
echo "$0: ERROR: cannot determine old status_update from status.json: $STATUS_JSON" 1>&2
|
||||
return 12
|
||||
fi
|
||||
|
||||
# update the status.json file modification date if the "contest_status" changed
|
||||
#
|
||||
@@ -306,11 +312,11 @@ function output_status_json
|
||||
status="$?"
|
||||
if [[ $status -ne 0 ]]; then
|
||||
echo "$0: ERROR: in output_status_json: modification date of $STATUS_PATH failed, error: $status" 1>&2
|
||||
return 11
|
||||
return 13
|
||||
fi
|
||||
if [[ -z $NEW_STATUS_UPDATE ]]; then
|
||||
echo "$0: ERROR: in output_status_json: modification date of $STATUS_PATH is empty" 1>&2
|
||||
return 12
|
||||
return 14
|
||||
fi
|
||||
|
||||
# only update the "status_update" If the "contest_status" changed
|
||||
@@ -788,7 +794,6 @@ if [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: REPO_TOP_URL=$REPO_TOP_URL" 1>&2
|
||||
echo "$0: debug[3]: REPO_URL=$REPO_URL" 1>&2
|
||||
echo "$0: debug[3]: SITE_URL=$SITE_URL" 1>&2
|
||||
echo "$0: debug[3]: JPARSE_TOOL=$JPARSE_TOOL" 1>&2
|
||||
echo "$0: debug[3]: IOCCC_STATUS_VERSION=$IOCCC_STATUS_VERSION" 1>&2
|
||||
echo "$0: debug[3]: NOOP=$NOOP" 1>&2
|
||||
echo "$0: debug[3]: DO_NOT_PROCESS=$DO_NOT_PROCESS" 1>&2
|
||||
@@ -819,23 +824,16 @@ if [[ $V_FLAG -ge 3 ]]; then
|
||||
fi
|
||||
|
||||
|
||||
# validate JSON in status.json
|
||||
#
|
||||
"$JPARSE_TOOL" -q "$STATUS_JSON"
|
||||
status="$?"
|
||||
if [[ $status -ne 0 ]]; then
|
||||
echo "$0: ERROR: status.json is not valid JSON, jparse error code: $status" 1>&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# obtain contest_status if CONTEST_STATUS was not set in the arg
|
||||
#
|
||||
if [[ -z $CONTEST_STATUS ]]; then
|
||||
|
||||
# obtain the contest_status from status.json
|
||||
#
|
||||
CONTEST_STATUS=$("$JVAL_WRAPPER" -w -b "$STATUS_JSON" '$..contest_status')
|
||||
if [[ -z $CONTEST_STATUS ]]; then
|
||||
PATTERN='$..contest_status'
|
||||
CONTEST_STATUS=$("$JVAL_WRAPPER" -b -q "$STATUS_JSON" "$PATTERN")
|
||||
status="$?"
|
||||
if [[ $status -ne 0 || -z $CONTEST_STATUS ]]; then
|
||||
echo "$0: ERROR: cannot determine contest_status from status.json: $STATUS_JSON" 1>&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
Reference in new issue
Block a user