release readme2index.sh version 1.0 2024-01-14
The initial release of `bin/readme2index.sh` tool that converts `READMD.md` and various JSON files such as `.winner.json` as well as "author/author_handle.json" files. Applied the `readme2index.sh` tool to generate `2020/ferguson1/index.html`. We generated the `2020/ferguson1/index.html` by executing: ```sh ./bin/readme2index.sh -v 3 2020/ferguson1 ``` NOTE: This above tool requires **pandoc to be installed**. See the [pandoc web site](https://pandoc.org) for how to install. We used _HomeBrew_ to install it, but there is also a _macports_ process as well. NOTE: While the above web page does have formatting issues, it is a first attempt data driven programmatic web page generation. Some adjustments can be made by mostly improving `ioccc.css`, and some perhaps by modifying how `bin/pandoc_wrapper.sh` invokes pandoc. But probably most of the improvements can be made by fixing and improving `ioccc.css`. TODO: Fix and improve the `2020/ferguson1/index.html` web page by fixing and improving `ioccc.css`. Moved scripts and tools needed to generate HTML pages into a new `/bin` subdirectory. TODO: Consider getting rid of the [build-ioccc repo](https://github.com/ioccc-src/build-ioccc) and just use the `/bin` subdirectory. NOTE: Some links in the `2020/ferguson1/index.html` web page do not (yet) exist. TODO: Once the basic format of the `2020/ferguson1/index.html` web page is somewhat reasonable, generate `index.html` pages for other winners. Generate the year level `index.html' files. Generate the top level HTML files, including the missing `location.html` file. Greatly simplified the `.winner.json` file format, removing a number of fields that were not strictly needed for generating an inventory on a `index.html` page. Updated various versions: * `bin/gen-author.sh` VERSION="1.3 2024-01-14" * `bin/gen-inventory.sh` VERSION="1.0 2024-01-14" * `bin/manifest.winner.json.awk` VERSION="1.0 2024-01-14" * `bin/pandoc_wrapper.sh` VERSION="1.3 2024-01-14" * `bin/readme2index.sh` VERSION="1.0 2024-01-14" * `bin/subst.default.sh` VERSION="1.1 2024-01-14" * `bin/subst.winner-index.sh` VERSION="1.1 2024-01-14" * `tmp/gen_path_list.found.sh` VERSION="1.1 2024-01-14" * `tmp/gen_winner_json.sh` VERSION="1.1 2024-01-14" The `.winner.json` file format from winner_JSON_format_version "0.5 2023-11-04" to "1.0 2024-01-14". All `.winner.json` files have been updated to use the new `.winner.json` file format. The manifest (`tmp/manifest.numbers` and `tmp/manifest.csv`) has updated to use the new `.winner.json` file format. The following procedure was used to generate the `tmp/manifest.csv` manifest file: 0. open [manifest.numbers](manifest.numbers) in numbers: modify if/as needed NOTE: Please try to keep the [manifest.numbers](manifest.numbers) spreadsheet in order order. Use Organize -> Sort -> ((Sort Now)) to sort. NOTE: Empty fields in the [manifest.numbers](manifest.numbers) spreadsheet are not allowed. By default, a cell that is empty is colored as a warning. 1. File -> Export To -> CSV... * 1a. `[ ]` Include table names (unset) * 1b. Text Encoding: Unicode (UTF-8) * 1c. Click ((Save..)) 2. Save As: manifest.csv * 2a. Select the `tmp` directory 3. Click ((Export)) * 3a. If needed click ((Replace)) 4. Execute ./tmp/[fix_csv.sh](fix_csv.sh) NOTE: You must run the [fix_csv.sh](fix_csv.sh) in the above step in order to make the CSV file usable by other tools. NOTE: If no changes are needed in the above step, the original file content will be restored. 5. Execute ./tmp/[gen_path_list.found.sh](gen_path_list.found.sh) NOTE: The above step silently performs a `make clobber` at the top level, and so it takes a bit of time. NOTE: If no changes are needed in the above step, the original file content will be restored. 6. Execute ./tmp/[check_path_list.sh](check_path_list.sh) NOTE: The above step prints nothing and for an exit 0, then all is well. If there is a problem, such as files found that are not part of the manifest, or files in the manifest that are not found, then an error message will be printed and the tool will exit non-zero. If errors are reported, go back to step 0 and correct the errors. 7. Execute ./tmp/[run_all.sh](run_all.sh) ./gen_winner_json.sh NOTE: The above step will take a few seconds to run. It will rebuild all `.winner.json` files for all winners. NOTE: If no changes are needed in the above step, the original file content will be restored. 8. Execute `git status` and inspect changes NOTE: Check in changes if and as needed. NOTE: In a number of shell scripts under `bin/*.sh`, and in the `bin/manifest.winner.json.awk` script you will find comments of the form: ``` ``` This is a note that when tools such as `jnamval` (and `jval` and `jfmt`) are ready, these gross hacks should be replaced with calls to those tools. Until they are ready, some must be careful with the layout of JSON files. A number of bugs and typos have been fixed. NOTE: Fixing typos in the files touched by this comment is welcome. This includes scripts, data files, text files, etc. The _inventory_order_, _display_as_, _display_via_github_. and _winners_text_ fields of the [manifest.numbers](manifest.numbers) spreadsheet were just set to have some initial values. As they impact the inventory displayed on `index.html` pages, their content needs to be improved. NOTE: Improving the above mentioned fields of the [manifest.numbers](manifest.numbers) spreadsheet is welcome.
This commit is contained in:
Landon Curt Noll committed
1 parent
f5a259d3b3
commit
c6e6201209
366 files changed
+29369
-31783
No files matched your search
Executable
+355
@@ -0,0 +1,355 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# subst.default.sh - print default substitutions
|
||||
#
|
||||
# This tool is used in conjunction with the inc/md2html.cfg configuration
|
||||
# file, and the tools that use that use that configuration file
|
||||
# such as "readme2index.sh".
|
||||
#
|
||||
# Copyright (c) 2024 by Landon Curt Noll. All Rights Reserved.
|
||||
#
|
||||
# Permission to use, copy, modify, and distribute this software and
|
||||
# its documentation for any purpose and without fee is hereby granted,
|
||||
# provided that the above copyright, this permission notice and text
|
||||
# this comment, and the disclaimer below appear in all of the following:
|
||||
#
|
||||
# supporting documentation
|
||||
# source copies
|
||||
# source works derived from this source
|
||||
# binaries derived from this source or from derived source
|
||||
#
|
||||
# LANDON CURT NOLL DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
|
||||
# INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO
|
||||
# EVENT SHALL LANDON CURT NOLL BE LIABLE FOR ANY SPECIAL, INDIRECT OR
|
||||
# CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF
|
||||
# USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
|
||||
# OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
|
||||
# PERFORMANCE OF THIS SOFTWARE.
|
||||
#
|
||||
# chongo (Landon Curt Noll, http://www.isthe.com/chongo/index.html) /\oo/\
|
||||
#
|
||||
# Share and enjoy! :-)
|
||||
|
||||
# firewall - must be bash with a version 4.2 or later
|
||||
#
|
||||
# We must declare arrays with -ag or -Ag
|
||||
#
|
||||
if [[ -z ${BASH_VERSINFO[0]} || ${BASH_VERSINFO[0]} -lt 4 || ${BASH_VERSINFO[0]} -eq 4 && ${BASH_VERSINFO[1]} -lt 2 ]]; then
|
||||
echo "$0: ERROR: bash version must be >= 4.2: $BASH_VERSION" 1>&2
|
||||
exit 4
|
||||
fi
|
||||
|
||||
# setup bash file matching
|
||||
#
|
||||
# Requires bash with a version 4.2 or later
|
||||
#
|
||||
shopt -s nullglob # enable expanded to nothing rather than remaining unexpanded
|
||||
shopt -u failglob # disable error message if no matches are found
|
||||
shopt -u dotglob # disable matching files starting with .
|
||||
shopt -s globskipdots # enable never matching . nor ..
|
||||
shopt -u nocaseglob # disable strict case matching
|
||||
shopt -u extglob # enable extended globbing patterns
|
||||
shopt -s globstar # enable ** to match all files and zero or more directories and subdirectories
|
||||
|
||||
# set variables referenced in the usage message
|
||||
#
|
||||
export VERSION="1.1 2024-01-14"
|
||||
NAME=$(basename "$0")
|
||||
export NAME
|
||||
export V_FLAG=0
|
||||
GIT_TOOL=$(type -P git)
|
||||
export GIT_TOOL
|
||||
if [[ -z "$GIT_TOOL" ]]; then
|
||||
echo "$0: FATAL: git tool is not installed or not in PATH" 1>&2
|
||||
exit 200
|
||||
fi
|
||||
"$GIT_TOOL" rev-parse --is-inside-work-tree >/dev/null 2>&1
|
||||
status="$?"
|
||||
if [[ $status -eq 0 ]]; then
|
||||
TOPDIR=$("$GIT_TOOL" rev-parse --show-toplevel)
|
||||
fi
|
||||
export REPO_URL="https://github.com/ioccc-src/temp-test-ioccc"
|
||||
export TOP_URL="https://ioccc-src.github.io/temp-test-ioccc"
|
||||
|
||||
# set usage message
|
||||
#
|
||||
export USAGE="usage: $0 [-h] [-v level] [-V] [-n] [-N]
|
||||
[-d topdir] [-p tool] [-P optstr]
|
||||
[-u repo_url] [-U top_url] [-e string ..] [-E exitcode]
|
||||
yyyy/dir
|
||||
|
||||
-h print help message and exit
|
||||
-v level set verbosity level (def level: 0)
|
||||
-V print version string and exit
|
||||
|
||||
-d topdir set topdir (def: $TOPDIR)
|
||||
|
||||
-n go thru the actions, but do not update any files (def: do the action)
|
||||
-N do not process file, just parse arguments and ignore the file (def: process the file)
|
||||
|
||||
-u repo_url Base level URL of target git repo (def: $REPO_URL)
|
||||
-U top_url Top level URL of web site where HTML files will be viewed (def: $TOP_URL)
|
||||
|
||||
-e string output string, followed by newline, to stderr (def: do not)
|
||||
-E exitcode force exit with exitcode (def: exit based on success or failure of the action)
|
||||
|
||||
yyyy/dir path from topdir to winner directory: must contain the files: README.md, .path and .winner.json
|
||||
|
||||
Exit codes:
|
||||
0 all OK
|
||||
1 pandoc exited non-zero
|
||||
2 -h and help string printed or -V and version string printed
|
||||
3 command line error
|
||||
4 bash version is < 4.2
|
||||
5 yyyy/dir is not a winner directory
|
||||
>= 10 < 200 ((not used))
|
||||
>= 200 internal error
|
||||
|
||||
$NAME version: $VERSION"
|
||||
|
||||
# setup
|
||||
#
|
||||
export NOOP=
|
||||
export DO_NOT_PROCESS=
|
||||
export DOCROOT_URL_SLASH="/"
|
||||
|
||||
# parse command line
|
||||
#
|
||||
while getopts :hv:Vd:nNu:U:e:E: flag; do
|
||||
case "$flag" in
|
||||
h) echo "$USAGE" 1>&2
|
||||
exit 2
|
||||
;;
|
||||
v) V_FLAG="$OPTARG"
|
||||
;;
|
||||
V) echo "$VERSION"
|
||||
exit 2
|
||||
;;
|
||||
d) TOPDIR="$OPTARG"
|
||||
;;
|
||||
n) NOOP="-n"
|
||||
;;
|
||||
N) DO_NOT_PROCESS="-N"
|
||||
;;
|
||||
u) REPO_URL="$OPTARG"
|
||||
;;
|
||||
U) TOP_URL="$OPTARG"
|
||||
;;
|
||||
e) echo "$OPTARG" 1>&2
|
||||
;;
|
||||
E) exit "$OPTARG"
|
||||
;;
|
||||
\?) echo "$0: ERROR: invalid option: -$OPTARG" 1>&2
|
||||
echo 1>&2
|
||||
print_usage 1>&2
|
||||
exit 3
|
||||
;;
|
||||
:) echo "$0: ERROR: option -$OPTARG requires an argument" 1>&2
|
||||
echo 1>&2
|
||||
echo "$USAGE" 1>&2
|
||||
exit 3
|
||||
;;
|
||||
*) echo "$0: ERROR: unexpected value from getopts: $flag" 1>&2
|
||||
echo 1>&2
|
||||
echo "$USAGE" 1>&2
|
||||
exit 3
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# parse the command line arguments
|
||||
#
|
||||
if [[ $V_FLAG -ge 1 ]]; then
|
||||
echo "$0: debug[1]: debug level: $V_FLAG" 1>&2
|
||||
fi
|
||||
#
|
||||
shift $(( OPTIND - 1 ));
|
||||
#
|
||||
if [[ $V_FLAG -ge 5 ]]; then
|
||||
echo "$0: debug[5]: file argument count: $#" 1>&2
|
||||
fi
|
||||
if [[ $# -ne 1 ]]; then
|
||||
echo "$0: ERROR: expected 1 arg, found: $#" 1>&2
|
||||
exit 3
|
||||
fi
|
||||
#
|
||||
export WINNER_PATH="$1"
|
||||
if [[ $V_FLAG -ge 1 ]]; then
|
||||
echo "$0: debug[1]: WINNER_PATH=$WINNER_PATH" 1>&2
|
||||
fi
|
||||
|
||||
# verify that we have a topdir directory
|
||||
#
|
||||
REPO_NAME=$(basename "$REPO_URL")
|
||||
export REPO_NAME
|
||||
if [[ -z $TOPDIR ]]; then
|
||||
echo "$0: ERROR: cannot find top of git repo directory" 1>&2
|
||||
echo "$0: Notice: if needed: $GIT_TOOL clone $REPO_URL; cd $REPO_NAME" 1>&2
|
||||
exit 201
|
||||
fi
|
||||
if [[ ! -d $TOPDIR ]]; then
|
||||
echo "$0: ERROR: TOPDIR is not a directory: $TOPDIR" 1>&2
|
||||
echo "$0: Notice: if needed: $GIT_TOOL clone $REPO_URL; cd $REPO_NAME" 1>&2
|
||||
exit 202
|
||||
fi
|
||||
|
||||
# verify that we have an author subdirectory
|
||||
#
|
||||
export AUTHOR_PATH="$TOPDIR/author"
|
||||
if [[ ! -d $AUTHOR_PATH ]]; then
|
||||
echo "$0: ERROR: author is not a directory under topdir: $AUTHOR_PATH" 1>&2
|
||||
exit 203
|
||||
fi
|
||||
export AUTHOR_DIR="author"
|
||||
|
||||
# verify that we have an inc subdirectory
|
||||
#
|
||||
export INC_PATH="$TOPDIR/inc"
|
||||
if [[ ! -d $INC_PATH ]]; then
|
||||
echo "$0: ERROR: inc is not a directory under topdir: $INC_PATH" 1>&2
|
||||
exit 204
|
||||
fi
|
||||
export INC_DIR="inc"
|
||||
|
||||
# verify that we have an bin subdirectory
|
||||
#
|
||||
export BIN_PATH="$TOPDIR/bin"
|
||||
if [[ ! -d $BIN_PATH ]]; then
|
||||
echo "$0: ERROR: bin is not a directory under topdir: $BIN_PATH" 1>&2
|
||||
exit 205
|
||||
fi
|
||||
export BIN_DIR="bin"
|
||||
|
||||
# cd to topdir
|
||||
#
|
||||
if [[ ! -e $TOPDIR ]]; then
|
||||
echo "$0: ERROR: cannot cd to non-existent path: $TOPDIR" 1>&2
|
||||
exit 206
|
||||
fi
|
||||
if [[ ! -d $TOPDIR ]]; then
|
||||
echo "$0: ERROR: cannot cd to a non-directory: $TOPDIR" 1>&2
|
||||
exit 207
|
||||
fi
|
||||
export CD_FAILED
|
||||
if [[ $V_FLAG -ge 5 ]]; then
|
||||
echo "$0: debug[5]: about to: cd $TOPDIR" 1>&2
|
||||
fi
|
||||
cd "$TOPDIR" || CD_FAILED="true"
|
||||
if [[ -n $CD_FAILED ]]; then
|
||||
echo "$0: ERROR: cd $TOPDIR failed" 1>&2
|
||||
exit 208
|
||||
fi
|
||||
if [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: now in directory: $(/bin/pwd)" 1>&2
|
||||
fi
|
||||
|
||||
# verify that WINNER_PATH is a winner directory
|
||||
#
|
||||
# WINNER_PATH must be in yyyy/dir form
|
||||
# yyyy must be a directory
|
||||
# yyyy must be a writable directory
|
||||
# yyyy/.year must be a non-empty file
|
||||
# yyyy/dir must be a directory
|
||||
# yyyy/dir/.path must be a non-empty file
|
||||
# WINNER_PATH must match the contents of yyyy/dir/.path
|
||||
# yyyy/dir/.winner.json must be a non-empty file
|
||||
#
|
||||
if [[ ! -d $WINNER_PATH ]]; then
|
||||
echo "$0: ERROR: arg is not a directory: $WINNER_PATH" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ ! -w $WINNER_PATH ]]; then
|
||||
echo "$0: ERROR: arg is not a writable directory: $WINNER_PATH" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
export YEAR_DIR=${WINNER_PATH%%/*}
|
||||
if [[ -z $YEAR_DIR ]]; then
|
||||
echo "$0: ERROR: arg not in yyyy/dir form: $WINNER_PATH" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
export WINNER_DIR=${WINNER_PATH#*/}
|
||||
if [[ -z $WINNER_DIR ]]; then
|
||||
echo "$0: ERROR: arg: $WINNER_PATH not in $YEAR_DIR/dir form: $WINNER_PATH" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ $WINNER_DIR = */* ]]; then
|
||||
echo "$0: ERROR: dir from arg: $WINNER_PATH contains a /: $WINNER_DIR" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ ! -d $YEAR_DIR ]]; then
|
||||
echo "$0: ERROR: yyyy from arg: $WINNER_PATH is not a directory: $YEAR_DIR" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
export WINNER_ID="${YEAR_DIR}_${WINNER_DIR}"
|
||||
export DOT_YEAR="$YEAR_DIR/.year"
|
||||
if [[ ! -s $DOT_YEAR ]]; then
|
||||
echo "$0: ERROR: not a non-empty file: $DOT_YEAR" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ ! -d $YEAR_DIR/$WINNER_DIR ]]; then
|
||||
echo "$0: ERROR: yyyy/dir from arg: $WINNER_PATH is not a directory: $YEAR_DIR/$WINNER_DIR" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
export DOT_PATH="$YEAR_DIR/$WINNER_DIR/.path"
|
||||
if [[ ! -s $DOT_PATH ]]; then
|
||||
echo "$0: ERROR: not a non-empty file: $DOT_PATH" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
DOT_PATH_CONTENT=$(< "$DOT_PATH")
|
||||
if [[ $WINNER_PATH != "$DOT_PATH_CONTENT" ]]; then
|
||||
echo "$0: ERROR: arg: $WINNER_PATH does not match $DOT_PATH contents: $DOT_PATH_CONTENT" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
export WINNER_JSON="$YEAR_DIR/$WINNER_DIR/.winner.json"
|
||||
if [[ ! -e $WINNER_JSON ]]; then
|
||||
echo "$0: ERROR: .winner.json does not exist: $WINNER_JSON" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ ! -f $WINNER_JSON ]]; then
|
||||
echo "$0: ERROR: .winner.json is not a file: $WINNER_JSON" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
if [[ ! -r $WINNER_JSON ]]; then
|
||||
echo "$0: ERROR: .winner.json is not a readable file: $WINNER_JSON" 1>&2
|
||||
exit 5
|
||||
fi
|
||||
|
||||
# parameter debugging
|
||||
#
|
||||
if [[ $V_FLAG -ge 3 ]]; then
|
||||
echo "$0: debug[3]: NAME=$NAME" 1>&2
|
||||
echo "$0: debug[3]: YEAR_DIR=$YEAR_DIR" 1>&2
|
||||
echo "$0: debug[3]: WINNER_DIR=$WINNER_DIR" 1>&2
|
||||
echo "$0: debug[3]: WINNER_ID=$WINNER_ID" 1>&2
|
||||
echo "$0: debug[3]: DOT_YEAR=$DOT_YEAR" 1>&2
|
||||
echo "$0: debug[3]: DOT_PATH=$DOT_PATH" 1>&2
|
||||
echo "$0: debug[3]: WINNER_JSON=$WINNER_JSON" 1>&2
|
||||
echo "$0: debug[3]: REPO_URL=$REPO_URL" 1>&2
|
||||
echo "$0: debug[3]: REPO_NAME=$REPO_NAME" 1>&2
|
||||
echo "$0: debug[3]: TOP_URL=$TOP_URL" 1>&2
|
||||
echo "$0: debug[3]: AUTHOR_PATH=$AUTHOR_PATH" 1>&2
|
||||
echo "$0: debug[3]: AUTHOR_DIR=$AUTHOR_DIR" 1>&2
|
||||
echo "$0: debug[3]: INC_PATH=$INC_PATH" 1>&2
|
||||
echo "$0: debug[3]: INC_DIR=$INC_DIR" 1>&2
|
||||
echo "$0: debug[3]: BIN_PATH=$BIN_PATH" 1>&2
|
||||
echo "$0: debug[3]: BIN_DIR=$BIN_DIR" 1>&2
|
||||
echo "$0: debug[3]: DO_NOT_PROCESS=$DO_NOT_PROCESS" 1>&2
|
||||
echo "$0: debug[3]: NOOP=$NOOP" 1>&2
|
||||
fi
|
||||
|
||||
# If -N, time to exit
|
||||
#
|
||||
if [[ -n $DO_NOT_PROCESS ]]; then
|
||||
if [[ $V_FLAG -ge 1 ]]; then
|
||||
echo "$0: debug[1]: arguments parsed, -N given, exit 0" 1>&2
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# output HEADER_1 substitution
|
||||
#
|
||||
echo "-s HEADER_1='The International Obfuscated C Code Contest'"
|
||||
|
||||
# All Done!!! -- Jessica Noll, Age 2
|
||||
#
|
||||
exit 0
|
||||
Reference in new issue
Block a user