Files
ioccc/bin/manifest.entry.json.awk
T
Landon Curt Noll aedb017223 improve markdown to HTML tools, build location.html, authors.html, years.html
Renamed `.winner.json` to `.entry.json` as per recent decisions made for issue #2156.

JSON element _winner_JSON_format_version_ renamed to _entry_JSON_format_version_
in `.winner.json`.

Changed _entry_JSON_format_version_ from "1.0 2024-01-14" to "1.1 2024-02-11".

JSON element _winner_id_ renamed to _entry_id_ in `.entry.json`
JSON element _winner_text renamed to _entry_text_ in `.entry.json`
JSON element _winner_id_ renamed to _entry_id_ in `author/author_handle.json` files.

Changed _author_JSON_format_version_ from "1.0 2023-06-10" to "1.1 2024-02-11".

Renamed `tmp/gen_winner_json.sh` to `tmp/gen_entry_json.sh`.

Renamed `tmp/example.dot_winner.json` to `tmp/example.dot_entry.json`.

Renamed `tmp/winner_id.txt` to `tmp/entry_id.txt`.

Renamed `tmp/winner_dir.txt` to `tmp/entry_dir.txt`.

Moved outdated `tmp/awards_check.sh` into `tmp/OLD_manifest/awards_check.sh`.
Moved outdated `tmp/awards_check.sh` into `tmp/OLD_manifest/awards_check.sh`.
Moved outdated `tmp/src/winner_name` into `tmp/OLD_manifest/winner_name`.
Moved outdated `tmp/sql` into `tmp/OLD_manifest/sql`.
Moved outdated `tmp/template-footer.html` into `tmp/OLD_manifest/template-footer.html`.
Moved outdated `tmp/template-header.html` into `tmp/OLD_manifest/template-header.html`.
Moved outdated `tmp/template-top.html` into `tmp/OLD_manifest/template-top.html`.
Moved outdated `tmp/test.html` into `tmp/OLD_manifest/test.html`.
Moved outdated `tmp/fake.gen_index.sh` into `tmp/OLD_manifest/fake.gen_index.sh`.
Moved outdated `tmp/old-ioccc.css` into `tmp/OLD_manifest/old-ioccc.css`.

Removed winner_name from `tmp/src/Makefile`.

Updated `tmp/example.dot_entry.json` to reflect the above JSON changes.

Updated `tmp/manifest.numbers` to reflect the above JSON changes.

Updated `tmp/README.md` to reflect the above JSON changes.
Sorted `tmp/README.md` by existing `tmp` filenames.

Updated all `.entry.json` files to reflect the above JSON changes.

Updated all `author/author_handle.json` files to reflect the above JSON changes.

Added new tool `bin/gen-location.sh` to form the top level file `location.html`
from `author/author_handle.json` files!

Added new tool `bin/gen-years.sh` to form the top level file `years.html`
using `.top`, `.year` and `.entry.json' files!

Added new tool `bin/gen-authors.sh` to form a new top level file
`/authors.html` from `author/author_handle.json` and related
`.entry.json' files!

The old `winners.html` file now redirects to the new `authors.html` file.

Renamed `bin/gen-author.sh` to `bin/output-index-author.sh`.
Renamed `bin/subst.winner-index.sh` to `subst.entry-index.sh`.
Renamed `bin/subst.winner-index.awk` to `subst.entry-index.awk`.
Renamed `bin/manifest.entry.json.awk` to `bin/manifest.entry.json.awk`.
Renamed `bin/gen-inventory.sh` to `bin/output-index-inventory.sh`.
Renamed `bin/sort.gitignore` to `bin/sort.gitignore.sh`.
Renamed `bin/sgi` to `bin/sgi.sh`.

The core logic of converting markdown to HTML has been
generalized into a new `bin/md2html.sh` tool.

The `bin/readme2index.sh` tool now calls the
new `bin/md2html.sh` tool to form entry's `index.html` files.

The `bin/quick-readme2index.sh` tool now calls the new `bin/md2html.sh`
tool, unless the `index.html` is newer than both `.entry.json` and
`README.md`, to form entry `index.html` files.

Increased, slightly, the speed of the `bin/quick-readme2index.sh` tool
with the entry's `index.html` file does not need to be updated.

The `bin/quick-readme2index.sh` tool will touch the `index.html` file,
even if the `bin/md2html.sh` tool determines that the `index.html` file
did not change, so that the `index.html` file will not appear
out of date with respect to `.entry.json`.

Slightly improved the look and feel of `index.html` files for entries.

The logic of `bin/all-run.sh` has been improved.  The initial
processing, from first bash version check, thru the -N test, serves
as the model for the `bin/readme2index.sh` tool and the
`bin/quick-readme2index.sh` tool.  There tools now share
a common set of command line options.

Changed the format of `inc/md2html.cfg` so that for a given "_file_glob_"
match, options follow, one per line.  The "_file_glob_" is on a line
by itself.  The option lines that follow must be intended:

```
[12][0-9][0-9][0-9]/README.md
	-H
	topnav=year
	-o
	bin/subst.default.sh
	-o
	bin/subst.year.sh
```

Added "-D ./" to top level markdown files in `inc/md2html.cfg`.

The above intending is performed with 1 or more whitespace
characters, although we use a leading tab "convention"
out of deference to Makefile formats. :-)

Once a new "_file_glob_" is found, the set of any previous
option lines is said to have ended.

The general argument processing logic now makes used of shell arrays.

Added checks in the `bin/md2html.sh` tool to disallow a "_file_glob_"
of . (dot) as this is used internally to help the shell process the
new `inc/md2html.cfg` format.

Changed `bin/all-run.sh` VERSION from "1.1.2 2024-01-20"
to "1.3 2024-02-11".

Changed `bin/output-index-author.sh` VERSION from "1.3.1 2024-01-16"
to "1.4 2024-02-11".

Changed `bin/pandoc-wrapper.sh` VERSION from "1.3.2 2024-01-19"
to "1.4 2024-02-05".

Changed `bin/quick-readme2index.sh` VERSION from "1.0 2024-01-20"
to "1.1 2024-02-11".

Changed `bin/readme2index.sh` VERSION from "1.1.1 2024-01-20"
to "1.2 2024-02-11".

Changed `bin/subst.default.sh` VERSION from "1.1.1 2024-01-16"
to "1.2 2024-02-05".

Changed `bin/subst.entry-index.sh` VERSION from "1.1.1 2024-01-16
to "1.2 2024-02-11".

Changed `bin/output-index-inventory.sh` VERSION from "1.0.1 2024-01-16"
to "1.1 2024-02-11".

Changed `bin/manifest.entry.json.awk` from "1.0 2024-01-14"
to "1.1 2024-02-11".

Changed `bin/sort.gitignore.sh` from "1.0 2024-02-01"
to "1.1 2024-02-11".

Changed substitution symbol _%%DOCROOT_URL_SLASH%%_ to _%%DOCROOT_SLASH%%_
in various "inc/*.html" files.

Moved setting of symbol _%%DOCROOT_SLASH%%_ from `bin/subst.entry-index.sh`
to `bin/subst.default.sh`.

Changed the variable _$DOCROOT_URL_SLASH_ to _$DOCROOT_SLASH_, in
bin tools that use it, to be more descriptive.

Removed `-P pandoc_opts` from `bin/*.sh` command lines.
The PANDOC_WRAPPER_OPTSTR value is only used in `bin/pandoc-wrapper.sh`.

Removed `-B optstr` and `-A optstr` from the `bin/md2html.sh` command line.

Removed `[-O tool=optstr ..]` from the `bin/md2html.sh` command line.

Removed old use of `-U top_url` from command lines.
Add `-w site_url` as sort of a "replacement" for the old `-U top_url`.

Added a new `-U URL` to `bin/gen-author.sh`, `bin/output-index-inventory.sh`,
`bin/md2html.sh`, `bin/subst.default.sh`, and `bin/subst.entry-index.sh`
to allow for the Nu HTML check can use the encoded absolute URL of
the HTML page.

Added `-t tagline` to most command lines.

Added `-T md2html.sh` to most command lines.

Added `-D docroot/` to most command lines to set DOCROOT_SLASH.
The DOCROOT_SLASH value defaults to "../../" for most in tools,
suitable for HTML files under a `YYYY/dir` directory, except for
`bin/gen-location.sh` and `gen-entries-html.sh` (as this produces a
top level HTML file) and `bin/md2html.sh` (forms files at any level)
who use "./" instead.

In the case of `bin/output-index-inventory.sh`, `bin/quick-readme2index.sh`,
`bin/readme2index.sh`, and `bin/all-run.sh`

Helped make the use of exit code in bin tools a bit more consistent.

Fixed a number of typos in the bin tools.

Added comments for lower level tools such as
`manifest.entry.json.awk`, `bin/subst.default.sh`,
`bin/subst.entry-index.sh`, and bin/`manifest.entry.json.awk` to
indicate how they are usually invoked by other tools.

Changed the comments that a new `bin/md2html.sh` tool (that was the
`bin/readme2index.sh` tool) would add to a entry's `index.html` file.
No longer will it add the tool version number to the comment.  Thus,
a change in tool version will not force all entry `index.html`
files to be modified.  Now both the tool adds two comments, one
is the _tagline_ to indicate how the markdown content was generated,
and one for the new `bin/md2html.sh` tool.  For example:

```
<!-- The original markdown content was formed by the tool: bin/quick-readme2index.sh -->
<!-- The HTML was converted from markdown by the tool: bin/md2html.sh -->
```

Improve the debugging, especially in regards to changing important
exported shell variables, in `bin/md2html.sh`.  Other bin tools
have their exported shell variables improved somewhat as well.

The `bin/README.md` now describes the general _converting markdown
to HTML_ procedure including _getopt phases_ and _HTML phases_.
Content from `inc/md2html.cfg` and `bin/readme2index.sh` that related
to this topic has been centralized into the `bin/README.md` file.

Improved formatting of the `bin/README.md` file.  Updated it as needed
according to the above.

Sorted `2011/.year`, `2018/.year`, and `2019/.year` to be in dictionary order.
Adjusted ${WINNERS} in year Makefile to use dictionary order so
that `make genpath` will produce `.year` files in dictionary order.

FYI: We tested this massive set of changes as best as we could.
     Nevertheless we might have missing something in the JSON changes,
     `.entry.json` renames, manifest updates, code comments, etc.
2024-02-11 16:00:31 -08:00

433 lines
13 KiB
Awk
Executable File

#!/usr/bin/env awk
#
# manifest.entry.json.awk - output manifest table from a entry's .entry.json file
#
# usage:
#
# awk -v github=REPO_URL -f bin/manifest.entry.json.awk YYYY/dir/.entry.json
#
# where:
#
# YYYY IOCCC year or mock
# dir name of entry's directory under YYYY containing .entry.json
# REPO_URL GitHub repo URL, e.g., https://github.com/ioccc-src/winner
# XXX - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - XXX
# XXX - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - XXX
# XXX - until we have the jnamval command, we must FAKE PARSE the .entry.json file - XXX
# XXX - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - XXX
# XXX - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - XXX
BEGIN {
# setup
#
VERSION="1.1 2024-02-11"
found_manifest_array = 0; # 1 ==> we found the manifest ARRAY
within_manifest_array = 0; # 1 ==> found start of "manifest" : [ JSON array
begin_manifest_element = 0; # 1 ==> we have found the start of the manifest array element
# error if github is not set via -v github=REPO_URL
#
if (length(github) == 0) {
exit 210; # END section will output ERROR message about github not set
}
# extract YYYY and dir from file argument
#
YYYY = "";
dir = "";
tmp = ARGV[1];
gsub(/\.entry\.json$/, "", tmp);
arg_path_count = split(tmp, part, "/");
if (arg_path_count >= 2) {
dir=part[arg_path_count-1];
YYYY=part[arg_path_count-2];
}
if (length(dir) == 0) {
exit 211; # END section will output ERROR message about cannot find dir
}
if (length(YYYY) == 0) {
exit 212; # END section will output ERROR message about cannot find YYYY
}
}
# look for the start of "manifest" JSON array
#
# We look for a line with 3 tokens:
#
# "manifest" : [
#
NF == 3 && $1 ~ /^"manifest"$/ && $2 ~ /^:$/ && $3 ~ /^\[$/ {
# note we have found the start of "manifest" JSON array
#
found_manifest_array = 1;
within_manifest_array = 1;
begin_manifest_element = 0;
}
# after start of "manifest" JSON array, process start of a manifest array element
#
# We have just seen the start of "manifest" JSON array:
#
# "manifest" : [
#
# Now, when we see a line of this form:
#
# {
#
# we know that we are about to see the start of one of the manifest array elements.
#
within_manifest_array == 1 && NF == 1 && $1 ~ /^{$/ {
# note the start of the manifest array element
#
begin_manifest_element = 1;
# clear values for the JSON members of this manifest array elements
#
file_path = "";
inventory_order = "";
OK_to_edit = "";
display_as = "";
display_via_github = "";
entry_text = "";
# clear unquoted values for the JSON members of this manifest array elements
#
unquote_file_path = "";
unquote_inventory_order = "";
unquote_OK_to_edit = "";
unquote_display_as = "";
unquote_display_via_github = "";
unquote_entry_text = "";
}
# record file_path from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "file_path" : something,
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"file_path"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
file_path = $0;
# Remove any trailing ,
#
if (file_path ~ /,$/) {
file_path = substr(file_path, 1, length(file_path)-1);
}
# remove any enclosing double quotes
#
if (file_path ~ /^".*"$/) {
unquote_file_path = substr(file_path, 2, length(file_path)-2);
} else {
unquote_file_path = file_path;
}
}
# record inventory_order from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "inventory_order" : something,
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"inventory_order"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
inventory_order = $0;
# Remove any trailing ,
#
if (inventory_order ~ /,$/) {
inventory_order = substr(inventory_order, 1, length(inventory_order)-1);
}
# remove any enclosing double quotes
#
if (inventory_order ~ /^".*"$/) {
unquote_inventory_order = substr(inventory_order, 2, length(inventory_order)-2);
} else {
unquote_inventory_order = inventory_order;
}
}
# record OK_to_edit from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "OK_to_edit" : something,
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"OK_to_edit"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
OK_to_edit = $0;
# Remove any trailing ,
#
if (OK_to_edit ~ /,$/) {
OK_to_edit = substr(OK_to_edit, 1, length(OK_to_edit)-1);
}
# remove any enclosing double quotes
#
if (OK_to_edit ~ /^".*"$/) {
unquote_OK_to_edit = substr(OK_to_edit, 2, length(OK_to_edit)-2);
} else {
unquote_OK_to_edit = OK_to_edit;
}
}
# record display_as from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "display_as" : something,
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"display_as"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
display_as = $0;
# Remove any trailing ,
#
if (display_as ~ /,$/) {
display_as = substr(display_as, 1, length(display_as)-1);
}
# remove any enclosing double quotes
#
if (display_as ~ /^".*"$/) {
unquote_display_as = substr(display_as, 2, length(display_as)-2);
} else {
unquote_display_as = display_as;
}
}
# record display_via_github from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "display_via_github" : something,
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"display_via_github"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
display_via_github = $0;
# Remove any trailing ,
#
if (display_via_github ~ /,$/) {
display_via_github = substr(display_via_github, 1, length(display_via_github)-1);
}
# remove any enclosing double quotes
#
if (display_via_github ~ /^".*"$/) {
unquote_display_via_github = substr(display_via_github, 2, length(display_via_github)-2);
} else {
unquote_display_via_github = display_via_github;
}
}
# record entry_text from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "entry_text" : something
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"entry_text"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
entry_text = $0;
# Remove any trailing ,
#
if (entry_text ~ /,$/) {
entry_text = substr(entry_text, 1, length(entry_text)-1);
}
# remove any enclosing double quotes
#
if (entry_text ~ /^".*"$/) {
unquote_entry_text = substr(entry_text, 2, length(entry_text)-2);
} else {
unquote_entry_text = entry_text;
}
}
# process end of manifest array element
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF == 1 && $1 ~ /^},?$/ {
# verify we have seen a file_path JSON member
#
if (length(file_path) == 0) {
print "ERROR: manifest array JSON member did not have a file_path JSON member in:", ARGV[1] > "/dev/stderr";
exit 213;
}
# verify we have seen a inventory_order JSON member
#
if (length(inventory_order) == 0) {
print "ERROR: manifest array JSON member did not have a inventory_order JSON member in:", ARGV[1] > "/dev/stderr";
exit 214;
}
# verify we have seen a OK_to_edit JSON member
#
if (length(OK_to_edit) == 0) {
print "ERROR: manifest array JSON member did not have a OK_to_edit JSON member in:", ARGV[1] > "/dev/stderr";
exit 215;
}
# verify we have seen a display_as JSON member
#
if (length(display_as) == 0) {
print "ERROR: manifest array JSON member did not have a display_as JSON member in:", ARGV[1] > "/dev/stderr";
exit 216;
}
# verify we have seen a display_via_github JSON member
#
if (length(display_via_github) == 0) {
print "ERROR: manifest array JSON member did not have a display_via_github JSON member in:", ARGV[1] > "/dev/stderr";
exit 217;
}
# verify we have seen a entry_text JSON member
#
if (length(entry_text) == 0) {
print "ERROR: manifest array JSON member did not have a entry_text JSON member in:", ARGV[1] > "/dev/stderr";
exit 218;
}
# print manifest information
#
# print "# debug: file_path:", file_path;
# print "# debug: inventory_order:", inventory_order;
# print "# debug: OK_to_edit:", OK_to_edit;
# print "# debug: display_as:", display_as;
# print "# debug: display_via_github:", display_via_github;
# print "# debug: entry_text:", entry_text;
#
# print "# debug: unquote_file_path:", unquote_file_path;
# print "# debug: unquote_inventory_order:", unquote_inventory_order;
# print "# debug: unquote_OK_to_edit:", unquote_OK_to_edit;
# print "# debug: unquote_display_as:", unquote_display_as;
# print "# debug: unquote_display_via_github:", unquote_display_via_github;
# print "# debug: unquote_entry_text:", unquote_entry_text;
#
if (unquote_display_via_github == "false") {
url = unquote_file_path;
} else {
url = github "/blob/master/" YYYY "/" dir "/" unquote_file_path;
}
print inventory_order, "* [" unquote_file_path "](" url ") -", unquote_entry_text;
# note the end of the manifest array element
#
begin_manifest_element = 0;
# note if we found the end of the manifest array
#
if ($1 ~ /^}$/) {
within_manifest_array = 0;
}
}
END {
# case: we quit due to github unset
#
if (length(github) == 0) {
print "ERROR: github variable not set, call with -v github=REPO_URL" > "/dev/stderr";
exit 219; # use 200 to match exit code from BEGIN
}
# case: we quit because we could not determine dir from ARGV[1]
#
if (length(dir) == 0) {
print "ERROR: cannot determine dir from:", ARGV[1] > "/dev/stderr";
exit 220; # END section will output ERROR message about cannot find dir
}
if (length(YYYY) == 0) {
print "ERROR: cannot determine YYYY from:", ARGV[1] > "/dev/stderr";
exit 221; # END section will output ERROR message about cannot find YYYY
}
# case: no manifest array was found
#
if (!found_manifest_array) {
print "ERROR: we did not find a manifest JSON array in:", ARGV[1] > "/dev/stderr";
exit 222;
}
# case: manifest array was found, but manifest array element did not finish
#
if (begin_manifest_element) {
print "ERROR: manifest JSON array element did not end in:", ARGV[1] > "/dev/stderr";
exit 223;
}
# case: manifest array did not end
#
if (within_manifest_array) {
print "ERROR: manifest JSON array did not end in:", ARGV[1] > "/dev/stderr";
exit 224;
}
}