mirror/ioccc
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b06023f3302dd67c65cf5577c7bb4d7a7e39fde4
ioccc/bin/manifest.winner.json.awk
T
Landon Curt Noll c6e6201209 release readme2index.sh version 1.0 2024-01-14 
The initial release of `bin/readme2index.sh` tool that converts
`READMD.md` and various JSON files such as `.winner.json`
as well as "author/author_handle.json" files.

Applied the `readme2index.sh` tool to generate `2020/ferguson1/index.html`.

We generated the  `2020/ferguson1/index.html` by executing:

```sh
./bin/readme2index.sh -v 3 2020/ferguson1
```

NOTE: This above tool requires **pandoc to be installed**.
See the [pandoc web site](https://pandoc.org) for how
to install.  We used _HomeBrew_ to install it,
but there is also a _macports_ process as well.

NOTE: While the above web page does have formatting issues, it is
a first attempt data driven programmatic web page generation.

Some adjustments can be made by mostly improving `ioccc.css`,
and some perhaps by modifying how `bin/pandoc_wrapper.sh`
invokes pandoc.  But probably most of the improvements can
be made by fixing and improving `ioccc.css`.

TODO: Fix and improve the `2020/ferguson1/index.html`
web page by fixing and improving `ioccc.css`.

Moved scripts and tools needed to generate HTML pages
into a new `/bin` subdirectory.

TODO: Consider getting rid of the [build-ioccc
repo](https://github.com/ioccc-src/build-ioccc) and just use the
`/bin` subdirectory.

NOTE: Some links in the `2020/ferguson1/index.html`
web page do not (yet) exist.

TODO: Once the basic format of the `2020/ferguson1/index.html` web
page is somewhat reasonable, generate `index.html` pages for other
winners.  Generate the year level `index.html' files.
Generate the top level HTML files, including the missing
`location.html` file.

Greatly simplified the `.winner.json` file format,
removing a number of fields that were not strictly needed
for generating an inventory on a `index.html` page.

Updated various versions:

* `bin/gen-author.sh` VERSION="1.3 2024-01-14"
* `bin/gen-inventory.sh` VERSION="1.0 2024-01-14"
* `bin/manifest.winner.json.awk` VERSION="1.0 2024-01-14"
* `bin/pandoc_wrapper.sh` VERSION="1.3 2024-01-14"
* `bin/readme2index.sh` VERSION="1.0 2024-01-14"
* `bin/subst.default.sh` VERSION="1.1 2024-01-14"
* `bin/subst.winner-index.sh` VERSION="1.1 2024-01-14"
* `tmp/gen_path_list.found.sh` VERSION="1.1 2024-01-14"
* `tmp/gen_winner_json.sh` VERSION="1.1 2024-01-14"

The `.winner.json` file format from winner_JSON_format_version
"0.5 2023-11-04" to "1.0 2024-01-14".

All `.winner.json` files have been updated to use the
new `.winner.json` file format.

The manifest (`tmp/manifest.numbers` and `tmp/manifest.csv`) has
updated to use the new `.winner.json` file format.

The following procedure was used to generate the
`tmp/manifest.csv` manifest file:

0. open [manifest.numbers](manifest.numbers) in numbers: modify if/as needed

NOTE: Please try to keep the [manifest.numbers](manifest.numbers) spreadsheet
      in order order.  Use Organize -> Sort -> ((Sort Now)) to sort.

NOTE: Empty fields in the [manifest.numbers](manifest.numbers) spreadsheet
      are not allowed.  By default, a cell that is empty is colored
      as a warning.

1. File -> Export To -> CSV...
    * 1a. `[ ]` Include table names (unset)
    * 1b. Text Encoding: Unicode (UTF-8)
    * 1c. Click ((Save..))

2. Save As: manifest.csv
    * 2a. Select the `tmp` directory

3. Click ((Export))
    * 3a. If needed click ((Replace))

4. Execute ./tmp/[fix_csv.sh](fix_csv.sh)

NOTE: You must run the [fix_csv.sh](fix_csv.sh) in the above
      step in order to make the CSV file usable by other tools.

NOTE: If no changes are needed in the above step, the
      original file content will be restored.

5. Execute ./tmp/[gen_path_list.found.sh](gen_path_list.found.sh)

NOTE: The above step silently performs a `make clobber` at the
      top level, and so it takes a bit of time.

NOTE: If no changes are needed in the above step, the
      original file content will be restored.

6. Execute ./tmp/[check_path_list.sh](check_path_list.sh)

NOTE: The above step prints nothing and for an exit 0, then
      all is well.  If there is a problem, such as files
      found that are not part of the manifest, or files
      in the manifest that are not found, then an error
      message will be printed and the tool will exit non-zero.
      If errors are reported, go back to step 0 and
      correct the errors.

7. Execute ./tmp/[run_all.sh](run_all.sh) ./gen_winner_json.sh

NOTE: The above step will take a few seconds to run.  It will rebuild
      all `.winner.json` files for all winners.

NOTE: If no changes are needed in the above step, the
      original file content will be restored.

8. Execute `git status` and inspect changes

NOTE: Check in changes if and as needed.

NOTE: In a number of shell scripts under `bin/*.sh`,
      and in the `bin/manifest.winner.json.awk` script
      you will find comments of the form:

```
```

This is a note that when tools such as `jnamval` (and `jval` and `jfmt`) are
ready, these gross hacks should be replaced with calls to those tools.
Until they are ready, some must be careful with the layout of JSON files.

A number of bugs and typos have been fixed.

NOTE: Fixing typos in the files touched by this comment is welcome.
      This includes scripts, data files, text files, etc.

The _inventory_order_, _display_as_, _display_via_github_.  and
_winners_text_ fields of the [manifest.numbers](manifest.numbers)
spreadsheet were just set to have some initial values.
As they impact the inventory displayed on `index.html` pages,
their content needs to be improved.

NOTE: Improving the above mentioned fields of the
      [manifest.numbers](manifest.numbers) spreadsheet is welcome.
2024-01-14 22:03:28 -08:00

433 lines
13 KiB
Awk
Executable File
Raw Blame History

#!/usr/bin/env awk
#
# manifest.winner.json.awk - output manifest table from a winner .winner.json file
#
# usage:
#
# awk -v github=REPO_URL -f bin/manifest.winner.json.awk YYYY/dir/.winner.json
#
# where:
#
# YYYY IOCCC year or mock
# dir name of winner directory under YYYY containing .winner.json
# REPO_URL GitHub repo URL, e.g., https://github.com/ioccc-src/winner
# XXX - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - XXX
# XXX - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - XXX
# XXX - until we have the jnamval command, we must FAKE PARSE the .winner.json file - XXX
# XXX - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - GROSS HACK - XXX
# XXX - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - XXX
BEGIN {
# setup
#
VERSION="1.0 2024-01-14"
found_manifest_array = 0; # 1 ==> we found the manifest ARRAY
within_manifest_array = 0; # 1 ==> found start of "manifest" : [ JSON array
begin_manifest_element = 0; # 1 ==> we have found the start of the manifest array element
# error if github is not set via -v github=REPO_URL
#
if (length(github) == 0) {
exit 200; # END section will output ERROR message about github not set
}
# extract YYYY and dir from file argument
#
yyyy = "";
dir = "";
tmp = ARGV[1];
gsub(/\.winner\.json$/, "", tmp);
arg_path_count = split(tmp, part, "/");
if (arg_path_count >= 2) {
dir=part[arg_path_count-1];
yyyy=part[arg_path_count-2];
}
if (length(dir) == 0) {
exit 201; # END section will output ERROR message about cannot find dir
}
if (length(yyyy) == 0) {
exit 202; # END section will output ERROR message about cannot find yyyy
}
}
# look for the start of "manifest" JSON array
#
# We look for a line with 3 tokens:
#
# "manifest" : [
#
NF == 3 && $1 ~ /^"manifest"$/ && $2 ~ /^:$/ && $3 ~ /^\[$/ {
# note we have found the start of "manifest" JSON array
#
found_manifest_array = 1;
within_manifest_array = 1;
begin_manifest_element = 0;
}
# after start of "manifest" JSON array, process start of a manifest array element
#
# We have just seen the start of "manifest" JSON array:
#
# "manifest" : [
#
# Now, when we see a line of this form:
#
# {
#
# we know that we are about to see the start of one of the manifest array elements.
#
within_manifest_array == 1 && NF == 1 && $1 ~ /^{$/ {
# note the start of the manifest array element
#
begin_manifest_element = 1;
# clear values for the JSON members of this manifest array elements
#
file_path = "";
inventory_order = "";
OK_to_edit = "";
display_as = "";
display_via_github = "";
winners_text = "";
# clear unquoted values for the JSON members of this manifest array elements
#
unquote_file_path = "";
unquote_inventory_order = "";
unquote_OK_to_edit = "";
unquote_display_as = "";
unquote_display_via_github = "";
unquote_winners_text = "";
}
# record file_path from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "file_path" : something,
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"file_path"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
file_path = $0;
# Remove any trailing ,
#
if (file_path ~ /,$/) {
file_path = substr(file_path, 1, length(file_path)-1);
}
# remove any enclosing double quotes
#
if (file_path ~ /^".*"$/) {
unquote_file_path = substr(file_path, 2, length(file_path)-2);
} else {
unquote_file_path = file_path;
}
}
# record inventory_order from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "inventory_order" : something,
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"inventory_order"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
inventory_order = $0;
# Remove any trailing ,
#
if (inventory_order ~ /,$/) {
inventory_order = substr(inventory_order, 1, length(inventory_order)-1);
}
# remove any enclosing double quotes
#
if (inventory_order ~ /^".*"$/) {
unquote_inventory_order = substr(inventory_order, 2, length(inventory_order)-2);
} else {
unquote_inventory_order = inventory_order;
}
}
# record OK_to_edit from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "OK_to_edit" : something,
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"OK_to_edit"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
OK_to_edit = $0;
# Remove any trailing ,
#
if (OK_to_edit ~ /,$/) {
OK_to_edit = substr(OK_to_edit, 1, length(OK_to_edit)-1);
}
# remove any enclosing double quotes
#
if (OK_to_edit ~ /^".*"$/) {
unquote_OK_to_edit = substr(OK_to_edit, 2, length(OK_to_edit)-2);
} else {
unquote_OK_to_edit = OK_to_edit;
}
}
# record display_as from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "display_as" : something,
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"display_as"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
display_as = $0;
# Remove any trailing ,
#
if (display_as ~ /,$/) {
display_as = substr(display_as, 1, length(display_as)-1);
}
# remove any enclosing double quotes
#
if (display_as ~ /^".*"$/) {
unquote_display_as = substr(display_as, 2, length(display_as)-2);
} else {
unquote_display_as = display_as;
}
}
# record display_via_github from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "display_via_github" : something,
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"display_via_github"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
display_via_github = $0;
# Remove any trailing ,
#
if (display_via_github ~ /,$/) {
display_via_github = substr(display_via_github, 1, length(display_via_github)-1);
}
# remove any enclosing double quotes
#
if (display_via_github ~ /^".*"$/) {
unquote_display_via_github = substr(display_via_github, 2, length(display_via_github)-2);
} else {
unquote_display_via_github = display_via_github;
}
}
# record winners_text from an manifest array element
#
# We are looking for a JSON member line of the form:
#
# "winners_text" : something
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF >= 3 && $1 ~ /^"winners_text"$/ && $2 ~ /^:$/ {
# save JSON member value
#
# The JSON member value starts after the 1st and 2nd fields.
# We remove any leading leading whitespace, such as the FS
# between the 1st and 2nd fields, and between the 2nd and 3rd field.
#
$1 = ""; # remove 1st field
$2 = ""; # remove 2nd field
gsub(/^[[:space:]]+/, "", $0); # any leading whitespace
winners_text = $0;
# Remove any trailing ,
#
if (winners_text ~ /,$/) {
winners_text = substr(winners_text, 1, length(winners_text)-1);
}
# remove any enclosing double quotes
#
if (winners_text ~ /^".*"$/) {
unquote_winners_text = substr(winners_text, 2, length(winners_text)-2);
} else {
unquote_winners_text = winners_text;
}
}
# process end of manifest array element
#
within_manifest_array == 1 && begin_manifest_element == 1 && NF == 1 && $1 ~ /^},?$/ {
# verify we have seen a file_path JSON member
#
if (length(file_path) == 0) {
print "ERROR: manifest array JSON member did not have a file_path JSON member in:", ARGV[1] > "/dev/stderr";
exit 203;
}
# verify we have seen a inventory_order JSON member
#
if (length(inventory_order) == 0) {
print "ERROR: manifest array JSON member did not have a inventory_order JSON member in:", ARGV[1] > "/dev/stderr";
exit 204;
}
# verify we have seen a OK_to_edit JSON member
#
if (length(OK_to_edit) == 0) {
print "ERROR: manifest array JSON member did not have a OK_to_edit JSON member in:", ARGV[1] > "/dev/stderr";
exit 206;
}
# verify we have seen a display_as JSON member
#
if (length(display_as) == 0) {
print "ERROR: manifest array JSON member did not have a display_as JSON member in:", ARGV[1] > "/dev/stderr";
exit 207;
}
# verify we have seen a display_via_github JSON member
#
if (length(display_via_github) == 0) {
print "ERROR: manifest array JSON member did not have a display_via_github JSON member in:", ARGV[1] > "/dev/stderr";
exit 209;
}
# verify we have seen a winners_text JSON member
#
if (length(winners_text) == 0) {
print "ERROR: manifest array JSON member did not have a winners_text JSON member in:", ARGV[1] > "/dev/stderr";
exit 210;
}
# print manifest information
#
# print "# debug: file_path:", file_path;
# print "# debug: inventory_order:", inventory_order;
# print "# debug: OK_to_edit:", OK_to_edit;
# print "# debug: display_as:", display_as;
# print "# debug: display_via_github:", display_via_github;
# print "# debug: winners_text:", winners_text;
#
# print "# debug: unquote_file_path:", unquote_file_path;
# print "# debug: unquote_inventory_order:", unquote_inventory_order;
# print "# debug: unquote_OK_to_edit:", unquote_OK_to_edit;
# print "# debug: unquote_display_as:", unquote_display_as;
# print "# debug: unquote_display_via_github:", unquote_display_via_github;
# print "# debug: unquote_winners_text:", unquote_winners_text;
#
if (unquote_display_via_github == "false") {
url = unquote_file_path;
} else {
url = github "/blob/master/" yyyy "/" dir "/" unquote_file_path;
}
print inventory_order, "* [" unquote_file_path "](" url ") -", unquote_winners_text;
# note the end of the manifest array element
#
begin_manifest_element = 0;
# note if we found the end of the manifest array
#
if ($1 ~ /^}$/) {
within_manifest_array = 0;
}
}
END {
# case: we quit due to github unset
#
if (length(github) == 0) {
print "ERROR: github variable not set, call with -v github=REPO_URL" > "/dev/stderr";
exit 200; # use 200 to match exit code from BEGIN
}
# case: we quit because we could not determine dir from ARGV[1]
#
if (length(dir) == 0) {
print "ERROR: cannot determine dir from:", ARGV[1] > "/dev/stderr";
exit 201; # END section will output ERROR message about cannot find dir
}
if (length(yyyy) == 0) {
print "ERROR: cannot determine yyyy from:", ARGV[1] > "/dev/stderr";
exit 202; # END section will output ERROR message about cannot find yyyy
}
# case: no manifest array was found
#
if (!found_manifest_array) {
print "ERROR: we did not find a manifest JSON array in:", ARGV[1] > "/dev/stderr";
exit 211;
}
# case: manifest array was found, but manifest array element did not finish
#
if (begin_manifest_element) {
print "ERROR: manifest JSON array element did not end in:", ARGV[1] > "/dev/stderr";
exit 212;
}
# case: manifest array did not end
#
if (within_manifest_array) {
print "ERROR: manifest JSON array did not end in:", ARGV[1] > "/dev/stderr";
exit 213;
}
}
Reference in new issue View Git Blame Copy Permalink