In conjunction with changes made to "the other repo",
we now set:
export LANG="C"
export LC_CTYPE="C"
export LC_NUMERIC="C"
export LC_TIME="C"
export LC_COLLATE="C"
export LC_MONETARY="C"
export LC_MESSAGES="C"
export LC_PAPER="C"
export LC_NAME="C"
export LC_ADDRESS="C"
export LC_TELEPHONE="C"
export LC_MEASUREMENT="C"
export LC_IDENTIFICATION="C"
export LC_ALL="C"
instead of just "$LC_ALL", "$LANG", "$LC_NUMERIC".
In all bin shell scripts, set `LC_ALL`, `LANG`, and `LC_NUMERIC` to "C".
Changed all bin shell script VERSION to "2.0.0 2025-03-13".
Tested the above via `make www` on macOS. Only the sitemap was updated.
This happened because when focusing on a bug for several days that had
to do with decoding code points (which so happens to be encoding but in
JSON it should be converted to unicode even when decoding) I lost sight
of the fact that jstrencode is meant to be **JSON** encoder. In the
process of this I have uncovered a number of other bugs but that's with
jstrencode and the website uses jstrdecode so this should not delay the
contest at all.
After the swap in the jparse repo of jstrencode and jstrdecode, due to
mix-up in terminology, originally in the mkiocccentry repo, the website
tools that used jstrdecode now have to use jstrencode, with a minimum
version of 2.0.0.
The scripts have been updated to do this. After doing a make install
from the jparse repo I did a make www with no html files changed. Then
doing the same thing after make install from the mkiocccentry/jparse
directory, no changes were made either, indicating that all should be
okay.
The bin/index.html was however modified to refer to jstrencode, not
jstrdecode.
The `bin/entry2csv.sh` too was modified to leave \uxxxx
in awards alone and to NOT convert it.
Changed `bin/entry2csv.sh` VERSION from "1.5.1 2024-10-10"
to "1.6 2024-10-14".
Performed `make entry2csv` and `make csv2entry` to test the above
under macOS.
We now use `jstrdecode -N` without the need for
using `-T` with `bin/jval-wrapper.sh`.
Now depending on `jstrdecode(1)` with a minimum version
of "1.2.3" (was "1.2.0").
Added information about bin tools missing from `bin/README.md`.
Changed `bin/entry2csv.sh` VERSION from "1.5 2024-10-08"
to "1.5.1 2024-10-10".
Changed `bin/gen-authors.sh` VERSION from "1.14 2024-10-08"
to "1.14.1 2024-10-10".
Changed `bin/gen-location.sh` VERSION from "1.8 2024-10-08"
to "1.8.1 2024-10-10".
Changed `bin/gen-years.sh` VERSION from "1.6 2024-10-08"
to "1.6.1 2024-10-10".
Changed `bin/output-index-author.sh` VERSION from "1.10 2024-10-08"
to "1.10.1 2024-10-10".
Changed `bin/output-year-index.sh` VERSION from "1.6 2024-10-08"
to "1.6.1 2024-10-10".
Changed `bin/subst.entry-index.sh` VERSION from "1.8 2024-10-08"
to "1.8.1 2024-10-10".
Perform `make all_jfmt` to format canoncailly, all `entry.json`
and all `author/author_handle.json` files. A side effect of this
is to replace all of the fancy non-ASCII characters within
JSON strings with their "bashslash-u hex hex hex hex" equivalent.
Thanks to the newly fixed `jstrdecode(1)` tool, there was
no change in the HTML output.
Perform `make www` to perform the above and test under macOS.
Use ``jstrdecode(1)` to decode certain JSON strings,
such as "Full Name", "Abstract", "Award" and "Organization".
We now rely on a correctly decoding `jstrdecode(1)`. The bin tools
now rely on `jstrdecode` with the following version or later:
> jstrdecode version: 1.0.6 2024-10-08
> JSON parser version: 1.1.7 2024-10-08
Removed `bin/unicode-fix.sed` and `bin/unicode-chk.sh`.
Removed `make unicode_chk`.
Fixed a few rule dependencies on `Makefile`.
Added information about `bin/jval-wrapper.sh` in `bin/README.md`.
Changed `bin/csv2entry.sh` VERSION from "1.4 2024-09-27"
to "1.5 2024-10-08".
Changed `bin/entry2csv.sh` VERSION from "1.4 2024-09-27"
to "1.5 2024-10-08".
Changed `bin/gen-authors.sh` VERSION from "1.13 2024-09-23"
to "1.14 2024-10-08".
Changed `bin/gen-location.sh` VERSION from "1.7 2024-09-23"
to "1.8 2024-10-08".
Changed `bin/gen-years.sh` VERSION from "1.5 2024-09-23"
to "1.6 2024-10-08".
Changed `bin/jfmt-wrapper.sh` VERSION from "1.2 2024-09-23"
to "1.3 2024-10-08".
Changed `bin/jprint-wrapper.sh` VERSION from "1.3 2024-09-23"
to "1.4 2024-10-08".
Changed `bin/jval-wrapper.sh` VERSION from "1.1 2024-09-23"
to "1.2 2024-10-08".
Changed `bin/output-index-author.sh` VERSION from "1.9 2024-09-23"
to "1.10 2024-10-08".
Changed `bin/output-year-index.sh` VERSION from "1.5 2024-09-23"
to "1.6 2024-10-08".
Changed `bin/subst.entry-index.sh` VERSION from "1.7 2024-09-27"
to "1.8 2024-10-08".
Perform `make www` to perform the above and test under macOS.
We add an initial "_title_" and "_abstract_" to all entry `.entry.json` files.
The `bin/csv2entry.sh` tool reads the `summary.csv` file and
loads the "_title_" and "_abstract_" to entry `.entry.json` files.
Changed `bin/csv2entry.sh` VERSION from "1.3 2024-09-23"
to "1.4 2024-09-27".
Perform `make csv2entry` to perform the above and test under macOS.
The `bin/cvt-submission.sh` tool adds "_title_" and "_abstract_" to the
.entry.json based on the `.info.json` file.
Changed `bin/cvt-submission.sh` VERSION from "1.2 2024-09-23"
to "1.3 2024-09-27".
The `bin/entry2csv.sh` tool forms the `summary.csv` file
based on the "_title_" and "_abstract_" fields found in .entry.json files.
Changed `bin/entry2csv.sh` VERSION from "1.3 2024-09-23"
to "1.4 2024-09-27".
Perform `make entry2csv` to perform the above and test under macOS.
Added `-H` and `header=with-header-3` to the `bin/md2html.cfg`
for the configuration entry for `[12][0-9][0-9][0-9]/*/README.md`.
Changed `bin/md2html.sh` VERSION from "1.6 2024-08-17"
to "1.7 2024-09-27".
All entry `index.html` files now list the "_abstract_" in a new
`<h3>abstract</h3>` header.
Changed `bin/subst.entry-index.sh` VERSION from "1.6 2024-09-23"
to "1.7 2024-09-27".
Removed `summary.csv`.
Added `summary.csv` and `summary.numbers` to `.gitignore`.
Sorted the result with `bin/sgi.sh`.
Perform `make quick_www` to perform the above and test under macOS.
Perform `make www` to perform the above and test under macOS.
If the [jsp tool](https://github.com/kjozsa/jsp) is
installed, it will be used in favor of the [JSONPath.sh
tool](https://github.com/lcn2/JSONPath.sh).
Updated `bin/jfmt-wrapper.sh`, `bin/jprint-wrapper.sh`, and
`bin/jval-wrapper.sh` to use jsp if found. These 3 tools may be given
`-t tool` where tool is either "_jsp_" or "_JSONPath.sh_" for force the
use of the tool. They also use `-Q` to perform a quick check tool using
trivial input.
The `bin/jval-wrapper.sh` tool had been simplified. The `-w`,
`-i` and `-u` options are gone. Instead it has:
```
-b print only values (def: print both JSON member and JSON value)
-q remove enclosing double quotes (def: keep any enclosing double quotes)
```
Add `make all_jfmt` to Makefile to canonically format all entry and
author handle JSON files.
A sed script `bin/unicode-fix.sed` has been added that converts
JSON string "_\u[0-9A-Fa-f]{4}_" encoding that may arise from known
JSON data. This is done as a work-a-round for [Unicode / UTF-8
issues](https://github.com/xexyl/jparse/issues/13) with `jstrdecode(1)`.
The `bin/unicode-chk.sh` tool, if given on file arguments, scans
all `.entry.json` and `author/author_handle.json` JSON files to be
sure that, after `bin/unicode-fix.sed` processing, no JSON string
"_\u[0-9A-Fa-f]{4}_" encodings remain.
Removed all `-j jparse` args from commands. Instead the `jparse(1)`
tool is looked for along $PATH.
Added `combine_author_handle.sh` tool to transform all of the
`author/author_handle.json` files into a temporary as a single JSON
file so that `bin/jval-wrapper.sh` tool may be used to extract all
"_sort_word_" or all "_entry_id_" values.
Changed `bin/all-jfmt.sh` VERSION from "1.0 2024-08-31"
to "1.1 2024-09-24".
Changed `bin/csv2entry.sh` VERSION from "1.2.3 2024-08-30"
to "1.3 2024-09-23".
Changed `bin/cvt-submission.sh` VERSION from "1.1 2024-08-31"
to "1.2 2024-09-23".
Changed `bin/entry2csv.sh` VERSION from "1.2.1 2024-08-17"
to "1.3 2024-09-23".
Changed `bin/gen-authors.sh` VERSION from "1.12 2024-08-18"
to "1.13 2024-09-23".
Changed `bin/gen-location.sh` VERSION from "1.6 2024-08-18"
to "1.7 2024-09-23".
Changed `bin/gen-status.sh` VERSION from "1.6 2024-08-17"
to "1.7 2024-09-23".
Changed `bin/gen-years.sh` VERSION from "1.4 2024-08-17"
to "1.5 2024-09-23".
Changed `bin/jfmt-wrapper.sh` VERSION from "1.1 2024-08-20"
to "1.2 2024-09-23".
Changed `bin/jprint-wrapper.sh` VERSION from "1.1 2024-08-20"
to "1.3 2024-09-23".
Changed `bin/jval-wrapper.sh` VERSION from "1.0.4 2024-08-20"
to "1.1 2024-09-23".
Changed `bin/manifest.entry.json.awk` VERSION from "1.1.2 2024-07-28"
to "1.2 2024-09-17".
Changed `bin/output-index-author.sh` VERSION from "1.8 2024-08-17"
to "1.9 2024-09-23".
Changed `bin/output-year-index.sh` VERSION from "1.4.4 2024-08-05"
to "1.5 2024-09-23".
Changed `bin/subst.entry-index.sh` VERSION from "1.5 2024-08-13"
to "1.6 2024-09-23".
Added `bin/combine_author_handle.sh` VERSION "1.0 2024-09-23".
Added `bin/unicode-chk.sh` VERSION "1.0 2024-09-23".
Added `bin/unicode-fix.sed` VERSION "1.0 2024-09-23".
Perform `make www` to perform the above and test under macOS.
FYI: On our MacBook Pro, `make www` took:
```
make www 631.79s user 264.95s system 94% cpu 15:53.71 total
```
FYI: On our MacBook Pro, `make quick_www` took:
```
make quick_www 81.43s user 58.26s system 102% cpu 2:16.02 total
```
We use the `bin/jval-wrapper.sh` tool to obtain information from JSON
files instead of attempting using a "GROSS HACK" that involved using
`grep(1)` (and friends) as those "GROSS HACK"s assumed that the layout of
the JSON was canonical. The `bin/jval-wrapper.sh` tool is able to deal
with various layouts of JSON including non-canonical layouts. Moreover,
by using the `bin/jval-wrapper.sh` tool we greatly reduce the places in
bin tools that contain "GROSS HACK"s.
Adjusted `bin/gen-status.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/gen-status.sh` VERSION from "1.5.8 2024-08-05"
to "1.6 2024-08-17".
Performed `make gen_status` to test the above under macOS.
Adjusted `bin/gen-years.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/gen-years.sh` VERSION from "1.3.7 2024-08-05"
to "1.4 2024-08-17".
Performed `make gen_years` to test the above under macOS.
Adjusted `bin/output-index-author.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/output-index-author.sh` VERSION from "1.7.3 2024-08-05"
to "1.8 2024-08-17".
Performed `make entry_index` to test the above under macOS.
Added `make quick_year_index` to generate year level index.html files
when year level `YYYY/README.md` is newer.
The `make quick_www` calls `quick_year_index` instead of `make gen_year_index`.
The `bin` directory is now sat as `BIN_PATH=$TOPDIR/bin`, where
`$TOPDIR` is determined by top of the git repo (as determined by
the command: `git rev-parse --is-inside-work-tree`) and potentially
modified by `-d topdir`. After `cd $TOPDIR`, we set `BIN_DIR=bin`.
All tools from this repo are now assumed to under "$BIN_DIR". Removed
command like options such as `-T md2html.sh`, and `-p tool` that used
to be able to change the location of a `bin` tool.
Changed `bin/gen-authors.sh` VERSION from "1.11 2024-08-13"
to "1.12 2024-08-18".
Changed `bin/gen-location.sh` VERSION from "1.5 2024-08-17"
to "1.6 2024-08-18".
Changed `bin/readme2index.sh` VERSION from "1.2.7 2024-08-05"
to "1.3 2024-08-17".
Modified `bin/all-run.sh` to use the `bin` directory mechanism.
Modified `bin/all-years.sh` to use the `bin` directory mechanism.
Modified `bin/gen-other-html.sh` to use the `bin` directory mechanism.
Modified `bin/gen-year-index.sh` to use the `bin` directory mechanism.
Modified `bin/md2html.sh` to use the `bin` directory mechanism.
Modified `bin/quick-readme2index.sh` to use the `bin` directory mechanism.
Modified `bin/readme2index.sh` to use the `bin` directory mechanism.
Changed `bin/all-run.sh` VERSION from "1.3.6 2024-08-05"
to "1.4 2024-08-17".
Changed `bin/all-years.sh` VERSION from "1.0.5 2024-08-05"
to "1.1 2024-08-17".
Changed `bin/gen-other-html.sh` VERSION from "1.3 2024-08-11"
to "1.4 2024-08-17".
Changed `bin/gen-year-index.sh` VERSION from "1.1.2 2024-08-05"
to "1.2 2024-08-17".
Changed `bin/quick-readme2index.sh` VERSION from "1.1.7 2024-08-05"
to "1.2 2024-08-17".
Changed `bin/md2html.sh` VERSION from "1.5.5 2024-08-05"
to "1.6 2024-08-17".
Added some whitespace in the form of empty lines to tools.
Made checks for tools under `bin` directory more consistent.
When using `-Q` (quick mode) and the prerequisite files are newer,
we force the target to be touched so that a later run will not
have to build the target HTML file.
Performed `make quick_www` to test the above under macOS.
Performed `make www` to test the above under macOS.
We make sure that when we sort, we sort with `LC_ALL=C` as in:
> LC_ALL=C sort ...
Changed `bin/entry2csv.sh` VERSION from "1.2 2024-08-13"
to "1.2.1 2024-08-17".
Performed `make entry2csv` to test the above under macOS.
Changed `bin/csv2entry.sh` VERSION from "1.2 2024-08-13"
to "1.2.1 2024-08-17".
Performed `make csv2entry` to test the above under macOS.
Changed `bin/cvt-submission.sh` VERSION from "1.0.4 2024-08-11"
to "1.0.5 2024-08-17".
Changed `bin/gen-sitemap.sh` VERSION from "1.3.13 2024-08-05"
to "1.3.14 2024-08-19".
Changed `bin/new-dir.sh` VERSION from "1.3.1 2024-08-11"
to "1.3.2 2024-08-17".
Performed `make www` to test the above under macOS.
We use the `bin/jval-wrapper.sh` tool to obtain information from JSON
files instead of attempting using a "GROSS HACK" that involved using
`grep(1)` (and friends) as those "GROSS HACK"s assumed that the layout of
the JSON was canonical. The `bin/jval-wrapper.sh` tool is able to deal
with various layouts of JSON including non-canonical layouts. Moreover,
by using the `bin/jval-wrapper.sh` tool we greatly reduce the places in
bin tools that contain "GROSS HACK"s.
Adjusted `bin/subst.entry-index.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/subst.entry-index.sh` VERSION from "1.4 2024-08-09"
to "1.5 2024-08-13".
Adjusted `bin/entry2csv.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/entry2csv.sh` VERSION from "1.1.1 2024-07-22"
to "1.2 2024-08-13".
Performed `make entry2csv` to test the above under macOS.
Adjusted `bin/csv2entry.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/entry2csv.sh` VERSION from "1.1.1 2024-07-22"
to "1.2 2024-08-13".
Performed `make csv2entry` to test the above under macOS.
In the spirit of pr #2589, we correct similar cases where the string
"status" instead of `$status` was being tested for a value.
In `bin/csv2entry.sh` changed VERSION from "1.1 2024-07-20"
to "1.1.1 2024-07-22".
In `bin/entry2csv.sh` changed VERSION from "1.1 2024-07-20"
to "1.1.1 2024-07-22".
In `bin/gen-sitemap.sh` changed VERSION from "1.3.10 2024-07-18"
to "1.3.11 2024-07-22".
In `bin/gen-status.sh` changed VERSION from "1.5.5 2024-07-18"
to "1.5.6 2024-07-22".
In `bin/md2html.sh` changed VERSION from "1.5.2 2024-07-18"
to "1.5.3 2024-07-22".
In `bin/tar-entry.sh` changed VERSION from "1.3.6 2024-04-28"
to "1.3.7 2024-07-22".
In `bin/tar-year.sh` changed VERSION from "1.2.4 2024-04-28"
to "1.2.5 2024-07-22".
Performed `make www` to accomplish the above and to test under macOS.
Any canonicalization errors (e.g., sort error, mv error, etc.) will
cause an immediate `exit 8`.
The `bin/entry2csv.sh` tool will verify the author_handles referred
to by `author_wins.csv` and check the `author/cauthor_handle.json' file set.
The `bin/csv2entry.sh` tool will verify the author_handles referred
to by `author_wins.csv` and check the `author/cauthor_handle.json' file set.
In `bin/entry2csv.sh` changed VERSION from 1.0.4 2024-07-19"
"1.1 2024-07-20".
In `bin/csv2entry.sh` changed VERSION from to "1.0.1 2024-07-19"
"1.1 2024-07-20".
Performed `make www` to accomplish the above and to test under macOS.
TODO: Remove the `tmp` directory (after saving contents into private storage)
after testing and improvement of `bin/csv2entry.sh` and
`bin/entry2csv.sh is complete.
Fixed typo in `make csv2entry` rule in `Makefile`.
Fixed how exit codes triggered deep within loops are realized.
Regularize how raising an exit codes with loops are announced.
The use of `-n` does NOT prevent the creation of temporary files:
only permanent files are not modified with `-n`. NOTE: While
CSV files are NOT permanent files, they too are not modified with `-n`.
Fixed the canonical sort of `manifest.csv`. Now running
`bin/csv2entry.sh` immediately after `bin/entry2csv.sh` will not
modify `manifest.csv`.
If `bin/csv2entry.sh` encounters an author handle for which
there is no `author/author_handle.json` file, the tool will
complete the formation of the CSV file set and then exit 9
at the very end.
Fixed a few comments in `bin/csv2entry.sh` and `bin/entry2csv.sh`.
In `bin/entry2csv.sh` changed VERSION from "1.0.3 2024-07-19"
to "1.0.4 2024-07-19".
In `bin/csv2entry.sh` changed VERSION from "1.0 2024-07-19"
to "1.0.1 2024-07-19".
Performed `make www` to accomplish the above and to test under macOS.
TODO: Incorporate as needed, more processing of `author/author_handle.json` files.
TODO: Remove the `tmp` directory (after saving contents into private storage).
Added `bin/csv2entry.sh` VERSION "1.0 2024-07-19".
In `bin/entry2csv.sh` changed VERSION from "1.0.2 2024-07-14"
to "1.0.3 2024-07-19".
The `bin/entry2csv.sh` tool converts all entry `.entry.json` files
into canonicalized `author_wins.csv`, `manifest.csv`, and `year_prize.csv`
CSV files.
The `bin/csv2entry.sh` tool converts `author_wins.csv`, `manifest.csv`,
and `year_prize.csv` (canonicalizing them if needed) `.entry.json`
files for all entries.
To change the manifest for one or more entries:
```sh
bin/entry2csv.sh -v 1
# ((import `manifest.csv` into a spreadsheet tool))
# ((modify in the spreadsheet tool, the content as needed))
# ((export the spreadsheet contents, as the CSV file `manifest.csv`))
bin/csv2entry.sh -v 1
make www
``
To change the award for one or more entries:
```sh
bin/entry2csv.sh -v 1
# ((import `year_prize.csv` into a spreadsheet tool))
# ((modify in the spreadsheet tool, the content as needed))
# ((export the spreadsheet contents, as the CSV file `year_prize.csv`))
bin/csv2entry.sh -v 1
make www
``
To change the authorship of one or more entries:
```sh
bin/entry2csv.sh -v 1
# ((import `author_wins.csv` into a spreadsheet tool))
# ((modify in the spreadsheet tool, the content as needed))
# ((export the spreadsheet contents, as the CSV file `author_wins.csv`))
bin/csv2entry.sh -v 1
# ((modify winning_entry_set of `author/author_handle.json` files as needed))
make www
``
Performed `make www` to accomplish the above and to test under macOS.
TODO: Review and improve/expand as needed, changes that were made to `bin/README.md`.
TODO: Remove the `tmp` directory (after saving contents into private storage).
We add a new `bin/entry2csv.sh` tool VERSION "1.0 2024-07-14"
that will convert entry `.entry.json` files into 3 CSV files
`author_wins.csv`, `manifest.csv`, and `year_prize.csv`.
The `bin/entry2csv.sh` tool uses `bin/manifest.csv.entry.awk`.
The new `bin/manifest.csv.entry.awk` VERSION is "1.0 2024-07-14".
This tool will be used to convert entry `.entry.json` file into
CSV files suitable for importing into a spreadsheet application
for further processing.
TODO: Write the `bin/csv2entry.sh` tool, for a given entry, to
convert 3 CSV files into that entry's c`.entry.json` file.
Landon Curt Noll