In conjunction with changes made to "the other repo",
we now set:
export LANG="C"
export LC_CTYPE="C"
export LC_NUMERIC="C"
export LC_TIME="C"
export LC_COLLATE="C"
export LC_MONETARY="C"
export LC_MESSAGES="C"
export LC_PAPER="C"
export LC_NAME="C"
export LC_ADDRESS="C"
export LC_TELEPHONE="C"
export LC_MEASUREMENT="C"
export LC_IDENTIFICATION="C"
export LC_ALL="C"
instead of just "$LC_ALL", "$LANG", "$LC_NUMERIC".
In all bin shell scripts, set `LC_ALL`, `LANG`, and `LC_NUMERIC` to "C".
Changed all bin shell script VERSION to "2.0.0 2025-03-13".
Tested the above via `make www` on macOS. Only the sitemap was updated.
Use ``jstrdecode(1)` to decode certain JSON strings,
such as "Full Name", "Abstract", "Award" and "Organization".
We now rely on a correctly decoding `jstrdecode(1)`. The bin tools
now rely on `jstrdecode` with the following version or later:
> jstrdecode version: 1.0.6 2024-10-08
> JSON parser version: 1.1.7 2024-10-08
Removed `bin/unicode-fix.sed` and `bin/unicode-chk.sh`.
Removed `make unicode_chk`.
Fixed a few rule dependencies on `Makefile`.
Added information about `bin/jval-wrapper.sh` in `bin/README.md`.
Changed `bin/csv2entry.sh` VERSION from "1.4 2024-09-27"
to "1.5 2024-10-08".
Changed `bin/entry2csv.sh` VERSION from "1.4 2024-09-27"
to "1.5 2024-10-08".
Changed `bin/gen-authors.sh` VERSION from "1.13 2024-09-23"
to "1.14 2024-10-08".
Changed `bin/gen-location.sh` VERSION from "1.7 2024-09-23"
to "1.8 2024-10-08".
Changed `bin/gen-years.sh` VERSION from "1.5 2024-09-23"
to "1.6 2024-10-08".
Changed `bin/jfmt-wrapper.sh` VERSION from "1.2 2024-09-23"
to "1.3 2024-10-08".
Changed `bin/jprint-wrapper.sh` VERSION from "1.3 2024-09-23"
to "1.4 2024-10-08".
Changed `bin/jval-wrapper.sh` VERSION from "1.1 2024-09-23"
to "1.2 2024-10-08".
Changed `bin/output-index-author.sh` VERSION from "1.9 2024-09-23"
to "1.10 2024-10-08".
Changed `bin/output-year-index.sh` VERSION from "1.5 2024-09-23"
to "1.6 2024-10-08".
Changed `bin/subst.entry-index.sh` VERSION from "1.7 2024-09-27"
to "1.8 2024-10-08".
Perform `make www` to perform the above and test under macOS.
We add an initial "_title_" and "_abstract_" to all entry `.entry.json` files.
The `bin/csv2entry.sh` tool reads the `summary.csv` file and
loads the "_title_" and "_abstract_" to entry `.entry.json` files.
Changed `bin/csv2entry.sh` VERSION from "1.3 2024-09-23"
to "1.4 2024-09-27".
Perform `make csv2entry` to perform the above and test under macOS.
The `bin/cvt-submission.sh` tool adds "_title_" and "_abstract_" to the
.entry.json based on the `.info.json` file.
Changed `bin/cvt-submission.sh` VERSION from "1.2 2024-09-23"
to "1.3 2024-09-27".
The `bin/entry2csv.sh` tool forms the `summary.csv` file
based on the "_title_" and "_abstract_" fields found in .entry.json files.
Changed `bin/entry2csv.sh` VERSION from "1.3 2024-09-23"
to "1.4 2024-09-27".
Perform `make entry2csv` to perform the above and test under macOS.
Added `-H` and `header=with-header-3` to the `bin/md2html.cfg`
for the configuration entry for `[12][0-9][0-9][0-9]/*/README.md`.
Changed `bin/md2html.sh` VERSION from "1.6 2024-08-17"
to "1.7 2024-09-27".
All entry `index.html` files now list the "_abstract_" in a new
`<h3>abstract</h3>` header.
Changed `bin/subst.entry-index.sh` VERSION from "1.6 2024-09-23"
to "1.7 2024-09-27".
Removed `summary.csv`.
Added `summary.csv` and `summary.numbers` to `.gitignore`.
Sorted the result with `bin/sgi.sh`.
Perform `make quick_www` to perform the above and test under macOS.
Perform `make www` to perform the above and test under macOS.
If the [jsp tool](https://github.com/kjozsa/jsp) is
installed, it will be used in favor of the [JSONPath.sh
tool](https://github.com/lcn2/JSONPath.sh).
Updated `bin/jfmt-wrapper.sh`, `bin/jprint-wrapper.sh`, and
`bin/jval-wrapper.sh` to use jsp if found. These 3 tools may be given
`-t tool` where tool is either "_jsp_" or "_JSONPath.sh_" for force the
use of the tool. They also use `-Q` to perform a quick check tool using
trivial input.
The `bin/jval-wrapper.sh` tool had been simplified. The `-w`,
`-i` and `-u` options are gone. Instead it has:
```
-b print only values (def: print both JSON member and JSON value)
-q remove enclosing double quotes (def: keep any enclosing double quotes)
```
Add `make all_jfmt` to Makefile to canonically format all entry and
author handle JSON files.
A sed script `bin/unicode-fix.sed` has been added that converts
JSON string "_\u[0-9A-Fa-f]{4}_" encoding that may arise from known
JSON data. This is done as a work-a-round for [Unicode / UTF-8
issues](https://github.com/xexyl/jparse/issues/13) with `jstrdecode(1)`.
The `bin/unicode-chk.sh` tool, if given on file arguments, scans
all `.entry.json` and `author/author_handle.json` JSON files to be
sure that, after `bin/unicode-fix.sed` processing, no JSON string
"_\u[0-9A-Fa-f]{4}_" encodings remain.
Removed all `-j jparse` args from commands. Instead the `jparse(1)`
tool is looked for along $PATH.
Added `combine_author_handle.sh` tool to transform all of the
`author/author_handle.json` files into a temporary as a single JSON
file so that `bin/jval-wrapper.sh` tool may be used to extract all
"_sort_word_" or all "_entry_id_" values.
Changed `bin/all-jfmt.sh` VERSION from "1.0 2024-08-31"
to "1.1 2024-09-24".
Changed `bin/csv2entry.sh` VERSION from "1.2.3 2024-08-30"
to "1.3 2024-09-23".
Changed `bin/cvt-submission.sh` VERSION from "1.1 2024-08-31"
to "1.2 2024-09-23".
Changed `bin/entry2csv.sh` VERSION from "1.2.1 2024-08-17"
to "1.3 2024-09-23".
Changed `bin/gen-authors.sh` VERSION from "1.12 2024-08-18"
to "1.13 2024-09-23".
Changed `bin/gen-location.sh` VERSION from "1.6 2024-08-18"
to "1.7 2024-09-23".
Changed `bin/gen-status.sh` VERSION from "1.6 2024-08-17"
to "1.7 2024-09-23".
Changed `bin/gen-years.sh` VERSION from "1.4 2024-08-17"
to "1.5 2024-09-23".
Changed `bin/jfmt-wrapper.sh` VERSION from "1.1 2024-08-20"
to "1.2 2024-09-23".
Changed `bin/jprint-wrapper.sh` VERSION from "1.1 2024-08-20"
to "1.3 2024-09-23".
Changed `bin/jval-wrapper.sh` VERSION from "1.0.4 2024-08-20"
to "1.1 2024-09-23".
Changed `bin/manifest.entry.json.awk` VERSION from "1.1.2 2024-07-28"
to "1.2 2024-09-17".
Changed `bin/output-index-author.sh` VERSION from "1.8 2024-08-17"
to "1.9 2024-09-23".
Changed `bin/output-year-index.sh` VERSION from "1.4.4 2024-08-05"
to "1.5 2024-09-23".
Changed `bin/subst.entry-index.sh` VERSION from "1.5 2024-08-13"
to "1.6 2024-09-23".
Added `bin/combine_author_handle.sh` VERSION "1.0 2024-09-23".
Added `bin/unicode-chk.sh` VERSION "1.0 2024-09-23".
Added `bin/unicode-fix.sed` VERSION "1.0 2024-09-23".
Perform `make www` to perform the above and test under macOS.
FYI: On our MacBook Pro, `make www` took:
```
make www 631.79s user 264.95s system 94% cpu 15:53.71 total
```
FYI: On our MacBook Pro, `make quick_www` took:
```
make quick_www 81.43s user 58.26s system 102% cpu 2:16.02 total
```
The `bin/cvt-submission.sh` tool is now complete.
Changed `bin/cvt-submission.sh` VERSION from "1.0.5 2024-08-17"
to "1.1 2024-08-31".
Make minor error message fix to `bin/csv2entry.sh`.
Performed `make quick_www` to accomplish the above and to test under macOS.
TODO: Complete the `bin/csv2entry.sh` tool.
We fixed `bin/csv2entry.sh` to output entry `.entry.json`
files in JSON canonical form.
Changed `bin/csv2entry.sh` VERSION from "1.2.2 2024-08-17"
to "1.2.3 2024-08-30".
Performed `make quick_www` to accomplish the above and to test under macOS.
We use the `bin/jval-wrapper.sh` tool to obtain information from JSON
files instead of attempting using a "GROSS HACK" that involved using
`grep(1)` (and friends) as those "GROSS HACK"s assumed that the layout of
the JSON was canonical. The `bin/jval-wrapper.sh` tool is able to deal
with various layouts of JSON including non-canonical layouts. Moreover,
by using the `bin/jval-wrapper.sh` tool we greatly reduce the places in
bin tools that contain "GROSS HACK"s.
Adjusted `bin/gen-status.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/gen-status.sh` VERSION from "1.5.8 2024-08-05"
to "1.6 2024-08-17".
Performed `make gen_status` to test the above under macOS.
Adjusted `bin/gen-years.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/gen-years.sh` VERSION from "1.3.7 2024-08-05"
to "1.4 2024-08-17".
Performed `make gen_years` to test the above under macOS.
Adjusted `bin/output-index-author.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/output-index-author.sh` VERSION from "1.7.3 2024-08-05"
to "1.8 2024-08-17".
Performed `make entry_index` to test the above under macOS.
Added `make quick_year_index` to generate year level index.html files
when year level `YYYY/README.md` is newer.
The `make quick_www` calls `quick_year_index` instead of `make gen_year_index`.
The `bin` directory is now sat as `BIN_PATH=$TOPDIR/bin`, where
`$TOPDIR` is determined by top of the git repo (as determined by
the command: `git rev-parse --is-inside-work-tree`) and potentially
modified by `-d topdir`. After `cd $TOPDIR`, we set `BIN_DIR=bin`.
All tools from this repo are now assumed to under "$BIN_DIR". Removed
command like options such as `-T md2html.sh`, and `-p tool` that used
to be able to change the location of a `bin` tool.
Changed `bin/gen-authors.sh` VERSION from "1.11 2024-08-13"
to "1.12 2024-08-18".
Changed `bin/gen-location.sh` VERSION from "1.5 2024-08-17"
to "1.6 2024-08-18".
Changed `bin/readme2index.sh` VERSION from "1.2.7 2024-08-05"
to "1.3 2024-08-17".
Modified `bin/all-run.sh` to use the `bin` directory mechanism.
Modified `bin/all-years.sh` to use the `bin` directory mechanism.
Modified `bin/gen-other-html.sh` to use the `bin` directory mechanism.
Modified `bin/gen-year-index.sh` to use the `bin` directory mechanism.
Modified `bin/md2html.sh` to use the `bin` directory mechanism.
Modified `bin/quick-readme2index.sh` to use the `bin` directory mechanism.
Modified `bin/readme2index.sh` to use the `bin` directory mechanism.
Changed `bin/all-run.sh` VERSION from "1.3.6 2024-08-05"
to "1.4 2024-08-17".
Changed `bin/all-years.sh` VERSION from "1.0.5 2024-08-05"
to "1.1 2024-08-17".
Changed `bin/gen-other-html.sh` VERSION from "1.3 2024-08-11"
to "1.4 2024-08-17".
Changed `bin/gen-year-index.sh` VERSION from "1.1.2 2024-08-05"
to "1.2 2024-08-17".
Changed `bin/quick-readme2index.sh` VERSION from "1.1.7 2024-08-05"
to "1.2 2024-08-17".
Changed `bin/md2html.sh` VERSION from "1.5.5 2024-08-05"
to "1.6 2024-08-17".
Added some whitespace in the form of empty lines to tools.
Made checks for tools under `bin` directory more consistent.
When using `-Q` (quick mode) and the prerequisite files are newer,
we force the target to be touched so that a later run will not
have to build the target HTML file.
Performed `make quick_www` to test the above under macOS.
Performed `make www` to test the above under macOS.
We make sure that when we uniq, we sort with `LC_ALL=C` as in:
> LC_ALL=C uniq ...
Changed `bin/csv2entry.sh` VERSION from "1.2.1 2024-08-17"
to "1.2.2 2024-08-17".
Performed `make csv2entry` to test the above under macOS.
We make sure that when we sort, we sort with `LC_ALL=C` as in:
> LC_ALL=C sort ...
Changed `bin/entry2csv.sh` VERSION from "1.2 2024-08-13"
to "1.2.1 2024-08-17".
Performed `make entry2csv` to test the above under macOS.
Changed `bin/csv2entry.sh` VERSION from "1.2 2024-08-13"
to "1.2.1 2024-08-17".
Performed `make csv2entry` to test the above under macOS.
Changed `bin/cvt-submission.sh` VERSION from "1.0.4 2024-08-11"
to "1.0.5 2024-08-17".
Changed `bin/gen-sitemap.sh` VERSION from "1.3.13 2024-08-05"
to "1.3.14 2024-08-19".
Changed `bin/new-dir.sh` VERSION from "1.3.1 2024-08-11"
to "1.3.2 2024-08-17".
Performed `make www` to test the above under macOS.
We use the `bin/jval-wrapper.sh` tool to obtain information from JSON
files instead of attempting using a "GROSS HACK" that involved using
`grep(1)` (and friends) as those "GROSS HACK"s assumed that the layout of
the JSON was canonical. The `bin/jval-wrapper.sh` tool is able to deal
with various layouts of JSON including non-canonical layouts. Moreover,
by using the `bin/jval-wrapper.sh` tool we greatly reduce the places in
bin tools that contain "GROSS HACK"s.
Adjusted `bin/subst.entry-index.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/subst.entry-index.sh` VERSION from "1.4 2024-08-09"
to "1.5 2024-08-13".
Adjusted `bin/entry2csv.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/entry2csv.sh` VERSION from "1.1.1 2024-07-22"
to "1.2 2024-08-13".
Performed `make entry2csv` to test the above under macOS.
Adjusted `bin/csv2entry.sh` to use `bin/jval-wrapper.sh`.
Changed `bin/entry2csv.sh` VERSION from "1.1.1 2024-07-22"
to "1.2 2024-08-13".
Performed `make csv2entry` to test the above under macOS.
In the spirit of pr #2589, we correct similar cases where the string
"status" instead of `$status` was being tested for a value.
In `bin/csv2entry.sh` changed VERSION from "1.1 2024-07-20"
to "1.1.1 2024-07-22".
In `bin/entry2csv.sh` changed VERSION from "1.1 2024-07-20"
to "1.1.1 2024-07-22".
In `bin/gen-sitemap.sh` changed VERSION from "1.3.10 2024-07-18"
to "1.3.11 2024-07-22".
In `bin/gen-status.sh` changed VERSION from "1.5.5 2024-07-18"
to "1.5.6 2024-07-22".
In `bin/md2html.sh` changed VERSION from "1.5.2 2024-07-18"
to "1.5.3 2024-07-22".
In `bin/tar-entry.sh` changed VERSION from "1.3.6 2024-04-28"
to "1.3.7 2024-07-22".
In `bin/tar-year.sh` changed VERSION from "1.2.4 2024-04-28"
to "1.2.5 2024-07-22".
Performed `make www` to accomplish the above and to test under macOS.
Any canonicalization errors (e.g., sort error, mv error, etc.) will
cause an immediate `exit 8`.
The `bin/entry2csv.sh` tool will verify the author_handles referred
to by `author_wins.csv` and check the `author/cauthor_handle.json' file set.
The `bin/csv2entry.sh` tool will verify the author_handles referred
to by `author_wins.csv` and check the `author/cauthor_handle.json' file set.
In `bin/entry2csv.sh` changed VERSION from 1.0.4 2024-07-19"
"1.1 2024-07-20".
In `bin/csv2entry.sh` changed VERSION from to "1.0.1 2024-07-19"
"1.1 2024-07-20".
Performed `make www` to accomplish the above and to test under macOS.
TODO: Remove the `tmp` directory (after saving contents into private storage)
after testing and improvement of `bin/csv2entry.sh` and
`bin/entry2csv.sh is complete.
The fixes in bin/README.md include typo fixes, link fixes, format fixes
as well as adding a couple terms that are now used by the IOCCC
(submission and submitter).
The typo fixes in bin/csv2entry.sh were deemed worth putting in this
commit as a lot of the changes in the document are related.
Fixed typo in `make csv2entry` rule in `Makefile`.
Fixed how exit codes triggered deep within loops are realized.
Regularize how raising an exit codes with loops are announced.
The use of `-n` does NOT prevent the creation of temporary files:
only permanent files are not modified with `-n`. NOTE: While
CSV files are NOT permanent files, they too are not modified with `-n`.
Fixed the canonical sort of `manifest.csv`. Now running
`bin/csv2entry.sh` immediately after `bin/entry2csv.sh` will not
modify `manifest.csv`.
If `bin/csv2entry.sh` encounters an author handle for which
there is no `author/author_handle.json` file, the tool will
complete the formation of the CSV file set and then exit 9
at the very end.
Fixed a few comments in `bin/csv2entry.sh` and `bin/entry2csv.sh`.
In `bin/entry2csv.sh` changed VERSION from "1.0.3 2024-07-19"
to "1.0.4 2024-07-19".
In `bin/csv2entry.sh` changed VERSION from "1.0 2024-07-19"
to "1.0.1 2024-07-19".
Performed `make www` to accomplish the above and to test under macOS.
TODO: Incorporate as needed, more processing of `author/author_handle.json` files.
TODO: Remove the `tmp` directory (after saving contents into private storage).
Added `bin/csv2entry.sh` VERSION "1.0 2024-07-19".
In `bin/entry2csv.sh` changed VERSION from "1.0.2 2024-07-14"
to "1.0.3 2024-07-19".
The `bin/entry2csv.sh` tool converts all entry `.entry.json` files
into canonicalized `author_wins.csv`, `manifest.csv`, and `year_prize.csv`
CSV files.
The `bin/csv2entry.sh` tool converts `author_wins.csv`, `manifest.csv`,
and `year_prize.csv` (canonicalizing them if needed) `.entry.json`
files for all entries.
To change the manifest for one or more entries:
```sh
bin/entry2csv.sh -v 1
# ((import `manifest.csv` into a spreadsheet tool))
# ((modify in the spreadsheet tool, the content as needed))
# ((export the spreadsheet contents, as the CSV file `manifest.csv`))
bin/csv2entry.sh -v 1
make www
``
To change the award for one or more entries:
```sh
bin/entry2csv.sh -v 1
# ((import `year_prize.csv` into a spreadsheet tool))
# ((modify in the spreadsheet tool, the content as needed))
# ((export the spreadsheet contents, as the CSV file `year_prize.csv`))
bin/csv2entry.sh -v 1
make www
``
To change the authorship of one or more entries:
```sh
bin/entry2csv.sh -v 1
# ((import `author_wins.csv` into a spreadsheet tool))
# ((modify in the spreadsheet tool, the content as needed))
# ((export the spreadsheet contents, as the CSV file `author_wins.csv`))
bin/csv2entry.sh -v 1
# ((modify winning_entry_set of `author/author_handle.json` files as needed))
make www
``
Performed `make www` to accomplish the above and to test under macOS.
TODO: Review and improve/expand as needed, changes that were made to `bin/README.md`.
TODO: Remove the `tmp` directory (after saving contents into private storage).
Landon Curt Noll