mirror/ioccc
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

release readme2index.sh version 1.0 2024-01-14

Browse Source 
The initial release of `bin/readme2index.sh` tool that converts
`READMD.md` and various JSON files such as `.winner.json`
as well as "author/author_handle.json" files.

Applied the `readme2index.sh` tool to generate `2020/ferguson1/index.html`.

We generated the  `2020/ferguson1/index.html` by executing:

```sh
./bin/readme2index.sh -v 3 2020/ferguson1
```

NOTE: This above tool requires **pandoc to be installed**.
See the [pandoc web site](https://pandoc.org) for how
to install.  We used _HomeBrew_ to install it,
but there is also a _macports_ process as well.

NOTE: While the above web page does have formatting issues, it is
a first attempt data driven programmatic web page generation.

Some adjustments can be made by mostly improving `ioccc.css`,
and some perhaps by modifying how `bin/pandoc_wrapper.sh`
invokes pandoc.  But probably most of the improvements can
be made by fixing and improving `ioccc.css`.

TODO: Fix and improve the `2020/ferguson1/index.html`
web page by fixing and improving `ioccc.css`.

Moved scripts and tools needed to generate HTML pages
into a new `/bin` subdirectory.

TODO: Consider getting rid of the [build-ioccc
repo](https://github.com/ioccc-src/build-ioccc) and just use the
`/bin` subdirectory.

NOTE: Some links in the `2020/ferguson1/index.html`
web page do not (yet) exist.

TODO: Once the basic format of the `2020/ferguson1/index.html` web
page is somewhat reasonable, generate `index.html` pages for other
winners.  Generate the year level `index.html' files.
Generate the top level HTML files, including the missing
`location.html` file.

Greatly simplified the `.winner.json` file format,
removing a number of fields that were not strictly needed
for generating an inventory on a `index.html` page.

Updated various versions:

* `bin/gen-author.sh` VERSION="1.3 2024-01-14"
* `bin/gen-inventory.sh` VERSION="1.0 2024-01-14"
* `bin/manifest.winner.json.awk` VERSION="1.0 2024-01-14"
* `bin/pandoc_wrapper.sh` VERSION="1.3 2024-01-14"
* `bin/readme2index.sh` VERSION="1.0 2024-01-14"
* `bin/subst.default.sh` VERSION="1.1 2024-01-14"
* `bin/subst.winner-index.sh` VERSION="1.1 2024-01-14"
* `tmp/gen_path_list.found.sh` VERSION="1.1 2024-01-14"
* `tmp/gen_winner_json.sh` VERSION="1.1 2024-01-14"

The `.winner.json` file format from winner_JSON_format_version
"0.5 2023-11-04" to "1.0 2024-01-14".

All `.winner.json` files have been updated to use the
new `.winner.json` file format.

The manifest (`tmp/manifest.numbers` and `tmp/manifest.csv`) has
updated to use the new `.winner.json` file format.

The following procedure was used to generate the
`tmp/manifest.csv` manifest file:

0. open [manifest.numbers](manifest.numbers) in numbers: modify if/as needed

NOTE: Please try to keep the [manifest.numbers](manifest.numbers) spreadsheet
      in order order.  Use Organize -> Sort -> ((Sort Now)) to sort.

NOTE: Empty fields in the [manifest.numbers](manifest.numbers) spreadsheet
      are not allowed.  By default, a cell that is empty is colored
      as a warning.

1. File -> Export To -> CSV...
    * 1a. `[ ]` Include table names (unset)
    * 1b. Text Encoding: Unicode (UTF-8)
    * 1c. Click ((Save..))

2. Save As: manifest.csv
    * 2a. Select the `tmp` directory

3. Click ((Export))
    * 3a. If needed click ((Replace))

4. Execute ./tmp/[fix_csv.sh](fix_csv.sh)

NOTE: You must run the [fix_csv.sh](fix_csv.sh) in the above
      step in order to make the CSV file usable by other tools.

NOTE: If no changes are needed in the above step, the
      original file content will be restored.

5. Execute ./tmp/[gen_path_list.found.sh](gen_path_list.found.sh)

NOTE: The above step silently performs a `make clobber` at the
      top level, and so it takes a bit of time.

NOTE: If no changes are needed in the above step, the
      original file content will be restored.

6. Execute ./tmp/[check_path_list.sh](check_path_list.sh)

NOTE: The above step prints nothing and for an exit 0, then
      all is well.  If there is a problem, such as files
      found that are not part of the manifest, or files
      in the manifest that are not found, then an error
      message will be printed and the tool will exit non-zero.
      If errors are reported, go back to step 0 and
      correct the errors.

7. Execute ./tmp/[run_all.sh](run_all.sh) ./gen_winner_json.sh

NOTE: The above step will take a few seconds to run.  It will rebuild
      all `.winner.json` files for all winners.

NOTE: If no changes are needed in the above step, the
      original file content will be restored.

8. Execute `git status` and inspect changes

NOTE: Check in changes if and as needed.

NOTE: In a number of shell scripts under `bin/*.sh`,
      and in the `bin/manifest.winner.json.awk` script
      you will find comments of the form:

```
```

This is a note that when tools such as `jnamval` (and `jval` and `jfmt`) are
ready, these gross hacks should be replaced with calls to those tools.
Until they are ready, some must be careful with the layout of JSON files.

A number of bugs and typos have been fixed.

NOTE: Fixing typos in the files touched by this comment is welcome.
      This includes scripts, data files, text files, etc.

The _inventory_order_, _display_as_, _display_via_github_.  and
_winners_text_ fields of the [manifest.numbers](manifest.numbers)
spreadsheet were just set to have some initial values.
As they impact the inventory displayed on `index.html` pages,
their content needs to be improved.

NOTE: Improving the above mentioned fields of the
      [manifest.numbers](manifest.numbers) spreadsheet is welcome.
This commit is contained in:
Landon Curt Noll committed 2024-01-14 22:03:28 -08:00
1 parent f5a259d3b3
commit c6e6201209
366 files changed
+29369 -31783
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tmp/fix_csv.sh
+47 -32
View File
@@ -104,7 +104,7 @@ fi
# check for ,TRUE,
#
rm -f "$TMP_CHECK"
grep ',TRUE.' "$TMP_CSV" > "$TMP_CHECK"
grep -E ',TRUE,|,TRUE$' "$TMP_CSV" > "$TMP_CHECK"
if [[ -s $TMP_CHECK ]]; then
echo "$0: ERROR: found ,TRUE, instead of ,true, in $MANIFEST_CSV" 1>&2
echo "$0: Warning: lines with ,TRUE, $MANIFEST_CSV starts below" 1>&2
@@ -116,7 +116,7 @@ fi
# check for ,FALSE,
#
rm -f "$TMP_CHECK"
grep ',FALSE.' "$TMP_CSV" > "$TMP_CHECK"
grep -E ',FALSE,|,FALSE$' "$TMP_CSV" > "$TMP_CHECK"
if [[ -s $TMP_CHECK ]]; then
echo "$0: ERROR: found ,FALSE, instead of ,false, in $MANIFEST_CSV" 1>&2
echo "$0: Warning: lines with ,FALSE, $MANIFEST_CSV starts below" 1>&2
@@ -125,16 +125,29 @@ if [[ -s $TMP_CHECK ]]; then
exit 20
fi
# check for ,null,
#
rm -f "$TMP_CHECK"
grep -E ',null,|.,null$' "$TMP_CSV" > "$TMP_CHECK"
if [[ -s $TMP_CHECK ]]; then
echo "$0: ERROR: found ,null, in $MANIFEST_CSV" 1>&2
echo "$0: Warning: lines with ,null, $MANIFEST_CSV starts below" 1>&2
cat "$TMP_CHECK" 1>&2
echo "$0: Warning: lines with ,null, $MANIFEST_CSV ends above" 1>&2
exit 21
fi
rm -f "$TMP_CHECK"
# check for ,NULL,
#
rm -f "$TMP_CHECK"
grep ',FALSE.' "$TMP_CSV" > "$TMP_CHECK"
grep -E ',NULL,|,NULL$' "$TMP_CSV" > "$TMP_CHECK"
if [[ -s $TMP_CHECK ]]; then
echo "$0: ERROR: found ,NULL, instead of ,null, in $MANIFEST_CSV" 1>&2
echo "$0: ERROR: found ,NULL, in $MANIFEST_CSV" 1>&2
echo "$0: Warning: lines with ,NULL, $MANIFEST_CSV starts below" 1>&2
cat "$TMP_CHECK" 1>&2
echo "$0: Warning: lines with ,NULL, $MANIFEST_CSV ends above" 1>&2
exit 21
exit 22
fi
rm -f "$TMP_CHECK"
@@ -143,11 +156,13 @@ rm -f "$TMP_CHECK"
# sort manifest CSV file
#
sort -t, -k1,1 -k2d,2 -k4g,4 -k3d,3 -k5d,10 "$TMP_CSV" -o "$TMP_CSV"
# sort by: year dir file_path inventory_order everything_else
#
sort -t, -k1,1 -k2d,2 -k4g,4 -k3d,3 -k5d,8 "$TMP_CSV" -o "$TMP_CSV"
status="$?"
if [[ $status -ne 0 ]]; then
printf "$0: ERROR: sort -t, -k1,1 -k2d,2 -k4g,4 -k3d,3 -k5d,10 %s -o %s failed, error: %s\n" "$TMP_CSV" "$TMP_CSV" "$status" 1>&2
exit 22
printf "$0: ERROR: sort -t, -k1,1 -k2d,2 -k4n,4 -k3d,3 -k5d,8 %s -o %s failed, error: %s\n" "$TMP_CSV" "$TMP_CSV" "$status" 1>&2
exit 23
fi
# append newline if last line of manifest CSV file if not empty
@@ -168,7 +183,7 @@ if ! cmp -s "$TMP_CSV" "$MANIFEST_CSV"; then
status="$?"
if [[ $status -ne 0 ]]; then
echo "$0: ERROR: mv -f -v $TMP_CSV $MANIFEST_CSV failed, error: $status" 1>&2
exit 23
exit 24
fi
else
@@ -178,7 +193,7 @@ else
rm -f "$TMP_CSV"
if [[ -e $TMP_CSV ]]; then
echo "$0: ERROR: TMP_CSV cannot be removed for cleanup: $TMP_CSV" 1>&2
exit 24
exit 25
fi
fi
@@ -190,7 +205,7 @@ sed -e '/^# -/d' "$MANIFEST_CSV" |
sort -t, -k1,1 -k2d,2 -k4g,4 -k3d,3 -k5d,10 > "$TMP_CSV"
if [[ ! -s $TMP_CSV ]]; then
echo "$0: ERROR: sorted $MANIFEST_CSV is empty" 1>&2
exit 25
exit 26
fi
if ! cmp -s "$TMP_CSV" "$MANIFEST_CSV"; then
@@ -201,7 +216,7 @@ if ! cmp -s "$TMP_CSV" "$MANIFEST_CSV"; then
status="$?"
if [[ $status -ne 0 ]]; then
echo "$0: ERROR: mv -f -v $TMP_CSV $MANIFEST_CSV failed, error: $status" 1>&2
exit 26
exit 27
fi
else
@@ -211,7 +226,7 @@ else
rm -f "$TMP_CSV"
if [[ -e $TMP_CSV ]]; then
echo "$0: ERROR: TMP_CSV cannot be removed for cleanup: $TMP_CSV" 1>&2
exit 27
exit 28
fi
fi
@@ -220,10 +235,10 @@ fi
export REQUIRED_PATH_LIST="path_list.required.txt"
export TMP_FILE="tmp.$$.$REQUIRED_PATH_LIST"
trap 'rm -f $TMP_FILE; exit' 0 1 2 3 15
awk -F, '$1 !~ "^#" && $5 == "null" { print $1 "/" $2 "/" $3; }' "$MANIFEST_CSV" | sort -t/ > "$TMP_FILE"
awk -F, '$1 !~ "^#" && ( $5 == "true" || $8 == "original source" ) { print $1 "/" $2 "/" $3; }' "$MANIFEST_CSV" | sort -t/ > "$TMP_FILE"
if [[ ! -s $TMP_FILE ]]; then
echo "$0: ERROR: TMP_FILE, for required file list, is empty: $TMP_CSV" 1>&2
exit 28
exit 29
fi
if ! cmp -s "$TMP_FILE" "$REQUIRED_PATH_LIST"; then
@@ -234,7 +249,7 @@ if ! cmp -s "$TMP_FILE" "$REQUIRED_PATH_LIST"; then
status="$?"
if [[ $status -ne 0 ]]; then
echo "$0: ERROR: mv -f -v $TMP_FILE $REQUIRED_PATH_LIST failed, error: $status" 1>&2
exit 29
exit 30
fi
else
@@ -244,7 +259,7 @@ else
rm -f "$TMP_FILE"
if [[ -e $TMP_CSV ]]; then
echo "$0: ERROR: TMP_FILE cannot be removed for cleanup: $TMP_FILE" 1>&2
exit 30
exit 31
fi
fi
@@ -253,10 +268,10 @@ fi
export BUILT_PATH_LIST="path_list.built.txt"
export TMP_FILE="tmp.$$.$BUILT_PATH_LIST"
trap 'rm -f $TMP_FILE; exit' 0 1 2 3 15
awk -F, '$1 !~ "^#" && $5 != "null" { print $1 "/" $2 "/" $3; }' "$MANIFEST_CSV" | sort -t/ > "$TMP_FILE"
awk -F, '$1 !~ "^#" && $5 == "false" && $8 != "original source" { print $1 "/" $2 "/" $3; }' "$MANIFEST_CSV" | sort -t/ > "$TMP_FILE"
if [[ ! -s $TMP_FILE ]]; then
echo "$0: ERROR: TMP_FILE, for built file list, is empty: $TMP_CSV" 1>&2
exit 31
exit 32
fi
if ! cmp -s "$TMP_FILE" "$BUILT_PATH_LIST"; then
@@ -267,7 +282,7 @@ if ! cmp -s "$TMP_FILE" "$BUILT_PATH_LIST"; then
status="$?"
if [[ $status -ne 0 ]]; then
echo "$0: ERROR: mv -f -v $TMP_FILE $BUILT_PATH_LIST failed, error: $status" 1>&2
exit 32
exit 33
fi
else
@@ -277,7 +292,7 @@ else
rm -f "$TMP_FILE"
if [[ -e $TMP_CSV ]]; then
echo "$0: ERROR: TMP_FILE cannot be removed for cleanup: $TMP_FILE" 1>&2
exit 33
exit 34
fi
fi
@@ -289,7 +304,7 @@ trap 'rm -f $TMP_FILE; exit' 0 1 2 3 15
cat "$REQUIRED_PATH_LIST" "$BUILT_PATH_LIST" | sort -t/ > "$TMP_FILE"
if [[ ! -s $TMP_FILE ]]; then
echo "$0: ERROR: TMP_FILE, for manifest file list, is empty: $TMP_CSV" 1>&2
exit 34
exit 35
fi
if ! cmp -s "$TMP_FILE" "$MANIFEST_LIST"; then
@@ -300,7 +315,7 @@ if ! cmp -s "$TMP_FILE" "$MANIFEST_LIST"; then
status="$?"
if [[ $status -ne 0 ]]; then
echo "$0: ERROR: mv -f -v $TMP_FILE $MANIFEST_LIST failed, error: $status" 1>&2
exit 35
exit 36
fi
else
@@ -310,7 +325,7 @@ else
rm -f "$TMP_FILE"
if [[ -e $TMP_CSV ]]; then
echo "$0: ERROR: TMP_FILE cannot be removed for cleanup: $TMP_FILE" 1>&2
exit 35
exit 37
fi
fi
@@ -324,11 +339,11 @@ for csv in author.csv author_wins.csv year_prize.csv; do
#
if [[ ! -f $csv ]]; then
echo "$0: ERROR: CSV is missing: $csv" 1>&2
exit 37
exit 38
fi
if [[ ! -s $csv ]]; then
echo "$0: ERROR: CSV is empty: $csv" 1>&2
exit 38
exit 39
fi
# setup temp CSV file
@@ -337,7 +352,7 @@ for csv in author.csv author_wins.csv year_prize.csv; do
rm -f "$TMP_CSV"
if [[ -e $TMP_CSV ]]; then
echo "$0: ERROR: TMP_CSV cannot be removed: $TMP_CSV" 1>&2
exit 39
exit 40
fi
# remove carriage returns in CSV file
@@ -347,11 +362,11 @@ for csv in author.csv author_wins.csv year_prize.csv; do
status="$?"
if [[ $status -ne 0 ]]; then
printf "$0: ERROR: TMP_CSV: tr -d '\\\015' < %s > %s failed, error: %s\n" "$csv" "$TMP_CSV" "$status" 1>&2
exit 40
exit 41
fi
if [[ ! -s $TMP_CSV ]]; then
printf "$0: ERROR: TMP_CSV: tr -d '\\\015' < %s > %s produced an empty file" "$csv" "$TMP_CSV" 1>&2
exit 41
exit 42
fi
# sort CSV file
@@ -360,7 +375,7 @@ for csv in author.csv author_wins.csv year_prize.csv; do
status="$?"
if [[ $status -ne 0 ]]; then
printf "$0: ERROR: sort -t, -k1d,1 -k2d,2 %s -o %s failed, error: %s\n" "$TMP_CSV" "$TMP_CSV" "$status" 1>&2
exit 42
exit 43
fi
# append newline if last line of CSV file if not empty
@@ -381,7 +396,7 @@ for csv in author.csv author_wins.csv year_prize.csv; do
status="$?"
if [[ $status -ne 0 ]]; then
echo "$0: ERROR: mv -f -v $TMP_CSV $csv failed, error: $status" 1>&2
exit 43
exit 44
fi
else
@@ -391,7 +406,7 @@ for csv in author.csv author_wins.csv year_prize.csv; do
rm -f "$TMP_CSV"
if [[ -e $TMP_CSV ]]; then
echo "$0: ERROR: TMP_CSV cannot be removed for cleanup: $TMP_CSV" 1>&2
exit 44
exit 45
fi
fi
done