add tools to manage .entry.json content

Added `bin/csv2entry.sh` VERSION "1.0 2024-07-19".

In `bin/entry2csv.sh` changed VERSION from "1.0.2 2024-07-14"
to "1.0.3 2024-07-19".

The `bin/entry2csv.sh` tool converts all entry `.entry.json` files
into canonicalized `author_wins.csv`, `manifest.csv`, and `year_prize.csv`
CSV files.

The `bin/csv2entry.sh` tool converts `author_wins.csv`, `manifest.csv`,
and `year_prize.csv` (canonicalizing them if needed)  `.entry.json`
files for all entries.

To change the manifest for one or more entries:

```sh
    bin/entry2csv.sh -v 1
    # ((import `manifest.csv` into a spreadsheet tool))
    # ((modify in the spreadsheet tool, the content as needed))
    # ((export the spreadsheet contents, as the CSV file  `manifest.csv`))
    bin/csv2entry.sh -v 1
    make www
``

To change the award for one or more entries:

```sh
    bin/entry2csv.sh -v 1
    # ((import `year_prize.csv` into a spreadsheet tool))
    # ((modify in the spreadsheet tool, the content as needed))
    # ((export the spreadsheet contents, as the CSV file  `year_prize.csv`))
    bin/csv2entry.sh -v 1
    make www
``

To change the authorship of one or more entries:

```sh
    bin/entry2csv.sh -v 1
    # ((import `author_wins.csv` into a spreadsheet tool))
    # ((modify in the spreadsheet tool, the content as needed))
    # ((export the spreadsheet contents, as the CSV file  `author_wins.csv`))
    bin/csv2entry.sh -v 1
    # ((modify winning_entry_set of `author/author_handle.json` files as needed))
    make www
``

Performed `make www` to accomplish the above and to test under macOS.

TODO: Review and improve/expand as needed, changes that were made to `bin/README.md`.

TODO: Remove the `tmp` directory (after saving contents into private storage).

bin/README.md

@@ -134,6 +134,109 @@ the top level `Makefile` by:
 ```
 
 
+<div id="csv2entry">
+### [csv2entry.sh](%%REPO_URL%%/bin/csv2entry.sh)
+</div>
+
+Convert CSV files into `.entry.json` for all entries.
+
+This tool takes as input, the following CSV files:
+
+- [author_wins.csv](#author_wins_csv) - author_handle followed by all their entry_ids
+- [manifest.csv](#manifest_csv) - information about files under a entry
+- [year_prize.csv](#year_prize_csv) - entry_id followed by the entry's award
+
+This tool updates `.entry.json` files for all entries.
+Only those `.entry.json` files whose content is modified are written.
+
+For example:
+
+``` <!---sh-->
+    bin/csv2entry.sh -v 1
+```
+
+There is no requirement to sort the CSV files, nor convert
+them to UNIX format, nor append a final newline to the file.
+
+This tool will canonicalize CSV files before using them
+as input.  Thus, if one wishes to import the CSV file into
+some spreadsheet such as the [macOS](https://www.apple.com/macos)
+[Numbers](https://www.apple.com/numbers/) spreadsheet,
+modifies the content and final exports back to the CSV file,
+this tool will modify the CSV file (if needed) in order
+to restore the CSV order and other canonicalizing processes.
+
+This tool will flag as an error, any empty fields, fields that are
+an un-quoted _NULL_ or _null_, fields that start with whitespace,
+fields that ends with whitespace, or fields that contain consecutive
+whitespace characters.
+
+
+#### Internal details of `bin/csv2entry.sh`
+
+We first canonicalize the CSV files by replacing any "carriage
+return line feeds" with "newlines.  We also make sure that the CSV
+file ends in a newline.  We do this because some spreadsheet
+applications, when exporting to a CSV file, do not do this.
+
+We also sort the CSV files in the same way that `bin/entry2csv.sh`
+sorts its CSV output files.  We do this in case the
+CSV files were imported into a spreadsheet where their order
+was changed before exporting.  This means one is free
+to order the CSV file content as you wish as this
+tool will reset these CSV file.
+
+Next this tool processes the non-CSV comment lines in manifest.csv.
+The 1st and 2nd fields of [manifest.csv](#manifest_csv) prefer to entry YYYY and
+entry sub-directory (i.e., the YYYY/dir directory under the
+root of the git repository).  From that list of YYYY/dir
+IOCCC entry directories, we will create the `.entry.json` files.
+We only modify those `.entry.json` files when their content changes.
+
+**NOTE**:
+
+While this tool uses `jparse(1)` to verify that the modified
+`.entry.json` contains valid JSON content, this tool does NOT
+perform any semantic checks.  For example, this tool does NOT
+verify that the manifest in the `.entry.json` file matches the
+files in the YYYY/dir directory, or even that the `.entry.json`
+contains a manifest (or any of the other required JSON content).
+
+
+<div id="entry2csv2">
+### [entry2csv.sh](%%REPO_URL%%/bin/entry2csv.sh)
+</div>
+
+This tool takes as input, all entry `.entry.json` files
+and updates 3 CSV files:
+
+- [author_wins.csv](#author_wins_csv) - author_handle followed by all their entry_ids
+- [manifest.csv](#manifest_csv) - information about files under a entry
+- [year_prize.csv](#year_prize_csv) - entry_id followed by the entry's award
+
+The CSV files are written in a canonical UNIX format form.
+
+Only those CSV files files whose content is modified are written.
+
+#### Internal details of `bin/entry2csv.sh`
+
+We generate CSV files from the `.entry.json` files from winning
+IOCCC entries listed under years listed in the `.top` file,
+and in sub-directories listed in the `YYYY/.year` file for the
+given year.  Only those entries so listed are processed.
+
+All IOCCC entry directories must have a `.path` file that lists
+the path of the entry's directory from the TOPDIR.
+
+**NOTE**:
+
+When adding a new IOCCC years entries, the `.top` file MUST
+be updated, and the new IOCCC year `YYYY/.year` files MUST
+reference the directory of the new IOCCC entries.  They
+must also contain a `.path` file that contains the path
+of the IOCCC entry directory from the TOPDIR.
+
+
 <div id="filelist-entry-json-awk">
 ### [filelist.entry.json.awk](%%REPO_URL%%/bin/filelist.entry.json.awk)
 </div>
@@ -1305,6 +1408,113 @@ A string that identifies the winning entry.  The string is of the form:
 ```
 
 
+<div id="csv">
+# CSV files
+</div>
+
+The `bin/csv2entry.sh` and `bin/entry2csv.sh` tools use the following
+3 CSV files.  In the case of `bin/entry2csv.sh`, these 3 CSV files
+are created / updated.  In the case of `bin/csv2entry.sh`, these 3
+CSV files are used as input.
+
+
+<div id="author_wins_csv">
+## author_wins.csv
+</div>
+
+A CSV spreadsheet: one line per `author`.
+
+The first field is an `author_handle`.
+
+The other fields are the `entry_id`'s of all _YYYY/dir_ entry's won by the `author`.
+
+
+<div id="manifest_csv">
+## manifest.csv
+</div>
+
+A CSV spreadsheet contains information about files in _YYYY/dir_ entry directories under year directories.
+This file has the following fields:
+
+1. year:
+
+   IOCCC year as a 4-character string.  Normally this would be a 4 digit year string,
+   however it may also be a string such as "mock".
+
+   NOTE: If year begins with "#", then dir is a comment, and the
+   rest of the row is to be ignored.  Rows of this form do NOT
+   contain manifest information for a file.
+
+2. dir:
+
+   Directory name number the IOCCC year.
+
+3. path:
+
+   Path under the IOCCC/directory.  In a few cases this is a path,
+   not just a simple filename under the IOCCC/directory.
+
+4. inventory_order:
+
+   This number is the rank showing the order
+   that this file is to be listed for the given entry's in
+   the winning entry's `index.html` file.
+
+   If the number is 9 digits or less, then the given file
+   is considered a primary file when listed in the inventory
+   of files for a given winning _YYYY/dir_ entry.
+
+   If the number is 10 digits or more, then the given file
+   is considered a secondary file when listed in the inventory
+   of files for a given winning _YYYY/dir_ entry  We recommend using the
+   value **4294967295** (2^32-1) for secondary files.
+
+5. OK_to_edit:
+
+   If the value is `true`, then the file is one that may be
+   edited directly.
+
+   If the value is `false`, then the file should **NOT** be
+   edited, because the given file is generated by a tool.
+   One should **NOT** modify such a file directly, but
+   instead modify source files (i.e., markdown files,
+   JSON files, etc.)
+
+6. display_as:
+
+   The type of given file.
+
+7. display_via_github
+
+   If `true`, then the contents of the file should be viewed
+   via the GitHub repo.
+
+   If `false`, then the contents of the file should be viewed
+   in the web browser directly.  In some cases this may
+   result in the file being downloaded instead of being displayed.
+
+8. entry_text:
+
+    Any text that should be displayed at the end of line in `index.html`
+    (with a preceding " - "), or null is no such text is to be displayed.
+
+
+NOTE: Cells containing true or false are JSON booleans.
+NOTE: All other cells are JSON strings that need to be double quoted, including the year.
+NOTE: Do not put commas, nor quotes, nor newlines in fields as these are bound to cause problems.
+
+
+<div id="year_prize_csv">
+## year_prize.csv
+</div>
+
+A CSV spreadsheet: one line per _YYYY/dir_ entry directory.
+
+The first field is a `entry_id`.
+
+The second field is the name of the award for given _YYYY/dir_ entry.
+
+
 <!--
 
     Copyright © 1984-2024 by Landon Curt Noll. All Rights Reserved.

bin/entry2csv.sh

@@ -2,6 +2,19 @@
 #
 # entry2csv.sh - convert .entry.json data into CSV files
 #
+# This tool takes as input, all entry `.entry.json` files
+# and updates 3 CSV files:
+#
+# author_wins.csv - author_handle followed by all their entry_ids
+# manifest.csv - information about files under a entry
+# year_prize.csv - entry_id followed by the entry's award
+#
+# The CSV files are written in a canonical UNIX format form.
+#
+# Only those CSV files files whose content is modified are written.
+#
+# Internal details of entry2csv.sh
+#
 # We generate CSV files from the `.entry.json` files from winning
 # IOCCC entries listed under years listed in the `.top` file,
 # and in sub-directories listed in the `YYYY/.year` file for the
@@ -10,11 +23,13 @@
 # All IOCCC entry directories must have a `.path` file that lists
 # the path of the entry's directory from the TOPDIR.
 #
-# NOTE: When adding a new IOCCC years entries, the `.top` file MUST
+# NOTE:
-#	be updated, and the new IOCCC year `YYYY/.year` files MUST
+#
-#	reference the directory of the new IOCCC entries.  They
+# When adding a new IOCCC years entries, the `.top` file MUST
-#	must also contain a `.path` file that contains the path
+# be updated, and the new IOCCC year `YYYY/.year` files MUST
-#	of the IOCCC entry directory from the TOPDIR.
+# reference the directory of the new IOCCC entries.  They
+# must also contain a `.path` file that contains the path
+# of the IOCCC entry directory from the TOPDIR.
 #
 # NOTE: All IOCCC entries MUST have a valid `.entry.json` file.
 #
@@ -42,6 +57,7 @@
 #
 # Share and enjoy! :-)
 
+
 # firewall - run only with a bash that is version 5.1.8 or later
 #
 # The "/usr/bin/env bash" command must result in using a bash that
@@ -100,7 +116,7 @@ shopt -s globstar	# enable ** to match all files and zero or more directories an
 
 # set variables referenced in the usage message
 #
-export VERSION="1.0.2 2024-07-14"
+export VERSION="1.0.3 2024-07-19"
 NAME=$(basename "$0")
 export NAME
 export V_FLAG=0
@@ -998,12 +1014,12 @@ fi
 if [[ -z $NOOP ]]; then
 
     if [[ $V_FLAG -ge 5 ]]; then
-	echo "$0: debug[5]: about to: sort -t, -k1,1 -k2d,2 -k4n,4 -k3d,3 -k5d,8 $TMP_MANIFEST_CSV -o $TMP_MANIFEST_CSV" 1>&2
+	echo "$0: debug[5]: about to: sort -t, -k1,1 -k2d,2 -k4n,4 -k3,3 -k5d,8 $TMP_MANIFEST_CSV -o $TMP_MANIFEST_CSV" 1>&2
     fi
     sort -t, -k1,1 -k2d,2 -k4g,4 -k3d,3 -k5d,8 "$TMP_MANIFEST_CSV" -o "$TMP_MANIFEST_CSV"
     status="$?"
     if [[ status -ne 0 ]]; then
-	echo "$0: ERROR: sort -t, -k1,1 -k2d,2 -k4n,4 -k3d,3 -k5d,8 $TMP_MANIFEST_CSV -o $TMP_MANIFEST_CSV failed," \
+	echo "$0: ERROR: sort -t, -k1,1 -k2d,2 -k4n,4 -k3,3 -k5d,8 $TMP_MANIFEST_CSV -o $TMP_MANIFEST_CSV failed," \
 	     "error code: $status" 1>&2
 	EXIT_CODE=8 # exit 8
     elif [[ $V_FLAG -ge 3 ]]; then

bin/index.html

@@ -459,6 +459,82 @@ need to do so.</p>
 <p>If you wish to run it on all entries, we recommend that this tool be invoked via
 the top level <code>Makefile</code> by:</p>
 <pre><code>    make verify_entry_files</code></pre>
+<div id="csv2entry">
+<h3 id="csv2entry.sh"><a href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/bin/csv2entry.sh">csv2entry.sh</a></h3>
+</div>
+<p>Convert CSV files into <code>.entry.json</code> for all entries.</p>
+<p>This tool takes as input, the following CSV files:</p>
+<ul>
+<li><a href="#author_wins_csv">author_wins.csv</a> - author_handle followed by all their entry_ids</li>
+<li><a href="#manifest_csv">manifest.csv</a> - information about files under a entry</li>
+<li><a href="#year_prize_csv">year_prize.csv</a> - entry_id followed by the entry’s award</li>
+</ul>
+<p>This tool updates <code>.entry.json</code> files for all entries.
+Only those <code>.entry.json</code> files whose content is modified are written.</p>
+<p>For example:</p>
+<pre><code>    bin/csv2entry.sh -v 1</code></pre>
+<p>There is no requirement to sort the CSV files, nor convert
+them to UNIX format, nor append a final newline to the file.</p>
+<p>This tool will canonicalize CSV files before using them
+as input. Thus, if one wishes to import the CSV file into
+some spreadsheet such as the <a href="https://www.apple.com/macos">macOS</a>
+<a href="https://www.apple.com/numbers/">Numbers</a> spreadsheet,
+modifies the content and final exports back to the CSV file,
+this tool will modify the CSV file (if needed) in order
+to restore the CSV order and other canonicalizing processes.</p>
+<p>This tool will flag as an error, any empty fields, fields that are
+an un-quoted <em>NULL</em> or <em>null</em>, fields that start with whitespace,
+fields that ends with whitespace, or fields that contain consecutive
+whitespace characters.</p>
+<h4 id="internal-details-of-bincsv2entry.sh">Internal details of <code>bin/csv2entry.sh</code></h4>
+<p>We first canonicalize the CSV files by replacing any “carriage
+return line feeds” with “newlines. We also make sure that the CSV
+file ends in a newline. We do this because some spreadsheet
+applications, when exporting to a CSV file, do not do this.</p>
+<p>We also sort the CSV files in the same way that <code>bin/entry2csv.sh</code>
+sorts its CSV output files. We do this in case the
+CSV files were imported into a spreadsheet where their order
+was changed before exporting. This means one is free
+to order the CSV file content as you wish as this
+tool will reset these CSV file.</p>
+<p>Next this tool processes the non-CSV comment lines in manifest.csv.
+The 1st and 2nd fields of <a href="#manifest_csv">manifest.csv</a> prefer to entry YYYY and
+entry sub-directory (i.e., the YYYY/dir directory under the
+root of the git repository). From that list of YYYY/dir
+IOCCC entry directories, we will create the <code>.entry.json</code> files.
+We only modify those <code>.entry.json</code> files when their content changes.</p>
+<p><strong>NOTE</strong>:</p>
+<p>While this tool uses <code>jparse(1)</code> to verify that the modified
+<code>.entry.json</code> contains valid JSON content, this tool does NOT
+perform any semantic checks. For example, this tool does NOT
+verify that the manifest in the <code>.entry.json</code> file matches the
+files in the YYYY/dir directory, or even that the <code>.entry.json</code>
+contains a manifest (or any of the other required JSON content).</p>
+<div id="entry2csv2">
+<h3 id="entry2csv.sh"><a href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/bin/entry2csv.sh">entry2csv.sh</a></h3>
+</div>
+<p>This tool takes as input, all entry <code>.entry.json</code> files
+and updates 3 CSV files:</p>
+<ul>
+<li><a href="#author_wins_csv">author_wins.csv</a> - author_handle followed by all their entry_ids</li>
+<li><a href="#manifest_csv">manifest.csv</a> - information about files under a entry</li>
+<li><a href="#year_prize_csv">year_prize.csv</a> - entry_id followed by the entry’s award</li>
+</ul>
+<p>The CSV files are written in a canonical UNIX format form.</p>
+<p>Only those CSV files files whose content is modified are written.</p>
+<h4 id="internal-details-of-binentry2csv.sh">Internal details of <code>bin/entry2csv.sh</code></h4>
+<p>We generate CSV files from the <code>.entry.json</code> files from winning
+IOCCC entries listed under years listed in the <code>.top</code> file,
+and in sub-directories listed in the <code>YYYY/.year</code> file for the
+given year. Only those entries so listed are processed.</p>
+<p>All IOCCC entry directories must have a <code>.path</code> file that lists
+the path of the entry’s directory from the TOPDIR.</p>
+<p><strong>NOTE</strong>:</p>
+<p>When adding a new IOCCC years entries, the <code>.top</code> file MUST
+be updated, and the new IOCCC year <code>YYYY/.year</code> files MUST
+reference the directory of the new IOCCC entries. They
+must also contain a <code>.path</code> file that contains the path
+of the IOCCC entry directory from the TOPDIR.</p>
 <div id="filelist-entry-json-awk">
 <h3 id="filelist.entry.json.awk"><a href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/bin/filelist.entry.json.awk">filelist.entry.json.awk</a></h3>
 </div>
@@ -1110,6 +1186,76 @@ is directly under a <code>year</code> directory.</p>
 </div>
 <p>A string that identifies the winning entry. The string is of the form:</p>
 <pre><code>    year_dir</code></pre>
+<div id="csv">
+<h1 id="csv-files">CSV files</h1>
+</div>
+<p>The <code>bin/csv2entry.sh</code> and <code>bin/entry2csv.sh</code> tools use the following
+3 CSV files. In the case of <code>bin/entry2csv.sh</code>, these 3 CSV files
+are created / updated. In the case of <code>bin/csv2entry.sh</code>, these 3
+CSV files are used as input.</p>
+<div id="author_wins_csv">
+<h2 id="author_wins.csv">author_wins.csv</h2>
+</div>
+<p>A CSV spreadsheet: one line per <code>author</code>.</p>
+<p>The first field is an <code>author_handle</code>.</p>
+<p>The other fields are the <code>entry_id</code>’s of all <em>YYYY/dir</em> entry’s won by the <code>author</code>.</p>
+<div id="manifest_csv">
+<h2 id="manifest.csv">manifest.csv</h2>
+</div>
+<p>A CSV spreadsheet contains information about files in <em>YYYY/dir</em> entry directories under year directories.
+This file has the following fields:</p>
+<ol type="1">
+<li><p>year:</p>
+<p>IOCCC year as a 4-character string. Normally this would be a 4 digit year string,
+however it may also be a string such as “mock”.</p>
+<p>NOTE: If year begins with “#”, then dir is a comment, and the
+rest of the row is to be ignored. Rows of this form do NOT
+contain manifest information for a file.</p></li>
+<li><p>dir:</p>
+<p>Directory name number the IOCCC year.</p></li>
+<li><p>path:</p>
+<p>Path under the IOCCC/directory. In a few cases this is a path,
+not just a simple filename under the IOCCC/directory.</p></li>
+<li><p>inventory_order:</p>
+<p>This number is the rank showing the order
+that this file is to be listed for the given entry’s in
+the winning entry’s <code>index.html</code> file.</p>
+<p>If the number is 9 digits or less, then the given file
+is considered a primary file when listed in the inventory
+of files for a given winning <em>YYYY/dir</em> entry.</p>
+<p>If the number is 10 digits or more, then the given file
+is considered a secondary file when listed in the inventory
+of files for a given winning <em>YYYY/dir</em> entry We recommend using the
+value <strong>4294967295</strong> (2^32-1) for secondary files.</p></li>
+<li><p>OK_to_edit:</p>
+<p>If the value is <code>true</code>, then the file is one that may be
+edited directly.</p>
+<p>If the value is <code>false</code>, then the file should <strong>NOT</strong> be
+edited, because the given file is generated by a tool.
+One should <strong>NOT</strong> modify such a file directly, but
+instead modify source files (i.e., markdown files,
+JSON files, etc.)</p></li>
+<li><p>display_as:</p>
+<p>The type of given file.</p></li>
+<li><p>display_via_github</p>
+<p>If <code>true</code>, then the contents of the file should be viewed
+via the GitHub repo.</p>
+<p>If <code>false</code>, then the contents of the file should be viewed
+in the web browser directly. In some cases this may
+result in the file being downloaded instead of being displayed.</p></li>
+<li><p>entry_text:</p>
+<p>Any text that should be displayed at the end of line in <code>index.html</code>
+(with a preceding ” - “), or null is no such text is to be displayed.</p></li>
+</ol>
+<p>NOTE: Cells containing true or false are JSON booleans.
+NOTE: All other cells are JSON strings that need to be double quoted, including the year.
+NOTE: Do not put commas, nor quotes, nor newlines in fields as these are bound to cause problems.</p>
+<div id="year_prize_csv">
+<h2 id="year_prize.csv">year_prize.csv</h2>
+</div>
+<p>A CSV spreadsheet: one line per <em>YYYY/dir</em> entry directory.</p>
+<p>The first field is a <code>entry_id</code>.</p>
+<p>The second field is the name of the award for given <em>YYYY/dir</em> entry.</p>
 <!--
 
     Copyright © 1984-2024 by Landon Curt Noll. All Rights Reserved.