From 06ceb851e4eede259d86657005192b47f0c31196 Mon Sep 17 00:00:00 2001 From: Cody Boone Ferguson <53008573+xexyl@users.noreply.github.com> Date: Fri, 19 Jul 2024 13:45:24 -0700 Subject: [PATCH] More updates for entry2csv and csv2entry The Makefile rules now have a comment for what they do. The help rule has not been updated. The bin/README.md has had some format fixes and some typo fixes as well. Rebuilt bin/index.html but these two files will have to be checked more thoroughly later. --- Makefile | 6 ++++-- bin/README.md | 57 ++++++++++++++++++++++++++------------------------ bin/index.html | 57 ++++++++++++++++++++++++++------------------------ 3 files changed, 64 insertions(+), 56 deletions(-) diff --git a/Makefile b/Makefile index 5ba00a9d2..37c1ee8ab 100644 --- a/Makefile +++ b/Makefile @@ -472,14 +472,16 @@ find_missing_links: ${FIND_MISSING_LINKS} -v 1 @echo '=-=-=-=-= IOCCC complete make $@ =-=-=-=-=' -# csv2entry.sh +# convert author_wins.csv, manifest.csv and year_prize.csv CSV files to +# .entry.json files. # csv2entry: @echo '=-=-=-=-= IOCCC begin make $@ =-=-=-=-=' ${CVS2ENTRY} -v 1 @echo '=-=-=-=-= IOCCC complete make $@ =-=-=-=-=' -# entry2csv.sh +# convert .entry.json files to author_wins.csv, manifest.csv and year_prize.csv +# CSV files. # entry2csv: @echo '=-=-=-=-= IOCCC begin make $@ =-=-=-=-=' diff --git a/bin/README.md b/bin/README.md index 027130af3..541b69c2b 100644 --- a/bin/README.md +++ b/bin/README.md @@ -142,9 +142,11 @@ Convert CSV files into `.entry.json` for all entries. This tool takes as input, the following CSV files: -- [author_wins.csv](#author_wins_csv) - author_handle followed by all their entry_ids -- [manifest.csv](#manifest_csv) - information about files under a entry -- [year_prize.csv](#year_prize_csv) - entry_id followed by the entry's award +- [author_wins.csv](#author_wins_csv) - `author_handle` followed by all their +`entry_id`s +- [manifest.csv](#manifest_csv) - information about files under an entry +- [year_prize.csv](#year_prize_csv) - `entry_id` followed by the entry's award +title This tool updates `.entry.json` files for all entries. Only those `.entry.json` files whose content is modified are written. @@ -167,7 +169,7 @@ this tool will modify the CSV file (if needed) in order to restore the CSV order and other canonicalizing processes. This tool will flag as an error, any empty fields, fields that are -an un-quoted _NULL_ or _null_, fields that start with whitespace, +an unquoted `NULL` or `null`, fields that start with whitespace, fields that ends with whitespace, or fields that contain consecutive whitespace characters. @@ -179,40 +181,41 @@ return line feeds" with "newlines. We also make sure that the CSV file ends in a newline. We do this because some spreadsheet applications, when exporting to a CSV file, do not do this. -We also sort the CSV files in the same way that `bin/entry2csv.sh` -sorts its CSV output files. We do this in case the -CSV files were imported into a spreadsheet where their order -was changed before exporting. This means one is free -to order the CSV file content as you wish as this -tool will reset these CSV file. +We also sort the CSV files in the same way that [entry2csv](#entry2csv) sorts +its CSV output files. We do this in case the CSV files were imported into a +spreadsheet where their order was changed before exporting. This means one is +free to order the CSV file content as you wish as this tool will reset these CSV +file. -Next this tool processes the non-CSV comment lines in manifest.csv. -The 1st and 2nd fields of [manifest.csv](#manifest_csv) prefer to entry YYYY and -entry sub-directory (i.e., the YYYY/dir directory under the -root of the git repository). From that list of YYYY/dir -IOCCC entry directories, we will create the `.entry.json` files. -We only modify those `.entry.json` files when their content changes. +Next this tool processes the non-CSV comment lines in +[manifest.csv](#manifest_csv). The 1st and 2nd fields of +[manifest.csv](#manifest_csv) refer to entry YYYY and entry subdirectory (i.e., +the `YYYY/dir` directory under the root of the git repository). From that list of +`YYYY/dir` IOCCC entry directories, we will create the `.entry.json` files. We +only modify those `.entry.json` files when their content changes. **NOTE**: While this tool uses `jparse(1)` to verify that the modified -`.entry.json` contains valid JSON content, this tool does NOT -perform any semantic checks. For example, this tool does NOT +`.entry.json` contains valid JSON content, **this tool does NOT +perform any semantic checks**. For example, this tool does NOT verify that the manifest in the `.entry.json` file matches the -files in the YYYY/dir directory, or even that the `.entry.json` +files in the `YYYY/dir directory`, or even that the `.entry.json` contains a manifest (or any of the other required JSON content). -
+
### [entry2csv.sh](%%REPO_URL%%/bin/entry2csv.sh)
This tool takes as input, all entry `.entry.json` files and updates 3 CSV files: -- [author_wins.csv](#author_wins_csv) - author_handle followed by all their entry_ids +- [author_wins.csv](#author_wins_csv) - `author_handle` followed by all their +`entry_id`s - [manifest.csv](#manifest_csv) - information about files under a entry -- [year_prize.csv](#year_prize_csv) - entry_id followed by the entry's award +- [year_prize.csv](#year_prize_csv) - `entry_id` followed by the entry's award +title The CSV files are written in a canonical UNIX format form. @@ -222,19 +225,19 @@ Only those CSV files files whose content is modified are written. We generate CSV files from the `.entry.json` files from winning IOCCC entries listed under years listed in the `.top` file, -and in sub-directories listed in the `YYYY/.year` file for the +and in subdirectories listed in the `YYYY/.year` file for the given year. Only those entries so listed are processed. All IOCCC entry directories must have a `.path` file that lists -the path of the entry's directory from the TOPDIR. +the path of the entry's directory from the `TOPDIR`. **NOTE**: -When adding a new IOCCC years entries, the `.top` file MUST -be updated, and the new IOCCC year `YYYY/.year` files MUST +When adding new IOCCC years' entries, the `.top` file **MUST** +be updated, and the new IOCCC year `YYYY/.year` files **MUST** reference the directory of the new IOCCC entries. They must also contain a `.path` file that contains the path -of the IOCCC entry directory from the TOPDIR. +of the IOCCC entry directory from the `TOPDIR`.
diff --git a/bin/index.html b/bin/index.html index 1a0d5983b..fe2555098 100644 --- a/bin/index.html +++ b/bin/index.html @@ -465,9 +465,11 @@ the top level Makefile by:

Convert CSV files into .entry.json for all entries.

This tool takes as input, the following CSV files:

This tool updates .entry.json files for all entries. Only those .entry.json files whose content is modified are written.

@@ -483,7 +485,7 @@ modifies the content and final exports back to the CSV file, this tool will modify the CSV file (if needed) in order to restore the CSV order and other canonicalizing processes.

This tool will flag as an error, any empty fields, fields that are -an un-quoted NULL or null, fields that start with whitespace, +an unquoted NULL or null, fields that start with whitespace, fields that ends with whitespace, or fields that contain consecutive whitespace characters.

Internal details of bin/csv2entry.sh

@@ -491,50 +493,51 @@ whitespace characters.

return line feeds” with “newlines. We also make sure that the CSV file ends in a newline. We do this because some spreadsheet applications, when exporting to a CSV file, do not do this.

-

We also sort the CSV files in the same way that bin/entry2csv.sh -sorts its CSV output files. We do this in case the -CSV files were imported into a spreadsheet where their order -was changed before exporting. This means one is free -to order the CSV file content as you wish as this -tool will reset these CSV file.

-

Next this tool processes the non-CSV comment lines in manifest.csv. -The 1st and 2nd fields of manifest.csv prefer to entry YYYY and -entry sub-directory (i.e., the YYYY/dir directory under the -root of the git repository). From that list of YYYY/dir -IOCCC entry directories, we will create the .entry.json files. -We only modify those .entry.json files when their content changes.

+

We also sort the CSV files in the same way that entry2csv sorts +its CSV output files. We do this in case the CSV files were imported into a +spreadsheet where their order was changed before exporting. This means one is +free to order the CSV file content as you wish as this tool will reset these CSV +file.

+

Next this tool processes the non-CSV comment lines in +manifest.csv. The 1st and 2nd fields of +manifest.csv refer to entry YYYY and entry subdirectory (i.e., +the YYYY/dir directory under the root of the git repository). From that list of +YYYY/dir IOCCC entry directories, we will create the .entry.json files. We +only modify those .entry.json files when their content changes.

NOTE:

While this tool uses jparse(1) to verify that the modified -.entry.json contains valid JSON content, this tool does NOT -perform any semantic checks. For example, this tool does NOT +.entry.json contains valid JSON content, this tool does NOT +perform any semantic checks. For example, this tool does NOT verify that the manifest in the .entry.json file matches the -files in the YYYY/dir directory, or even that the .entry.json +files in the YYYY/dir directory, or even that the .entry.json contains a manifest (or any of the other required JSON content).

-
+

This tool takes as input, all entry .entry.json files and updates 3 CSV files:

The CSV files are written in a canonical UNIX format form.

Only those CSV files files whose content is modified are written.

Internal details of bin/entry2csv.sh

We generate CSV files from the .entry.json files from winning IOCCC entries listed under years listed in the .top file, -and in sub-directories listed in the YYYY/.year file for the +and in subdirectories listed in the YYYY/.year file for the given year. Only those entries so listed are processed.

All IOCCC entry directories must have a .path file that lists -the path of the entry’s directory from the TOPDIR.

+the path of the entry’s directory from the TOPDIR.

NOTE:

-

When adding a new IOCCC years entries, the .top file MUST -be updated, and the new IOCCC year YYYY/.year files MUST +

When adding new IOCCC years’ entries, the .top file MUST +be updated, and the new IOCCC year YYYY/.year files MUST reference the directory of the new IOCCC entries. They must also contain a .path file that contains the path -of the IOCCC entry directory from the TOPDIR.

+of the IOCCC entry directory from the TOPDIR.

filelist.entry.json.awk