mirror/ioccc
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #2580 from xexyl/entry-csv-make

Browse Source 
Update Makefile for csv2entry and entry2csv
This commit is contained in:
Landon Curt Noll authored and GitHub committed 2024-07-19 14:11:13 -07:00
2 parents 097dd49cc7 06ceb851e4
commit bbd09c7e46
3 files changed
+83 -58
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Makefile
+23 -4
View File
@@ -66,6 +66,9 @@ GEN_STATUS= bin/gen-status.sh
GEN_SITEMAP= bin/gen-sitemap.sh
SORT_GITIGNORE= bin/sort.gitignore.sh
FIND_MISSING_LINKS= bin/find-missing-links.sh
CSV2ENTRY= bin/csv2entry.sh
ENTRY2CSV= bin/entry2csv.sh
#############
@@ -242,10 +245,9 @@ indent.c:
.PHONY: help genpath genfilelist verify_entry_files gen_authors gen_location gen_years \
find_missing_links test entry_index gen_top_html thanks gen_other_html quick_entry_index \
gen_year_index quick_www www \
untar_entry_tarball untar_year_tarball \
form_entry_tarball form_year_tarball tar \
gen_status gen_sitemap sitemap timestamp update
gen_year_index quick_www www untar_entry_tarball untar_year_tarball \
form_entry_tarball form_year_tarball tar gen_status gen_sitemap \
sitemap timestamp update csv2entry entry2csv
# Suggest rules in this section
#
@@ -470,6 +472,23 @@ find_missing_links:
${FIND_MISSING_LINKS} -v 1
@echo '=-=-=-=-= IOCCC complete make $@ =-=-=-=-='
# convert author_wins.csv, manifest.csv and year_prize.csv CSV files to
# .entry.json files.
#
csv2entry:
@echo '=-=-=-=-= IOCCC begin make $@ =-=-=-=-='
${CVS2ENTRY} -v 1
@echo '=-=-=-=-= IOCCC complete make $@ =-=-=-=-='
# convert .entry.json files to author_wins.csv, manifest.csv and year_prize.csv
# CSV files.
#
entry2csv:
@echo '=-=-=-=-= IOCCC begin make $@ =-=-=-=-='
${ENTRY2CSV} -v 1
@echo '=-=-=-=-= IOCCC complete make $@ =-=-=-=-='
# mostly harmless tests
#
test:
bin/README.md
+30 -27
View File
@@ -142,9 +142,11 @@ Convert CSV files into `.entry.json` for all entries.
This tool takes as input, the following CSV files:
- [author_wins.csv](#author_wins_csv) - author_handle followed by all their entry_ids
- [manifest.csv](#manifest_csv) - information about files under a entry
- [year_prize.csv](#year_prize_csv) - entry_id followed by the entry's award
- [author_wins.csv](#author_wins_csv) - `author_handle` followed by all their
`entry_id`s
- [manifest.csv](#manifest_csv) - information about files under an entry
- [year_prize.csv](#year_prize_csv) - `entry_id` followed by the entry's award
title
This tool updates `.entry.json` files for all entries.
Only those `.entry.json` files whose content is modified are written.
@@ -167,7 +169,7 @@ this tool will modify the CSV file (if needed) in order
to restore the CSV order and other canonicalizing processes.
This tool will flag as an error, any empty fields, fields that are
an un-quoted _NULL_ or _null_, fields that start with whitespace,
an unquoted `NULL` or `null`, fields that start with whitespace,
fields that ends with whitespace, or fields that contain consecutive
whitespace characters.
@@ -179,40 +181,41 @@ return line feeds" with "newlines. We also make sure that the CSV
file ends in a newline. We do this because some spreadsheet
applications, when exporting to a CSV file, do not do this.
We also sort the CSV files in the same way that `bin/entry2csv.sh`
sorts its CSV output files. We do this in case the
CSV files were imported into a spreadsheet where their order
was changed before exporting. This means one is free
to order the CSV file content as you wish as this
tool will reset these CSV file.
We also sort the CSV files in the same way that [entry2csv](#entry2csv) sorts
its CSV output files. We do this in case the CSV files were imported into a
spreadsheet where their order was changed before exporting. This means one is
free to order the CSV file content as you wish as this tool will reset these CSV
file.
Next this tool processes the non-CSV comment lines in manifest.csv.
The 1st and 2nd fields of [manifest.csv](#manifest_csv) prefer to entry YYYY and
entry sub-directory (i.e., the YYYY/dir directory under the
root of the git repository). From that list of YYYY/dir
IOCCC entry directories, we will create the `.entry.json` files.
We only modify those `.entry.json` files when their content changes.
Next this tool processes the non-CSV comment lines in
[manifest.csv](#manifest_csv). The 1st and 2nd fields of
[manifest.csv](#manifest_csv) refer to entry YYYY and entry subdirectory (i.e.,
the `YYYY/dir` directory under the root of the git repository). From that list of
`YYYY/dir` IOCCC entry directories, we will create the `.entry.json` files. We
only modify those `.entry.json` files when their content changes.
**NOTE**:
While this tool uses `jparse(1)` to verify that the modified
`.entry.json` contains valid JSON content, this tool does NOT
perform any semantic checks. For example, this tool does NOT
`.entry.json` contains valid JSON content, **this tool does NOT
perform any semantic checks**. For example, this tool does NOT
verify that the manifest in the `.entry.json` file matches the
files in the YYYY/dir directory, or even that the `.entry.json`
files in the `YYYY/dir directory`, or even that the `.entry.json`
contains a manifest (or any of the other required JSON content).
<div id="entry2csv2">
<div id="entry2csv">
### [entry2csv.sh](%%REPO_URL%%/bin/entry2csv.sh)
</div>
This tool takes as input, all entry `.entry.json` files
and updates 3 CSV files:
- [author_wins.csv](#author_wins_csv) - author_handle followed by all their entry_ids
- [author_wins.csv](#author_wins_csv) - `author_handle` followed by all their
`entry_id`s
- [manifest.csv](#manifest_csv) - information about files under a entry
- [year_prize.csv](#year_prize_csv) - entry_id followed by the entry's award
- [year_prize.csv](#year_prize_csv) - `entry_id` followed by the entry's award
title
The CSV files are written in a canonical UNIX format form.
@@ -222,19 +225,19 @@ Only those CSV files files whose content is modified are written.
We generate CSV files from the `.entry.json` files from winning
IOCCC entries listed under years listed in the `.top` file,
and in sub-directories listed in the `YYYY/.year` file for the
and in subdirectories listed in the `YYYY/.year` file for the
given year. Only those entries so listed are processed.
All IOCCC entry directories must have a `.path` file that lists
the path of the entry's directory from the TOPDIR.
the path of the entry's directory from the `TOPDIR`.
**NOTE**:
When adding a new IOCCC years entries, the `.top` file MUST
be updated, and the new IOCCC year `YYYY/.year` files MUST
When adding new IOCCC years' entries, the `.top` file **MUST**
be updated, and the new IOCCC year `YYYY/.year` files **MUST**
reference the directory of the new IOCCC entries. They
must also contain a `.path` file that contains the path
of the IOCCC entry directory from the TOPDIR.
of the IOCCC entry directory from the `TOPDIR`.
<div id="filelist-entry-json-awk">
bin/index.html
+30 -27
View File
@@ -465,9 +465,11 @@ the top level <code>Makefile</code> by:</p>
<p>Convert CSV files into <code>.entry.json</code> for all entries.</p>
<p>This tool takes as input, the following CSV files:</p>
<ul>
<li><a href="#author_wins_csv">author_wins.csv</a> - author_handle followed by all their entry_ids</li>
<li><a href="#manifest_csv">manifest.csv</a> - information about files under a entry</li>
<li><a href="#year_prize_csv">year_prize.csv</a> - entry_id followed by the entry’s award</li>
<li><a href="#author_wins_csv">author_wins.csv</a> - <code>author_handle</code> followed by all their
<code>entry_id</code>s</li>
<li><a href="#manifest_csv">manifest.csv</a> - information about files under an entry</li>
<li><a href="#year_prize_csv">year_prize.csv</a> - <code>entry_id</code> followed by the entry’s award
title</li>
</ul>
<p>This tool updates <code>.entry.json</code> files for all entries.
Only those <code>.entry.json</code> files whose content is modified are written.</p>
@@ -483,7 +485,7 @@ modifies the content and final exports back to the CSV file,
this tool will modify the CSV file (if needed) in order
to restore the CSV order and other canonicalizing processes.</p>
<p>This tool will flag as an error, any empty fields, fields that are
an un-quoted <em>NULL</em> or <em>null</em>, fields that start with whitespace,
an unquoted <code>NULL</code> or <code>null</code>, fields that start with whitespace,
fields that ends with whitespace, or fields that contain consecutive
whitespace characters.</p>
<h4 id="internal-details-of-bincsv2entry.sh">Internal details of <code>bin/csv2entry.sh</code></h4>
@@ -491,50 +493,51 @@ whitespace characters.</p>
return line feeds” with “newlines. We also make sure that the CSV
file ends in a newline. We do this because some spreadsheet
applications, when exporting to a CSV file, do not do this.</p>
<p>We also sort the CSV files in the same way that <code>bin/entry2csv.sh</code>
sorts its CSV output files. We do this in case the
CSV files were imported into a spreadsheet where their order
was changed before exporting. This means one is free
to order the CSV file content as you wish as this
tool will reset these CSV file.</p>
<p>Next this tool processes the non-CSV comment lines in manifest.csv.
The 1st and 2nd fields of <a href="#manifest_csv">manifest.csv</a> prefer to entry YYYY and
entry sub-directory (i.e., the YYYY/dir directory under the
root of the git repository). From that list of YYYY/dir
IOCCC entry directories, we will create the <code>.entry.json</code> files.
We only modify those <code>.entry.json</code> files when their content changes.</p>
<p>We also sort the CSV files in the same way that <a href="#entry2csv">entry2csv</a> sorts
its CSV output files. We do this in case the CSV files were imported into a
spreadsheet where their order was changed before exporting. This means one is
free to order the CSV file content as you wish as this tool will reset these CSV
file.</p>
<p>Next this tool processes the non-CSV comment lines in
<a href="#manifest_csv">manifest.csv</a>. The 1st and 2nd fields of
<a href="#manifest_csv">manifest.csv</a> refer to entry YYYY and entry subdirectory (i.e.,
the <code>YYYY/dir</code> directory under the root of the git repository). From that list of
<code>YYYY/dir</code> IOCCC entry directories, we will create the <code>.entry.json</code> files. We
only modify those <code>.entry.json</code> files when their content changes.</p>
<p><strong>NOTE</strong>:</p>
<p>While this tool uses <code>jparse(1)</code> to verify that the modified
<code>.entry.json</code> contains valid JSON content, this tool does NOT
perform any semantic checks. For example, this tool does NOT
<code>.entry.json</code> contains valid JSON content, <strong>this tool does NOT
perform any semantic checks</strong>. For example, this tool does NOT
verify that the manifest in the <code>.entry.json</code> file matches the
files in the YYYY/dir directory, or even that the <code>.entry.json</code>
files in the <code>YYYY/dir directory</code>, or even that the <code>.entry.json</code>
contains a manifest (or any of the other required JSON content).</p>
<div id="entry2csv2">
<div id="entry2csv">
<h3 id="entry2csv.sh"><a href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/bin/entry2csv.sh">entry2csv.sh</a></h3>
</div>
<p>This tool takes as input, all entry <code>.entry.json</code> files
and updates 3 CSV files:</p>
<ul>
<li><a href="#author_wins_csv">author_wins.csv</a> - author_handle followed by all their entry_ids</li>
<li><a href="#author_wins_csv">author_wins.csv</a> - <code>author_handle</code> followed by all their
<code>entry_id</code>s</li>
<li><a href="#manifest_csv">manifest.csv</a> - information about files under a entry</li>
<li><a href="#year_prize_csv">year_prize.csv</a> - entry_id followed by the entry’s award</li>
<li><a href="#year_prize_csv">year_prize.csv</a> - <code>entry_id</code> followed by the entry’s award
title</li>
</ul>
<p>The CSV files are written in a canonical UNIX format form.</p>
<p>Only those CSV files files whose content is modified are written.</p>
<h4 id="internal-details-of-binentry2csv.sh">Internal details of <code>bin/entry2csv.sh</code></h4>
<p>We generate CSV files from the <code>.entry.json</code> files from winning
IOCCC entries listed under years listed in the <code>.top</code> file,
and in sub-directories listed in the <code>YYYY/.year</code> file for the
and in subdirectories listed in the <code>YYYY/.year</code> file for the
given year. Only those entries so listed are processed.</p>
<p>All IOCCC entry directories must have a <code>.path</code> file that lists
the path of the entry’s directory from the TOPDIR.</p>
the path of the entry’s directory from the <code>TOPDIR</code>.</p>
<p><strong>NOTE</strong>:</p>
<p>When adding a new IOCCC years entries, the <code>.top</code> file MUST
be updated, and the new IOCCC year <code>YYYY/.year</code> files MUST
<p>When adding new IOCCC years’ entries, the <code>.top</code> file <strong>MUST</strong>
be updated, and the new IOCCC year <code>YYYY/.year</code> files <strong>MUST</strong>
reference the directory of the new IOCCC entries. They
must also contain a <code>.path</code> file that contains the path
of the IOCCC entry directory from the TOPDIR.</p>
of the IOCCC entry directory from the <code>TOPDIR</code>.</p>
<div id="filelist-entry-json-awk">
<h3 id="filelist.entry.json.awk"><a href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/bin/filelist.entry.json.awk">filelist.entry.json.awk</a></h3>
</div>