This is a HUGE commit and this task is not done. There are still some
changes that might need to be discussed and since so much is done here I
think it's better to let this go through and then make sure everything
is okay. When the remaining references are discussed they can also be
changed.
I did NOT include changes in the tmp/OLD_manifest directory.
Some of the changes in this commit (it might be exhaustive but it might
not be - not sure):
- In the Makefiles the WINNERS was renamed AUTHORS.
- References to winners.html were changed to authors.html.
- The FAQ named anchor 'fix_a_winner' was renamed 'fix_an_entry'.
- The JSON fields (or whatever they're called) 'winner_id' and
'winner_JSON_format_version' were changed so that 'winner' is 'entry'.
- In scripts for entries 'IOCCC winner' was changed to 'IOCCC entry'.
- 'winners_text' is now 'entries_text'. It might be better to be
'entry_text' as it's per entry but this was not noticed at first. Since
this is easy to fix I am holding off on this for the meantime.
- The script tmp/update-manifest.sh now has the commit log messages
changed to the correct terms as well.
The index.html files were updated as well with any of the above
changes.
There are some references that should NOT be updated. Two examples
caught (and maybe more have slipped through that will have to be changed
back): in 1984/anonymous/README.md the author wrote back to Landon after
being told about the famous tattoo and they used the words:
.. my entry will be the first IOCCC winner ..
but since this comes from the author it should remain so.
The other one is that winners.html should still refer to winners.html as
it informs people that it's been moved to authors.html (it does redirect
too but even so if it was changed it would defeat the purpose of the
message).
With paths like authors.html I tried to copy paste so that hopefully
there are no typos. It is possible that I did not end up doing this with
all but these can be fixed when discovered.
Added `bin/bin/tar-entry.sh` to build a compressed tarball for an
entry. The compressed tarball is formed by gnu tar with options
so that the same entry file content will always produce the same
compressed tarball, byte for byte.
As some files are missing from some entries, one should
use the `-W` option as in:
```sh
bin/tar-entry.sh -v 5 -W 2020/ferguson1
```
Added `make form_entry_tarball` to the top level `Makefile`
to build entry tarballs even when files are still missing.
Added `make verify_entry_files` to verify that all entries have all
of their files listed in their manifest, plus the top level ioccc.css
and var.mk files. As file entry files are still missing (due to
HTML files that need to be built), the overall `make quick_www` and
`make www` do **NOT** (yet) exit when the `make verify_entry_files`
fails.
Improved `bin/chk-entry.sh` by removing some variables
that were not needed.
TODO: Improve `bin/` tool exit codes as the current use is a bit silly.
TODO: Clean out some variables used in `bin/` tools as some of then
are only set printed in debug statements.
TODO: Clean up usage messages, some of the options listed to not
match the `getopts(1)` parsing. In some cases the option needs to
be parsed but needs to be ignored by the tool because another tool
automatically passes along certain options that really have no
effect. Such ignored options need to be show as "This option is
ignored" in the usage message.
The -Wno-foo options were in the CDEFINE variable which although works
it is not congruent with the other Makefiles and it is slightly more
confusing (even though not confusing).
Added `bin/all-years.sh` to run a command over all years.
Added `bin/gen-year-index.sh` to build an `index.html`
for a given year.
Added support code to build an `index.html`
for a given year.
Added `make gen_year_index` to to level Makefile
and related rules.
Using the above, added `index.html` files for all IOCCC years.
Updated `bin/README.md` according to the above.
Improved comments in `bin/all-run.sh`.
Exported TOPDIR in all `bin/` tools that use TOPDIR.
Removed incorrect lines in `bin/quick-readme2index.sh`
usage message.
Improve -v 3 printed important variables messages,
in order that important variables are exported,
from all `bin/` tools.
Rebuild all HTML pages via `make www`.
We add to the top level Makefile, some initial rules that
will facilitate the building of HTML content.
The `bin/md2html.sh` tool as been updated to output HTML comments
about the output during HTML phases 20, 21 and 22.
All entry `index.html` files were updated with some additional HTML
comments as a result of the above. The top level `location.html`,
`years.html` and `authors.html` have had HTML comments added also
a result.
Sorry (tm Canada 🇨🇦) for the mass update of entry `index.html` files.
Hopefully the times when all of them (often) need to be updated
will soon draw to a close.
A _YYYY/.filelist_ is a simple list of files for a given IOCCC year
that need to be present when building a _YYYY_ level compressed
tarball. The list is relative to _docroot_ (top level repo directory).
We do not need a complex JSON file for such a simple list of files.
We only care about checking for the existence of files when forming
_YYYY_ level compressed tarball.
Renamed `.winner.json` to `.entry.json` as per recent decisions made for issue #2156.
JSON element _winner_JSON_format_version_ renamed to _entry_JSON_format_version_
in `.winner.json`.
Changed _entry_JSON_format_version_ from "1.0 2024-01-14" to "1.1 2024-02-11".
JSON element _winner_id_ renamed to _entry_id_ in `.entry.json`
JSON element _winner_text renamed to _entry_text_ in `.entry.json`
JSON element _winner_id_ renamed to _entry_id_ in `author/author_handle.json` files.
Changed _author_JSON_format_version_ from "1.0 2023-06-10" to "1.1 2024-02-11".
Renamed `tmp/gen_winner_json.sh` to `tmp/gen_entry_json.sh`.
Renamed `tmp/example.dot_winner.json` to `tmp/example.dot_entry.json`.
Renamed `tmp/winner_id.txt` to `tmp/entry_id.txt`.
Renamed `tmp/winner_dir.txt` to `tmp/entry_dir.txt`.
Moved outdated `tmp/awards_check.sh` into `tmp/OLD_manifest/awards_check.sh`.
Moved outdated `tmp/awards_check.sh` into `tmp/OLD_manifest/awards_check.sh`.
Moved outdated `tmp/src/winner_name` into `tmp/OLD_manifest/winner_name`.
Moved outdated `tmp/sql` into `tmp/OLD_manifest/sql`.
Moved outdated `tmp/template-footer.html` into `tmp/OLD_manifest/template-footer.html`.
Moved outdated `tmp/template-header.html` into `tmp/OLD_manifest/template-header.html`.
Moved outdated `tmp/template-top.html` into `tmp/OLD_manifest/template-top.html`.
Moved outdated `tmp/test.html` into `tmp/OLD_manifest/test.html`.
Moved outdated `tmp/fake.gen_index.sh` into `tmp/OLD_manifest/fake.gen_index.sh`.
Moved outdated `tmp/old-ioccc.css` into `tmp/OLD_manifest/old-ioccc.css`.
Removed winner_name from `tmp/src/Makefile`.
Updated `tmp/example.dot_entry.json` to reflect the above JSON changes.
Updated `tmp/manifest.numbers` to reflect the above JSON changes.
Updated `tmp/README.md` to reflect the above JSON changes.
Sorted `tmp/README.md` by existing `tmp` filenames.
Updated all `.entry.json` files to reflect the above JSON changes.
Updated all `author/author_handle.json` files to reflect the above JSON changes.
Added new tool `bin/gen-location.sh` to form the top level file `location.html`
from `author/author_handle.json` files!
Added new tool `bin/gen-years.sh` to form the top level file `years.html`
using `.top`, `.year` and `.entry.json' files!
Added new tool `bin/gen-authors.sh` to form a new top level file
`/authors.html` from `author/author_handle.json` and related
`.entry.json' files!
The old `winners.html` file now redirects to the new `authors.html` file.
Renamed `bin/gen-author.sh` to `bin/output-index-author.sh`.
Renamed `bin/subst.winner-index.sh` to `subst.entry-index.sh`.
Renamed `bin/subst.winner-index.awk` to `subst.entry-index.awk`.
Renamed `bin/manifest.entry.json.awk` to `bin/manifest.entry.json.awk`.
Renamed `bin/gen-inventory.sh` to `bin/output-index-inventory.sh`.
Renamed `bin/sort.gitignore` to `bin/sort.gitignore.sh`.
Renamed `bin/sgi` to `bin/sgi.sh`.
The core logic of converting markdown to HTML has been
generalized into a new `bin/md2html.sh` tool.
The `bin/readme2index.sh` tool now calls the
new `bin/md2html.sh` tool to form entry's `index.html` files.
The `bin/quick-readme2index.sh` tool now calls the new `bin/md2html.sh`
tool, unless the `index.html` is newer than both `.entry.json` and
`README.md`, to form entry `index.html` files.
Increased, slightly, the speed of the `bin/quick-readme2index.sh` tool
with the entry's `index.html` file does not need to be updated.
The `bin/quick-readme2index.sh` tool will touch the `index.html` file,
even if the `bin/md2html.sh` tool determines that the `index.html` file
did not change, so that the `index.html` file will not appear
out of date with respect to `.entry.json`.
Slightly improved the look and feel of `index.html` files for entries.
The logic of `bin/all-run.sh` has been improved. The initial
processing, from first bash version check, thru the -N test, serves
as the model for the `bin/readme2index.sh` tool and the
`bin/quick-readme2index.sh` tool. There tools now share
a common set of command line options.
Changed the format of `inc/md2html.cfg` so that for a given "_file_glob_"
match, options follow, one per line. The "_file_glob_" is on a line
by itself. The option lines that follow must be intended:
```
[12][0-9][0-9][0-9]/README.md
-H
topnav=year
-o
bin/subst.default.sh
-o
bin/subst.year.sh
```
Added "-D ./" to top level markdown files in `inc/md2html.cfg`.
The above intending is performed with 1 or more whitespace
characters, although we use a leading tab "convention"
out of deference to Makefile formats. :-)
Once a new "_file_glob_" is found, the set of any previous
option lines is said to have ended.
The general argument processing logic now makes used of shell arrays.
Added checks in the `bin/md2html.sh` tool to disallow a "_file_glob_"
of . (dot) as this is used internally to help the shell process the
new `inc/md2html.cfg` format.
Changed `bin/all-run.sh` VERSION from "1.1.2 2024-01-20"
to "1.3 2024-02-11".
Changed `bin/output-index-author.sh` VERSION from "1.3.1 2024-01-16"
to "1.4 2024-02-11".
Changed `bin/pandoc-wrapper.sh` VERSION from "1.3.2 2024-01-19"
to "1.4 2024-02-05".
Changed `bin/quick-readme2index.sh` VERSION from "1.0 2024-01-20"
to "1.1 2024-02-11".
Changed `bin/readme2index.sh` VERSION from "1.1.1 2024-01-20"
to "1.2 2024-02-11".
Changed `bin/subst.default.sh` VERSION from "1.1.1 2024-01-16"
to "1.2 2024-02-05".
Changed `bin/subst.entry-index.sh` VERSION from "1.1.1 2024-01-16
to "1.2 2024-02-11".
Changed `bin/output-index-inventory.sh` VERSION from "1.0.1 2024-01-16"
to "1.1 2024-02-11".
Changed `bin/manifest.entry.json.awk` from "1.0 2024-01-14"
to "1.1 2024-02-11".
Changed `bin/sort.gitignore.sh` from "1.0 2024-02-01"
to "1.1 2024-02-11".
Changed substitution symbol _%%DOCROOT_URL_SLASH%%_ to _%%DOCROOT_SLASH%%_
in various "inc/*.html" files.
Moved setting of symbol _%%DOCROOT_SLASH%%_ from `bin/subst.entry-index.sh`
to `bin/subst.default.sh`.
Changed the variable _$DOCROOT_URL_SLASH_ to _$DOCROOT_SLASH_, in
bin tools that use it, to be more descriptive.
Removed `-P pandoc_opts` from `bin/*.sh` command lines.
The PANDOC_WRAPPER_OPTSTR value is only used in `bin/pandoc-wrapper.sh`.
Removed `-B optstr` and `-A optstr` from the `bin/md2html.sh` command line.
Removed `[-O tool=optstr ..]` from the `bin/md2html.sh` command line.
Removed old use of `-U top_url` from command lines.
Add `-w site_url` as sort of a "replacement" for the old `-U top_url`.
Added a new `-U URL` to `bin/gen-author.sh`, `bin/output-index-inventory.sh`,
`bin/md2html.sh`, `bin/subst.default.sh`, and `bin/subst.entry-index.sh`
to allow for the Nu HTML check can use the encoded absolute URL of
the HTML page.
Added `-t tagline` to most command lines.
Added `-T md2html.sh` to most command lines.
Added `-D docroot/` to most command lines to set DOCROOT_SLASH.
The DOCROOT_SLASH value defaults to "../../" for most in tools,
suitable for HTML files under a `YYYY/dir` directory, except for
`bin/gen-location.sh` and `gen-entries-html.sh` (as this produces a
top level HTML file) and `bin/md2html.sh` (forms files at any level)
who use "./" instead.
In the case of `bin/output-index-inventory.sh`, `bin/quick-readme2index.sh`,
`bin/readme2index.sh`, and `bin/all-run.sh`
Helped make the use of exit code in bin tools a bit more consistent.
Fixed a number of typos in the bin tools.
Added comments for lower level tools such as
`manifest.entry.json.awk`, `bin/subst.default.sh`,
`bin/subst.entry-index.sh`, and bin/`manifest.entry.json.awk` to
indicate how they are usually invoked by other tools.
Changed the comments that a new `bin/md2html.sh` tool (that was the
`bin/readme2index.sh` tool) would add to a entry's `index.html` file.
No longer will it add the tool version number to the comment. Thus,
a change in tool version will not force all entry `index.html`
files to be modified. Now both the tool adds two comments, one
is the _tagline_ to indicate how the markdown content was generated,
and one for the new `bin/md2html.sh` tool. For example:
```
<!-- The original markdown content was formed by the tool: bin/quick-readme2index.sh -->
<!-- The HTML was converted from markdown by the tool: bin/md2html.sh -->
```
Improve the debugging, especially in regards to changing important
exported shell variables, in `bin/md2html.sh`. Other bin tools
have their exported shell variables improved somewhat as well.
The `bin/README.md` now describes the general _converting markdown
to HTML_ procedure including _getopt phases_ and _HTML phases_.
Content from `inc/md2html.cfg` and `bin/readme2index.sh` that related
to this topic has been centralized into the `bin/README.md` file.
Improved formatting of the `bin/README.md` file. Updated it as needed
according to the above.
Sorted `2011/.year`, `2018/.year`, and `2019/.year` to be in dictionary order.
Adjusted ${WINNERS} in year Makefile to use dictionary order so
that `make genpath` will produce `.year` files in dictionary order.
FYI: We tested this massive set of changes as best as we could.
Nevertheless we might have missing something in the JSON changes,
`.entry.json` renames, manifest updates, code comments, etc.
We sort with lines starting with # first.
We sort with lines starting with * second.
We sort with lines that do not start with [#!*] third.
We sort with lines starting with ! fourth.
Example usage in vim with a keyboard macro <escape>f:
```vim
:map ^[f 1G!G../../bin/sgi^M
```
or via .vimrc:
```vimrc
" sort the entire .gitignore file using ../../bin/sgi
:map ^[f 1G!G../../bin/sgi^M
```
NOTE: The ^[ above is ((Escape)) and ^M is ((Control-M)).
Example usage in shell:
```sh
rm -f t && ../../bin/sgi < .gitignore > t && mv -f t .gitignore
```
Suggested usage:
```sh
bin/all-run.sh bin/sort.gitignore -v 1
```
Sorted all winner .gitignore according to this method.
Sorted all year level .gitignore according to this method.
Sorted top level .gitignore according to this method.
Improved `all-run.sh` and `readme2index.sh` so that
certain command line options (e.g., -p, -P, -u, -U ...)
are only passed along to other tools if they are
given on the command line.
all-run.sh VERSION changed from "1.1.1 2024-01-19"
to "1.1.2 2024-01-20",
readme2index.sh VERSION changed from "1.1 2024-01-16"
to "1.1.1 2024-01-20".
Added `quick-readme2index.sh` that only calls
`readme2index.sh` if the `index.html` is not
a non-empty file, or of either `.winner.json`
and/or `README.md` is newer than `index.html`.
Updated all `index.html` files because the
the file contains a comment that includes
the newly modified readme2index.sh VERSION.
The try.alt.sh script is slightly different from try.sh in that the
order is reversed.
The try.sh script now at the end always starts out in a sequence of
northern hemisphere then southern hemisphere so that one can guarantee
to get that form. The try.alt.sh does the same but starts out with the
southern hemisphere first.
The bugs.md file is all that's needed; the information is already there
in bugs.md and the purpose of having the 'STATUS: ...' lines in the bugs
and (mis)features section of the README.md files was to not have that.
Still having the reference to DISPLAY being unset being a problem seems
like a good idea.
Updated manifest as per recent file changes.
Updated winner `.winner.json` to reflect the manifest changes.
Added [bin/all_run.sh](bin/all_run.sh) tool to run a command over
all winning entries.
These tools had slight improvements in the help usage message:
* bin/gen-author.sh VERSION="1.3.1 2024-01-16"
* bin/gen-inventory.sh VERSION="1.0.1 2024-01-16"
* bin/pandoc_wrapper.sh VERSION="1.3.1 2024-01-16"
* bin/subst.default.sh VERSION="1.1.1 2024-01-16"
* bin/subst.winner-index.sh VERSION="1.1.1 2024-01-16"
The `bin/readme2index.sh` tool was improved so that
if the `index.html` file exists and the tool forms
the exact same content, then the file `index.html`
file is NOT touched.
* bin/readme2index.sh VERSION="1.1 2024-01-16"
We used the following command to generate `index.html` files
for all winning entries:
```sh
bin/all_run.sh -v 3 bin/readme2index.sh
```
NOTE: We have not (yet) built HTML files above the winner directory
level (e.g., no year level nor top level HTML), nor have be built
non-index HTML files for winners.
NOTE: By default we are running `pandoc(1)` with:
```
--fail-if-warnings=true
```
This means that any problems with markdown to HTML
generates an error. This is done to be sure that
the HTML conversion is clean.
We had to fix the following `README.md` files:
* `2000/rince/README.md`
The reference [^2] was never used. We made a guess that [^2] applies
to the same spot as [^1]. Perhaps there is a better spot where
[^2] applies?
NOTE: For every reference, there MUST be some place where that
reference is used.
* 2012/vik/README.md
The string "$...$" is assumed to be a TeX encoded string.
NOTE: Every "$..$" needs to be backslashed as in "\$..\$".
* 2018/burton1/README.md
The reference [2] was duplicated. We changed the
second [2] to [3].
NOTE: Every reference needs to have a unique number.
* 2020/carlini/README.md
The string "$...$" is assumed to be a TeX encoded string.
NOTE: Every "$..$" needs to be backslashed as in "\$..\$".
We also found 2 errors in the `author/author_handle.json` file set:
* author/Anonymous_2001.json
The `author/Anonymous_2001.json` file was missing.
* author/Jari_Arkko.json
The winner ID for `1989_jar.1` had a typo.
The initial release of `bin/readme2index.sh` tool that converts
`READMD.md` and various JSON files such as `.winner.json`
as well as "author/author_handle.json" files.
Applied the `readme2index.sh` tool to generate `2020/ferguson1/index.html`.
We generated the `2020/ferguson1/index.html` by executing:
```sh
./bin/readme2index.sh -v 3 2020/ferguson1
```
NOTE: This above tool requires **pandoc to be installed**.
See the [pandoc web site](https://pandoc.org) for how
to install. We used _HomeBrew_ to install it,
but there is also a _macports_ process as well.
NOTE: While the above web page does have formatting issues, it is
a first attempt data driven programmatic web page generation.
Some adjustments can be made by mostly improving `ioccc.css`,
and some perhaps by modifying how `bin/pandoc_wrapper.sh`
invokes pandoc. But probably most of the improvements can
be made by fixing and improving `ioccc.css`.
TODO: Fix and improve the `2020/ferguson1/index.html`
web page by fixing and improving `ioccc.css`.
Moved scripts and tools needed to generate HTML pages
into a new `/bin` subdirectory.
TODO: Consider getting rid of the [build-ioccc
repo](https://github.com/ioccc-src/build-ioccc) and just use the
`/bin` subdirectory.
NOTE: Some links in the `2020/ferguson1/index.html`
web page do not (yet) exist.
TODO: Once the basic format of the `2020/ferguson1/index.html` web
page is somewhat reasonable, generate `index.html` pages for other
winners. Generate the year level `index.html' files.
Generate the top level HTML files, including the missing
`location.html` file.
Greatly simplified the `.winner.json` file format,
removing a number of fields that were not strictly needed
for generating an inventory on a `index.html` page.
Updated various versions:
* `bin/gen-author.sh` VERSION="1.3 2024-01-14"
* `bin/gen-inventory.sh` VERSION="1.0 2024-01-14"
* `bin/manifest.winner.json.awk` VERSION="1.0 2024-01-14"
* `bin/pandoc_wrapper.sh` VERSION="1.3 2024-01-14"
* `bin/readme2index.sh` VERSION="1.0 2024-01-14"
* `bin/subst.default.sh` VERSION="1.1 2024-01-14"
* `bin/subst.winner-index.sh` VERSION="1.1 2024-01-14"
* `tmp/gen_path_list.found.sh` VERSION="1.1 2024-01-14"
* `tmp/gen_winner_json.sh` VERSION="1.1 2024-01-14"
The `.winner.json` file format from winner_JSON_format_version
"0.5 2023-11-04" to "1.0 2024-01-14".
All `.winner.json` files have been updated to use the
new `.winner.json` file format.
The manifest (`tmp/manifest.numbers` and `tmp/manifest.csv`) has
updated to use the new `.winner.json` file format.
The following procedure was used to generate the
`tmp/manifest.csv` manifest file:
0. open [manifest.numbers](manifest.numbers) in numbers: modify if/as needed
NOTE: Please try to keep the [manifest.numbers](manifest.numbers) spreadsheet
in order order. Use Organize -> Sort -> ((Sort Now)) to sort.
NOTE: Empty fields in the [manifest.numbers](manifest.numbers) spreadsheet
are not allowed. By default, a cell that is empty is colored
as a warning.
1. File -> Export To -> CSV...
* 1a. `[ ]` Include table names (unset)
* 1b. Text Encoding: Unicode (UTF-8)
* 1c. Click ((Save..))
2. Save As: manifest.csv
* 2a. Select the `tmp` directory
3. Click ((Export))
* 3a. If needed click ((Replace))
4. Execute ./tmp/[fix_csv.sh](fix_csv.sh)
NOTE: You must run the [fix_csv.sh](fix_csv.sh) in the above
step in order to make the CSV file usable by other tools.
NOTE: If no changes are needed in the above step, the
original file content will be restored.
5. Execute ./tmp/[gen_path_list.found.sh](gen_path_list.found.sh)
NOTE: The above step silently performs a `make clobber` at the
top level, and so it takes a bit of time.
NOTE: If no changes are needed in the above step, the
original file content will be restored.
6. Execute ./tmp/[check_path_list.sh](check_path_list.sh)
NOTE: The above step prints nothing and for an exit 0, then
all is well. If there is a problem, such as files
found that are not part of the manifest, or files
in the manifest that are not found, then an error
message will be printed and the tool will exit non-zero.
If errors are reported, go back to step 0 and
correct the errors.
7. Execute ./tmp/[run_all.sh](run_all.sh) ./gen_winner_json.sh
NOTE: The above step will take a few seconds to run. It will rebuild
all `.winner.json` files for all winners.
NOTE: If no changes are needed in the above step, the
original file content will be restored.
8. Execute `git status` and inspect changes
NOTE: Check in changes if and as needed.
NOTE: In a number of shell scripts under `bin/*.sh`,
and in the `bin/manifest.winner.json.awk` script
you will find comments of the form:
```
```
This is a note that when tools such as `jnamval` (and `jval` and `jfmt`) are
ready, these gross hacks should be replaced with calls to those tools.
Until they are ready, some must be careful with the layout of JSON files.
A number of bugs and typos have been fixed.
NOTE: Fixing typos in the files touched by this comment is welcome.
This includes scripts, data files, text files, etc.
The _inventory_order_, _display_as_, _display_via_github_. and
_winners_text_ fields of the [manifest.numbers](manifest.numbers)
spreadsheet were just set to have some initial values.
As they impact the inventory displayed on `index.html` pages,
their content needs to be improved.
NOTE: Improving the above mentioned fields of the
[manifest.numbers](manifest.numbers) spreadsheet is welcome.
Landon Curt Noll