Landon Curt Noll
7712db4a17
Given how problematic leading tabs in markdown files can be,
we have expanded all leading tabs in markdown files by:
```sh
find * -type f -name '*.md' -print |
while read -r file; do
rm -f "$file.tmp" ;
expand < "$file" > "$file.tmp" ;
mv -f -v "$file.tmp" "$file" ;
done
```
Given the previous changes to `bin/pandoc-wrapper.sh` and other recent
commits and recent pull requests, the were were **NO** changes to
any HTML files. This is a **good thing** as the expression goes.
No there are no leading tabs in any markdown files.
A check to catch any re-introduction of leading tabs
into any markdown file is on the way.
BTW: We do not have anything against leading tabs in files,
even though we have had to deal with their impact on forming
HTML from markdown files (as well as the calc source code).
It is just that this is one of those cases where tabs conflict
(sometimes caused by people wanting to change the classic
8-space tab model) with a desired outcome.
Perform `make www` to perform the above and test under macOS.
bin
The bin directory holds tools that build files, such as HTML content, for the official IOCCC website.
For HTML content, the bin directory tools make use of HTML fragments from the inc directory as well as various JSON files and other content from the IOCCC GitHub repo.
bin/ tools
All tools in this directory support a number of options that can be used to get help, diagnose problems, see progress etc. These options are as follows:
Get help / usage string of a tool
If you need to remember the syntax of the tool or get certain notes about
different options, you can use the -h option.
For instance if you want help on the all-run.sh tool from the root directory, you would do:
bin/all-run.sh -h
Set verbosity level of a tool
If, however, you want verbosity, say for debugging purposes or to see what is
going on more, you should use the -v level option. For instance if you wish to
see what is going on with the
quick-readme2index.sh tool, you might do:
bin/quick-readme2index.sh -v 3
to set the verbosity level to 3. The default for verbosity is 0, no
verbosity, though using the top level Makefile will,
for some tools, set a verbosity level.
Get version string of a tool
If you just want to know what version the tool is, you can use the -V option.
For instance to see what version the chk-entry.sh tool
is, you would do:
bin/chk-entry.sh -V
Other notes
These options, and especially -h and -v level, can be very useful to get
basic usage information and to see what is going on when the tool is running.
For more details on each tool, including the ones mentioned above, see below. As
you go through each tool, if you need to understand more of it, we recommend
that you use the -h option on it first.
There are some scripts that are invoked by the bin/md2html.cfg file but some of these tools can be directly invoked as well, should you wish to see their output or if you have some odd need to do so.
Run a command on all winning entries.
For example:
bin/all-run.sh -v 3 bin/quick-readme2index.sh -v 1
Or for example:
bin/all-run.sh -v 3 bin/readme2index.sh -v 1
Run a command on all IOCCC years.
For example:
bin/all-years.sh -v 1 bin/gen-year-index.sh -v 1
Or for example:
bin/all-years.sh -v 1 bin/chk-entry.sh
Check an entry directory to verify that both the files in its manifest
(.entry.json) exist and that no other files exist.
For example:
bin/chk-entry.sh 2020/ferguson1
If you wish to run it on all entries, we recommend that this tool be invoked via
the top level Makefile by:
make verify_entry_files
Generate a list of files in an entry's manifest (the .entry.json file).
For example:
awk -f bin/filelist.entry.json.awk 2020/ferguson1/.entry.json
In this case the command will list all the files of the 2020/ferguson1 winning entry.
Find all markdown links to local files that do not exist.
This tool does NOT check that a remote URL exists. This tool only checks on links to local files.
This tool does NOT check links in a given place in a file. This tool only checks that local files linked by markdown exist.
This tool does NOT check HTML file links. This tool only checks markdown based links.
Usage:
bin/find-missing-links.sh -v 1
If no missing links are found, this tool exits 0 with no output (debug messages not withstanding), otherwise this tool will exit non-zero.
NOTE: If the markdown link is malformed, this tool might generate an error about a file that does exist. If this tool claims that a file is missing that does exist, look for a malformed markdown line and/or use of markdown that is NOT an IOCCC markdown best practice.
See also the IOCCC markdown best practices document for more details.
We recommend that this tool be invoked via the top level Makefile by:
make find_missing_links
Generate the top level authors.html page.
Usage:
bin/gen-authors.sh -v 1
We recommend that this tool be invoked via the top level Makefile by:
make gen_authors
Generate the top level location.html page.
Usage:
bin/gen-location.sh -v 1
We recommend that this tool be invoked via the top level Makefile by:
make gen_location
Generate the HTML files (other than the README.md to index.html files) from markdown files, of all entries.
Usage:
bin/gen-other-html.sh -v 1
We recommend that this tool be invoked via the top level Makefile by:
make gen_other_html
Generate an XML sitemap for the IOCCC website.
Usage:
bin/gen-sitemap.sh -v 1
This would generate the sitemap.xml file.
We recommend that this tool be invoked via the top level Makefile by:
make gen_sitemap
IMPORTANT NOTE: if a file is open with vim or some editor that creates a swap file, this will cause a problem with the output.
Generate status.json according to the modification date of
news.md and/or whether the contest_status is changed
at the command line.
Without an argument, the contest_status is unchanged.
Usage:
bin/gen-status.sh -v 1
To force the contest_status to be closed:
bin/gen-status.sh -v 1 closed
To force the contest_status to be open:
bin/gen-status.sh -v 1 open
To see other valid statuses:
bin/gen-status -h
We recommend that this tool be invoked via the top level Makefile by:
make gen_status
unless the contest_status is to be changed, but since only the judges should
do that, that is not a problem.
Generate a number of top level HTML pages for the IOCCC websites.
Usage:
bin/gen-top-html.sh -v 1
Examples of top level HTML pages built by this tool include:
- CODE_OF_CONDUCT.html
- archive/historic/index.html
- bin/index.html
- bugs.html
- contact.html
- faq.html
- inc/index.html
- index.html
- judges.html
- news.html
- thanks-for-help.html
We recommend that this tool be invoked via the top level Makefile by:
make gen_top_html
Generate an index.html page for a given IOCCC year.
Usage:
bin/gen-year-index.sh -v 1 2020
This would create the 2020/index.html file.
We recommend that this tool be invoked via the top level Makefile by:
make gen_year_index
which will create the index.html for every IOCCC year (1984/index.html,
1985/index.html etc.).
Generate the top level years.html page.
Usage:
bin/gen-years.sh -v 1
We recommend that this tool be invoked via the top level Makefile by:
make gen_years
This is a configuration file for the md2html tool. Although a configuration file it is also something of a script file.
To not down descend into the depths of pandoc(1) and to have to explain how
this file can change all the HTML files, we advise you to let
Smaug (the dragon of
Erebor), famously visited by a
Hobbit and 13
Dwarves in the
T.A. 2941 or 1341
in the Shire-reckoning, slumber as
"here be dragons". If you still need convincing, we wish to remind you of Bilbo
Baggins' famous words:
"Never laugh at live dragons, Bilbo you fool!" he said to himself, and it
became a favourite saying of his later, and passed into a proverb.
This is the primary tool that forms IOCCC generated HTML content from markdown files (permanent markdown files or temporarily generated markdown files) and HTML fragments from the inc directory.
The md2html.cfg configuration file is used by md2html.sh to drive the generation process.
Output author's or authors' related HTML details for an entry's index.html page.
For an example, see the author details in 1984/anonymous.
This tool is used in the bin/md2html.cfg file as part of the md2html tool.
Output the inventory in HTML form for an entry's index.html page. For an example inventory, see the inventory in 1984/anonymous.
This tool is used in the bin/md2html.cfg file as part of the md2html tool.
output-year-index.sh
Output the inventory for a given year's winning entries in HTML form. In other words in 1984 it would list, as links, the four winning entries, which you can see directly at 1984 inventory.
This tool is used in the bin/md2html.cfg file as part of the md2html tool.
pandoc-wrapper.sh
Wrapper tool to run pandoc(1).
Build an entry's index.html file if the entry directory
does not have a non-empty index.hmtl file, or if either
.entry.json or README.md is newer than the index.hmtl file.
This is useful when only a few entries have been
modified (resulting in an updated .entry.json file)
or if the README.md of a few entries have been changed: while the
readme2index.sh script takes a few seconds
to run for a few entries, when applied to 300+ entries, the extra time can add
up.
If only a few index.hmtl files need updating, then
this command will only briefly pause while the
readme2index.sh can take much longer.
Usage:
# For all entries:
bin/all-run.sh -v 3 bin/quick-readme2index.sh -v 1
# For an individual entry:
bin/quick-readme2index.sh -v 1 2020/ferguson2
NOTE: This command assumes that the relative
modification times for README.md, .entry.json,
and index.html are correct. If in doubt, use
readme2index.sh.
We recommend that this tool be invoked via the top level Makefile by:
make quick_entry_index
Convert an entry's README.md into its index.html file.
Usage:
bin/readme2index.sh -v 3 1984/mullender
To build index.html files for all entries:
bin/all-run.sh -v 3 bin/readme2index.sh -v 1
We recommend that this tool be invoked via the top level Makefile by:
make entry_index
Convert status.json into HTML.
This tool is a 'before tool' (-b tool) that is intended
to be used by gen-status.sh.
Sort .gitignore content from stdin to stdout.
We sort with lines starting with # first.
We sort with lines starting with * second.
We sort with lines that do not start with [#!*] third.
We sort with lines starting with ! fourth.
This tool is used by sort.gitignore.sh.
Sort a .gitignore in a entry directory.
Usage:
bin/sort.gitignore.sh -v 1 YYYY/dir
Suggested usage (for all .gitignore files):
bin/all-run.sh bin/sort.gitignore.sh -v 1
We recommend that this tool be invoked via the top level Makefile by:
make sort_gitignore
Print default substitutions.
This tool is used in the bin/md2html.cfg file.
Print substitutions for an entry's index.html.
This tool is used in the bin/md2html.cfg file.
Output substitutions for navbar on behalf of an entry.
This tool is used in subst.entry-index.sh.
Print substitutions for a year level index.html.
This tool is used in the bin/md2html.cfg file.
Output substitutions for navbar on behalf of a year level index.html.
This tool is used in subst.year-index.sh.
Form a compressed tarball for an entry.
Usage:
bin/tar-entry.sh -v 1 YYYY/dir
Suggested usage:
bin/all-run.sh -v 3 bin/tar-entry.sh -v 1 -W
Form a compressed tarball for an IOCCC year.
Usage:
bin/tar-year.sh -v 1 YYYY
Suggested usage:
bin/all-years.sh -v 3 bin/tar-year.sh -v 1 -W
We recommend that this tool be invoked via the top level Makefile by:
make form_year_tarball
Untar an entry's compressed tarball.
Usage:
bin/untar-entry.sh -v 1 YYYY/dir
Suggested usage:
bin/all-run.sh -v 3 bin/untar-entry.sh -v 1
We recommend that this tool be invoked via the top level Makefile by:
make untar_entry_tarball
Untar an IOCCC year's compressed tarball.
Usage:
bin/untar-year.sh -v 1 YYYY
Suggested usage:
bin/all-years.sh -v 3 bin/untar-year.sh -v 1
We recommend that this tool be invoked via the top level Makefile by:
make untar_year_tarball
The md2html.sh tool is the primary tool that is used to form all IOCCC related HTML pages for the official IOCCC web site.
Nearly all IOCCC related HTML pages are built from markdown files, from either permanent markdown files or temporary generated markdown files, as well as HTML fragments from the inc directory.
Most HTML content is built from permanent markdown files, such as a README.md
markdown file found in each entry directory. Some HTML content are generated
from temporary markdown files. These temporary markdown files are produced by
tools in the bin directory and exist only while the tool is running.
In addition to converting markdown to HTML, the canonical way that HTML content is built uses, by default, files from the inc directory, of the file name form:
*.default.html
By using command line options of the form:
-H phase=name
where phase is the name of an HTML phase,
a non-default file may be used. For example:
-H navbar=up2index
will cause inc/navbar.up2index.html instead of inc/navbar.default.html to be used during the navbar HTML phase.
If name is dot (i.e., .), then the given HTML phase is skipped.
For example:
-H footer=.
will cause no HTML content to be produced during the footer HTML phase.
The following HTML phase files are used to build HTML content:
- inc/top.default.html
- inc/head.default.html
- inc/body.default.html
- inc/topbar.default.html
- inc/header.default.html
- inc/navbar.default.html
- inc/before-content.default.html
Phases 7-19 are reserved for future use.
- ((insert 'before tool' output))
- ((insert 'pandoc wrapper tool' output))
- ((insert 'after tool' output))
Phases 23-29 are reserved for future use.
Phases 33-39 are reserved for future use.
In all of the above HTML phase numbers, symbols of the form %%TOKEN%% are
substituted.
Substitutions of the form %%TOKEN%% are performed on included HTML content,
content generated by the 'before tool' (-b tool),
content generated by the 'after tool' (-a tool),
as well as the markdown content given to the pandoc wrapper tool (-p tool).
It is an error, unless -S is given, for any phase, except phases HTML
20-29, to not substitute all %%TOKEN%%s. For phases HTML 20-29, any
%%TOKEN%% that is not substituted are passed thru without substitution.
See the tool readme2index.sh for an example of how HTML phases are implemented.
The command options are evaluated in the following getopt phases:
In getopt phase 0 we parse command line options and save all arguments for the
end of the final command line.
Depending on the program, the argument at the end is directory under topdir, or it is a file under topdir.
getopt phase 0 is the only getopt phase where -d topdir and -c md2html.cfg may be used.
In getopt phase 1 we execute each -o "output tool" from getopt phase 0,
parsing the output as a command line.
The -o "output tool" may be given optstr from "-O tool=optstr", followed
by the phase 0 argument.
The -o "output tool" is not allowed to output another "-o tool" or a "-O tool=optstr". Instead a -o "output tool" may execute another -o "output tool" and merge the output into its own.
In getopt phase 2 we parse the cfg_options from the first md2html.cfg line
matched by a saved argument.
The match is made with the (possibly modified) phase 0 argument.
In getopt phase 3 we execute each -o "output tool" from getopt phase 2,
parsing the output as a command line.
The -o "output tool" may be given optstr from "-O tool=optstr", followed
by the phase 0 argument.
The -o "output tool" is not allowed to output another "-o tool" or a "-O tool=optstr". Instead a -o "output tool" may execute another -o "output tool" and merge the output into its own.
Most command line options override earlier copies of the same option. However
in the case of -H phase=name, a later -H phase=name only overrides an
earlier use of -H for the same phase only. Also in the case of -s token=value, a later -s token=value only overrides an earlier use of -s for
the same token only.
An "output tool" may be used to add certain additional command line options to be processed.
IMPORTANT: An "output tool" will print each command line option / argument on a separate line. So for example, if an "output tool" wishes to convey:
-s TITLE='IOCCC entry locations' -H navbar=top-of-site
The "output tool" would output the following 4 lines:
-s
TITLE='IOCCC entry locations'
-H
navbar=top-of-site
The command line options printed by an "output tool" are processed after all command line options, and all options from a matching line from the md2html.cfg file, and before any filename arguments on the command line.
For example:
command [command_options ..] [cfg_options ..] [output_tool_options ..] [--] [filename_arg ..]
Here the command first processes any command line options, then it processes
cfg_options obtained from this file from a first match of a given file_glob,
then it processes output tool (i.e., -o tool) options found in
command_options, and cfg_options, then an optional -- (end of all
options), then zero of more filename_arg filename arguments.
The -o tool must NOT output any -o tool options as -o tool is NOT
RECURSIVE. A -o tool is free, however, to execute other -o tool output
tools and merge the output from those tools into its own.
When -E exitcode is evaluated, the application will exit with the exitcode
value. If one wishes to also output a message to stderr, the -e string must
come BEFORE any -E exitcode in the command line.
All tokens (i.e., strings of the form %%token%%) MUST be substituted (by
some -s token=value) in all HTML output, except during phase numbers 20-29
(i.e., except before tool output, pandoc wrapper output, and after tool
output), or the command will exit non-zero, unless -S is given. If -S is
given, only a warning about non-substituted tokens will be written to stderr.
The command line of tools in the bin directory, and perhaps modified via the md2html config file may change to using a different filename for a given phase.
For example when forming the HTML from
2020/ferguson1/chocolate-cake.md,
a different navbar navigation bar is needed. So instead of the
usual top navigation bar that normally directs people to the previous
entry for the year, or go up to the year page, or to the next entry
for the year, a top navigation bar to just go up to the entry's
main page is needed. A line in the md2html config file
that refers to
2020/ferguson1/chocolate-cake.md may
specify use of navbar.up2index.html (as navbar.up2index)instead of using the
navbar.default.html (navbar.default.html) default.
The HTML phase may be skipped resulting in no HTML output during a given phase and furthermore, forming no HTML content from a given markdown file altogether.
See comments in the md2html config file for details. See also, the tool readme2index.sh for an example of how such command lines are used.
Use CAUTION when modifying inc/ files
Some of the files under this directory are used to form MOST of the HTML content on the official IOCCC website.
These files are used to form MOST of the HTML content on the experimental website.
... and in particular files under inc that are of the form (called default HTML files) ...:
*.default.html
... contain default content used to form IOCCC HTML / IOCCC web pages.
Instead of editing the default HTML files in order to fix a special web page, consider making a copy of the default file and modifying the md2html config file to refer to the copy instead. That way your special case situation will not impact MOST of the HTML content.
Why we do not use certain well-known HTML technologies
You may wonder why we need these files in the first place. You may wonder why we even need the tools in the bin directory when there other solutions available to form web pages.
Here are some reasons why we are using these files and special tools to create HTML content / IOCCC web pages:
We host official IOCCC website via GitHub pages. As of the time this file written, only static web pages are supported.
We cannot use server side include
We use static web pages, so use of "server side include" is not available to the IOCCC.
For example, Apache SSI #include does not work on GitHub
pages.
We cannot use a back-end database
We use static web pages, so use of a "back-end database" is not available to the IOCCC.
The GitHub pages have the distributed server capacity needed to handle the huge download volume or Slashdot effect that happens when a IOCCC entries are published.
Most so-called low cost web hosting sites have a somewhat hidden excessive bandwidth charge and/or cap bandwidth when the volume gets too high, and/or do not have the infrastructure that can handle the Slashdot effect.
The IOCCC had experience with people offering web mirrors and even free hosting only to find that the generous benefactor moves on, or their service is purchased by someone else who is not so generous. While it is possible that GitHub might someday suffer a similar fate, for the time being we are betting that GitHub will remain willing to host the IOCCC.
The official IOCCC website is, after all, primarily C source code with some supporting documentation (sometimes :-) ). As such it is a natural fit for GitHub and GitHub pages.
We cannot use the HTML <object> element
The <object> HTML element does not work for our needs.
HTML elements do not extend into the content that they include. For example, menu bars (see the ioccc.css stylesheet) will not operate under an HTML element.
We cannot use the HTML <embed> element
The <embed> HTML element does not work for our needs.
This element wants one to specify the width and height in pixels.
Use of a percentage is not officially supported even if some browsers
might do so. Our Responsive Web Design in the
ioccc.css stylesheet
needs to be responsive to small-sized cell phone-like screens,
mid-sized table-like screens, as well as large-sized desktop-like screens.
Specifying a width and height in pixels will not work well in
all of those screen size contexts.
We cannot use the HTML <iframe> element
The <iframe> HTML element does not work for our needs.
This element wants one to specify the width and height in pixels. Use of a
percentage is not officially supported even if some browsers might do so. Our
Responsive Web Design in the ioccc.css stylesheet
needs to be responsive to small-sized cell phone-like screens, mid-sized
table-like screens, as well as large-sized desktop-like screens. Specifying a
width and height in pixels will not work well in all of those screen size
contexts.
We cannot use JavaScript to include content
We do not use JavaScript to include HTML content.
While the IOCCC may use JavaScript in the future to directly render things like C source code, we will do so in such a way that someone will be able to view official IOCCC website content with JavaScript disabled.
The IOCCC will NOT MANDATE USE OF JavaScript to view official IOCCC web site (except for some mobile devices for the menu).
For this reason, we cannot use JavaScript include HTML content.
The following IOCCC terms apply to tools, JSON files, and the directory structure of this repo.
An individual who was an author of at least one winning IOCCC entry.
Some authors have submitted more than one IOCCC entry that won. Some winning
IOCCC entries have more than one author. In that case we might use the word
authors.
An author_handle is string that refers to a given author and is unique to the
IOCCC. Each author has EXACTLY ONE author_handle.
For each author_handle, there will be a JSON file located at:
author/author_handle.json
where author_handle is the author's author_handle.
Because the author_handle is used to form a JSON filename, the string must be
a POSIX safe string with the addition of + (for technical reasons beyond this
document). In particular, the author_handle must be an ASCII
string that matches this regexp:
^[0-9A-Za-z][0-9A-Za-z._+-]*$"
Default author_handles do not have multiple consecutive _ (underscore)
characters. Nevertheless, multiple consecutive _ (underscore) characters are
allowed. Contest submitters who wish to override the author_handle may do so.
The author_handle is derived from the name of the author. While there is a
algorithm that maps the name of the author (which can contain any UTF-8
characters) into a default author_handle string, those who submit an entry to
the IOCCC are free to choose a different author_handle string if they so
desire.
An author who has won a previous IOCCC is encouraged to reuse their
author_handle so that new winning entries can be associated with the same
author.
For an anonymous author, their handle is one of these forms:
Anonymous_year
or:
Anonymous_year.digits
The latter form is in case there are more than one anonymous author in a given year.
NOTE: even if the directory name is not anonymous the above rules apply as in
in the case of 2005/anon.
Anonymous author_handle's match this regexp:
Anonymous_[0-9][0-9][0-9][0-9][.0-9]*$
An IOCCC entry that won an award for a given IOCCC.
An entry has one or more authors.
Each entry is contained under its own directory. The parent of the entry
directory is the year's directory.
While in most cases an entry consists of files under an entry's directory,
there are a few cases where an entry directory contains subdirectories.
Use of subdirectories under an entry directory is discouraged and
may be limited to previous entrys that used them.
NOTE: In the past, the term winner was used in instead of today's term of
entry.
A year is a 4 character string. Most years are 4-digit strings that
represent the year. Some special year strings are possible, such as mock.
Non-numeric year strings are lower case.
A year string matches this regexp:
[0-9a-z][0-9a-z][0-9a-z][0-9a-z]
The year directories reside directly below the top level directory.
Each year directory will have a file under it named .year.
The contents of the .year file is the year string of the directory. For
instance, 2020/.year lists, one directory per line,
the directories of the winning entries of that year. For instance, for 2020:
$ cat 2020/.year
2020/burton
2020/carlini
2020/endoh1
2020/endoh2
2020/endoh3
2020/ferguson1
2020/ferguson2
2020/giles
2020/kurdyukov1
2020/kurdyukov2
2020/kurdyukov3
2020/kurdyukov4
2020/otterness
2020/tsoj
2020/yang
A dir is a POSIX safe string that holds an entry.
A dir is a string that matches this regexp:
^[a-z][0-9a-z.-]*$
Each entry is under its own individual directory. This directory
is directly under a year directory.
A string that identifies the winning entry. The string is of the form:
year_dir