build index.html for all winners

Updated manifest as per recent file changes.

Updated winner `.winner.json` to reflect the manifest changes.

Added [bin/all_run.sh](bin/all_run.sh) tool to run a command over
all winning entries.

These tools had slight improvements in the help usage message:

* bin/gen-author.sh VERSION="1.3.1 2024-01-16"
* bin/gen-inventory.sh VERSION="1.0.1 2024-01-16"
* bin/pandoc_wrapper.sh VERSION="1.3.1 2024-01-16"
* bin/subst.default.sh VERSION="1.1.1 2024-01-16"
* bin/subst.winner-index.sh VERSION="1.1.1 2024-01-16"

The `bin/readme2index.sh` tool was improved so that
if the `index.html` file exists and the tool forms
the exact same content, then the file `index.html`
file is NOT touched.

* bin/readme2index.sh VERSION="1.1 2024-01-16"

We used the following command to generate `index.html` files
for all winning entries:

```sh
bin/all_run.sh -v 3 bin/readme2index.sh
```

NOTE: We have not (yet) built HTML files above the winner directory
level (e.g., no year level nor top level HTML), nor have be built
non-index HTML files for winners.

NOTE: By default we are running `pandoc(1)` with:

```
--fail-if-warnings=true
```

This means that any problems with markdown to HTML
generates an error.  This is done to be sure that
the HTML conversion is clean.

We had to fix the following `README.md` files:

* `2000/rince/README.md`

The reference [^2] was never used.  We made a guess that [^2] applies
to the same spot as [^1].  Perhaps there is a better spot where
[^2] applies?

NOTE: For every reference, there MUST be some place where that
reference is used.

* 2012/vik/README.md

The string "$...$" is assumed to be a TeX encoded string.

NOTE: Every "$..$" needs to be backslashed as in "\$..\$".

* 2018/burton1/README.md

The reference [2] was duplicated.  We changed the
second [2] to [3].

NOTE: Every reference needs to have a unique number.

* 2020/carlini/README.md

The string "$...$" is assumed to be a TeX encoded string.

NOTE: Every "$..$" needs to be backslashed as in "\$..\$".

We also found 2 errors in the `author/author_handle.json` file set:

* author/Anonymous_2001.json

The `author/Anonymous_2001.json` file was missing.

* author/Jari_Arkko.json

The winner ID for `1989_jar.1` had a typo.
This commit is contained in:
Landon Curt Noll committed 2024-01-17 00:32:39 -08:00
1 parent 893cd915b6
commit c1466f5165
350 files changed
+95242 -4217

No files matched your search

+570 -13
View File
@@ -1,4 +1,6 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<!-- START: two lines up starts content from: inc/top.default.html -->
<!-- -->
<!-- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -->
<!-- !!! DO NOT MODIFY THIS FILE - This file is generated by a tool !!! -->
@@ -6,27 +8,582 @@
<!-- !!! DO NOT MODIFY THIS FILE - This file is generated by a tool !!! -->
<!-- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -->
<!-- -->
<!-- file: 2011/dlowe/index.html created by: fake.gen_index.sh version: 1.0.5 2023-10-31 -->
<html lang="en">
<!-- END: this line ends content from: inc/top.default.html -->
<!-- -->
<!-- This file was formed by the tool: bin/readme2index.sh version: 1.1 2024-01-16 -->
<!-- -->
<!-- START: this line starts content from: inc/head.default.html -->
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>2011 dlowe</title>
<meta name="description" content="IOCCC year: 2011 IOCCC winner: dlowe winner index page">
<meta name="keywords" content="IOCCC,2011,dlowe,winner index page">
<link rel="stylesheet" href="/temp-test-ioccc/ioccc.css">
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<title>2011/dlowe</title>
<meta name="description" content="2011 IOCCC winner dlowe">
<meta name="keywords" content="IOCCC, 2011, IOCCC 2011, IOCCC winner, dlowe">
</head>
<!-- END: this line ends content from: inc/head.default.html -->
<!-- START: this line starts content from: inc/body.default.html -->
<body>
<h1>2011 dlowe</h1>
<!-- END: this line ends content from: inc/body.default.html -->
<!-- START: this line starts content from: inc/header.default.html -->
<p>XXX - fake content until readme2index is working - XXX</p>
<div class="header">
<img src="/temp-test-ioccc/png/ioccc.png" alt="IOCCC image by Matt Zucker" width=300 height=110>
<h1>The International Obfuscated C Code Contest</h1>
<h2>2011/dlowe</h2>
</div>
<hr style="width:50%;text-align:left;margin-left:0">
<!-- END: this line ends content from: inc/header.default.html -->
<!-- START: this line starts content from: inc/topnav.mid.html -->
<!-- Start Nu Html Checker -->
<p><a href="https://validator.w3.org/nu/?doc=https%3A%2F%2Fioccc-src.github.io%2Ftemp-test-ioccc%2F2011%2Fdlowe%2Findex.html">Nu Html check https://ioccc-src.github.io/temp-test-ioccc/2011/dlowe/index.html</a></p>
<!-- End Nu Html Checker -->
<div class="topnav">
<a class="Left" href="../borsanyi/index.html">&larr; borsanyi</a>
<a class="Left" href="../index.html">&uarr; 2011 &uarr;</a>
<a class="Left" href="../eastman/index.html">eastman &rarr;</a>
<a class="Right" href="#inventory">Inventory</a>
</div>
<!-- END: this line ends content from: inc/topnav.mid.html -->
<!-- START: this line starts content from: inc/begin-row.default.html -->
<div class="row">
<!-- END: this line ends content from: inc/begin-row.default.html -->
<!-- START: this line starts content from: inc/begin-leftcolumn.default.html -->
<div class="leftcolumn">
<!-- END: this line ends content from: inc/begin-leftcolumn.default.html -->
<!-- START: this line starts content from: inc/sidenav.default.html -->
<div class="sidenav">
<a href="#content">Jump to contents</a>
</div>
<hr>
<div class="sidenav">
<a href="/temp-test-ioccc/index.html">IOCCC Home</a>
<a href="/temp-test-ioccc/years.html">Winning Entries</a>
<a href="/temp-test-ioccc/winners.html">People who have won</a>
</div>
<hr>
<div class="sidenav">
<a href="/temp-test-ioccc/news.html">IOCCC News</a>
<a href="/temp-test-ioccc/faq.html#submit">How to enter the IOCCC</a>
<a href="/temp-test-ioccc/faq.html">FAQ</a>
</div>
<hr>
<div class="sidenav">
<a href="/temp-test-ioccc/bugs.html">Bugs &amp; (mis)features</a>
<a href="/temp-test-ioccc/faq.html#fix_a_winner">Fixing IOCCC winners</a>
<a href="/temp-test-ioccc/faq.html#fix_web_site">Fixing the web site</a>
<a href="/temp-test-ioccc/faq.html#fix_author">Fixing winner info</a>
<a href="/temp-test-ioccc/thanks-for-help.html">Thanks for the help</a>
</div>
<hr>
<div class="sidenav">
<a href="/temp-test-ioccc/judges.html">The IOCCC Judges</a>
<a href="/temp-test-ioccc/contact.html">Contacting the IOCCC</a>
</div>
<hr>
<div class="sidenav">
<a href="#top">Jump to top</a>
</div>
<!-- END: this line ends content from: inc/sidenav.default.html -->
<!-- START: this line starts content from: inc/end-leftcolumn.default.html -->
</div>
<!-- END: this line ends content from: inc/end-leftcolumn.default.html -->
<!-- START: this line starts content from: inc/begin-rightcolumn.default.html -->
<div class="rightcolumn">
<!-- END: this line ends content from: inc/begin-rightcolumn.default.html -->
<!-- START: this line starts content from: inc/before-content.default.html -->
<a name="content"></a>
<!-- END: this line ends content from: inc/before-content.default.html -->
<!-- START: this line starts content generated by: bin/gen-author.sh -->
<p><a name="author"><strong>Author:</strong></a> <br><br>Name: <a
href="/temp-test-ioccc/winners.html#J_David_Lowe">J. David Lowe</a><br>
Location: <a href="/temp-test-ioccc/location.html#US">US</a></p>
<!-- END: next line ends content generated by: bin/gen-author.sh -->
<h2 id="to-build">To build:</h2>
<div class="sourceCode" id="cb1"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="fu">make</span></span></code></pre></div>
<h3 id="bugs-and-misfeatures">Bugs and (Mis)features:</h3>
<p>The current status of this entry is:</p>
<pre><code>STATUS: missing or dead link - please provide them
STATUS: INABIAF - please **DO NOT** fix</code></pre>
<p>For more detailed information see <a href="/bugs.md#2011-dlowe">2011
dlowe in bugs.md</a>.</p>
<h2 id="to-use">To use:</h2>
<div class="sourceCode" id="cb3"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-</span><span class="op">&lt;</span>n_iterations<span class="op">&gt;</span> corpus1/ <span class="pp">[</span><span class="ss">...</span><span class="pp">]</span> corpus0/ <span class="op">&lt;</span> start.net <span class="op">&gt;</span> trained.net</span></code></pre></div>
<p>NOTE: In the above command, the directory args MUST end in a
<code>/</code>.</p>
<p>Then to use <code>trained.net</code>:</p>
<div class="sourceCode" id="cb4"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> file [file ...] <span class="op">&lt;</span> trained.net</span></code></pre></div>
<h2 id="try">Try:</h2>
<p>Try these pre-trained networks:</p>
<div class="sourceCode" id="cb5"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./try.sh</span></span></code></pre></div>
<p>NOTE: The <code>dlowe-aux-data/</code> directory, which <a
href="try.sh">try.sh</a> refers to, was created by:</p>
<div class="sourceCode" id="cb6"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="fu">tar</span> <span class="at">-jxf</span> dlowe-aux-data.tar.bz2</span></code></pre></div>
<h2 id="judges-remarks">Judges’ remarks:</h2>
<p>This entry is a ghoulish example of a brain (dead?) neural network
classifier. It comes with a set of corpora for you to try.</p>
<h3
id="ioccc-winlose-trained.net"><code>ioccc-winlose-trained.net</code></h3>
<p>You can create your own trained networks. For example, we trained a
network on the C code of entries that were supplied to the 20Th
IOCCC.</p>
<p>The <code>ioccc-winlose-trained.net</code> was trained using the <a
href="earlystop.pl">earlystop.pl</a> tool:</p>
<div class="sourceCode" id="cb7"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rm</span> <span class="at">-f</span> ioccc-winlose-trained.net</span>
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a><span class="ex">./earlystop.pl</span> ioccc-winlose-trained.net ioccc_won_training/ ioccc_lost_training/ ioccc_won_test/ ioccc_lost_test/</span></code></pre></div>
<p>where:</p>
<ul>
<li><p><code>ioccc_won_training/</code> 1/2 of C code of the 20th IOCCC
winners (except for <a href="dlowe.c">dlowe.c</a> from 2011) plus 1/2 of
the C code winners from prior IOCCC contests</p></li>
<li><p><code>ioccc_won_test/</code> 1/2 of C code of the 20th IOCCC
winners (except for <a href="dlowe.c">dlowe.c</a> from 2011) plus 1/2 of
the C code winners from prior IOCCC contests</p></li>
<li><p><code>ioccc_lost_training/</code> 1/2 of C code from entries that
that did not win</p></li>
<li><p><code>ioccc_lost_test/</code> 1/2 of C code from entries that
that did not win</p></li>
</ul>
<p>NOTE: The above directories were not shipped and everything from the
entries that did not win were flushed to <code>/dev/null</code>.</p>
<p>The <a href="earlystop.pl">earlystop.pl</a> tool wrote:</p>
<pre><code>training 1 to 1001...
training 1001 to 2001...
prior error = 56.746974766043; current error = 58.095691135305</code></pre>
<p>The resulting <code>ioccc-winlose-trained.net</code> file was
included in the <a
href="dlowe-aux-data.tar.bz2">dlowe-aux-data.tar.bz2</a> tarball.</p>
<p>The <a href="dlowe.c">dlowe.c</a> was explicitly excluded from this
trailing set, so this test is interesting:</p>
<div class="sourceCode" id="cb9"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> dlowe.c <span class="op">&lt;</span> dlowe-aux-data/ioccc-winlose-trained.net</span></code></pre></div>
<p>NOTE: this is done in <a href="try.sh">try.sh</a>.</p>
<p>The match of <a href="dlowe.c">dlowe.c</a> to the lose/win trailed
network was:</p>
<pre><code>dlowe.c 0.125108</code></pre>
<p>NOTE: The non-artificially intelligent decisions of the IOCCC judges
include random quirks and their non-linear judgment calls not found in
the ioccc trained network. Just because C code is or isn’t scored highly
by this network does not mean it will win or lose a future IOCCC. On the
other hand … it just might!</p>
<h4 id="other-data-sets-to-try">other data sets to try</h4>
<p>While we cannot supply you with the losing IOCCC source code, the
author did supply a number data sets on which to test:</p>
<p>We created the <a
href="dlowe-aux-data/english-trained.net">english-trained.net</a> as
follows:</p>
<div class="sourceCode" id="cb11"><pre
class="sourceCode sh"><code class="sourceCode bash"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-8000</span> dlowe-aux-data/english-1/ dlowe-aux-data/english-0/ <span class="op">&lt;</span> /dev/null <span class="op">&gt;</span> dlowe-aux-data/english-trained.net</span></code></pre></div>
<ul>
<li><p><a href="dlowe-aux-data/english-0/">dlowe-aux-data/english-0/</a>
Non-English (French) text</p></li>
<li><p><a href="dlowe-aux-data/english-1">dlowe-aux-data/english-1/</a>
English (non-French) text</p></li>
</ul>
<p>We created the <a
href="dlowe-aux-data/png-trained.net">png-trained.net</a> as
follows:</p>
<div class="sourceCode" id="cb12"><pre
class="sourceCode sh"><code class="sourceCode bash"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-8000</span> png-1/ png-0/ <span class="op">&lt;</span> /dev/null <span class="op">&gt;</span> dlowe-aux-data/png-trained.net</span></code></pre></div>
<ul>
<li><p><a href="dlowe-aux-data/png-0/">dlowe-aux-data/png-0/</a> Non-png
(gif) images</p></li>
<li><p><a href="dlowe-aux-data/png-1/">dlowe-aux-data/png-1/</a> png
(non-gif) images</p></li>
</ul>
<p>We created the <a
href="dlowe-aux-data/xor-trained.net">xor-trained.net</a> as
follows:</p>
<div class="sourceCode" id="cb13"><pre
class="sourceCode sh"><code class="sourceCode bash"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-8000</span> dlowe-aux-data/xor-1/ dlowe-aux-data/xor-0/ <span class="op">&lt;</span> /dev/null <span class="op">&gt;</span> dlowe-aux-data/xor-trained.net</span></code></pre></div>
<ul>
<li><p><a href="dlowe-aux-data/xor-0/">dlowe-aux-data/xor-0</a> Data
that XORs to 0</p></li>
<li><p><a href="dlowe-aux-data/xor-1/">dlowe-aux-data/xor-1/</a> Data
that XORs to 1</p></li>
</ul>
<p><strong>NOTE</strong>: The directory args <strong>must end in
a</strong> <code>/</code> for them to work.</p>
<h2 id="authors-remarks">Author’s remarks:</h2>
<h3 id="synopsis">Synopsis</h3>
<p>This is an <a
href="https://en.wikipedia.org/wiki/Artificial_intelligence">artificially
intelligent</a> judging tool to help the IOCCC judges. Here’s to
shorter, more frequent contests!</p>
<h3 id="description">Description</h3>
<p>This is a multilayer <a
href="https://en.wikipedia.org/wiki/Perceptron">perceptron</a> (a <a
href="https://en.wikipedia.org/wiki/Feedforward_neural_network">feedforward
artificial neural network</a>) which can be trained, using on-line <a
href="https://en.wikipedia.org/wiki/Backpropagation">backpropagation</a>,
to classify input files.</p>
<p>It has a fixed topology of <code>2^16</code> input neurons, 6 hidden
neurons and 1 output neuron.</p>
<p>The neurons’ activation function is the logistic function
<code>1 / (1 + e ^ -x)</code>.</p>
<h3 id="classifying">Classifying</h3>
<div class="sourceCode" id="cb14"><pre
class="sourceCode sh"><code class="sourceCode bash"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> file [file ...] <span class="op">&lt;</span> trained.net</span></code></pre></div>
<p>To classify files, one specifies a trained network (on
<code>stdin</code>) and one or more files to classify. The program will
output one line per successfully-classified file to <code>stderr</code>,
with the filename and the classification: a number between 0 and 1.</p>
<p>The interpretation of the classification number depends on how the
network was trained, but it’s geared toward interpretation as a
<em>probability</em> or a <em>confidence</em>.</p>
<h3 id="training">Training</h3>
<div class="sourceCode" id="cb15"><pre
class="sourceCode sh"><code class="sourceCode bash"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-</span><span class="op">&lt;</span>n_iterations<span class="op">&gt;</span> corpus1/ <span class="pp">[</span><span class="ss">...</span><span class="pp">]</span> corpus0/ <span class="op">&lt;</span> start.net <span class="op">&gt;</span> end.net</span></code></pre></div>
<p><strong>NOTE</strong>: The directory args <strong>must end in
a</strong> <code>/</code> for them to work.</p>
<p>To train a network, one specifies a starting network (on
<code>stdin</code>), two or more corpora (directories containing
training data), and the number of training iterations to run. The
program will write some progress data to <code>stderr</code> and, when
it’s done, will serialize the updated network to
<code>stdout</code>.</p>
<p>If no input network is given, a random new network will be generated
as a starting point (i.e. provide an empty <code>stdin</code>).</p>
<p>The first corpus will be assigned a target value of <code>1</code>.
The last will be assigned a target value of <code>0</code>. Intervening
directories (if any) will be assigned intermediate target values.</p>
<p>The learning rate is hard-coded as <code>0.3</code>. No momentum
factor is used.</p>
<h4 id="png-corpora">png corpora</h4>
<p>The <a href="dlowe-aux-data/png-1">png-1</a> corpus was obtained by
manually scraping the first results from an <a
href="https://images.google.com"
class="uri">https://images.google.com</a> search for “<a
href="https://www.google.com/search?tbm=isch&amp;sxsrf=APwXEddqIU34dxhpFdjEa8emNJKHOkYXIw:1683049158657&amp;source=hp&amp;biw=1727&amp;bih=952&amp;ei=xkpRZLapJtLKkPIP8bGKkAk&amp;iflsig=AOEireoAAAAAZFFY1nbmEUHC74Cmg3Y5rEY1fV7ocASZ&amp;ved=0ahUKEwi2pPXZltf-AhVSJUQIHfGYApIQ4dUDCAY&amp;uact=5&amp;oq=obfuscate+filetype:png&amp;gs_lcp=CgNpbWcQA1AAWABg9QNoAHAAeACAATuIATuSAQExmAEAoAECoAEBqgELZ3dzLXdpei1pbWc&amp;sclient=img&amp;q=obfuscate&amp;tbs=ift:png">obfuscate
filetype:png</a>”.</p>
<p>The <a href="dlowe-aux-data/png-0">png-0</a> corpus was obtained by
manually scraping the first results from an <a
href="https://images.google.com"
class="uri">https://images.google.com</a> search for “<a
href="https://www.google.com/search?tbm=isch&amp;ved=2ahUKEwiowo7fltf-AhXSOUQIHbJBDp8Q2-cCegQIABAA&amp;oq=obfuscate+filetype:gif&amp;gs_lcp=CgNpbWcQAzoECCMQJzoHCAAQigUQQzoFCAAQgARQ6AxY6AxgpBJoAHAAeACAAUeIAcsBkgEBM5gBAKABAaoBC2d3cy13aXotaW1nwAEB&amp;sclient=img&amp;ei=0UpRZOiVINLzkPIPsoO5-Ak&amp;bih=952&amp;biw=1727&amp;q=obfuscate&amp;tbs=ift:gif">obfuscate
filetype:gif</a>”.</p>
<h5 id="results">Results:</h5>
<ul>
<li>Using these corpora as a training set;</li>
<li>Using several hundred random <code>*.gif</code> and
<code>*.png</code> files from my home computer as a test set;</li>
<li>Using the included <a href="earlystop.pl">earlystop.pl</a> script,
which stopped after 67000 iterations.</li>
<li>Interpreting output of <code>&gt; 0.5</code> as “probably a
<code>.png</code>” and <code>&lt; 0.5</code> as “probably a
<code>.gif</code>”</li>
</ul>
<p>produced a network with about 87% accuracy</p>
<h4 id="english-corpora">English corpora</h4>
<p>The <a href="dlowe-aux-data/english-1">english-1</a> corpus was
obtained by manually scraping the first results from a <a
href="https://www.google.com/webhp?lr=lang_en"
class="uri">https://www.google.com/webhp?lr=lang_en</a> search for “<a
href="https://www.google.com/search?q=paris+filetype%3Atxt&amp;lr=lang_en&amp;tbs=lr%3Alang_1en&amp;sxsrf=APwXEdeHXyUYdUxCYQQDHmJhbHzX9aARSA%3A1683049834430&amp;source=hp&amp;ei=ak1RZJHUGN_DkPIP26OtkA8&amp;iflsig=AOEireoAAAAAZFFbejA0Xfqoq__FXx--Ok0S6GQjnGnA&amp;ved=0ahUKEwiRsZOcmdf-AhXfIUQIHdtRC_IQ4dUDCAo&amp;uact=5&amp;oq=paris+filetype%3Atxt&amp;gs_lcp=Cgdnd3Mtd2l6EANQAFgAYN4DaABwAHgAgAE-iAE-kgEBMZgBAKABAqABAQ&amp;sclient=gws-wiz">paris
filetype:txt</a>”.</p>
<p>The <a href="dlowe-aux-data/english-0">english-0</a> corpus was
obtained by manually scraping the first results from a <a
href="https://www.google.com/webhp?lr=lang_fr"
class="uri">https://www.google.com/webhp?lr=lang_fr</a> search for “<a
href="https://www.google.com/search?q=paris+filetype%3Atxt&amp;lr=lang_fr&amp;tbs=lr%3Alang_1fr&amp;sxsrf=APwXEddcsQ-gMhrqSUJo6omWFUl7jQMASg%3A1683049870736&amp;source=hp&amp;ei=jk1RZLevK_fKkPIPjbWN-AU&amp;iflsig=AOEireoAAAAAZFFbnvExHGkcavB5H8F0xsLAWN2KfYFK&amp;ved=0ahUKEwi3rrutmdf-AhV3JUQIHY1aA18Q4dUDCAo&amp;uact=5&amp;oq=paris+filetype%3Atxt&amp;gs_lcp=Cgdnd3Mtd2l6EANQAFgAYLMDaABwAHgAgAE-iAE-kgEBMZgBAKABAqABAQ&amp;sclient=gws-wiz">paris
filetype:txt</a>”.</p>
<h5 id="results-1">Results:</h5>
<ul>
<li>Using these corpora as a training set;</li>
<li>Using several dozen additional text files (obtained by the same
methods) as a test set;</li>
<li>Using the included <a href="earlystop.pl">earlystop.pl</a> script,
which stopped after 10000 iterations;</li>
<li>Interpreting output of <code>&gt; 0.5</code> as “probably English”
and <code>&lt; 0.5</code> as “probably French”</li>
</ul>
<p>produced a network with 100% accuracy.</p>
<h3 id="xor-corpora">xor corpora</h3>
<p>The <a href="dlowe-aux-data/xor-1">xor-1</a> corpus consists of two
files containing <code>01</code> and <code>10</code> respectively.</p>
<p>The <a href="dlowe-aux-data/xor-0">xor-0</a> corpus consists of two
files containing <code>00</code> and <code>11</code> respectively.</p>
<h5 id="results-2">Results:</h5>
<p>Using these corpora as a training set takes about 8000 iterations to
learn xor to within a tolerance of <code>&lt;0.01</code>.</p>
<h3 id="limitations">Limitations</h3>
<p>The program can’t tell you anything meaningful about files with less
than two bytes in them (I’m looking at you, <a
href="/1994/smr/smr.c">smr.c</a>!)</p>
<p>You must include the trailing directory separator on training
directories (this allows the program to be portable without wasting
precious bytes on figuring out how to concatenate directory and file
names…).</p>
<p>Serialized network files are only portable between systems with the
same <a
href="https://en.wikipedia.org/wiki/Floating-point_arithmetic">floating-point</a>
representation and <a
href="https://en.wikipedia.org/wiki/Endianness">endianness</a>.</p>
<p>Making sure not to <a
href="https://en.wikipedia.org/wiki/Overfitting#Machine_learning">overfit</a>
the network to the training data is a bit of a black art. I have
enclosed <a href="earlystop.pl">earlystop.pl</a>, a wrapper script that
implements a simple ‘early stopping’ algorithm; other techniques are
possible.</p>
<p>Bad input (e.g. nonexistent files, non-numeric number of iterations,
etc.) tends to result in empty output.</p>
<p>Given exactly one corpus, the program will crash or produce
garbage.</p>
<p>Leaks memory and file descriptors while processing files.</p>
<p>Will crash and die horribly if it runs out of memory.</p>
<p>The Microsoft C compiler doesn’t provide a <code>dirent</code> API,
so to get this working on a Windows system you’ll need cygwin+gcc
(tested) or a <code>dirent</code> compatibility library (untested, but
they do exist).</p>
<p>Backpropagation doesn’t always converge: if you play with this long
enough, you’ll eventually have a training session that completely fails
to converge.</p>
<h3 id="obfuscation">Obfuscation</h3>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Zombie">Zombies</a>! (Since
neural networks are modeled after <a
href="https://en.wikipedia.org/wiki/Brain">BRAINS</a>, ya know? And
corpus sounds a lot like corpse. And I have 4- and 7-year-old kids ;)
)</li>
<li>Neural networks are interesting, and while the math isn’t terribly
difficult, their behavior is difficult to fully understand.</li>
<li>Had some extra space in one of the major data structures, and it
seemed a shame to waste it.</li>
<li>Similarly, I think three file pointers ought to be enough for
anyone.</li>
<li>Lots of magic numbers expressed in various ways.</li>
<li>Random abuse of random C trivia.</li>
<li>Some textual changes were made to maximize the classification of
this program by a network trained to recognize <a
href="/winners.html">IOCCC winners</a> ;)</li>
</ul>
<p>… but mostly zombies!</p>
<h2 id="copyright-and-cc-by-sa-4.0-license">Copyright and CC BY-SA 4.0
License:</h2>
<p>This file is Copyright (c) 2023 by Landon Curt Noll. All Rights
Reserved. You are free to share and adapt this file under the terms of
this license:</p>
<pre><code>Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)</code></pre>
<p>For more information, see:
https://creativecommons.org/licenses/by-sa/4.0/</p>
<!-- START: this line starts content generated by: bin/gen-inventory.sh -->
<h1 id="inventory-for-2011dlowe"><a name="inventory"></a>Inventory for
2011/dlowe</h1>
<h2 id="primary-files">Primary files</h2>
<ul>
<li><a href="index.html">index.html</a> - winner home page</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe.c">dlowe.c</a>
- winning source</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/Makefile">Makefile</a>
- winner Makefile</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe.orig.c">dlowe.orig.c</a>
- original source</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/earlystop.pl">earlystop.pl</a>
- wrapper script to stop training early</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/try.sh">try.sh</a>
- how to run</li>
<li><a
href="dlowe-aux-data/png-0/5756029-DMZ-overview-B.gif">dlowe-aux-data/png-0/5756029-DMZ-overview-B.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/M2.gif">dlowe-aux-data/png-0/M2.gif</a> -
Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/Obfuscate.gif">dlowe-aux-data/png-0/Obfuscate.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/figure_49.gif">dlowe-aux-data/png-0/figure_49.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/klockhuset.gif">dlowe-aux-data/png-0/klockhuset.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/obfuscate_1.gif">dlowe-aux-data/png-0/obfuscate_1.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/podcastimage_150215.gif">dlowe-aux-data/png-0/podcastimage_150215.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/prjman1.gif">dlowe-aux-data/png-0/prjman1.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/proguard_obfuscate.gif">dlowe-aux-data/png-0/proguard_obfuscate.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/scripts_encryptor__screnc_-31918.gif">dlowe-aux-data/png-0/scripts_encryptor_<em>screnc</em>-31918.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/visual_panel.gif">dlowe-aux-data/png-0/visual_panel.gif</a>
- Non-png gif images</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/040604.txt">dlowe-aux-data/english-0/040604.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/2001-2002.txt">dlowe-aux-data/english-0/2001-2002.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/CFP_TSI_Graphes_appel.txt">dlowe-aux-data/english-0/CFP_TSI_Graphes_appel.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/CIR_APE_08_doc%5B1%5D.txt">dlowe-aux-data/english-0/CIR_APE_08_doc%5B1%5D.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/admin_fr.txt">dlowe-aux-data/english-0/admin_fr.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/agenda.txt">dlowe-aux-data/english-0/agenda.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/paris.txt">dlowe-aux-data/english-0/paris.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/paris2.txt">dlowe-aux-data/english-0/paris2.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/structures3D-Paris.txt">dlowe-aux-data/english-0/structures3D-Paris.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/02-chronology.txt">dlowe-aux-data/english-1/02-chronology.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/36C215.txt">dlowe-aux-data/english-1/36C215.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/Wireless_Days_2011_CFP.txt">dlowe-aux-data/english-1/Wireless_Days_2011_CFP.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/cpb.txt">dlowe-aux-data/english-1/cpb.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/get-there.txt">dlowe-aux-data/english-1/get-there.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/pa_fv010.txt">dlowe-aux-data/english-1/pa_fv010.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/the_poor_people_of_paris.txt">dlowe-aux-data/english-1/the_poor_people_of_paris.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/utmost.txt">dlowe-aux-data/english-1/utmost.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/walljpar.txt">dlowe-aux-data/english-1/walljpar.txt</a>
- training text for neural networks</li>
<li><a href="dlowe-aux-data/xor-0/00">dlowe-aux-data/xor-0/00</a> -
training digits for neural networks</li>
<li><a href="dlowe-aux-data/xor-0/11">dlowe-aux-data/xor-0/11</a> -
training digits for neural networks</li>
<li><a href="dlowe-aux-data/xor-1/01">dlowe-aux-data/xor-1/01</a> -
training digits for neural networks</li>
<li><a href="dlowe-aux-data/xor-1/10">dlowe-aux-data/xor-1/10</a> -
training digits for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/JavaScript-obfuscation-code.png">dlowe-aux-data/png-1/JavaScript-obfuscation-code.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/control_flow_obfuscation_before.png">dlowe-aux-data/png-1/control_flow_obfuscation_before.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/deceive-inveigle-obfuscate-t-shirts.png">dlowe-aux-data/png-1/deceive-inveigle-obfuscate-t-shirts.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/help_obfuscation1.png">dlowe-aux-data/png-1/help_obfuscation1.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/image_thumb%5B40%5D.png">dlowe-aux-data/png-1/image_thumb[40].png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/jquery_after_deobfuscation.png">dlowe-aux-data/png-1/jquery_after_deobfuscation.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/obfuscate4e.png">dlowe-aux-data/png-1/obfuscate4e.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/obfuscation-decoder.png">dlowe-aux-data/png-1/obfuscation-decoder.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/obfuscation_methods.png">dlowe-aux-data/png-1/obfuscation_methods.png</a>
- training image for neural networks</li>
<li><a href="2011_dlowe.tar.bz2">2011_dlowe.tar.bz2</a> - download
winning source</li>
</ul>
<h2 id="secondary-files">Secondary files</h2>
<ul>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/.gitignore">.gitignore</a>
- list of files that should not be committed under git</li>
<li><a href=".path">.path</a> - directory path from top directory</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/.winner.json">.winner.json</a>
- description of winner in JSON</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/README.md">README.md</a>
- markdown source for index.html</li>
<li><a href="dlowe-aux-data.tar.bz2">dlowe-aux-data.tar.bz2</a> - IOCCC
win lose trained network</li>
<li><a
href="dlowe-aux-data/english-trained.net">dlowe-aux-data/english-trained.net</a>
- trained neural network for English text</li>
<li><a
href="dlowe-aux-data/ioccc-winlose-trained.net">dlowe-aux-data/ioccc-winlose-trained.net</a>
- trained neural network for C code from 20Th IOCCC</li>
<li><a
href="dlowe-aux-data/png-trained.net">dlowe-aux-data/png-trained.net</a>
- trained neural network for png-0 and png-1 images</li>
<li><a
href="dlowe-aux-data/xor-trained.net">dlowe-aux-data/xor-trained.net</a>
- trained neural network for xor-0 and xor-1 data</li>
</ul>
<!-- END: next line ends content generated by: bin/gen-inventory.sh -->
<!-- START: this line starts content from: inc/after-content.default.html -->
<!-- END: this line ends content from: inc/after-content.default.html -->
<!-- START: this line starts content from: inc/end-rightcolumn.default.html -->
</div>
<!-- END: this line ends content from: inc/end-rightcolumn.default.html -->
<!-- START: this line starts content from: inc/end-row.default.html -->
</div>
<!-- END: this line ends content from: inc/end-row.default.html -->
<!-- START: this line starts content from: inc/footer.default.html -->
<div class="footer">
<hr>
<h2>Coda</h2>
<p><a href="https://validator.w3.org/nu/?doc=https%3A%2F%2Fioccc-src.github.io%2Ftemp-test-ioccc%2F2011%2Fdlowe%2Findex.html">Nu HTML check this web page</a></p>
<p><a href="#top">Jump to top</a> <a href="#content">Jump to Content</a> <a href="#inventory">Jump to Inventory</a></p>
</div>
<!-- END: this line ends content from: inc/footer.default.html -->
<!-- START: this line starts content from: inc/bottom.default.html -->
</body>
</html>
<!-- END: this line ends content from: inc/bottom.default.html -->