build index.html for all winners
Updated manifest as per recent file changes. Updated winner `.winner.json` to reflect the manifest changes. Added [bin/all_run.sh](bin/all_run.sh) tool to run a command over all winning entries. These tools had slight improvements in the help usage message: * bin/gen-author.sh VERSION="1.3.1 2024-01-16" * bin/gen-inventory.sh VERSION="1.0.1 2024-01-16" * bin/pandoc_wrapper.sh VERSION="1.3.1 2024-01-16" * bin/subst.default.sh VERSION="1.1.1 2024-01-16" * bin/subst.winner-index.sh VERSION="1.1.1 2024-01-16" The `bin/readme2index.sh` tool was improved so that if the `index.html` file exists and the tool forms the exact same content, then the file `index.html` file is NOT touched. * bin/readme2index.sh VERSION="1.1 2024-01-16" We used the following command to generate `index.html` files for all winning entries: ```sh bin/all_run.sh -v 3 bin/readme2index.sh ``` NOTE: We have not (yet) built HTML files above the winner directory level (e.g., no year level nor top level HTML), nor have be built non-index HTML files for winners. NOTE: By default we are running `pandoc(1)` with: ``` --fail-if-warnings=true ``` This means that any problems with markdown to HTML generates an error. This is done to be sure that the HTML conversion is clean. We had to fix the following `README.md` files: * `2000/rince/README.md` The reference [^2] was never used. We made a guess that [^2] applies to the same spot as [^1]. Perhaps there is a better spot where [^2] applies? NOTE: For every reference, there MUST be some place where that reference is used. * 2012/vik/README.md The string "$...$" is assumed to be a TeX encoded string. NOTE: Every "$..$" needs to be backslashed as in "\$..\$". * 2018/burton1/README.md The reference [2] was duplicated. We changed the second [2] to [3]. NOTE: Every reference needs to have a unique number. * 2020/carlini/README.md The string "$...$" is assumed to be a TeX encoded string. NOTE: Every "$..$" needs to be backslashed as in "\$..\$". We also found 2 errors in the `author/author_handle.json` file set: * author/Anonymous_2001.json The `author/Anonymous_2001.json` file was missing. * author/Jari_Arkko.json The winner ID for `1989_jar.1` had a typo.
This commit is contained in:
1 parent
893cd915b6
commit
c1466f5165
350 files changed
+95242
-4217
No files matched your search
+570
-13
@@ -1,4 +1,6 @@
|
||||
<!DOCTYPE html>
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
||||
<!-- START: two lines up starts content from: inc/top.default.html -->
|
||||
<!-- -->
|
||||
<!-- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -->
|
||||
<!-- !!! DO NOT MODIFY THIS FILE - This file is generated by a tool !!! -->
|
||||
@@ -6,27 +8,582 @@
|
||||
<!-- !!! DO NOT MODIFY THIS FILE - This file is generated by a tool !!! -->
|
||||
<!-- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -->
|
||||
<!-- -->
|
||||
<!-- file: 2011/dlowe/index.html created by: fake.gen_index.sh version: 1.0.5 2023-10-31 -->
|
||||
<html lang="en">
|
||||
<!-- END: this line ends content from: inc/top.default.html -->
|
||||
<!-- -->
|
||||
<!-- This file was formed by the tool: bin/readme2index.sh version: 1.1 2024-01-16 -->
|
||||
<!-- -->
|
||||
<!-- START: this line starts content from: inc/head.default.html -->
|
||||
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>2011 dlowe</title>
|
||||
<meta name="description" content="IOCCC year: 2011 IOCCC winner: dlowe winner index page">
|
||||
<meta name="keywords" content="IOCCC,2011,dlowe,winner index page">
|
||||
<link rel="stylesheet" href="/temp-test-ioccc/ioccc.css">
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
<title>2011/dlowe</title>
|
||||
<meta name="description" content="2011 IOCCC winner dlowe">
|
||||
<meta name="keywords" content="IOCCC, 2011, IOCCC 2011, IOCCC winner, dlowe">
|
||||
</head>
|
||||
|
||||
<!-- END: this line ends content from: inc/head.default.html -->
|
||||
<!-- START: this line starts content from: inc/body.default.html -->
|
||||
|
||||
<body>
|
||||
|
||||
<h1>2011 dlowe</h1>
|
||||
<!-- END: this line ends content from: inc/body.default.html -->
|
||||
<!-- START: this line starts content from: inc/header.default.html -->
|
||||
|
||||
<p>XXX - fake content until readme2index is working - XXX</p>
|
||||
<div class="header">
|
||||
<img src="/temp-test-ioccc/png/ioccc.png" alt="IOCCC image by Matt Zucker" width=300 height=110>
|
||||
<h1>The International Obfuscated C Code Contest</h1>
|
||||
<h2>2011/dlowe</h2>
|
||||
</div>
|
||||
|
||||
<hr style="width:50%;text-align:left;margin-left:0">
|
||||
<!-- END: this line ends content from: inc/header.default.html -->
|
||||
<!-- START: this line starts content from: inc/topnav.mid.html -->
|
||||
|
||||
<!-- Start Nu Html Checker -->
|
||||
<p><a href="https://validator.w3.org/nu/?doc=https%3A%2F%2Fioccc-src.github.io%2Ftemp-test-ioccc%2F2011%2Fdlowe%2Findex.html">Nu Html check https://ioccc-src.github.io/temp-test-ioccc/2011/dlowe/index.html</a></p>
|
||||
<!-- End Nu Html Checker -->
|
||||
<div class="topnav">
|
||||
<a class="Left" href="../borsanyi/index.html">← borsanyi</a>
|
||||
<a class="Left" href="../index.html">↑ 2011 ↑</a>
|
||||
<a class="Left" href="../eastman/index.html">eastman →</a>
|
||||
<a class="Right" href="#inventory">Inventory</a>
|
||||
</div>
|
||||
|
||||
<!-- END: this line ends content from: inc/topnav.mid.html -->
|
||||
<!-- START: this line starts content from: inc/begin-row.default.html -->
|
||||
|
||||
<div class="row">
|
||||
|
||||
<!-- END: this line ends content from: inc/begin-row.default.html -->
|
||||
<!-- START: this line starts content from: inc/begin-leftcolumn.default.html -->
|
||||
|
||||
<div class="leftcolumn">
|
||||
|
||||
<!-- END: this line ends content from: inc/begin-leftcolumn.default.html -->
|
||||
<!-- START: this line starts content from: inc/sidenav.default.html -->
|
||||
|
||||
<div class="sidenav">
|
||||
<a href="#content">Jump to contents</a>
|
||||
</div>
|
||||
<hr>
|
||||
<div class="sidenav">
|
||||
<a href="/temp-test-ioccc/index.html">IOCCC Home</a>
|
||||
<a href="/temp-test-ioccc/years.html">Winning Entries</a>
|
||||
<a href="/temp-test-ioccc/winners.html">People who have won</a>
|
||||
</div>
|
||||
<hr>
|
||||
<div class="sidenav">
|
||||
<a href="/temp-test-ioccc/news.html">IOCCC News</a>
|
||||
<a href="/temp-test-ioccc/faq.html#submit">How to enter the IOCCC</a>
|
||||
<a href="/temp-test-ioccc/faq.html">FAQ</a>
|
||||
</div>
|
||||
<hr>
|
||||
<div class="sidenav">
|
||||
<a href="/temp-test-ioccc/bugs.html">Bugs & (mis)features</a>
|
||||
<a href="/temp-test-ioccc/faq.html#fix_a_winner">Fixing IOCCC winners</a>
|
||||
<a href="/temp-test-ioccc/faq.html#fix_web_site">Fixing the web site</a>
|
||||
<a href="/temp-test-ioccc/faq.html#fix_author">Fixing winner info</a>
|
||||
<a href="/temp-test-ioccc/thanks-for-help.html">Thanks for the help</a>
|
||||
</div>
|
||||
<hr>
|
||||
<div class="sidenav">
|
||||
<a href="/temp-test-ioccc/judges.html">The IOCCC Judges</a>
|
||||
<a href="/temp-test-ioccc/contact.html">Contacting the IOCCC</a>
|
||||
</div>
|
||||
<hr>
|
||||
<div class="sidenav">
|
||||
<a href="#top">Jump to top</a>
|
||||
</div>
|
||||
|
||||
<!-- END: this line ends content from: inc/sidenav.default.html -->
|
||||
<!-- START: this line starts content from: inc/end-leftcolumn.default.html -->
|
||||
|
||||
</div>
|
||||
|
||||
<!-- END: this line ends content from: inc/end-leftcolumn.default.html -->
|
||||
<!-- START: this line starts content from: inc/begin-rightcolumn.default.html -->
|
||||
|
||||
<div class="rightcolumn">
|
||||
|
||||
<!-- END: this line ends content from: inc/begin-rightcolumn.default.html -->
|
||||
<!-- START: this line starts content from: inc/before-content.default.html -->
|
||||
|
||||
<a name="content"></a>
|
||||
|
||||
<!-- END: this line ends content from: inc/before-content.default.html -->
|
||||
<!-- START: this line starts content generated by: bin/gen-author.sh -->
|
||||
|
||||
<p><a name="author"><strong>Author:</strong></a> <br><br>Name: <a
|
||||
href="/temp-test-ioccc/winners.html#J_David_Lowe">J. David Lowe</a><br>
|
||||
Location: <a href="/temp-test-ioccc/location.html#US">US</a></p>
|
||||
|
||||
<!-- END: next line ends content generated by: bin/gen-author.sh -->
|
||||
<h2 id="to-build">To build:</h2>
|
||||
<div class="sourceCode" id="cb1"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="fu">make</span></span></code></pre></div>
|
||||
<h3 id="bugs-and-misfeatures">Bugs and (Mis)features:</h3>
|
||||
<p>The current status of this entry is:</p>
|
||||
<pre><code>STATUS: missing or dead link - please provide them
|
||||
STATUS: INABIAF - please **DO NOT** fix</code></pre>
|
||||
<p>For more detailed information see <a href="/bugs.md#2011-dlowe">2011
|
||||
dlowe in bugs.md</a>.</p>
|
||||
<h2 id="to-use">To use:</h2>
|
||||
<div class="sourceCode" id="cb3"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-</span><span class="op"><</span>n_iterations<span class="op">></span> corpus1/ <span class="pp">[</span><span class="ss">...</span><span class="pp">]</span> corpus0/ <span class="op"><</span> start.net <span class="op">></span> trained.net</span></code></pre></div>
|
||||
<p>NOTE: In the above command, the directory args MUST end in a
|
||||
<code>/</code>.</p>
|
||||
<p>Then to use <code>trained.net</code>:</p>
|
||||
<div class="sourceCode" id="cb4"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> file [file ...] <span class="op"><</span> trained.net</span></code></pre></div>
|
||||
<h2 id="try">Try:</h2>
|
||||
<p>Try these pre-trained networks:</p>
|
||||
<div class="sourceCode" id="cb5"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./try.sh</span></span></code></pre></div>
|
||||
<p>NOTE: The <code>dlowe-aux-data/</code> directory, which <a
|
||||
href="try.sh">try.sh</a> refers to, was created by:</p>
|
||||
<div class="sourceCode" id="cb6"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="fu">tar</span> <span class="at">-jxf</span> dlowe-aux-data.tar.bz2</span></code></pre></div>
|
||||
<h2 id="judges-remarks">Judges’ remarks:</h2>
|
||||
<p>This entry is a ghoulish example of a brain (dead?) neural network
|
||||
classifier. It comes with a set of corpora for you to try.</p>
|
||||
<h3
|
||||
id="ioccc-winlose-trained.net"><code>ioccc-winlose-trained.net</code></h3>
|
||||
<p>You can create your own trained networks. For example, we trained a
|
||||
network on the C code of entries that were supplied to the 20Th
|
||||
IOCCC.</p>
|
||||
<p>The <code>ioccc-winlose-trained.net</code> was trained using the <a
|
||||
href="earlystop.pl">earlystop.pl</a> tool:</p>
|
||||
<div class="sourceCode" id="cb7"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rm</span> <span class="at">-f</span> ioccc-winlose-trained.net</span>
|
||||
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a><span class="ex">./earlystop.pl</span> ioccc-winlose-trained.net ioccc_won_training/ ioccc_lost_training/ ioccc_won_test/ ioccc_lost_test/</span></code></pre></div>
|
||||
<p>where:</p>
|
||||
<ul>
|
||||
<li><p><code>ioccc_won_training/</code> 1/2 of C code of the 20th IOCCC
|
||||
winners (except for <a href="dlowe.c">dlowe.c</a> from 2011) plus 1/2 of
|
||||
the C code winners from prior IOCCC contests</p></li>
|
||||
<li><p><code>ioccc_won_test/</code> 1/2 of C code of the 20th IOCCC
|
||||
winners (except for <a href="dlowe.c">dlowe.c</a> from 2011) plus 1/2 of
|
||||
the C code winners from prior IOCCC contests</p></li>
|
||||
<li><p><code>ioccc_lost_training/</code> 1/2 of C code from entries that
|
||||
that did not win</p></li>
|
||||
<li><p><code>ioccc_lost_test/</code> 1/2 of C code from entries that
|
||||
that did not win</p></li>
|
||||
</ul>
|
||||
<p>NOTE: The above directories were not shipped and everything from the
|
||||
entries that did not win were flushed to <code>/dev/null</code>.</p>
|
||||
<p>The <a href="earlystop.pl">earlystop.pl</a> tool wrote:</p>
|
||||
<pre><code>training 1 to 1001...
|
||||
training 1001 to 2001...
|
||||
prior error = 56.746974766043; current error = 58.095691135305</code></pre>
|
||||
<p>The resulting <code>ioccc-winlose-trained.net</code> file was
|
||||
included in the <a
|
||||
href="dlowe-aux-data.tar.bz2">dlowe-aux-data.tar.bz2</a> tarball.</p>
|
||||
<p>The <a href="dlowe.c">dlowe.c</a> was explicitly excluded from this
|
||||
trailing set, so this test is interesting:</p>
|
||||
<div class="sourceCode" id="cb9"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> dlowe.c <span class="op"><</span> dlowe-aux-data/ioccc-winlose-trained.net</span></code></pre></div>
|
||||
<p>NOTE: this is done in <a href="try.sh">try.sh</a>.</p>
|
||||
<p>The match of <a href="dlowe.c">dlowe.c</a> to the lose/win trailed
|
||||
network was:</p>
|
||||
<pre><code>dlowe.c 0.125108</code></pre>
|
||||
<p>NOTE: The non-artificially intelligent decisions of the IOCCC judges
|
||||
include random quirks and their non-linear judgment calls not found in
|
||||
the ioccc trained network. Just because C code is or isn’t scored highly
|
||||
by this network does not mean it will win or lose a future IOCCC. On the
|
||||
other hand … it just might!</p>
|
||||
<h4 id="other-data-sets-to-try">other data sets to try</h4>
|
||||
<p>While we cannot supply you with the losing IOCCC source code, the
|
||||
author did supply a number data sets on which to test:</p>
|
||||
<p>We created the <a
|
||||
href="dlowe-aux-data/english-trained.net">english-trained.net</a> as
|
||||
follows:</p>
|
||||
<div class="sourceCode" id="cb11"><pre
|
||||
class="sourceCode sh"><code class="sourceCode bash"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-8000</span> dlowe-aux-data/english-1/ dlowe-aux-data/english-0/ <span class="op"><</span> /dev/null <span class="op">></span> dlowe-aux-data/english-trained.net</span></code></pre></div>
|
||||
<ul>
|
||||
<li><p><a href="dlowe-aux-data/english-0/">dlowe-aux-data/english-0/</a>
|
||||
Non-English (French) text</p></li>
|
||||
<li><p><a href="dlowe-aux-data/english-1">dlowe-aux-data/english-1/</a>
|
||||
English (non-French) text</p></li>
|
||||
</ul>
|
||||
<p>We created the <a
|
||||
href="dlowe-aux-data/png-trained.net">png-trained.net</a> as
|
||||
follows:</p>
|
||||
<div class="sourceCode" id="cb12"><pre
|
||||
class="sourceCode sh"><code class="sourceCode bash"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-8000</span> png-1/ png-0/ <span class="op"><</span> /dev/null <span class="op">></span> dlowe-aux-data/png-trained.net</span></code></pre></div>
|
||||
<ul>
|
||||
<li><p><a href="dlowe-aux-data/png-0/">dlowe-aux-data/png-0/</a> Non-png
|
||||
(gif) images</p></li>
|
||||
<li><p><a href="dlowe-aux-data/png-1/">dlowe-aux-data/png-1/</a> png
|
||||
(non-gif) images</p></li>
|
||||
</ul>
|
||||
<p>We created the <a
|
||||
href="dlowe-aux-data/xor-trained.net">xor-trained.net</a> as
|
||||
follows:</p>
|
||||
<div class="sourceCode" id="cb13"><pre
|
||||
class="sourceCode sh"><code class="sourceCode bash"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-8000</span> dlowe-aux-data/xor-1/ dlowe-aux-data/xor-0/ <span class="op"><</span> /dev/null <span class="op">></span> dlowe-aux-data/xor-trained.net</span></code></pre></div>
|
||||
<ul>
|
||||
<li><p><a href="dlowe-aux-data/xor-0/">dlowe-aux-data/xor-0</a> Data
|
||||
that XORs to 0</p></li>
|
||||
<li><p><a href="dlowe-aux-data/xor-1/">dlowe-aux-data/xor-1/</a> Data
|
||||
that XORs to 1</p></li>
|
||||
</ul>
|
||||
<p><strong>NOTE</strong>: The directory args <strong>must end in
|
||||
a</strong> <code>/</code> for them to work.</p>
|
||||
<h2 id="authors-remarks">Author’s remarks:</h2>
|
||||
<h3 id="synopsis">Synopsis</h3>
|
||||
<p>This is an <a
|
||||
href="https://en.wikipedia.org/wiki/Artificial_intelligence">artificially
|
||||
intelligent</a> judging tool to help the IOCCC judges. Here’s to
|
||||
shorter, more frequent contests!</p>
|
||||
<h3 id="description">Description</h3>
|
||||
<p>This is a multilayer <a
|
||||
href="https://en.wikipedia.org/wiki/Perceptron">perceptron</a> (a <a
|
||||
href="https://en.wikipedia.org/wiki/Feedforward_neural_network">feedforward
|
||||
artificial neural network</a>) which can be trained, using on-line <a
|
||||
href="https://en.wikipedia.org/wiki/Backpropagation">backpropagation</a>,
|
||||
to classify input files.</p>
|
||||
<p>It has a fixed topology of <code>2^16</code> input neurons, 6 hidden
|
||||
neurons and 1 output neuron.</p>
|
||||
<p>The neurons’ activation function is the logistic function
|
||||
<code>1 / (1 + e ^ -x)</code>.</p>
|
||||
<h3 id="classifying">Classifying</h3>
|
||||
<div class="sourceCode" id="cb14"><pre
|
||||
class="sourceCode sh"><code class="sourceCode bash"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> file [file ...] <span class="op"><</span> trained.net</span></code></pre></div>
|
||||
<p>To classify files, one specifies a trained network (on
|
||||
<code>stdin</code>) and one or more files to classify. The program will
|
||||
output one line per successfully-classified file to <code>stderr</code>,
|
||||
with the filename and the classification: a number between 0 and 1.</p>
|
||||
<p>The interpretation of the classification number depends on how the
|
||||
network was trained, but it’s geared toward interpretation as a
|
||||
<em>probability</em> or a <em>confidence</em>.</p>
|
||||
<h3 id="training">Training</h3>
|
||||
<div class="sourceCode" id="cb15"><pre
|
||||
class="sourceCode sh"><code class="sourceCode bash"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-</span><span class="op"><</span>n_iterations<span class="op">></span> corpus1/ <span class="pp">[</span><span class="ss">...</span><span class="pp">]</span> corpus0/ <span class="op"><</span> start.net <span class="op">></span> end.net</span></code></pre></div>
|
||||
<p><strong>NOTE</strong>: The directory args <strong>must end in
|
||||
a</strong> <code>/</code> for them to work.</p>
|
||||
<p>To train a network, one specifies a starting network (on
|
||||
<code>stdin</code>), two or more corpora (directories containing
|
||||
training data), and the number of training iterations to run. The
|
||||
program will write some progress data to <code>stderr</code> and, when
|
||||
it’s done, will serialize the updated network to
|
||||
<code>stdout</code>.</p>
|
||||
<p>If no input network is given, a random new network will be generated
|
||||
as a starting point (i.e. provide an empty <code>stdin</code>).</p>
|
||||
<p>The first corpus will be assigned a target value of <code>1</code>.
|
||||
The last will be assigned a target value of <code>0</code>. Intervening
|
||||
directories (if any) will be assigned intermediate target values.</p>
|
||||
<p>The learning rate is hard-coded as <code>0.3</code>. No momentum
|
||||
factor is used.</p>
|
||||
<h4 id="png-corpora">png corpora</h4>
|
||||
<p>The <a href="dlowe-aux-data/png-1">png-1</a> corpus was obtained by
|
||||
manually scraping the first results from an <a
|
||||
href="https://images.google.com"
|
||||
class="uri">https://images.google.com</a> search for “<a
|
||||
href="https://www.google.com/search?tbm=isch&sxsrf=APwXEddqIU34dxhpFdjEa8emNJKHOkYXIw:1683049158657&source=hp&biw=1727&bih=952&ei=xkpRZLapJtLKkPIP8bGKkAk&iflsig=AOEireoAAAAAZFFY1nbmEUHC74Cmg3Y5rEY1fV7ocASZ&ved=0ahUKEwi2pPXZltf-AhVSJUQIHfGYApIQ4dUDCAY&uact=5&oq=obfuscate+filetype:png&gs_lcp=CgNpbWcQA1AAWABg9QNoAHAAeACAATuIATuSAQExmAEAoAECoAEBqgELZ3dzLXdpei1pbWc&sclient=img&q=obfuscate&tbs=ift:png">obfuscate
|
||||
filetype:png</a>”.</p>
|
||||
<p>The <a href="dlowe-aux-data/png-0">png-0</a> corpus was obtained by
|
||||
manually scraping the first results from an <a
|
||||
href="https://images.google.com"
|
||||
class="uri">https://images.google.com</a> search for “<a
|
||||
href="https://www.google.com/search?tbm=isch&ved=2ahUKEwiowo7fltf-AhXSOUQIHbJBDp8Q2-cCegQIABAA&oq=obfuscate+filetype:gif&gs_lcp=CgNpbWcQAzoECCMQJzoHCAAQigUQQzoFCAAQgARQ6AxY6AxgpBJoAHAAeACAAUeIAcsBkgEBM5gBAKABAaoBC2d3cy13aXotaW1nwAEB&sclient=img&ei=0UpRZOiVINLzkPIPsoO5-Ak&bih=952&biw=1727&q=obfuscate&tbs=ift:gif">obfuscate
|
||||
filetype:gif</a>”.</p>
|
||||
<h5 id="results">Results:</h5>
|
||||
<ul>
|
||||
<li>Using these corpora as a training set;</li>
|
||||
<li>Using several hundred random <code>*.gif</code> and
|
||||
<code>*.png</code> files from my home computer as a test set;</li>
|
||||
<li>Using the included <a href="earlystop.pl">earlystop.pl</a> script,
|
||||
which stopped after 67000 iterations.</li>
|
||||
<li>Interpreting output of <code>> 0.5</code> as “probably a
|
||||
<code>.png</code>” and <code>< 0.5</code> as “probably a
|
||||
<code>.gif</code>”</li>
|
||||
</ul>
|
||||
<p>produced a network with about 87% accuracy</p>
|
||||
<h4 id="english-corpora">English corpora</h4>
|
||||
<p>The <a href="dlowe-aux-data/english-1">english-1</a> corpus was
|
||||
obtained by manually scraping the first results from a <a
|
||||
href="https://www.google.com/webhp?lr=lang_en"
|
||||
class="uri">https://www.google.com/webhp?lr=lang_en</a> search for “<a
|
||||
href="https://www.google.com/search?q=paris+filetype%3Atxt&lr=lang_en&tbs=lr%3Alang_1en&sxsrf=APwXEdeHXyUYdUxCYQQDHmJhbHzX9aARSA%3A1683049834430&source=hp&ei=ak1RZJHUGN_DkPIP26OtkA8&iflsig=AOEireoAAAAAZFFbejA0Xfqoq__FXx--Ok0S6GQjnGnA&ved=0ahUKEwiRsZOcmdf-AhXfIUQIHdtRC_IQ4dUDCAo&uact=5&oq=paris+filetype%3Atxt&gs_lcp=Cgdnd3Mtd2l6EANQAFgAYN4DaABwAHgAgAE-iAE-kgEBMZgBAKABAqABAQ&sclient=gws-wiz">paris
|
||||
filetype:txt</a>”.</p>
|
||||
<p>The <a href="dlowe-aux-data/english-0">english-0</a> corpus was
|
||||
obtained by manually scraping the first results from a <a
|
||||
href="https://www.google.com/webhp?lr=lang_fr"
|
||||
class="uri">https://www.google.com/webhp?lr=lang_fr</a> search for “<a
|
||||
href="https://www.google.com/search?q=paris+filetype%3Atxt&lr=lang_fr&tbs=lr%3Alang_1fr&sxsrf=APwXEddcsQ-gMhrqSUJo6omWFUl7jQMASg%3A1683049870736&source=hp&ei=jk1RZLevK_fKkPIPjbWN-AU&iflsig=AOEireoAAAAAZFFbnvExHGkcavB5H8F0xsLAWN2KfYFK&ved=0ahUKEwi3rrutmdf-AhV3JUQIHY1aA18Q4dUDCAo&uact=5&oq=paris+filetype%3Atxt&gs_lcp=Cgdnd3Mtd2l6EANQAFgAYLMDaABwAHgAgAE-iAE-kgEBMZgBAKABAqABAQ&sclient=gws-wiz">paris
|
||||
filetype:txt</a>”.</p>
|
||||
<h5 id="results-1">Results:</h5>
|
||||
<ul>
|
||||
<li>Using these corpora as a training set;</li>
|
||||
<li>Using several dozen additional text files (obtained by the same
|
||||
methods) as a test set;</li>
|
||||
<li>Using the included <a href="earlystop.pl">earlystop.pl</a> script,
|
||||
which stopped after 10000 iterations;</li>
|
||||
<li>Interpreting output of <code>> 0.5</code> as “probably English”
|
||||
and <code>< 0.5</code> as “probably French”</li>
|
||||
</ul>
|
||||
<p>produced a network with 100% accuracy.</p>
|
||||
<h3 id="xor-corpora">xor corpora</h3>
|
||||
<p>The <a href="dlowe-aux-data/xor-1">xor-1</a> corpus consists of two
|
||||
files containing <code>01</code> and <code>10</code> respectively.</p>
|
||||
<p>The <a href="dlowe-aux-data/xor-0">xor-0</a> corpus consists of two
|
||||
files containing <code>00</code> and <code>11</code> respectively.</p>
|
||||
<h5 id="results-2">Results:</h5>
|
||||
<p>Using these corpora as a training set takes about 8000 iterations to
|
||||
learn xor to within a tolerance of <code><0.01</code>.</p>
|
||||
<h3 id="limitations">Limitations</h3>
|
||||
<p>The program can’t tell you anything meaningful about files with less
|
||||
than two bytes in them (I’m looking at you, <a
|
||||
href="/1994/smr/smr.c">smr.c</a>!)</p>
|
||||
<p>You must include the trailing directory separator on training
|
||||
directories (this allows the program to be portable without wasting
|
||||
precious bytes on figuring out how to concatenate directory and file
|
||||
names…).</p>
|
||||
<p>Serialized network files are only portable between systems with the
|
||||
same <a
|
||||
href="https://en.wikipedia.org/wiki/Floating-point_arithmetic">floating-point</a>
|
||||
representation and <a
|
||||
href="https://en.wikipedia.org/wiki/Endianness">endianness</a>.</p>
|
||||
<p>Making sure not to <a
|
||||
href="https://en.wikipedia.org/wiki/Overfitting#Machine_learning">overfit</a>
|
||||
the network to the training data is a bit of a black art. I have
|
||||
enclosed <a href="earlystop.pl">earlystop.pl</a>, a wrapper script that
|
||||
implements a simple ‘early stopping’ algorithm; other techniques are
|
||||
possible.</p>
|
||||
<p>Bad input (e.g. nonexistent files, non-numeric number of iterations,
|
||||
etc.) tends to result in empty output.</p>
|
||||
<p>Given exactly one corpus, the program will crash or produce
|
||||
garbage.</p>
|
||||
<p>Leaks memory and file descriptors while processing files.</p>
|
||||
<p>Will crash and die horribly if it runs out of memory.</p>
|
||||
<p>The Microsoft C compiler doesn’t provide a <code>dirent</code> API,
|
||||
so to get this working on a Windows system you’ll need cygwin+gcc
|
||||
(tested) or a <code>dirent</code> compatibility library (untested, but
|
||||
they do exist).</p>
|
||||
<p>Backpropagation doesn’t always converge: if you play with this long
|
||||
enough, you’ll eventually have a training session that completely fails
|
||||
to converge.</p>
|
||||
<h3 id="obfuscation">Obfuscation</h3>
|
||||
<ul>
|
||||
<li><a href="https://en.wikipedia.org/wiki/Zombie">Zombies</a>! (Since
|
||||
neural networks are modeled after <a
|
||||
href="https://en.wikipedia.org/wiki/Brain">BRAINS</a>, ya know? And
|
||||
corpus sounds a lot like corpse. And I have 4- and 7-year-old kids ;)
|
||||
)</li>
|
||||
<li>Neural networks are interesting, and while the math isn’t terribly
|
||||
difficult, their behavior is difficult to fully understand.</li>
|
||||
<li>Had some extra space in one of the major data structures, and it
|
||||
seemed a shame to waste it.</li>
|
||||
<li>Similarly, I think three file pointers ought to be enough for
|
||||
anyone.</li>
|
||||
<li>Lots of magic numbers expressed in various ways.</li>
|
||||
<li>Random abuse of random C trivia.</li>
|
||||
<li>Some textual changes were made to maximize the classification of
|
||||
this program by a network trained to recognize <a
|
||||
href="/winners.html">IOCCC winners</a> ;)</li>
|
||||
</ul>
|
||||
<p>… but mostly zombies!</p>
|
||||
<h2 id="copyright-and-cc-by-sa-4.0-license">Copyright and CC BY-SA 4.0
|
||||
License:</h2>
|
||||
<p>This file is Copyright (c) 2023 by Landon Curt Noll. All Rights
|
||||
Reserved. You are free to share and adapt this file under the terms of
|
||||
this license:</p>
|
||||
<pre><code>Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)</code></pre>
|
||||
<p>For more information, see:
|
||||
https://creativecommons.org/licenses/by-sa/4.0/</p>
|
||||
<!-- START: this line starts content generated by: bin/gen-inventory.sh -->
|
||||
|
||||
<h1 id="inventory-for-2011dlowe"><a name="inventory"></a>Inventory for
|
||||
2011/dlowe</h1>
|
||||
<h2 id="primary-files">Primary files</h2>
|
||||
<ul>
|
||||
<li><a href="index.html">index.html</a> - winner home page</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe.c">dlowe.c</a>
|
||||
- winning source</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/Makefile">Makefile</a>
|
||||
- winner Makefile</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe.orig.c">dlowe.orig.c</a>
|
||||
- original source</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/earlystop.pl">earlystop.pl</a>
|
||||
- wrapper script to stop training early</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/try.sh">try.sh</a>
|
||||
- how to run</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-0/5756029-DMZ-overview-B.gif">dlowe-aux-data/png-0/5756029-DMZ-overview-B.gif</a>
|
||||
- Non-png gif images</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-0/M2.gif">dlowe-aux-data/png-0/M2.gif</a> -
|
||||
Non-png gif images</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-0/Obfuscate.gif">dlowe-aux-data/png-0/Obfuscate.gif</a>
|
||||
- Non-png gif images</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-0/figure_49.gif">dlowe-aux-data/png-0/figure_49.gif</a>
|
||||
- Non-png gif images</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-0/klockhuset.gif">dlowe-aux-data/png-0/klockhuset.gif</a>
|
||||
- Non-png gif images</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-0/obfuscate_1.gif">dlowe-aux-data/png-0/obfuscate_1.gif</a>
|
||||
- Non-png gif images</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-0/podcastimage_150215.gif">dlowe-aux-data/png-0/podcastimage_150215.gif</a>
|
||||
- Non-png gif images</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-0/prjman1.gif">dlowe-aux-data/png-0/prjman1.gif</a>
|
||||
- Non-png gif images</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-0/proguard_obfuscate.gif">dlowe-aux-data/png-0/proguard_obfuscate.gif</a>
|
||||
- Non-png gif images</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-0/scripts_encryptor__screnc_-31918.gif">dlowe-aux-data/png-0/scripts_encryptor_<em>screnc</em>-31918.gif</a>
|
||||
- Non-png gif images</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-0/visual_panel.gif">dlowe-aux-data/png-0/visual_panel.gif</a>
|
||||
- Non-png gif images</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/040604.txt">dlowe-aux-data/english-0/040604.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/2001-2002.txt">dlowe-aux-data/english-0/2001-2002.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/CFP_TSI_Graphes_appel.txt">dlowe-aux-data/english-0/CFP_TSI_Graphes_appel.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/CIR_APE_08_doc%5B1%5D.txt">dlowe-aux-data/english-0/CIR_APE_08_doc%5B1%5D.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/admin_fr.txt">dlowe-aux-data/english-0/admin_fr.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/agenda.txt">dlowe-aux-data/english-0/agenda.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/paris.txt">dlowe-aux-data/english-0/paris.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/paris2.txt">dlowe-aux-data/english-0/paris2.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/structures3D-Paris.txt">dlowe-aux-data/english-0/structures3D-Paris.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/02-chronology.txt">dlowe-aux-data/english-1/02-chronology.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/36C215.txt">dlowe-aux-data/english-1/36C215.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/Wireless_Days_2011_CFP.txt">dlowe-aux-data/english-1/Wireless_Days_2011_CFP.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/cpb.txt">dlowe-aux-data/english-1/cpb.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/get-there.txt">dlowe-aux-data/english-1/get-there.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/pa_fv010.txt">dlowe-aux-data/english-1/pa_fv010.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/the_poor_people_of_paris.txt">dlowe-aux-data/english-1/the_poor_people_of_paris.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/utmost.txt">dlowe-aux-data/english-1/utmost.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/walljpar.txt">dlowe-aux-data/english-1/walljpar.txt</a>
|
||||
- training text for neural networks</li>
|
||||
<li><a href="dlowe-aux-data/xor-0/00">dlowe-aux-data/xor-0/00</a> -
|
||||
training digits for neural networks</li>
|
||||
<li><a href="dlowe-aux-data/xor-0/11">dlowe-aux-data/xor-0/11</a> -
|
||||
training digits for neural networks</li>
|
||||
<li><a href="dlowe-aux-data/xor-1/01">dlowe-aux-data/xor-1/01</a> -
|
||||
training digits for neural networks</li>
|
||||
<li><a href="dlowe-aux-data/xor-1/10">dlowe-aux-data/xor-1/10</a> -
|
||||
training digits for neural networks</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-1/JavaScript-obfuscation-code.png">dlowe-aux-data/png-1/JavaScript-obfuscation-code.png</a>
|
||||
- training image for neural networks</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-1/control_flow_obfuscation_before.png">dlowe-aux-data/png-1/control_flow_obfuscation_before.png</a>
|
||||
- training image for neural networks</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-1/deceive-inveigle-obfuscate-t-shirts.png">dlowe-aux-data/png-1/deceive-inveigle-obfuscate-t-shirts.png</a>
|
||||
- training image for neural networks</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-1/help_obfuscation1.png">dlowe-aux-data/png-1/help_obfuscation1.png</a>
|
||||
- training image for neural networks</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-1/image_thumb%5B40%5D.png">dlowe-aux-data/png-1/image_thumb[40].png</a>
|
||||
- training image for neural networks</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-1/jquery_after_deobfuscation.png">dlowe-aux-data/png-1/jquery_after_deobfuscation.png</a>
|
||||
- training image for neural networks</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-1/obfuscate4e.png">dlowe-aux-data/png-1/obfuscate4e.png</a>
|
||||
- training image for neural networks</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-1/obfuscation-decoder.png">dlowe-aux-data/png-1/obfuscation-decoder.png</a>
|
||||
- training image for neural networks</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-1/obfuscation_methods.png">dlowe-aux-data/png-1/obfuscation_methods.png</a>
|
||||
- training image for neural networks</li>
|
||||
<li><a href="2011_dlowe.tar.bz2">2011_dlowe.tar.bz2</a> - download
|
||||
winning source</li>
|
||||
</ul>
|
||||
<h2 id="secondary-files">Secondary files</h2>
|
||||
<ul>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/.gitignore">.gitignore</a>
|
||||
- list of files that should not be committed under git</li>
|
||||
<li><a href=".path">.path</a> - directory path from top directory</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/.winner.json">.winner.json</a>
|
||||
- description of winner in JSON</li>
|
||||
<li><a
|
||||
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/README.md">README.md</a>
|
||||
- markdown source for index.html</li>
|
||||
<li><a href="dlowe-aux-data.tar.bz2">dlowe-aux-data.tar.bz2</a> - IOCCC
|
||||
win lose trained network</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/english-trained.net">dlowe-aux-data/english-trained.net</a>
|
||||
- trained neural network for English text</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/ioccc-winlose-trained.net">dlowe-aux-data/ioccc-winlose-trained.net</a>
|
||||
- trained neural network for C code from 20Th IOCCC</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/png-trained.net">dlowe-aux-data/png-trained.net</a>
|
||||
- trained neural network for png-0 and png-1 images</li>
|
||||
<li><a
|
||||
href="dlowe-aux-data/xor-trained.net">dlowe-aux-data/xor-trained.net</a>
|
||||
- trained neural network for xor-0 and xor-1 data</li>
|
||||
</ul>
|
||||
|
||||
<!-- END: next line ends content generated by: bin/gen-inventory.sh -->
|
||||
<!-- START: this line starts content from: inc/after-content.default.html -->
|
||||
|
||||
<!-- END: this line ends content from: inc/after-content.default.html -->
|
||||
<!-- START: this line starts content from: inc/end-rightcolumn.default.html -->
|
||||
|
||||
</div>
|
||||
|
||||
<!-- END: this line ends content from: inc/end-rightcolumn.default.html -->
|
||||
<!-- START: this line starts content from: inc/end-row.default.html -->
|
||||
|
||||
</div>
|
||||
|
||||
<!-- END: this line ends content from: inc/end-row.default.html -->
|
||||
<!-- START: this line starts content from: inc/footer.default.html -->
|
||||
|
||||
<div class="footer">
|
||||
<hr>
|
||||
<h2>Coda</h2>
|
||||
<p><a href="https://validator.w3.org/nu/?doc=https%3A%2F%2Fioccc-src.github.io%2Ftemp-test-ioccc%2F2011%2Fdlowe%2Findex.html">Nu HTML check this web page</a></p>
|
||||
<p><a href="#top">Jump to top</a> <a href="#content">Jump to Content</a> <a href="#inventory">Jump to Inventory</a></p>
|
||||
</div>
|
||||
|
||||
<!-- END: this line ends content from: inc/footer.default.html -->
|
||||
<!-- START: this line starts content from: inc/bottom.default.html -->
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
||||
<!-- END: this line ends content from: inc/bottom.default.html -->
|
||||
Reference in new issue
Block a user