mirror/ioccc
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e6040e083c712f1ea2cbfcbc930044c098c2c83b
ioccc/2011/dlowe/index.html
T
Landon Curt Noll 2b92c89be8 fix and improve links within the web site 
Entry source is now prog.c for IOCCC years 2014 thru 2020,
or is now dirname.c IOCCC years 1984 thru 2013.

To achieve this state, we renamed the following:

renamed:    2004/hibachi/src/hibachi.c -> 2004/hibachi/hibachi.c
renamed:    2012/omoikane/nyaruko.alt.c -> 2012/omoikane/omoikane.alt.c
renamed:    2012/omoikane/nyaruko.alt2.c -> 2012/omoikane/omoikane.alt2.c
renamed:    2012/omoikane/nyaruko.c -> 2012/omoikane/omoikane.c
renamed:    2012/omoikane/nyaruko.orig.c -> 2012/omoikane/omoikane.orig.c

Modified `bin/subst.entry-index.sh` to add to entry `index.html` file,
links to the `Makefile` and the _C code_ to the top navigation bar.
Changed `bin/subst.entry-index.sh` VERSION from "1.2.2 2024-02-25"
to "1.3 2024-03-02".

Moved printing the Inventory link in the top navigation bar
from `bin/subst.entry-navbar.awk` to `bin/subst.entry-index.sh`.
Changed `bin/subst.entry-navbar.awk` VERSION from "1.0.1 2024-02-23"
to "1.1 2024-03-02".

Moved printing the Inventory link in the top navigation bar
from `bin/subst.year-navbar.awk` to `bin/subst.year-index.sh`.
Changed `bin/subst.year-index.sh` VERSION from "1.0 2024-02-11"
to "1.1 2024-03-02".
Changed `bin/subst.year-navbar.awk` VERSION from "1.0 2024-02-11"
to "1.1 2024-03-02".

Added location names (both default and common) to `author.html`.
Changed `tmp/gen_author_json.sh` VERSION from "1.0 2023-06-06"
to "1.1 2024-03-04".

Added location names (both default and common) to `author.html`.
Changed `bin/gen-authors.sh` VERSION from "1.2 2024-03-09"
to "1.3 2024-03-04".

The manifest and the resulting `.entry.json` files have been changed
for `2004/hibachi` and `2012/omoikane`.

The Makefiles for  `2004/hibachi` and `2012/omoikane` copy the
source file into the "previously used location".  The `make clobber`
rule removes the copied source.  The `.gitignore` files have been
updated accordingly.

Improved how the concept of IOCCC terms _entry_, _author_ and _submission_ are
used in markdown files.

Links to markdown files (files ending in .md) within the repo were
changed to link to HTML files (files ending in .html).  Special cases
where `README.md` became `index.html` were modified accordingly.

Fixed URL for `bugs.html` in Makefile comments.

Fixed name of `bin/sgi.sh` referenced in comments in `.gitignore` files.

Links to locations within the repo are now relative.  For example a
link to `/contact.html` may become `../contact.html' or '../../contact.html`
depending on the directory level of the markdown file.

Nearly all links with absolute URLs under _https://www.ioccc.org_
or `https://ioccc-src.github.io/temp-test-ioccc_ have been changed to
relative paths within the repo.  Some exceptions to this (such as
referring to a GitHub repo) were preserved.

Fixed a number of inside `faq.md` and `bugs.md`.

Added HTML name tags (in the form of an _entry_id_) for each entry
referenced in `bugs.md`.  Fixed links from outside of `bugs.html`
into `bugs.html` accordingly.

Where FAQ entries are referenced, a named tag is used so that if a
FAQ is ever renumbered, links outside of `faq.html` will still work.

Changed the _inventory_order_ of a compressed tarball from _300_
to _100000_.  While this may have been done for the manifest,
this changed had not yet been propagated into the `.entry.json` files.

Links to web sites that were in the for <URL> are now [URL](URL).

Performed `name www` to process all of the above.

Improved format of `authors.html`.  Full names are now bold.
Location codes are now links back into `location.html` and
location names (both default and common) are now italics.
Added _author_handle_ each author.

Changed `bin/gen-authors.sh` VERSION from "1.1 2024-02-25"
to "1.2 2024-03-09".

Improved format of `location.html` as location codes are now bold,
author counts in a given location are now bold, and location names
(both default and common) are now italics.

Changed `bin/gen-location.sh` VERSION from "1.0.1 2024-02-23"
to "1.1 2024-03-04".

Added italic location name after location code in entry
`index.html` files.

Changed `bin/output-index-author.sh` VERSION from "1.4.1 2024-02-23"
to "1.5 2024-03-04".

Added _Location of authors_ to the sidebar as a link to `location.html`.

TODO: While nearly all of the above mentioned changes are likely correct,
      they likely introduced a few errors that need correcting.
2024-03-04 23:45:28 -08:00

610 lines
33 KiB
HTML
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<!-- START: two lines up starts content from: inc/top.default.html -->
<!-- -->
<!-- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -->
<!-- !!! DO NOT MODIFY THIS FILE - This file is generated by a tool !!! -->
<!-- !!! DO NOT MODIFY THIS FILE - This file is generated by a tool !!! -->
<!-- !!! DO NOT MODIFY THIS FILE - This file is generated by a tool !!! -->
<!-- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -->
<!-- -->
<!-- END: this line ends content from: inc/top.default.html -->
<!-- -->
<!-- The original markdown content was formed by the tool: bin/readme2index.sh -->
<!-- The HTML was converted from markdown by the tool: bin/md2html.sh -->
<!-- -->
<!-- START: this line starts content from: inc/head.default.html -->
<head>
<link rel="stylesheet" href="../../ioccc.css">
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<title>2011/dlowe - Most self deprecating</title>
<meta name="description" content="2011 IOCCC entry dlowe - Most self deprecating">
<meta name="keywords" content="IOCCC, 2011, IOCCC 2011, IOCCC entry, dlowe, Most self deprecating">
</head>
<!-- END: this line ends content from: inc/head.default.html -->
<!-- START: this line starts content from: inc/body.default.html -->
<body>
<!-- END: this line ends content from: inc/body.default.html -->
<!-- START: this line starts content from: inc/header.default.html -->
<div class="header">
<img src="../../png/ioccc.png" alt="IOCCC image by Matt Zucker" width=300 height=110>
<h1>The International Obfuscated C Code Contest</h1>
<h2>2011/dlowe - Most self deprecating</h2>
</div>
<!-- END: this line ends content from: inc/header.default.html -->
<!-- START: this line starts content from: inc/topnav.mid.html -->
<div class="topnav">
<a class="Left" href="../borsanyi/index.html">&larr; borsanyi</a>
<a class="Left" href="../index.html">&uarr; 2011 &uarr;</a>
<a class="Left" href="../eastman/index.html">eastman &rarr;</a>
<a class="Right" href="https://github.com/ioccc-src/temp-test-ioccc/2011/dlowe/dlowe.c">C code</a>
<a class="Right" href="https://github.com/ioccc-src/temp-test-ioccc/2011/dlowe/Makefile">Makefile</a>
<a class="Right" href="#inventory">Inventory</a>
</div>
<!-- END: this line ends content from: inc/topnav.mid.html -->
<!-- START: this line starts content from: inc/begin-row.default.html -->
<div class="row">
<!-- END: this line ends content from: inc/begin-row.default.html -->
<!-- START: this line starts content from: inc/begin-leftcolumn.default.html -->
<div class="leftcolumn">
<!-- END: this line ends content from: inc/begin-leftcolumn.default.html -->
<!-- START: this line starts content from: inc/sidenav.default.html -->
<div class="sidenav">
<a href="#content">Jump to contents</a>
</div>
<hr>
<div class="sidenav">
<a href="../../index.html">IOCCC Home</a>
<a href="../../years.html">Winning Entries</a>
<a href="../../authors.html">People who have won</a>
<a href="../../location.html">Location of authors</a>
</div>
<hr>
<div class="sidenav">
<a href="../../news.html">IOCCC News</a>
<a href="../../faq.html#submit">How to enter the IOCCC</a>
<a href="../../faq.html">FAQ</a>
</div>
<hr>
<div class="sidenav">
<a href="../../bugs.html">Bugs &amp; (mis)features</a>
<a href="../../faq.html#fix_an_entry">Fixing IOCCC entries</a>
<a href="../../faq.html#fix_web_site">Fixing the web site</a>
<a href="../../faq.html#fix_author">Fixing author info</a>
<a href="../../thanks-for-help.html">Thanks for the help</a>
</div>
<hr>
<div class="sidenav">
<a href="../../judges.html">The IOCCC Judges</a>
<a href="../../contact.html">Contacting the IOCCC</a>
</div>
<hr>
<div class="sidenav">
<a href="#top">Jump to top</a>
</div>
<!-- END: this line ends content from: inc/sidenav.default.html -->
<!-- START: this line starts content from: inc/end-leftcolumn.default.html -->
</div>
<!-- END: this line ends content from: inc/end-leftcolumn.default.html -->
<!-- START: this line starts content from: inc/begin-rightcolumn.default.html -->
<div class="rightcolumn">
<!-- END: this line ends content from: inc/begin-rightcolumn.default.html -->
<!-- START: this line starts content from: inc/before-content.default.html -->
<a name="content"></a>
<!-- END: this line ends content from: inc/before-content.default.html -->
<!-- START: this line starts content for HTML phase 20 by: bin/output-index-author.sh via bin/md2html.sh -->
<!-- START: this line starts content generated by: bin/output-index-author.sh -->
<h2 id="author"><a name="author"></a>Author:</h2>
<ul>
<li>Name: <a href="../../authors.html#J_David_Lowe">J. David
Lowe</a><br> Location: <a href="../../location.html#US">US</a> -
<em>United States of America</em> (<em>United States</em>) <br><br></li>
</ul>
<!-- END: next line ends content generated by: bin/output-index-author.sh -->
<!-- END: this line ends content for HTML phase 20 by: bin/output-index-author.sh via bin/md2html.sh -->
<!-- START: this line starts content for HTML phase 21 by: bin/pandoc-wrapper.sh via bin/md2html.sh -->
<h2 id="to-build">To build:</h2>
<div class="sourceCode" id="cb1"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="fu">make</span></span></code></pre></div>
<h3 id="bugs-and-misfeatures">Bugs and (Mis)features:</h3>
<p>The current status of this entry is:</p>
<pre><code>STATUS: missing or dead link - please provide them
STATUS: INABIAF - please **DO NOT** fix</code></pre>
<p>For more detailed information see <a
href="../../bugs.html#2011_dlowe">2011 dlowe bugs</a>.</p>
<h2 id="to-use">To use:</h2>
<div class="sourceCode" id="cb3"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-</span><span class="op">&lt;</span>n_iterations<span class="op">&gt;</span> corpus1/ <span class="pp">[</span><span class="ss">...</span><span class="pp">]</span> corpus0/ <span class="op">&lt;</span> start.net <span class="op">&gt;</span> trained.net</span></code></pre></div>
<p>NOTE: In the above command, the directory args MUST end in a
<code>/</code>.</p>
<p>Then to use <code>trained.net</code>:</p>
<div class="sourceCode" id="cb4"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> file [file ...] <span class="op">&lt;</span> trained.net</span></code></pre></div>
<h2 id="try">Try:</h2>
<p>Try these pre-trained networks:</p>
<div class="sourceCode" id="cb5"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./try.sh</span></span></code></pre></div>
<p>NOTE: The <code>dlowe-aux-data/</code> directory, which <a
href="try.sh">try.sh</a> refers to, was created by:</p>
<div class="sourceCode" id="cb6"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="fu">tar</span> <span class="at">-jxf</span> dlowe-aux-data.tar.bz2</span></code></pre></div>
<h2 id="judges-remarks">Judges’ remarks:</h2>
<p>This entry is a ghoulish example of a brain (dead?) neural network
classifier. It comes with a set of corpora for you to try.</p>
<h3
id="ioccc-winlose-trained.net"><code>ioccc-winlose-trained.net</code></h3>
<p>You can create your own trained networks. For example, we trained a
network on the C code of entries that were supplied to the 20Th
IOCCC.</p>
<p>The <code>ioccc-winlose-trained.net</code> was trained using the <a
href="earlystop.pl">earlystop.pl</a> tool:</p>
<div class="sourceCode" id="cb7"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rm</span> <span class="at">-f</span> ioccc-winlose-trained.net</span>
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a><span class="ex">./earlystop.pl</span> ioccc-winlose-trained.net ioccc_won_training/ ioccc_lost_training/ ioccc_won_test/ ioccc_lost_test/</span></code></pre></div>
<p>where:</p>
<ul>
<li><p><code>ioccc_won_training/</code> 1/2 of C code of the 20th IOCCC
entries (except for <a href="dlowe.c">dlowe.c</a> from 2011) plus 1/2 of
the C code winning entries from prior IOCCC contests</p></li>
<li><p><code>ioccc_won_test/</code> 1/2 of C code of the 20th IOCCC
entries (except for <a href="dlowe.c">dlowe.c</a> from 2011) plus 1/2 of
the C code winning entries from prior IOCCC contests</p></li>
<li><p><code>ioccc_lost_training/</code> 1/2 of C code from entries that
that did not win</p></li>
<li><p><code>ioccc_lost_test/</code> 1/2 of C code from entries that
that did not win</p></li>
</ul>
<p>NOTE: The above directories were not shipped and everything from the
entries that did not win were flushed to <code>/dev/null</code>.</p>
<p>The <a href="earlystop.pl">earlystop.pl</a> tool wrote:</p>
<pre><code>training 1 to 1001...
training 1001 to 2001...
prior error = 56.746974766043; current error = 58.095691135305</code></pre>
<p>The resulting <code>ioccc-winlose-trained.net</code> file was
included in the <a
href="dlowe-aux-data.tar.bz2">dlowe-aux-data.tar.bz2</a> tarball.</p>
<p>The <a href="dlowe.c">dlowe.c</a> was explicitly excluded from this
trailing set, so this test is interesting:</p>
<div class="sourceCode" id="cb9"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> dlowe.c <span class="op">&lt;</span> dlowe-aux-data/ioccc-winlose-trained.net</span></code></pre></div>
<p>NOTE: this is done in <a href="try.sh">try.sh</a>.</p>
<p>The match of <a href="dlowe.c">dlowe.c</a> to the lose/win trailed
network was:</p>
<pre><code>dlowe.c 0.125108</code></pre>
<p>NOTE: The non-artificially intelligent decisions of the IOCCC judges
include random quirks and their non-linear judgment calls not found in
the ioccc trained network. Just because C code is or isn’t scored highly
by this network does not mean it will win or lose a future IOCCC. On the
other hand … it just might!</p>
<h4 id="other-data-sets-to-try">other data sets to try</h4>
<p>While we cannot supply you with the losing IOCCC source code, the
author did supply a number data sets on which to test:</p>
<p>We created the <a
href="dlowe-aux-data/english-trained.net">english-trained.net</a> as
follows:</p>
<div class="sourceCode" id="cb11"><pre
class="sourceCode sh"><code class="sourceCode bash"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-8000</span> dlowe-aux-data/english-1/ dlowe-aux-data/english-0/ <span class="op">&lt;</span> /dev/null <span class="op">&gt;</span> dlowe-aux-data/english-trained.net</span></code></pre></div>
<ul>
<li><p><a href="dlowe-aux-data/english-0/">dlowe-aux-data/english-0/</a>
Non-English (French) text</p></li>
<li><p><a href="dlowe-aux-data/english-1">dlowe-aux-data/english-1/</a>
English (non-French) text</p></li>
</ul>
<p>We created the <a
href="dlowe-aux-data/png-trained.net">png-trained.net</a> as
follows:</p>
<div class="sourceCode" id="cb12"><pre
class="sourceCode sh"><code class="sourceCode bash"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-8000</span> png-1/ png-0/ <span class="op">&lt;</span> /dev/null <span class="op">&gt;</span> dlowe-aux-data/png-trained.net</span></code></pre></div>
<ul>
<li><p><a href="dlowe-aux-data/png-0/">dlowe-aux-data/png-0/</a> Non-png
(gif) images</p></li>
<li><p><a href="dlowe-aux-data/png-1/">dlowe-aux-data/png-1/</a> png
(non-gif) images</p></li>
</ul>
<p>We created the <a
href="dlowe-aux-data/xor-trained.net">xor-trained.net</a> as
follows:</p>
<div class="sourceCode" id="cb13"><pre
class="sourceCode sh"><code class="sourceCode bash"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-8000</span> dlowe-aux-data/xor-1/ dlowe-aux-data/xor-0/ <span class="op">&lt;</span> /dev/null <span class="op">&gt;</span> dlowe-aux-data/xor-trained.net</span></code></pre></div>
<ul>
<li><p><a href="dlowe-aux-data/xor-0/">dlowe-aux-data/xor-0</a> Data
that XORs to 0</p></li>
<li><p><a href="dlowe-aux-data/xor-1/">dlowe-aux-data/xor-1/</a> Data
that XORs to 1</p></li>
</ul>
<p><strong>NOTE</strong>: The directory args <strong>must end in
a</strong> <code>/</code> for them to work.</p>
<h2 id="authors-remarks">Author’s remarks:</h2>
<h3 id="synopsis">Synopsis</h3>
<p>This is an <a
href="https://en.wikipedia.org/wiki/Artificial_intelligence">artificially
intelligent</a> judging tool to help the IOCCC judges. Here’s to
shorter, more frequent contests!</p>
<h3 id="description">Description</h3>
<p>This is a multilayer <a
href="https://en.wikipedia.org/wiki/Perceptron">perceptron</a> (a <a
href="https://en.wikipedia.org/wiki/Feedforward_neural_network">feedforward
artificial neural network</a>) which can be trained, using on-line <a
href="https://en.wikipedia.org/wiki/Backpropagation">backpropagation</a>,
to classify input files.</p>
<p>It has a fixed topology of <code>2^16</code> input neurons, 6 hidden
neurons and 1 output neuron.</p>
<p>The neurons’ activation function is the logistic function
<code>1 / (1 + e ^ -x)</code>.</p>
<h3 id="classifying">Classifying</h3>
<div class="sourceCode" id="cb14"><pre
class="sourceCode sh"><code class="sourceCode bash"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> file [file ...] <span class="op">&lt;</span> trained.net</span></code></pre></div>
<p>To classify files, one specifies a trained network (on
<code>stdin</code>) and one or more files to classify. The program will
output one line per successfully-classified file to <code>stderr</code>,
with the filename and the classification: a number between 0 and 1.</p>
<p>The interpretation of the classification number depends on how the
network was trained, but it’s geared toward interpretation as a
<em>probability</em> or a <em>confidence</em>.</p>
<h3 id="training">Training</h3>
<div class="sourceCode" id="cb15"><pre
class="sourceCode sh"><code class="sourceCode bash"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="ex">./dlowe</span> <span class="at">-</span><span class="op">&lt;</span>n_iterations<span class="op">&gt;</span> corpus1/ <span class="pp">[</span><span class="ss">...</span><span class="pp">]</span> corpus0/ <span class="op">&lt;</span> start.net <span class="op">&gt;</span> end.net</span></code></pre></div>
<p><strong>NOTE</strong>: The directory args <strong>must end in
a</strong> <code>/</code> for them to work.</p>
<p>To train a network, one specifies a starting network (on
<code>stdin</code>), two or more corpora (directories containing
training data), and the number of training iterations to run. The
program will write some progress data to <code>stderr</code> and, when
it’s done, will serialize the updated network to
<code>stdout</code>.</p>
<p>If no input network is given, a random new network will be generated
as a starting point (i.e. provide an empty <code>stdin</code>).</p>
<p>The first corpus will be assigned a target value of <code>1</code>.
The last will be assigned a target value of <code>0</code>. Intervening
directories (if any) will be assigned intermediate target values.</p>
<p>The learning rate is hard-coded as <code>0.3</code>. No momentum
factor is used.</p>
<h4 id="png-corpora">png corpora</h4>
<p>The <a href="dlowe-aux-data/png-1">png-1</a> corpus was obtained by
manually scraping the first results from an <a
href="https://images.google.com"
class="uri">https://images.google.com</a> search for “<a
href="https://www.google.com/search?tbm=isch&amp;sxsrf=APwXEddqIU34dxhpFdjEa8emNJKHOkYXIw:1683049158657&amp;source=hp&amp;biw=1727&amp;bih=952&amp;ei=xkpRZLapJtLKkPIP8bGKkAk&amp;iflsig=AOEireoAAAAAZFFY1nbmEUHC74Cmg3Y5rEY1fV7ocASZ&amp;ved=0ahUKEwi2pPXZltf-AhVSJUQIHfGYApIQ4dUDCAY&amp;uact=5&amp;oq=obfuscate+filetype:png&amp;gs_lcp=CgNpbWcQA1AAWABg9QNoAHAAeACAATuIATuSAQExmAEAoAECoAEBqgELZ3dzLXdpei1pbWc&amp;sclient=img&amp;q=obfuscate&amp;tbs=ift:png">obfuscate
filetype:png</a>”.</p>
<p>The <a href="dlowe-aux-data/png-0">png-0</a> corpus was obtained by
manually scraping the first results from an <a
href="https://images.google.com"
class="uri">https://images.google.com</a> search for “<a
href="https://www.google.com/search?tbm=isch&amp;ved=2ahUKEwiowo7fltf-AhXSOUQIHbJBDp8Q2-cCegQIABAA&amp;oq=obfuscate+filetype:gif&amp;gs_lcp=CgNpbWcQAzoECCMQJzoHCAAQigUQQzoFCAAQgARQ6AxY6AxgpBJoAHAAeACAAUeIAcsBkgEBM5gBAKABAaoBC2d3cy13aXotaW1nwAEB&amp;sclient=img&amp;ei=0UpRZOiVINLzkPIPsoO5-Ak&amp;bih=952&amp;biw=1727&amp;q=obfuscate&amp;tbs=ift:gif">obfuscate
filetype:gif</a>”.</p>
<h5 id="results">Results:</h5>
<ul>
<li>Using these corpora as a training set;</li>
<li>Using several hundred random <code>*.gif</code> and
<code>*.png</code> files from my home computer as a test set;</li>
<li>Using the included <a href="earlystop.pl">earlystop.pl</a> script,
which stopped after 67000 iterations.</li>
<li>Interpreting output of <code>&gt; 0.5</code> as “probably a
<code>.png</code>” and <code>&lt; 0.5</code> as “probably a
<code>.gif</code>”</li>
</ul>
<p>produced a network with about 87% accuracy</p>
<h4 id="english-corpora">English corpora</h4>
<p>The <a href="dlowe-aux-data/english-1">english-1</a> corpus was
obtained by manually scraping the first results from a <a
href="https://www.google.com/webhp?lr=lang_en"
class="uri">https://www.google.com/webhp?lr=lang_en</a> search for “<a
href="https://www.google.com/search?q=paris+filetype%3Atxt&amp;lr=lang_en&amp;tbs=lr%3Alang_1en&amp;sxsrf=APwXEdeHXyUYdUxCYQQDHmJhbHzX9aARSA%3A1683049834430&amp;source=hp&amp;ei=ak1RZJHUGN_DkPIP26OtkA8&amp;iflsig=AOEireoAAAAAZFFbejA0Xfqoq__FXx--Ok0S6GQjnGnA&amp;ved=0ahUKEwiRsZOcmdf-AhXfIUQIHdtRC_IQ4dUDCAo&amp;uact=5&amp;oq=paris+filetype%3Atxt&amp;gs_lcp=Cgdnd3Mtd2l6EANQAFgAYN4DaABwAHgAgAE-iAE-kgEBMZgBAKABAqABAQ&amp;sclient=gws-wiz">paris
filetype:txt</a>”.</p>
<p>The <a href="dlowe-aux-data/english-0">english-0</a> corpus was
obtained by manually scraping the first results from a <a
href="https://www.google.com/webhp?lr=lang_fr"
class="uri">https://www.google.com/webhp?lr=lang_fr</a> search for “<a
href="https://www.google.com/search?q=paris+filetype%3Atxt&amp;lr=lang_fr&amp;tbs=lr%3Alang_1fr&amp;sxsrf=APwXEddcsQ-gMhrqSUJo6omWFUl7jQMASg%3A1683049870736&amp;source=hp&amp;ei=jk1RZLevK_fKkPIPjbWN-AU&amp;iflsig=AOEireoAAAAAZFFbnvExHGkcavB5H8F0xsLAWN2KfYFK&amp;ved=0ahUKEwi3rrutmdf-AhV3JUQIHY1aA18Q4dUDCAo&amp;uact=5&amp;oq=paris+filetype%3Atxt&amp;gs_lcp=Cgdnd3Mtd2l6EANQAFgAYLMDaABwAHgAgAE-iAE-kgEBMZgBAKABAqABAQ&amp;sclient=gws-wiz">paris
filetype:txt</a>”.</p>
<h5 id="results-1">Results:</h5>
<ul>
<li>Using these corpora as a training set;</li>
<li>Using several dozen additional text files (obtained by the same
methods) as a test set;</li>
<li>Using the included <a href="earlystop.pl">earlystop.pl</a> script,
which stopped after 10000 iterations;</li>
<li>Interpreting output of <code>&gt; 0.5</code> as “probably English”
and <code>&lt; 0.5</code> as “probably French”</li>
</ul>
<p>produced a network with 100% accuracy.</p>
<h3 id="xor-corpora">xor corpora</h3>
<p>The <a href="dlowe-aux-data/xor-1">xor-1</a> corpus consists of two
files containing <code>01</code> and <code>10</code> respectively.</p>
<p>The <a href="dlowe-aux-data/xor-0">xor-0</a> corpus consists of two
files containing <code>00</code> and <code>11</code> respectively.</p>
<h5 id="results-2">Results:</h5>
<p>Using these corpora as a training set takes about 8000 iterations to
learn xor to within a tolerance of <code>&lt;0.01</code>.</p>
<h3 id="limitations">Limitations</h3>
<p>The program can’t tell you anything meaningful about files with less
than two bytes in them (I’m looking at you, <a
href="../../1994/smr/smr.c">smr.c</a>!)</p>
<p>You must include the trailing directory separator on training
directories (this allows the program to be portable without wasting
precious bytes on figuring out how to concatenate directory and file
names…).</p>
<p>Serialized network files are only portable between systems with the
same <a
href="https://en.wikipedia.org/wiki/Floating-point_arithmetic">floating-point</a>
representation and <a
href="https://en.wikipedia.org/wiki/Endianness">endianness</a>.</p>
<p>Making sure not to <a
href="https://en.wikipedia.org/wiki/Overfitting#Machine_learning">overfit</a>
the network to the training data is a bit of a black art. I have
enclosed <a href="earlystop.pl">earlystop.pl</a>, a wrapper script that
implements a simple ‘early stopping’ algorithm; other techniques are
possible.</p>
<p>Bad input (e.g. nonexistent files, non-numeric number of iterations,
etc.) tends to result in empty output.</p>
<p>Given exactly one corpus, the program will crash or produce
garbage.</p>
<p>Leaks memory and file descriptors while processing files.</p>
<p>Will crash and die horribly if it runs out of memory.</p>
<p>The Microsoft C compiler doesn’t provide a <code>dirent</code> API,
so to get this working on a Windows system you’ll need cygwin+gcc
(tested) or a <code>dirent</code> compatibility library (untested, but
they do exist).</p>
<p>Backpropagation doesn’t always converge: if you play with this long
enough, you’ll eventually have a training session that completely fails
to converge.</p>
<h3 id="obfuscation">Obfuscation</h3>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Zombie">Zombies</a>! (Since
neural networks are modeled after <a
href="https://en.wikipedia.org/wiki/Brain">BRAINS</a>, ya know? And
corpus sounds a lot like corpse. And I have 4- and 7-year-old kids ;)
)</li>
<li>Neural networks are interesting, and while the math isn’t terribly
difficult, their behavior is difficult to fully understand.</li>
<li>Had some extra space in one of the major data structures, and it
seemed a shame to waste it.</li>
<li>Similarly, I think three file pointers ought to be enough for
anyone.</li>
<li>Lots of magic numbers expressed in various ways.</li>
<li>Random abuse of random C trivia.</li>
<li>Some textual changes were made to maximize the classification of
this program by a network trained to recognize <a
href="../../authors.html">IOCCC entries</a> ;)</li>
</ul>
<p>… but mostly zombies!</p>
<h2 id="copyright-and-cc-by-sa-4.0-license">Copyright and CC BY-SA 4.0
License:</h2>
<p>This file is Copyright (c) 2023 by Landon Curt Noll. All Rights
Reserved. You are free to share and adapt this file under the terms of
this license:</p>
<pre><code>Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)</code></pre>
<p>For more information, see:
https://creativecommons.org/licenses/by-sa/4.0/</p>
<!-- END: this line ends content for HTML phase 21 by: bin/pandoc-wrapper.sh via bin/md2html.sh -->
<!-- START: this line starts content for HTML phase 22 by: bin/output-index-inventory.sh via bin/md2html.sh -->
<!-- START: this line starts content generated by: bin/output-index-inventory.sh -->
<h1 id="inventory-for-2011dlowe"><a name="inventory"></a>Inventory for
2011/dlowe</h1>
<p><a href="#top">Jump to top</a></p>
<h2 id="primary-files">Primary files</h2>
<ul>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe.c">dlowe.c</a>
- entry source</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/Makefile">Makefile</a>
- entry Makefile</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe.orig.c">dlowe.orig.c</a>
- original source</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/earlystop.pl">earlystop.pl</a>
- wrapper script to stop training early</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/try.sh">try.sh</a>
- how to run</li>
<li><a
href="dlowe-aux-data/png-0/5756029-DMZ-overview-B.gif">dlowe-aux-data/png-0/5756029-DMZ-overview-B.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/M2.gif">dlowe-aux-data/png-0/M2.gif</a> -
Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/Obfuscate.gif">dlowe-aux-data/png-0/Obfuscate.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/figure_49.gif">dlowe-aux-data/png-0/figure_49.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/klockhuset.gif">dlowe-aux-data/png-0/klockhuset.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/obfuscate_1.gif">dlowe-aux-data/png-0/obfuscate_1.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/podcastimage_150215.gif">dlowe-aux-data/png-0/podcastimage_150215.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/prjman1.gif">dlowe-aux-data/png-0/prjman1.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/proguard_obfuscate.gif">dlowe-aux-data/png-0/proguard_obfuscate.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/scripts_encryptor__screnc_-31918.gif">dlowe-aux-data/png-0/scripts_encryptor_<em>screnc</em>-31918.gif</a>
- Non-png gif images</li>
<li><a
href="dlowe-aux-data/png-0/visual_panel.gif">dlowe-aux-data/png-0/visual_panel.gif</a>
- Non-png gif images</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/040604.txt">dlowe-aux-data/english-0/040604.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/2001-2002.txt">dlowe-aux-data/english-0/2001-2002.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/CFP_TSI_Graphes_appel.txt">dlowe-aux-data/english-0/CFP_TSI_Graphes_appel.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/CIR_APE_08_doc%5B1%5D.txt">dlowe-aux-data/english-0/CIR_APE_08_doc%5B1%5D.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/admin_fr.txt">dlowe-aux-data/english-0/admin_fr.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/agenda.txt">dlowe-aux-data/english-0/agenda.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/paris.txt">dlowe-aux-data/english-0/paris.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/paris2.txt">dlowe-aux-data/english-0/paris2.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-0/structures3D-Paris.txt">dlowe-aux-data/english-0/structures3D-Paris.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/02-chronology.txt">dlowe-aux-data/english-1/02-chronology.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/36C215.txt">dlowe-aux-data/english-1/36C215.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/Wireless_Days_2011_CFP.txt">dlowe-aux-data/english-1/Wireless_Days_2011_CFP.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/cpb.txt">dlowe-aux-data/english-1/cpb.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/get-there.txt">dlowe-aux-data/english-1/get-there.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/pa_fv010.txt">dlowe-aux-data/english-1/pa_fv010.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/the_poor_people_of_paris.txt">dlowe-aux-data/english-1/the_poor_people_of_paris.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/utmost.txt">dlowe-aux-data/english-1/utmost.txt</a>
- training text for neural networks</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/dlowe-aux-data/english-1/walljpar.txt">dlowe-aux-data/english-1/walljpar.txt</a>
- training text for neural networks</li>
<li><a href="dlowe-aux-data/xor-0/00">dlowe-aux-data/xor-0/00</a> -
training digits for neural networks</li>
<li><a href="dlowe-aux-data/xor-0/11">dlowe-aux-data/xor-0/11</a> -
training digits for neural networks</li>
<li><a href="dlowe-aux-data/xor-1/01">dlowe-aux-data/xor-1/01</a> -
training digits for neural networks</li>
<li><a href="dlowe-aux-data/xor-1/10">dlowe-aux-data/xor-1/10</a> -
training digits for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/JavaScript-obfuscation-code.png">dlowe-aux-data/png-1/JavaScript-obfuscation-code.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/control_flow_obfuscation_before.png">dlowe-aux-data/png-1/control_flow_obfuscation_before.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/deceive-inveigle-obfuscate-t-shirts.png">dlowe-aux-data/png-1/deceive-inveigle-obfuscate-t-shirts.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/help_obfuscation1.png">dlowe-aux-data/png-1/help_obfuscation1.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/image_thumb%5B40%5D.png">dlowe-aux-data/png-1/image_thumb[40].png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/jquery_after_deobfuscation.png">dlowe-aux-data/png-1/jquery_after_deobfuscation.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/obfuscate4e.png">dlowe-aux-data/png-1/obfuscate4e.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/obfuscation-decoder.png">dlowe-aux-data/png-1/obfuscation-decoder.png</a>
- training image for neural networks</li>
<li><a
href="dlowe-aux-data/png-1/obfuscation_methods.png">dlowe-aux-data/png-1/obfuscation_methods.png</a>
- training image for neural networks</li>
<li><a href="2011_dlowe.tar.bz2">2011_dlowe.tar.bz2</a> - download entry
source</li>
</ul>
<h2 id="secondary-files">Secondary files</h2>
<ul>
<li><a href="index.html">index.html</a> - entry home page</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/.entry.json">.entry.json</a>
- description of entry in JSON</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/.gitignore">.gitignore</a>
- list of files that should not be committed under git</li>
<li><a href=".path">.path</a> - directory path from top directory</li>
<li><a
href="https://github.com/ioccc-src/temp-test-ioccc/blob/master/2011/dlowe/README.md">README.md</a>
- markdown source for index.html</li>
<li><a href="dlowe-aux-data.tar.bz2">dlowe-aux-data.tar.bz2</a> - IOCCC
win lose trained network</li>
<li><a
href="dlowe-aux-data/english-trained.net">dlowe-aux-data/english-trained.net</a>
- trained neural network for English text</li>
<li><a
href="dlowe-aux-data/ioccc-winlose-trained.net">dlowe-aux-data/ioccc-winlose-trained.net</a>
- trained neural network for C code from 20Th IOCCC</li>
<li><a
href="dlowe-aux-data/png-trained.net">dlowe-aux-data/png-trained.net</a>
- trained neural network for png-0 and png-1 images</li>
<li><a
href="dlowe-aux-data/xor-trained.net">dlowe-aux-data/xor-trained.net</a>
- trained neural network for xor-0 and xor-1 data</li>
</ul>
<!-- END: next line ends content generated by: bin/output-index-inventory.sh -->
<!-- END: this line ends content for HTML phase 22 by: bin/output-index-inventory.sh via bin/md2html.sh -->
<!-- START: this line starts content from: inc/after-content.default.html -->
<!-- END: this line ends content from: inc/after-content.default.html -->
<!-- START: this line starts content from: inc/end-rightcolumn.default.html -->
</div>
<!-- END: this line ends content from: inc/end-rightcolumn.default.html -->
<!-- START: this line starts content from: inc/end-row.default.html -->
</div>
<!-- END: this line ends content from: inc/end-row.default.html -->
<!-- START: this line starts content from: inc/footer.default.html -->
<div class="footer">
<hr>
<h2>Coda</h2>
<p><a href="https://validator.w3.org/nu/?doc=https%3A%2F%2Fioccc-src.github.io%2Ftemp-test-ioccc%2F2011%2Fdlowe%2Findex.html">Nu HTML check this web page</a></p>
<p><a href="#top">Jump to top</a> <a href="#content">Jump to Content</a> <a href="#inventory">Jump to Inventory</a></p>
</div>
<!-- END: this line ends content from: inc/footer.default.html -->
<!-- START: this line starts content from: inc/bottom.default.html -->
</body>
</html>
<!-- END: this line ends content from: inc/bottom.default.html -->
Reference in new issue View Git Blame Copy Permalink