mirror/ioccc
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

force use of pandoc version >= 3.1.12.2

Browse Source 
The `bin/pandoc-wrapper.sh` tool will verify that the ssytem
has a recent enough version of `pandoc(1)`.

We explain why we insist on `pandoc(1)`. version >= 3.1.12.2
in comments where the $PANDOC_MIN_VERSION is set.

Added `--wrap=preserve` as a default argument to `pandoc(1)`.
While this change will impact every HTML file generated,
we believe this will improve the ability of someone to look
at the generated HTML and correlate with the original markdown.

Changed `bin/pandoc-wrapper.sh` VERSION from "1.4.2 2024-03-17"
to "1.4.3 2024-03-18".

Performed `make www` to achieve all of the above and to generate
HTML using the new  `--wrap=preserve` `pandoc(1)` mode.
This commit is contained in:
Landon Curt Noll committed 2024-03-18 01:26:39 -07:00
1 parent 9eb7a54e9f
commit b0b212acb1
402 files changed
+37061 -55132
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

2015/burton/road.to.obscurity.html
+132 -161
View File
@@ -120,28 +120,26 @@
<!-- END: this line ends content from: inc/before-content.default.html -->
<!-- START: this line starts content for HTML phase 21 by: bin/pandoc-wrapper.sh via bin/md2html.sh -->
<h1 id="development-of-the-obfuscated-calculator.">Development of the
obfuscated calculator.</h1>
<h1 id="development-of-the-obfuscated-calculator.">Development of the obfuscated calculator.</h1>
<p>I wanted to enter the IOCCC….</p>
<p>I think I can write something clever. Possibly even obscure. The
competition is fierce, but not a closed door.</p>
<p>I needed an interesting program, that had a reasonable chance of
being shrunk within 4096 bytes, and lent itself to obfuscation. I had
been revisiting utilities I’ve written over the years. One useful
program was <code>calc</code>, a simple recursive descent signed integer
expression parser I developed from first principles as a demonstration
of how computers work for my son. That program did the basic four
operations, but as a programmer, I needed more, so those were added over
time.</p>
<p><code>calc</code> is a useful, functioning utility. The operators
before the obfuscation dance began (this is called foreshadowing):</p>
<p>I think I can write something clever. Possibly even obscure. The competition
is fierce, but not a closed door.</p>
<p>I needed an interesting program, that had a reasonable chance of being shrunk
within 4096 bytes, and lent itself to obfuscation. I had been revisiting
utilities I’ve written over the years. One useful program was <code>calc</code>, a simple
recursive descent signed integer expression parser I developed from first
principles as a demonstration of how computers work for my son. That program
did the basic four operations, but as a programmer, I needed more, so those
were added over time.</p>
<p><code>calc</code> is a useful, functioning utility. The operators before the obfuscation
dance began (this is called foreshadowing):</p>
<pre><code>() forced precedence
- ~ ? unary minus, compl, byteswap
* / % &amp; &lt;&lt; &gt;&gt; mul, div, mod, bitand, shift: left, right
+ - | ^ add, sub, bitor, xor
NUM 0-9</code></pre>
<p>This seemed like a reasonable candidate to experiment with
obfuscation; perhaps it might yield good results.</p>
<p>This seemed like a reasonable candidate to experiment with obfuscation; perhaps
it might yield good results.</p>
<h2 id="principle-of-operation">Principle of operation</h2>
<p>The algorithm was and is rather straightforward:</p>
<pre><code>expr() {
@@ -180,8 +178,8 @@ factor() {
error()
}</code></pre>
<h2 id="obfuscation-is-a-process">Obfuscation is a process</h2>
<p>Within the above algorithm, there are many details unmentioned, all
ripe for obfuscation:</p>
<p>Within the above algorithm, there are many details unmentioned, all ripe for
obfuscation:</p>
<ul>
<li>input</li>
<li>tokenization</li>
@@ -191,164 +189,137 @@ ripe for obfuscation:</p>
<li>swapping bytes</li>
<li>user interface (command line vs <code>stdin</code>)</li>
</ul>
<p>Uncertain as to how this would progress, I started
obfuscating/condensing everything except the conversion operations,
leaving those to <code>sprintf(3)</code> and <code>strtol(3)</code>. The
first version of the obfuscated code used an homage to 1984 anon for
I/O, and it featured a <code>#define</code> for <code>else</code>s and
the types, to help minimize the repetition. And of course, most
functions were renamed with one or two letters, and most of the
variables became single letter globals. Tokenization was rather easy
with a declared array, just initialized with unique values for each.</p>
<p>This was when the obfuscation took a turn for the better (worse): it
was easy to hide the operations via a simple transformation. The code
now dealt with small integer values instead of direct ASCII
representation of the operations. And the strings were collapsed into a
single combined error message and operator tokenization table. This was
relatively obscure.</p>
<p>Next, <code>sprintf(3)</code> was replaced with a custom formatter.
This was rather simple, but at least the formatting was not obvious.
Formatting and printing were split early. <code>strtol(3)</code> was
replaced with custom logic, limited to 3 practical bases: octal,
decimal, and hex. The last result was remembered in a special variable,
dot.</p>
<p>By this time, I had a version which I considered obscure, and shared
this early version with a couple of colleagues for comments. I suggested
I could improve the obfuscation of a <code>term()</code> and
<code>factor()</code>, and would like to have assignable memories.</p>
<p>This code was pretty dense, but inspired by previous IOCCC entries, I
thought I might get rid of all the <code>else</code>s, which showed the
structure of the parser rather clearly. Investing in the ternary
operator, the code got quite a bit more difficult to follow, used only
<code>for</code> loops, and I had to start taking notes to follow it.
This was a good sign! ;-)</p>
<p>I started testing the code. I wrote a test suite to ensure the
changes I made did not corrupt the processing (oh so easy to do with
this obfuscated mess). Why does <code>-1&gt;&gt;32</code> yield
<code>-1</code>? Oh, yeah: signed arithmetic. Enter Java’s
<code>unsigned</code> shift: <code>&gt;&gt;&gt;</code>. Better. But with
more features, more to get wrong, and harder to know which expression is
causing the error. The easy solution: if the name begins with
<code>e</code>, echo the input. Now I can see the failing expression.
<p>Uncertain as to how this would progress, I started obfuscating/condensing
everything except the conversion operations, leaving those to <code>sprintf(3)</code> and
<code>strtol(3)</code>. The first version of the obfuscated code used an homage to 1984 anon
for I/O, and it featured a <code>#define</code> for <code>else</code>s and the types, to help minimize
the repetition. And of course, most functions were renamed with one or two
letters, and most of the variables became single letter globals. Tokenization
was rather easy with a declared array, just initialized with unique values for
each.</p>
<p>This was when the obfuscation took a turn for the better (worse): it was easy
to hide the operations via a simple transformation. The code now dealt with
small integer values instead of direct ASCII representation of the operations.
And the strings were collapsed into a single combined error message and
operator tokenization table. This was relatively obscure.</p>
<p>Next, <code>sprintf(3)</code> was replaced with a custom formatter. This was rather simple,
but at least the formatting was not obvious. Formatting and printing were
split early. <code>strtol(3)</code> was replaced with custom logic, limited to 3 practical
bases: octal, decimal, and hex. The last result was remembered in a special
variable, dot.</p>
<p>By this time, I had a version which I considered obscure, and shared this early
version with a couple of colleagues for comments. I suggested I could improve
the obfuscation of a <code>term()</code> and <code>factor()</code>, and would like to have assignable
memories.</p>
<p>This code was pretty dense, but inspired by previous IOCCC entries, I thought I
might get rid of all the <code>else</code>s, which showed the structure of the parser
rather clearly. Investing in the ternary operator, the code got quite a bit
more difficult to follow, used only <code>for</code> loops, and I had to start taking notes
to follow it. This was a good sign! ;-)</p>
<p>I started testing the code. I wrote a test suite to ensure the changes I made
did not corrupt the processing (oh so easy to do with this obfuscated mess).
Why does <code>-1&gt;&gt;32</code> yield <code>-1</code>? Oh, yeah: signed arithmetic. Enter Java’s
<code>unsigned</code> shift: <code>&gt;&gt;&gt;</code>. Better. But with more features, more to get wrong,
and harder to know which expression is causing the error. The easy solution: if
the name begins with <code>e</code>, echo the input. Now I can see the failing expression.
Wait, what is this test expression supposed to be testing?</p>
<p>Wouldn’t it be nice to have comments?</p>
<p>Adding new functionality to an obscure program is … going about it
backwards. Get the functionality correct and complete, THEN obscure.
Sounds obvious, but that did not happen. I got comments working, then
desired to have multiple statements on a single line (read: semicolon
expression terminators). This involved quite a bit of rework of the
input code, but now allowed very convenient user interaction.</p>
<h2 id="better-obscurity-is-a-laborious-process">Better Obscurity is a
laborious process</h2>
<p>Satisfied with the functionality, but unsatisfied with the obscurity,
I pondered if I could get rid of all the constants? This had not been
done before (to my knowledge), and I was able to work out the how-tos
and the compromises. Now I can no longer declare my arrays, I have to
allocate them. Obscurity now increased!</p>
<p>Adding new functionality to an obscure program is … going about it backwards.
Get the functionality correct and complete, THEN obscure. Sounds obvious, but
that did not happen. I got comments working, then desired to have multiple
statements on a single line (read: semicolon expression terminators). This
involved quite a bit of rework of the input code, but now allowed very
convenient user interaction.</p>
<h2 id="better-obscurity-is-a-laborious-process">Better Obscurity is a laborious process</h2>
<p>Satisfied with the functionality, but unsatisfied with the obscurity, I
pondered if I could get rid of all the constants? This had not been done
before (to my knowledge), and I was able to work out the how-tos and the
compromises. Now I can no longer declare my arrays, I have to allocate them.
Obscurity now increased!</p>
<p>But wouldn’t it be nice to have assignable memories?</p>
<p>Around this time, the formatted code no longer fit with the 2053
limit (due to a lack of comprehension of the significance of the
space-after-brace-and-semi rule, and a layout program that did not know
how to do this). Frustrated, I wrote another program to reformat this
code into <a href="rule2.c">rule2.c</a>, which was a single C token per
line. This solved the formatting problem, but intriguingly introduced
another. Now I was faced with the 4096 rule as well as the 2053 rule.
But this was a challenge!</p>
<p>I wrote a de-obfuscator, and worked on both the unobfuscated version
and the master code. I worked on the template formatter to make this as
automated as possible, and worked to achieve both a perfectly presented
picture AND the single-line exactly-at-4096 <a
href="rule2.c">rule2.c</a>.</p>
<p>Finally, “perfection”, with assignable memories! Where perfection is
defined as a nicely formatted picture at less than 2053, and exactly
4096 token-per-line <a href="rule2.c">rule2.c</a>.</p>
<p>Around this time, the formatted code no longer fit with the 2053 limit (due to a
lack of comprehension of the significance of the space-after-brace-and-semi
rule, and a layout program that did not know how to do this). Frustrated, I
wrote another program to reformat this code into <a href="rule2.c">rule2.c</a>, which was a
single C token per line. This solved the formatting problem, but intriguingly
introduced another. Now I was faced with the 4096 rule as well as the 2053
rule. But this was a challenge!</p>
<p>I wrote a de-obfuscator, and worked on both the unobfuscated version and the
master code. I worked on the template formatter to make this as automated as
possible, and worked to achieve both a perfectly presented picture AND the
single-line exactly-at-4096 <a href="rule2.c">rule2.c</a>.</p>
<p>Finally, “perfection”, with assignable memories! Where perfection is defined
as a nicely formatted picture at less than 2053, and exactly 4096
token-per-line <a href="rule2.c">rule2.c</a>.</p>
<h2 id="testing-testing-testing">Testing Testing Testing</h2>
<p>Cross platform, it works on Mac OSX, fails on Linux. It works if
<code>#include &lt;stdlib.h&gt;</code>, but it dumps core without it
(the only change!)?. 64-bits without proper declaration of
<code>int</code>s in syscalls yields unpredictable results. 32-bit
<p>Cross platform, it works on Mac OSX, fails on Linux. It works if <code>#include &lt;stdlib.h&gt;</code>, but it dumps core without it (the only change!)?. 64-bits without
proper declaration of <code>int</code>s in syscalls yields unpredictable results. 32-bit
compilation showed problems with byte-swapping in using addition versus
bitwise-or. In 32-bit land, <code>x&lt;&lt;32 == x</code>, so
<code>x&lt;&lt;32 + x&gt;&gt;32</code> does not yield the desired
result, but <code>x&lt;&lt;32 | x&gt;&gt;32</code> does (with
appropriate care and feeding). And the different compilers on different
hosts complained loudly and differently, although all produced correct
bits in spite of the noise.</p>
<p>Further, the program did not do UNIX pipelines well, due to the
unbuffered homage code (byte-at-a-time I/O), of which
<code>write(2)</code> was also the source of the portability problem.
Replace <code>write(2)</code> with
<code>getchar(3)</code>/<code>putchar(3)</code>, and it piped well but
lost some obscurity and no longer paid (direct) homage to history.</p>
<p>Another problem: only one expression at a time was allowed on the
command line, no semis, no comments, and it turns out <code>$2</code>,
<code>$3</code>, … were not handled correctly. Given the new I/O code,
it was now possible to copy the command line arguments into a buffer,
and write a small routine to retrieve bytes from this buffer like
<code>getchar(3)</code>. Voila! Better obfuscation with much better
functionality!</p>
bitwise-or. In 32-bit land, <code>x&lt;&lt;32 == x</code>, so <code>x&lt;&lt;32 + x&gt;&gt;32</code> does not yield the
desired result, but <code>x&lt;&lt;32 | x&gt;&gt;32</code> does (with appropriate care and feeding).
And the different compilers on different hosts complained loudly and
differently, although all produced correct bits in spite of the noise.</p>
<p>Further, the program did not do UNIX pipelines well, due to the unbuffered
homage code (byte-at-a-time I/O), of which <code>write(2)</code> was also the source of the
portability problem. Replace <code>write(2)</code> with <code>getchar(3)</code>/<code>putchar(3)</code>, and it
piped well but lost some obscurity and no longer paid (direct) homage to
history.</p>
<p>Another problem: only one expression at a time was allowed on the command line,
no semis, no comments, and it turns out <code>$2</code>, <code>$3</code>, … were not handled correctly.
Given the new I/O code, it was now possible to copy the command line arguments
into a buffer, and write a small routine to retrieve bytes from this buffer
like <code>getchar(3)</code>. Voila! Better obfuscation with much better functionality!</p>
<p>But wouldn’t it be nice to show what the current variables hold?</p>
<p>Bytes are now hard to find to keep exactly at 4096, and adding
functionality to obscured code is not easy. Restructuring the logic to
gain the bytes necessary, code to display the variables was added.</p>
<p>Test. Test. Test. Oh, noes! More bugs! Typing “<code>1 + 2 5</code>”
yields <code>3</code>, but no syntax error. “<code>g = 1</code>” yields
a syntax error, and “<code>0xg</code>” does not! “<code>)</code>” causes
a segfault! Why didn’t I test these before I started obfuscating?! By
this point, everything I did to the code perturbed the 4096 bytes, all
variables were already a single character, and it was really difficult
to follow the logic in a debugger (single-stepping an expression did the
entire line of code, even if it involved multiple statements – and it
did, everywhere, even in the unobfuscated version; think: a complex
chain of ternary expressions is a single statement). This involved
careful examination of all the expressions within the logic, trying to
find a different way of getting the same result, but typically within
fewer bytes than before to allow for the added logic to fix the
bugs.</p>
<p>The bugs were fixed, the code re-formatted, and <a
href="rule2.c">rule2.c</a> was again 4096 bytes.</p>
<p>Bytes are now hard to find to keep exactly at 4096, and adding functionality to
obscured code is not easy. Restructuring the logic to gain the bytes
necessary, code to display the variables was added.</p>
<p>Test. Test. Test. Oh, noes! More bugs! Typing “<code>1 + 2 5</code>” yields <code>3</code>, but
no syntax error. “<code>g = 1</code>” yields a syntax error, and “<code>0xg</code>” does not! “<code>)</code>”
causes a segfault! Why didn’t I test these before I started obfuscating?! By
this point, everything I did to the code perturbed the 4096 bytes, all variables
were already a single character, and it was really difficult to follow the logic
in a debugger (single-stepping an expression did the entire line of code, even
if it involved multiple statements – and it did, everywhere, even in the
unobfuscated version; think: a complex chain of ternary expressions is a single
statement). This involved careful examination of all the expressions within the
logic, trying to find a different way of getting the same result, but typically
within fewer bytes than before to allow for the added logic to fix the bugs.</p>
<p>The bugs were fixed, the code re-formatted, and <a href="rule2.c">rule2.c</a> was again
4096 bytes.</p>
<p>But wouldn’t it be nice for unary plus to work?</p>
<p>Add unary plus, re-balance. Little things now take a fair bit of
work. Adding anything requires taking away in others, but since all the
logic is functional, this means rewriting the logic to be more concise.
After several rounds of this, bytes are scarce to come by.</p>
<p>Add unary plus, re-balance. Little things now take a fair bit of work. Adding
anything requires taking away in others, but since all the logic is functional,
this means rewriting the logic to be more concise. After several rounds of
this, bytes are scarce to come by.</p>
<p>But wouldn’t it be nice for assignments to be silent?</p>
<p>Made them silent (easier said than done!). Why does
“<code>1 + 2 5</code>” print a syntax error, but
“<code>a = 1 + 2 5</code>” does not? Rewrite(!) the assignment logic –
and this requires more bytes!</p>
<h2 id="better-obscurity-through-simplification">Better Obscurity
Through Simplification</h2>
<p>Except there were not enough bytes to do this, unless
<code>#include &lt;unistd.h&gt;</code> is removed. But then
<code>sbrk()</code> must be declared, eating into the savings. If only
there were 512 bytes of memory already allocated that could be portably
touched for free….</p>
<p>Light bulb! Memory allocation was excised, and the code became more
obscure.</p>
<p>It was working again! It formats beautifully, it compiled cleanly
(with apropos <code>-Wno-guards</code>), it remains obscure after
<code>cpp(1)</code> and <code>clang-format(1)</code>, it measures within
the limits and exactly at 4096 (this latter was really getting
<p>Made them silent (easier said than done!). Why does “<code>1 + 2 5</code>” print a syntax
error, but “<code>a = 1 + 2 5</code>” does not? Rewrite(!) the assignment logic – and
this requires more bytes!</p>
<h2 id="better-obscurity-through-simplification">Better Obscurity Through Simplification</h2>
<p>Except there were not enough bytes to do this, unless <code>#include &lt;unistd.h&gt;</code> is
removed. But then <code>sbrk()</code> must be declared, eating into the savings. If only
there were 512 bytes of memory already allocated that could be portably touched
for free….</p>
<p>Light bulb! Memory allocation was excised, and the code became more obscure.</p>
<p>It was working again! It formats beautifully, it compiled cleanly (with apropos
<code>-Wno-guards</code>), it remains obscure after <code>cpp(1)</code> and <code>clang-format(1)</code>, it
measures within the limits and exactly at 4096 (this latter was really getting
challenging to maintain). No more bugs known, it is feature complete,
documentation written.</p>
<p>But wouldn’t it be nice to have a logical not?</p>
<p>Sigh. This was the last operation added. Another round of code churn,
expressions are difficult to rewrite because they are both obscure and
relatively tight. But changing around the stable initialization code –
by now the largest piece of logic – yielded the necessary bytes, and
improved obscurity by double duty encoding of the operations
initialization.</p>
<p>The utility is now truly functional, and feature complete, and
obscure. Without <a href="src.doc.txt">src.doc.txt</a> (basic
documentation of the code structure), and some commented expanded logic
stashed away, and the familiarity I have with the code having written
and revised it repeatedly, I would not even want to attempt to decipher
the logic. Many times in the debugging, I was thwarted by my own
relatively tight. But changing around the stable initialization code – by now
the largest piece of logic – yielded the necessary bytes, and improved
obscurity by double duty encoding of the operations initialization.</p>
<p>The utility is now truly functional, and feature complete, and obscure. Without
<a href="src.doc.txt">src.doc.txt</a> (basic documentation of the code structure), and some
commented expanded logic stashed away, and the familiarity I have with the code
having written and revised it repeatedly, I would not even want to attempt to
decipher the logic. Many times in the debugging, I was thwarted by my own
obfuscations, and went down blind paths in trying to fix a bug or add
functionality. Now I think I have a good entry.</p>
<p>But wouldn’t it be nice to import pre-set definitions from a
file?</p>
<p>But wouldn’t it be nice to import pre-set definitions from a file?</p>
<!-- END: this line ends content for HTML phase 21 by: bin/pandoc-wrapper.sh via bin/md2html.sh -->
<!-- START: this line starts content from: inc/after-content.default.html -->