mirror/ioccc
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

improved indentation and whitespace for markdown code blocks

Browse Source 
All code blocks in markdown have been intended by at least 4 spaces.

We add `--tab-stop=8` to all `pandoc(1)` executions in order to set
a better tab stop indentation, especially for tab intended code blocks.

Changed `bin/pandoc-wrapper.sh` VERSION from "1.5.3 2024-03-26"
to "1.6 2024-03-31".

Fixed `1991/brnstnd/sorta.README.md` so that the sorta code
is code blocks.

Removed unneeded paragraph group in copyright in `inc/footer.default.html`.

Fixed excess whitespace after list Author on entry `index.html` page.

Changed `bin/output-index-author.sh` VERSION from "1.5.2 2024-03-26"
to "1.6 2024-03-31".

Performed `make www` to perform the above and test under macOS.

TODO: Fix the problem of the initial line of code being extra intended.
TODO: Fix excess whitespace around code that is inline.
This commit is contained in:
Landon Curt Noll committed 2024-04-01 04:48:48 -07:00
1 parent b84c27ce02
commit a996f8d2b8
734 files changed
+16723 -16963
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

2019/mills/README.md
+141 -141
View File
@@ -1,7 +1,7 @@
## To build:
```sh
make
make
```
@@ -10,7 +10,7 @@ make
The current status of this entry is:
```
STATUS: INABIAF - please **DO NOT** fix
STATUS: INABIAF - please **DO NOT** fix
```
For more detailed information see [2019 mills bugs](../../bugs.html#2019_mills).
@@ -19,16 +19,16 @@ For more detailed information see [2019 mills bugs](../../bugs.html#2019_mills).
## To use:
```sh
make cpclean
# Let this run for about about an hour and then kill it:
./prog Shakespeare.txt
make cpclean
# Let this run for about about an hour and then kill it:
./prog Shakespeare.txt
```
## Try:
```sh
./try.sh
./try.sh
```
However, as the binary model files used to produce the output are in an
@@ -93,56 +93,56 @@ For your first OMLET experiment, we will try training a simple single-level
RNN to write some [Shakespeare][5] plays. Start by typing
```sh
make
make
```
After it builds, train it using
```sh
./prog Shakespere.txt
./prog Shakespere.txt
```
This will immediately start outputting gibberish to the output, e.g.
```sh
./prog Shakespere.txt
./prog Shakespere.txt
```
produces:
```
sins ohennAu
T-teooclelp tiThoWy
sins ohennAu
T-teooclelp tiThoWy
g
nlakuafy
e
sselW usnsofueB Aoee pasfUsuslhe ooM ot Wou moy
me neltAl -no IoyI mhuyakse inT-l chu ghenn ffo? fnsoe yhyye
ue nnfrlass heUthole saounlcesyee pee
t,
T0:0% 3.210888
o',,vU
An ,hTf lnm Far rur:s moilt WoEgrv wonds mith Aog thernw
Rni So
co Nnd :
For an bImy pgafoun:
Wf'r hom wortiverita
int fod mous Eheledet,
Tho he theket nonS wnu-ang dorlaMSp
nrocWiSe tflg 'o.
T0:0% 2.995950
d whecedhencrysesil yr bn,
we hh y thiwt
g
nlakuafy
e
sselW usnsofueB Aoee pasfUsuslhe ooM ot Wou moy
me neltAl -no IoyI mhuyakse inT-l chu ghenn ffo? fnsoe yhyye
ue nnfrlass heUthole saounlcesyee pee
t,
T0:0% 3.210888
o',,vU
An ,hTf lnm Far rur:s moilt WoEgrv wonds mith Aog thernw
Rni So
co Nnd :
For an bImy pgafoun:
Wf'r hom wortiverita
int fod mous Eheledet,
Tho he theket nonS wnu-ang dorlaMSp
nrocWiSe tflg 'o.
T0:0% 2.995950
d whecedhencrysesil yr bn,
we hh y thiwt
hut fithlot,
Fmdy s he alt
Vh th no dh foud bobt werw:s Aotnf Fhwi't whe, eusu
hut fithlot,
Fmdy s he alt
Vh th no dh foud bobt werw:s Aotnf Fhwi't whe, eusu
lhh thele wewcond ary soupfy wind tDont couc ths:
er fucwald oncli hen bos, f
T0:1% 2.878945
lhh thele wewcond ary soupfy wind tDont couc ths:
er fucwald oncli hen bos, f
T0:1% 2.878945
```
The gibberish is the networks attempt to write a play. So far, it's not very
@@ -150,7 +150,7 @@ successful! Between the chunks of gibberish are training progress reports
that look like
```
T0:1% 2.878945
T0:1% 2.878945
```
This tells you that we are 1% through training epoch 0, and that the training
@@ -158,31 +158,31 @@ This tells you that we are 1% through training epoch 0, and that the training
reduce and the generated snippets will improve quickly:
```
ses, kuth
LAs of the wish,
As, I nos you,
Yov not to nalll,
Tr tot wonds.
ses, kuth
LAs of the wish,
As, I nos you,
Yov not to nalll,
Tr tot wonds.
First Sondy, llt lrte, our, tw.
First Sondy, llt lrte, our, tw.
First.
First.
BRUTUS:
Helsting the kith gops of hoch Whay, fars surd what to,
The cownens golt te.
T0:12% 2.259114
eplerrotrur tandans one wiok thy or thach and cullice ded yourssting
And wours:
Whed ur surt.
BRUTUS:
Helsting the kith gops of hoch Whay, fars surd what to,
The cownens golt te.
T0:12% 2.259114
eplerrotrur tandans one wiok thy or thach and cullice ded yourssting
And wours:
Whed ur surt.
SINENIUS:
On we lain bith
rerytund: tich lon hyivetetgor.
SINENIUS:
On we lain bith
rerytund: tich lon hyivetetgor.
VOLUONIA:
He brich nom dove worthan then wise,
T0:13% 2.254926
VOLUONIA:
He brich nom dove worthan then wise,
T0:13% 2.254926
```
It's already started to figure out things about Shakespeare plays -- how to
@@ -194,40 +194,40 @@ Eventually, we will finish with the training data set and move on to a
validation cycle:
```
y fath onother,
I sucess,
For I me the west crare.
y fath onother,
I sucess,
For I me the west crare.
TRANIO:
Whow and'd have not to had you you one in my lapteny she very ame come me a gut and shourd aghir you as ignested; shend to make I strem
To h
T0:99% 1.924063
notnce gaud and is nicked thou day, ha the dusing you disaid: in thim, you things in ere thee thus erile Iht that tare theme my hast thesp thou shay: thou not eaten-or-ho-bess resing: I the but had d
T0:99% 1.923128
V0:0% 1.678907
V0:9% 1.700527
V0:18% 1.733179
V0:27% 1.714891
V0:36% 1.716672
V0:45% 1.782946
V0:54% 1.835629
V0:63% 1.876108
V0:72% 1.906814
V0:81% 1.924096
V0:90% 1.954492
V0:99% 1.969287
serfs you'll alliencseard:
We
got you? before
I say.
TRANIO:
Whow and'd have not to had you you one in my lapteny she very ame come me a gut and shourd aghir you as ignested; shend to make I strem
To h
T0:99% 1.924063
notnce gaud and is nicked thou day, ha the dusing you disaid: in thim, you things in ere thee thus erile Iht that tare theme my hast thesp thou shay: thou not eaten-or-ho-bess resing: I the but had d
T0:99% 1.923128
V0:0% 1.678907
V0:9% 1.700527
V0:18% 1.733179
V0:27% 1.714891
V0:36% 1.716672
V0:45% 1.782946
V0:54% 1.835629
V0:63% 1.876108
V0:72% 1.906814
V0:81% 1.924096
V0:90% 1.954492
V0:99% 1.969287
serfs you'll alliencseard:
We
got you? before
I say.
Farstred dentlentecaly, sir, I it one bosticield
All me the backnour mino,
Whith capitaned mid! but stell the ifvemion
Willerity.
Farstred dentlentecaly, sir, I it one bosticield
All me the backnour mino,
Whith capitaned mid! but stell the ifvemion
Willerity.
First Cumfol of
T1:0% 1.885619
First Cumfol of
T1:0% 1.885619
```
Validation cycles are used to test the network to see if it has learned to
@@ -237,7 +237,7 @@ on the validation set is also displayed with a validation progress report that
looks like
```
V0:36% 1.716672
V0:36% 1.716672
```
which means we are 36% of the way through the validation for epoch 0, and the
@@ -260,7 +260,7 @@ You can continue training from a previous checkpoint by providing the
checkpoint file name as the second parameter, for example:
```sh
./prog Shakespeare.txt cp01_1.970
./prog Shakespeare.txt cp01_1.970
```
After the validation cycle finishes, OMLET begins the next epoch by restarting
@@ -276,7 +276,7 @@ parameters to generate data. Inference mode takes the checkpoint file as
command like
```sh
./prog < cp55_1.807
./prog < cp55_1.807
```
Running it produces an infinite amount of generated output, until you hit
@@ -295,8 +295,8 @@ executable name `prog`) is the simplest recurrent neural network. It looks
like
```c
h = tanh(Wxh * x + Whh * h' + Bh)
y = Why * h + By
h = tanh(Wxh * x + Whh * h' + Bh)
y = Why * h + By
```
where
@@ -318,53 +318,53 @@ It is the presence of the hidden state vector that allows the RNN to
state. If you type
```sh
make lin1
make lin1
```
OMLET will create an [ADALINE][10] network that does
```c
y = Wxy * x + By
y = Wxy * x + By
```
This is a simple [linear][11] [feed-forward network][12]. You can run it with
```sh
./lin1 Shakespeare.txt
./lin1 Shakespeare.txt
```
The linear network won't be able to get past the gibberish stage, because it
lacks history:
```sh
./lin1 Shakespeare.txt
./lin1 Shakespeare.txt
```
produces:
```
UERond w,
UERond w,
Gir:
KINof s, mesther s thouth.
E:
KINTret, at fu,
GOMy t, as sth kesewit sooos atse ang k, ck,
Sotheouserivesthecowhet been's, t he, h nre; t and, har wiread of pincer cedst sur has, ut:
Gir:
KINof s, mesther s thouth.
E:
KINTret, at fu,
GOMy t, as sth kesewit sooos atse ang k, ck,
Sotheouserivesthecowhet been's, t he, h nre; t and, har wiread of pincer cedst sur has, ut:
T14:67% 2.465115
UKESpan,
NGaromy soreate e m esewfoure pamitherarjulthengeoly tl.
NG s at e! w.
T14:67% 2.465115
UKESpan,
NGaromy soreate e m esewfoure pamitherarjulthengeoly tl.
NG s at e! w.
WAllinoully?
Wamisw ofilem:
I'delandinarrstath har aksubly s cath Whern t Is, weciss:
WAllinoully?
Wamisw ofilem:
I'delandinarrstath har aksubly s cath Whern t Is, weciss:
GLat s; llde.
Y aterit dsthence
T14:67% 2.465404
GLat s; llde.
Y aterit dsthence
T14:67% 2.465404
```
It is able to guess at what character is likely to follow the current one
@@ -389,9 +389,9 @@ We can try to improve the RNN's performance by stacking RNN modules atop each
other:
```c
h1 = RNN(h1', x)
h2 = RNN(h2', h1)
y = Why * h2 + By
h1 = RNN(h1', x)
h2 = RNN(h2', h1)
y = Why * h2 + By
```
with `RNN(h, x)` defined as above. Each RNN module has its own set of
@@ -415,13 +415,13 @@ the hard limit with `limit -h stacksize` and set it with
You can try the deeper network by doing
```sh
make rnn2
make rnn2
```
(or even `make rnn3` if you want a three-layer RNN) and train it with
```sh
./rnn2 Shakespeare.txt
./rnn2 Shakespeare.txt
```
The additional depth should allow the network to make better predictions (it
@@ -445,13 +445,13 @@ solve this problem. Christopher Olah gives a good description of them
at his [blog posting][20]. You can build a two-level LSTM by doing
```sh
make lstm2
make lstm2
```
and train with it with
```sh
./lstm2 Shakespeare.txt
./lstm2 Shakespeare.txt
```
The LSTM is much easier to train because it explicitly decides how to update
@@ -465,11 +465,11 @@ its hidden state via "gates". These gates are called
The basic LSTM equations are
```c
f = sigmoid(Wxf * x + Whf * h' + Bf)
i = sigmoid(Wxi * x + Whi * h' + Bi)
o = sigmoid(Wxo * x + Who * h' + Bo)
c = f * c' + i * tanh(Wxc * x + Whc * h' + Bc)
h = o * tanh(c)
f = sigmoid(Wxf * x + Whf * h' + Bf)
i = sigmoid(Wxi * x + Whi * h' + Bi)
o = sigmoid(Wxo * x + Who * h' + Bo)
c = f * c' + i * tanh(Wxc * x + Whc * h' + Bc)
h = o * tanh(c)
```
Where
@@ -497,7 +497,7 @@ require fewer learned parameters. This allows them to train faster than a
generic LSTM. You can build a two-layer GRU with
```sh
make gru2
make gru2
```
### Building your own networks
@@ -511,9 +511,9 @@ a `-DNW='...'` command which consists of a series of comma-separated
assignments. For example, the simple one-layer RNN could be defined like
```make
-DNW=' x = I(n), hp = I(128), \
h = C(hp, T(A(L(128, x), L(128, hp)))), \
y = L(n, h)'
-DNW=' x = I(n), hp = I(128), \
h = C(hp, T(A(L(128, x), L(128, hp)))), \
y = L(n, h)'
```
The network declares `x` as an input vector (there must be a declaration for
@@ -556,16 +556,16 @@ As an example of a more complicated network, we can look at a two-layer
GRU network:
```make
-DHS=128, \
-DNW=' x = I(n), \
y = L(n, MD(MD(x)))' \
-DBK=' hp = I(HS), \
z = S(A(L(HS, x), L(HS, hp))), \
r = S(A(L(HS, x), L(HS, hp))), \
c = T(A(L(HS, x), L(HS, hp))), \
zc = OG(1, -1, z), \
h = C(hp, A(M(zc, hp), M(z, c))), \
y = h'
-DHS=128, \
-DNW=' x = I(n), \
y = L(n, MD(MD(x)))' \
-DBK=' hp = I(HS), \
z = S(A(L(HS, x), L(HS, hp))), \
r = S(A(L(HS, x), L(HS, hp))), \
c = T(A(L(HS, x), L(HS, hp))), \
zc = OG(1, -1, z), \
h = C(hp, A(M(zc, hp), M(z, c))), \
y = h'
```
We are using a few new tricks here -- first, we are defining `HS` as the size