From f098cf0872df84f100e0e7b559841d26d8a4ded4 Mon Sep 17 00:00:00 2001 From: Landon Curt Noll Date: Wed, 6 Aug 2025 16:11:29 -0700 Subject: [PATCH] improve prog.c and try.sh for 2024/kurdyukov4 We improve `prog.c` beyond the 4M byte patch to reach 64M bytes. The `try.sh` is made more robust, detecting both compiler and executable problems that are **NOT** bugs in the code. The code is now declared **INABIAF - please DO NOT fix**. A detailed description of compiler, linker and resource issues has been added to the `README.md` and `index.html`. --- 2024/kurdyukov4/.gitignore | 18 +- 2024/kurdyukov4/Makefile | 11 +- 2024/kurdyukov4/README.md | 63 ++++- 2024/kurdyukov4/index.html | 62 +++-- 2024/kurdyukov4/prog.c | 4 +- 2024/kurdyukov4/try.sh | 455 ++++++++++++++++++++++++++++++++++--- bugs.md | 39 +--- 7 files changed, 569 insertions(+), 83 deletions(-) diff --git a/2024/kurdyukov4/.gitignore b/2024/kurdyukov4/.gitignore index 1fbcdaea0..3ff0d18b8 100644 --- a/2024/kurdyukov4/.gitignore +++ b/2024/kurdyukov4/.gitignore @@ -5,7 +5,11 @@ *.o .DS_Store .*.swp -random.bin +random.64K.bin +random.1M.bin +random.4M.bin +random.16M.bin +random.64M.bin test0 test0.c test1 @@ -16,3 +20,15 @@ test2.out.c test3 test3.c test3.out +test4 +test4.c +test4.out +test5 +test5.c +test5.out +test6 +test6.c +test6.out +test7 +test7.c +test7.out diff --git a/2024/kurdyukov4/Makefile b/2024/kurdyukov4/Makefile index 2a491e1e9..42292ddcf 100644 --- a/2024/kurdyukov4/Makefile +++ b/2024/kurdyukov4/Makefile @@ -203,9 +203,14 @@ clean: clobber: clean ${RM} -f ${TARGET} ${ALT_TARGET} ${RM} -rf *.dSYM - ${RM} -f random.bin - ${RM} -f test0 test0.c test1 test1.c test2 - ${RM} -f test2.c test2.out.c test3 test3.c test3.out + ${RM} -f random.64K.bin random.1M.bin random.4M.bin random.16M.bin random.64M.bin + ${RM} -f test0 test0.c test1 test1.c + ${RM} -f test2 test2.c test2.out.c + ${RM} -f test3 test3.c test3.out.c + ${RM} -f test4 test4.c test4.out.c + ${RM} -f test5 test5.c test5.out.c + ${RM} -f test6 test6.c test6.out.c + ${RM} -f test7 test7.c test7.out.c ###################################### # optional include of 1337 hacker rulz diff --git a/2024/kurdyukov4/README.md b/2024/kurdyukov4/README.md index 3cf8c01f2..719adb2ec 100644 --- a/2024/kurdyukov4/README.md +++ b/2024/kurdyukov4/README.md @@ -7,13 +7,11 @@ ### Bugs and (Mis)features: -It is necessary to avoid collisions with C keywords when generating names for variables, for example: if do asm int for EOF. +The current status of this entry is: -The original code had a collision with the keywords "asm" and "int", now it is fixed. +> **STATUS: INABIAF - please DO NOT fix** -Also, the size of the static buffers has increased from 1MB to 16MB, which should ensure processing of at least 4MB of input. If you want to increase the limit, look for `24` in the source code (this is a power of two, for 8MB of input data you would need `25`, and so on). - -Current code was tested with an input of 1MB. +For more detailed information see [2024/kurdyukov4 in bugs.html](../../bugs.html#2024_kurdyukov4). ## To use: @@ -51,6 +49,61 @@ re-assemble the original input **WITHOUT USING ANY LITERALS** in the C code it p You mind may go into a spiral as you attempt to read and understand how this is done. :-) +### Large input size issues + +The author, [Ilya Kurdyukov](../../authors.html#Ilya_Kurdyukov), was +kind enough to provide us with an improvement to his amazing code that +allows for larger input sizes. To see what was patched run the following +within the `2024/kurdyukov4` directory: + +``` + make make diff_orig_prog +``` + +The above patch extended the original standard input size limit from 128K bytes to 64M bytes. + +While the [2024/kurdyukov4/try.sh](%%REPO_URL%%/2024/kurdyukov4/try.sh) script will attempt +to run the code in steps starting from 64K of random data up to 64M bytes of random data, +it is unlikely that you will be able to limitations by the compiler, linker, processor +architecture, memory, system, other resource limits if not run out of patience. + +On fairly normal systems, we experienced the following while running +[2024/kurdyukov4/try.sh](%%REPO_URL%%/2024/kurdyukov4/try.sh): + +``` + | random | | C code | compile time | compile time | | + | input | try.sh | size in | clang 17.0.0 | gcc 11.5.0 | | + | bytes | stage | bytes | Apple silicon | x86 64 | notes | + |--------|--------|-----------|---------------|----------------|----------------------------| + | 64K | test3 | 515764 | 10 sec | 75 sec | gcc 7.5x slower than clang | + | 1M | test4 | 3939019 | 80 sec | 1 hour 15 min | gcc 56x slower than clang | + | 4M | test5 | 13296449 | 1 min 50 sec | 14 hour 2 min | gcc 460x slower than clang | + | 16M | test6 | 136786072 | 4 min 40 sec | very long | clang excutable crashes | + | 64M | test7 | 200412713 | 44 min 40 sec | extremely long | clang excutable crashes | +``` + +We never had the time nor resources to attempt to compile using `gcc(1)`, the code needed for +any random input larger than 4M bytes. + +For 16M bytes and larger on Apple silicon, for random input size of 16M bytes and 64M bytes, +the `clang(1)` linker phase issues the following warning: + +> ld: warning: atom '\_main' ... is larger than the max code size between branch island clusters, +> this may lead to unreachable branches + +If you attempt to run the `clang(1)` generated executable anyway, is crashes with: + +> segmentation fault + +Compiling `gcc(1)` with a random input size of 16M bytes and 64M bytes was **never attempted** +because such a compile attempt would **take too long** and require a **large amount of ram**. + +**PLEASE NOTE**: The above problems are **NOT** code bugs. They represent compiler mis-features, +linker limitations, and/or architecture limitations. + +**macOS NOTE**: The macOS `/usr/bin/gcc` is actually `clang(1)`, and so using `gcc(1)` under macOS doesn't count. + + ## Author's remarks: diff --git a/2024/kurdyukov4/index.html b/2024/kurdyukov4/index.html index a7c323ee1..33d2204ac 100644 --- a/2024/kurdyukov4/index.html +++ b/2024/kurdyukov4/index.html @@ -25,14 +25,14 @@ - - + + - - - + + + @@ -447,7 +447,7 @@ C code Makefile Inventory - ✓ + ✓ @@ -474,14 +474,7 @@ Location: RU - Russian Federation

To build:

    make all
-

Bugs and (Mis)features:

-

The current status of this entry is:

-
-

STATUS: known bug - please help us fix

-
-

For more detailed information see 2024/kurdyukov4 in bugs.html.

-

In fixing this bug you will come to a better appreciation of how the -code was able to process random data files as large as 64K.

+

XXX - It is necessary to avoid collisions with C keywords when generating names for variables, for example: if do asm int for EOF.

To use:

    ./prog < data > out.c

Try:

@@ -495,6 +488,45 @@ the resulting program when run:

./out > output

re-assemble the original input WITHOUT USING ANY LITERALS in the C code it produced!

You mind may go into a spiral as you attempt to read and understand how this is done. :-)

+

Large input size issues

+

The author, Ilya Kurdyukov, was +kind enough to provide us with an improvement to his amazing code that +allows for larger input sizes. To see what was patched run the following +within the 2024/kurdyukov4 directory:

+
    make make diff_orig_prog
+

The above patch extended the original standard input size limit from 128K bytes to 64M bytes.

+

While the 2024/kurdyukov4/try.sh script will attempt +to run the code in steps starting from 64K of random data up to 64M bytes of random data, +it is unlikely that you will be able to limitations by the compiler, linker, processor +architecture, memory, system, other resource limits if not run out of patience.

+

On fairly normal systems, we experienced the following while running +2024/kurdyukov4/try.sh:

+
    | random |        |    C code |  compile time |   compile time |                            |
+    |  input | try.sh |   size in |  clang 17.0.0 |     gcc 11.5.0 |                            |
+    |  bytes |  stage |     bytes | Apple silicon |         x86 64 | notes                      |
+    |--------|--------|-----------|---------------|----------------|----------------------------|
+    |    64K |  test3 |    515764 |        10 sec |         75 sec | gcc 7.5x slower than clang |
+    |     1M |  test4 |   3939019 |        80 sec |  1 hour 15 min | gcc  56x slower than clang |
+    |     4M |  test5 |  13296449 |  1 min 50 sec | 14 hour  2 min | gcc 460x slower than clang |
+    |    16M |  test6 | 136786072 |  4 min 40 sec |      very long | clang excutable crashes    |
+    |    64M |  test7 | 200412713 | 44 min 40 sec | extremely long | clang excutable crashes    |
+

We never had the time nor resources to attempt to compile using gcc(1), the code needed for +any random input larger than 4M bytes.

+

For 16M bytes and larger on Apple silicon, for random input size of 16M bytes and 64M bytes, +the clang(1) linker phase issues the following warning:

+
+

ld: warning: atom ‘_main’ … is larger than the max code size between branch island clusters, +this may lead to unreachable branches

+
+

If you attempt to run the clang(1) generated executable anyway, is crashes with:

+
+

segmentation fault

+
+

Compiling gcc(1) with a random input size of 16M bytes and 64M bytes was never attempted +because such a compile attempt would take too long and require a large amount of ram.

+

PLEASE NOTE: The above problems are NOT code bugs. They represent compiler mis-features, +linker limitations, and/or architecture limitations.

+

macOS NOTE: The macOS /usr/bin/gcc is actually clang(1), and so using gcc(1) under macOS doesn’t count.

Author’s remarks:

No Literals Encoder

This program creates C code that will output the specified data to stdout. There are no literals used in the generated code.

@@ -607,7 +639,7 @@ Jump to: top

Coda

- Nu HTML check this web page
+ Nu HTML check this web page
Jump to top Jump to Content Jump to Inventory

diff --git a/2024/kurdyukov4/prog.c b/2024/kurdyukov4/prog.c index 8439318df..25fbfa669 100644 --- a/2024/kurdyukov4/prog.c +++ b/2024/kurdyukov4/prog.c @@ -4,7 +4,7 @@ #define o(x,o)x##o char X[ 1<<16 - ][4],Z[1<<24],*N= X[1] + ][4],Z[1<<28],*N= X[1] ,*P=Z,*H,A[12];char*C(int i,o(u ,nsigned) x){char*B =0,* D;if((x >>16)|| 0==* @@ -23,7 +23,7 @@ =98; for( printf("#inclu" "de " "" "\nint" " a;i" "nt" " ma" "in()" "{int" " b=" - "-~a") ;i>3&&P- Z+33<1<< 24;C( + "-~a") ;i>3&&P- Z+33<1<< 28;C( i,a)) for(a=i=0;i<4&&0<=(c= o(get, char)( ));n++)((char *)&a)[ i++]=c; H=P;C(1 diff --git a/2024/kurdyukov4/try.sh b/2024/kurdyukov4/try.sh index 2e449846a..d911d3526 100755 --- a/2024/kurdyukov4/try.sh +++ b/2024/kurdyukov4/try.sh @@ -33,8 +33,8 @@ echo 1>&2 read -r -n 1 -p 'Press any key to compile prog.c: ' echo 1>&2 echo 1>&2 -echo "${CC} -O3 prog.c -o prog -lm" -"${CC}" -O3 prog.c -o prog -lm +echo "time ${CC} -O3 prog.c -o prog -lm" +time "${CC}" -O3 prog.c -o prog -lm echo 1>&2 # get ready for try #0 @@ -66,12 +66,20 @@ echo 1>&2 # compile and run result #0 # -read -r -n 1 -p 'Press any key to compile and run the result: ' +read -r -n 1 -p 'Press any key to compile and run: ' echo 1>&2 echo 1>&2 rm -f test0 -echo "${CC} -O3 test0.c -o test0; ./test0" 1>&2 -"${CC}" -O3 test0.c -o test0 +echo "time ${CC} -O3 test0.c -o test0" 1>&2 +time "${CC}" -O3 test0.c -o test0 +status="$?" +if [[ $status -ne 0 || ! -x test0 ]]; then + echo 1>&2 + echo "ERROR: time ${CC} -O3 test0.c -o test0 error: $status" 1>&2 + exit 2 +fi +echo 1>&2 +echo './test0' 1>&2 echo 1>&2 ./test0 echo 1>&2 @@ -116,12 +124,20 @@ echo 1>&2 # compile and run result #1 # -read -r -n 1 -p 'Press any key to compile and run the result: ' +read -r -n 1 -p 'Press any key to compile and run: ' echo 1>&2 echo 1>&2 rm -f test1 -echo "${CC} -O3 test1.c -o test1; ./test1" 1>&2 -"${CC}" -O3 test1.c -o test1 +echo "time ${CC} -O3 test1.c -o test1" 1>&2 +time "${CC}" -O3 test1.c -o test1 +status="$?" +if [[ $status -ne 0 || ! -x test1 ]]; then + echo 1>&2 + echo "ERROR: time ${CC} -O3 test1.c -o test1 error: $status" 1>&2 + exit 3 +fi +echo 1>&2 +echo './test1' 1>&2 echo 1>&2 ./test1 echo 1>&2 @@ -154,12 +170,18 @@ echo 1>&2 # compile and run result #2 # -read -r -n 1 -p 'Press any key to compile and run the result: ' 1>&2 +read -r -n 1 -p 'Press any key to compile and run: ' 1>&2 echo 1>&2 echo 1>&2 rm -f test2 -echo "${CC} -O3 test2.c -o test2; ./test2" 1>&2 -"${CC}" -O3 test2.c -o test2 +echo "time ${CC} -O3 test2.c -o test2" 1>&2 +time "${CC}" -O3 test2.c -o test2 +status="$?" +if [[ $status -ne 0 || ! -x test2 ]]; then + echo 1>&2 + echo "ERROR: time ${CC} -O3 test2.c -o test2 error: $status" 1>&2 + exit 4 +fi echo 1>&2 rm -f test2.out echo ./test2 '>' test2.out.c 1>&2 @@ -172,9 +194,10 @@ read -r -n 1 -p 'Press any key to compare prog.c and test2.out: ' 1>&2 echo 1>&2 echo 1>&2 if cmp prog.c test2.out.c; then - echo 'prog.c and test2.out.c are the same!' 1>&2 + echo 'PASS: prog.c and test2.out.c are the same' 1>&2 else - echo 'ERROR: test2.out.c is NOT the same as prog.c!' 1>&2 + echo 'ERROR: test2.out.c is NOT the same as prog.c !!!' 1>&2 + exit 5 fi echo 1>&2 @@ -185,22 +208,22 @@ clear # generate 64K of random data for #3 # -read -r -n 1 -p 'Press any key to generate 64K of random data: ' 1>&2 +read -r -n 1 -p 'Press any key to generate 64K bytes of random data: ' 1>&2 echo 1>&2 echo 1>&2 -rm -f random.bin -echo dd if=/dev/urandom bs=64K count=1 '>' random.bin -dd if=/dev/urandom bs=64K count=1 > random.bin +rm -f random.64K.bin +echo 'dd if=/dev/urandom bs=64K count=1 of=random.64K.bin' 1>&2 +dd if=/dev/urandom bs=64K count=1 of=random.64K.bin echo 1>&2 # verify the result #3 # -read -r -n 1 -p 'Press any key to ./prog < random.bin > test3.c: ' 1>&2 +read -r -n 1 -p 'Press any key to ./prog < random.64K.bin > test3.c: ' 1>&2 echo 1>&2 echo 1>&2 rm -f test3.c -echo ./prog '<' random.bin '>' test3.c -./prog < random.bin > test3.c +echo ./prog '<' random.64K.bin '>' test3.c +./prog < random.64K.bin > test3.c echo 1>&2 # view the result #3 @@ -214,28 +237,404 @@ echo 1>&2 ls -l test3.c echo 1>&2 -# compile and run the result #3 +# compile and run #3 # -read -r -n 1 -p 'Press any key to compile and run the result (this will take a bit of time to compile): ' 1>&2 +echo 'NOTICE: Compiling and running the 64K byte random input data test' 1>&2 +echo 1>&2 +echo 'NOTICE: Because test3.c can take so long to compile, we will compile without optimization.' 1>&2 +echo 1>&2 +echo 'NOTICE: clang might take about 10 seconds to compile' 1>&2 +echo 'NOTICE: gcc might take about 75 seconds to compile' 1>&2 +echo 1>&2 +read -r -n 1 -p 'Press any key to compile and run: ' 1>&2 echo 1>&2 echo 1>&2 rm -f test3 -echo "time ${CC} -O3 test3.c -o test3; ./test3" 1>&2 -time "${CC}" -O3 test3.c -o test3 +echo "time ${CC} -O0 test3.c -o test3" 1>&2 +time "${CC}" -O0 test3.c -o test3 +status="$?" +if [[ $status -ne 0 || ! -x test3 ]]; then + echo 1>&2 + echo "ERROR: time ${CC} -O0 test3.c -o test3 error: $status" 1>&2 + exit 6 +fi echo 1>&2 rm -f test3.out echo ./test3 '>' test3.out 1>&2 ./test3 > test3.out +status="$?" +if [[ $status -ne 0 ]]; then + echo 1>&2 + echo "ERROR: ./test3 > test3.out error: $status" 1>&2 + exit 7 +fi echo 1>&2 # verify the result #3 # -read -r -n 1 -p 'Press any key to compare random.bin and test3.out: ' 1>&2 +read -r -n 1 -p 'Press any key to compare random.64K.bin and test3.out: ' 1>&2 echo 1>&2 echo 1>&2 -if cmp random.bin test3.out; then - echo 'random.bin and test3.out are the same!' 1>&2 +if cmp random.64K.bin test3.out; then + echo 'PASS: random.64K.bin and test3.out are the same' 1>&2 else - echo 'ERROR: random.bin is NOT the same as test3.out!' 1>&2 + echo 'ERROR: random.64K.bin is NOT the same as test3.out !!!' 1>&2 + exit 8 +fi +echo 1>&2 + +# get ready for try #4 +# +read -r -n 1 -p '^^^^^ Press any key after viewing the results above ^^^^^: ' +clear + +# generate 1M of random data for #4 +# +read -r -n 1 -p 'Press any key to generate 1M bytes of random data: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f random.1M.bin +echo 'dd if=/dev/urandom bs=64K count=16 of=random.1M.bin' 1>&2 +dd if=/dev/urandom bs=64K count=16 of=random.1M.bin +echo 1>&2 + +# verify the result #4 +# +read -r -n 1 -p 'Press any key to ./prog < random.1M.bin > test4.c: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f test4.c +echo ./prog '<' random.1M.bin '>' test4.c +./prog < random.1M.bin > test4.c +echo 1>&2 + +# view the result #4 +# +read -r -n 1 -p 'Press any key to view the last 10000 characters of test4.c: ' +echo 1>&2 +echo 1>&2 +echo ... 1>&2 +tail -n 1 test4.c | rev | cut -c 1-10000 | rev +echo 1>&2 +ls -l test4.c +echo 1>&2 + +# compile and run #4 +# +echo 'NOTICE: Compiling and running the 1M byte random input data test' 1>&2 +echo 1>&2 +echo 'NOTICE: Because test4.c can take so long to compile, we will compile without optimization.' 1>&2 +echo 1>&2 +echo 'NOTICE: clang might take about 80 seconds to compile' 1>&2 +echo 'NOTICE: gcc might take about 1 hour 15 minutes to compile !!!' 1>&2 +echo 1>&2 +read -r -n 1 -p 'Press any key to compile and run: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f test4.o +echo "time ${CC} -O0 test4.c -o test4" 1>&2 +time "${CC}" -O0 test4.c -o test4 +status="$?" +if [[ $status -ne 0 || ! -x test4 ]]; then + echo 1>&2 + echo 'NOTICE: This is not a code bug, this is a compiler mis-feature !!!' 1>&2 + echo 1>&2 + echo "ERROR: time ${CC} -O0 test4.c -o test4 error: $status" 1>&2 + exit 9 +fi +echo 1>&2 +rm -f test4.out +echo ./test4 '>' test4.out 1>&2 +./test4 > test4.out +status="$?" +if [[ $status -ne 0 ]]; then + echo 1>&2 + echo 'NOTICE: This is not a code bug, this is a linker and/or architecture limitation !!!' 1>&2 + echo 1>&2 + echo "ERROR: ./test4 > test4.out error: $status" 1>&2 + exit 10 +fi +echo 1>&2 + +# verify the result #4 +# +read -r -n 1 -p 'Press any key to compare random.1M.bin and test4.out: ' 1>&2 +echo 1>&2 +echo 1>&2 +if cmp random.1M.bin test4.out; then + echo 'PASS: random.1M.bin and test4.out are the same' 1>&2 +else + echo 'ERROR: random.1M.bin is NOT the same as test4.out !!!' 1>&2 + exit 11 +fi +echo 1>&2 + +# get ready for try #5 +# +read -r -n 1 -p '^^^^^ Press any key after viewing the results above ^^^^^: ' +clear + +# generate 4M of random data for #5 +# +read -r -n 1 -p 'Press any key to generate 4M bytes of random data: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f random.4M.bin +echo 'dd if=/dev/urandom bs=64K count=64 of=random.4M.bin' 1>&2 +dd if=/dev/urandom bs=64K count=64 of=random.4M.bin +echo 1>&2 + +# verify the result #5 +# +read -r -n 1 -p 'Press any key to ./prog < random.4M.bin > test5.c: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f test5.c +echo ./prog '<' random.4M.bin '>' test5.c +./prog < random.4M.bin > test5.c +echo 1>&2 + +# view the result #5 +# +read -r -n 1 -p 'Press any key to view the last 10000 characters of test5.c: ' +echo 1>&2 +echo 1>&2 +echo ... 1>&2 +tail -n 1 test5.c | rev | cut -c 1-10000 | rev +echo 1>&2 +ls -l test5.c +echo 1>&2 + +# compile and run #5 +# +echo 'NOTICE: Compiling and running the 4M byte random input data test' 1>&2 +echo 1>&2 +echo 'NOTICE: Because test5.c can take so long to compile, we will compile without optimization.' 1>&2 +echo 1>&2 +echo 'NOTICE: clang might take about 1 minute 50 seconds to compile' 1>&2 +echo 'NOTICE: gcc might take about 14 hours !!!' 1>&2 +echo 1>&2 +echo 'WARNING: Some buggy compilers may crash, especially in resource limited environments, while compiling test5.c !!!' 1>&2 +echo 1>&2 +read -r -n 1 -p 'Press any key to compile and run: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f test5 +echo "time ${CC} -O0 test5.c -o test5" 1>&2 +time "${CC}" -O0 test5.c -o test5 +status="$?" +if [[ $status -ne 0 || ! -x test5 ]]; then + echo 1>&2 + echo 'NOTICE: This is not a code bug, this is a compiler mis-feature !!!' 1>&2 + echo 1>&2 + echo "ERROR: time ${CC} -O0 test5.c -o test5 error: $status" 1>&2 + exit 12 +fi +echo 1>&2 +rm -f test5.out +echo ./test5 '>' test5.out 1>&2 +./test5 > test5.out +status="$?" +if [[ $status -ne 0 ]]; then + echo 1>&2 + echo 'NOTICE: This is not a code bug, this is a linker and/or architecture limitation !!!' 1>&2 + echo 1>&2 + echo "ERROR: ./test5 > test5.out error: $status" 1>&2 + exit 13 +fi +echo 1>&2 + +# verify the result #5 +# +read -r -n 1 -p 'Press any key to compare random.4M.bin and test5.out: ' 1>&2 +echo 1>&2 +echo 1>&2 +if cmp random.4M.bin test5.out; then + echo 'PASS: random.4M.bin and test5.out are the same' 1>&2 +else + echo 'ERROR: random.4M.bin is NOT the same as test5.out !!!' 1>&2 + exit 14 +fi +echo 1>&2 + +# get ready for try #6 +# +read -r -n 1 -p '^^^^^ Press any key after viewing the results above ^^^^^: ' +clear + +# generate 16M of random data for #6 +# +read -r -n 1 -p 'Press any key to generate 16M bytes of random data: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f random.16M.bin +echo 'dd if=/dev/urandom bs=64K count=256 of=random.16M.bin' 1>&2 +dd if=/dev/urandom bs=64K count=256 of=random.16M.bin +echo 1>&2 + +# verify the result #6 +# +read -r -n 1 -p 'Press any key to ./prog < random.16M.bin > test6.c: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f test6.c +echo ./prog '<' random.16M.bin '>' test6.c +./prog < random.16M.bin > test6.c +echo 1>&2 + +# view the result #6 +# +read -r -n 1 -p 'Press any key to view the last 10000 characters of test6.c: ' +echo 1>&2 +echo 1>&2 +echo ... 1>&2 +tail -n 1 test6.c | rev | cut -c 1-10000 | rev +echo 1>&2 +ls -l test6.c +echo 1>&2 + +# compile and run #6 +# +echo 'NOTICE: Compiling and running the 16M byte random input data test' 1>&2 +echo 1>&2 +echo 'NOTICE: Because test6.c can take so long to compile, we will compile without optimization.' 1>&2 +echo 1>&2 +echo 'NOTICE: clang might take about 4 minutes 40 seconds to compile' 1>&2 +echo 'NOTICE: gcc took so long to compile at the 4M level that we did not even try gcc the 16M level !!!' 1>&2 +echo 1>&2 +echo 'WARNING: Some buggy compilers may crash, especially in resource limited environments, while compiling test6.c !!!' 1>&2 +echo 1>&2 +echo 'WARNING: The executable may crash on some architectures because main() is larger than the max' 1>&2 +echo ' code size between branch island clusters, leading to unreachable branches !!!' 1>&2 +echo 1>&2 +read -r -n 1 -p 'Press any key to compile and run: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f test6 +echo "time ${CC} -O0 test6.c -o test6" 1>&2 +time "${CC}" -O0 test6.c -o test6 +status="$?" +if [[ $status -ne 0 || ! -x test6 ]]; then + echo 1>&2 + echo 'NOTICE: This is not a code bug, this is a compiler mis-feature !!!' 1>&2 + echo 1>&2 + echo "ERROR: time ${CC} -O0 test6.c -o test6 error: $status" 1>&2 + exit 15 +fi +echo 1>&2 +rm -f test6.out +echo ./test6 '>' test6.out 1>&2 +./test6 > test6.out +status="$?" +if [[ $status -ne 0 ]]; then + echo 1>&2 + echo 'NOTICE: This is not a code bug, this is a linker and/or architecture limitation !!!' 1>&2 + echo 1>&2 + echo "ERROR: ./test6 > test6.out error: $status" 1>&2 + exit 16 +fi +echo 1>&2 + +# verify the result #6 +# +read -r -n 1 -p 'Press any key to compare random.16M.bin and test6.out: ' 1>&2 +echo 1>&2 +echo 1>&2 +if cmp random.16M.bin test6.out; then + echo 'PASS: random.16M.bin and test6.out are the same' 1>&2 +else + echo 'ERROR: random.16M.bin is NOT the same as test6.out !!!' 1>&2 + exit 17 +fi +echo 1>&2 + +# get ready for try #7 +# +read -r -n 1 -p '^^^^^ Press any key after viewing the results above ^^^^^: ' +clear + +# generate 64M of random data for #7 +# +read -r -n 1 -p 'Press any key to generate 64M bytes of random data: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f random.64M.bin +echo 'dd if=/dev/urandom bs=64K count=1024 of=random.64M.bin' 1>&2 +dd if=/dev/urandom bs=64K count=1024 of=random.64M.bin +echo 1>&2 + +# verify the result #7 +# +read -r -n 1 -p 'Press any key to ./prog < random.64M.bin > test7.c: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f test7.c +echo ./prog '<' random.64M.bin '>' test7.c +./prog < random.64M.bin > test7.c +echo 1>&2 + +# view the result #7 +# +read -r -n 1 -p 'Press any key to view the last 10000 characters of test7.c: ' +echo 1>&2 +echo 1>&2 +echo ... 1>&2 +tail -n 1 test7.c | rev | cut -c 1-10000 | rev +echo 1>&2 +ls -l test7.c +echo 1>&2 + +# compile and run #7 +# +echo 'NOTICE: Compiling and running the 64M byte random input data test' 1>&2 +echo 1>&2 +echo 'NOTICE: Because test7.c can take so long to compile, we will compile without optimization.' 1>&2 +echo 1>&2 +echo 'NOTICE: clang might take about 45 minutes to compile' 1>&2 +echo 'NOTICE: gcc took so long to compile at the 4M level that we did not even try gcc the 64M level !!!' 1>&2 +echo 1>&2 +echo 'WARNING: Some buggy compilers may crash, especially in resource limited environments, while compiling test7.c !!!' 1>&2 +echo 1>&2 +echo 'WARNING: The executable may crash on some architectures because main() is larger than the max' 1>&2 +echo ' code size between branch island clusters, leading to unreachable branches !!!' 1>&2 +echo 1>&2 +read -r -n 1 -p 'Press any key to compile and run: ' 1>&2 +echo 1>&2 +echo 1>&2 +rm -f test7 +echo "time ${CC} -O0 test7.c -o test7" 1>&2 +time "${CC}" -O0 test7.c -o test7 +status="$?" +if [[ $status -ne 0 || ! -x test7 ]]; then + echo 1>&2 + echo 'NOTICE: This is not a code bug, this is a compiler mis-feature !!!' 1>&2 + echo 1>&2 + echo "ERROR: time ${CC} -O0 test7.c -o test7 error: $status" 1>&2 + exit 18 +fi +echo 1>&2 +rm -f test7.out +echo ./test7 '>' test7.out 1>&2 +./test7 > test7.out +status="$?" +if [[ $status -ne 0 ]]; then + echo 1>&2 + echo 'NOTICE: This is not a code bug, this is a linker and/or architecture limitation !!!' 1>&2 + echo 1>&2 + echo "ERROR: ./test7 > test7.out error: $status" 1>&2 + exit 19 +fi +echo 1>&2 + +# verify the result #7 +# +read -r -n 1 -p 'Press any key to compare random.64M.bin and test7.out: ' 1>&2 +echo 1>&2 +echo 1>&2 +if cmp random.64M.bin test7.out; then + echo 'PASS: random.64M.bin and test7.out are the same' 1>&2 +else + echo 'ERROR: random.64M.bin is NOT the same as test7.out !!!' 1>&2 + exit 20 fi echo 1>&2 diff --git a/bugs.md b/bugs.md index 516d67e0c..8988ba31d 100644 --- a/bugs.md +++ b/bugs.md @@ -4915,40 +4915,18 @@ Jump to: [top](#) Jump to: [top](#) -### STATUS: known bug - please help us fix +### STATUS: INABIAF - please **DO NOT** fix ### Source code: [2024/kurdyukov4/prog.c](%%REPO_URL%%/2024/kurdyukov4/prog.c) ### Information: [2024/kurdyukov4/index.html](2024/kurdyukov4/index.html) -In fixing this bug you will come to a better appreciation of how the -code was able to process random data files as large as 64K. +The [2024/kurdyukov4/try.sh](%%REPO_URL%%/2024/kurdyukov4/try.sh) script +will very likely run into problems such as compile taking a long time, +compilers crashing and/or executables crashing when used with random +input sizes larger than 64K bytes. -If you provide a random data set of about 100K bytes, the resulting C program, after taking some time, will compile: +**PLEASE NOTE**: The above problems are **NOT** code bugs. They represent compiler mis-features, +linker limitations, and/or architecture limitations. -``` - dd if=/dev/urandom bs=100K count=1 | ./prog > out.c -``` - -However if increase the size of the random data set to 128K bytes, the resulting `out.c` C program: - -``` - dd if=/dev/urandom bs=128K count=1 | ./prog > out.c - cc out.c -o out -``` - -will fail to compile with errors such as: - -``` - out.c:2:...: error: expected identifier or '(' - out.c:2:...: error: expected ';' at end of declaration - out.c:2:...: error: use of undeclared identifier 'Z' -``` - -Can you fix `prog.c` to successfully process very large random data sets: perhaps into the gigabyte size range or larger? - -How long does it take to compile the resulting `out.c` source? - -Even better, can you fix `prog.c` while maintaining its style and source code layout, all without -making the source code significantly larger? Jump to: [top](#) @@ -4967,6 +4945,9 @@ request](https://github.com/ioccc-src/winner/pulls) or otherwise, we thank you as well for the help! We will happily add you to the [thanks](thanks-for-help.html) file as well. +For more information see the "**Large input size issues**" section of +[2024/kurdyukov4/index.html](2024/kurdyukov4/index.html#large-input-size-issues). + Jump to: [top](#)