Fixes and updates to 1998/schweikh3

Fixed a command to try.

Fixed formatting.

Updated .gitignore.

Fixed URL.
This commit is contained in:
Cody Boone Ferguson committed 2023-04-18 09:18:35 -07:00
1 parent e45c0d1755
commit 591d8774ef
2 files changed
+78 -48

No files matched your search

+4
View File
@@ -1 +1,5 @@
schweikh3 schweikh3
file1
file2
file3
file4
+74 -48
View File
@@ -5,83 +5,97 @@
Schlartaeckerweg 3 (Home address) Schlartaeckerweg 3 (Home address)
D-71384 Weinstadt D-71384 Weinstadt
Germany Germany
<http://www.schweikhardt.net>
http://www.shuttle.de/schweikh/
## To build: ## To build:
```sh
make all make all
```
## To run; ## To run:
If "dir" is the directory (or a directory tree) where you keep If "dir" is the directory (or a directory tree) where you keep
all your favorite pictures off the Internet or from all your favorite pictures off the Internet or from
`alt.binaries.pictures.*`, do `alt.binaries.pictures.*`, do
```sh
find dir -type f -print | ./schweikh3 find dir -type f -print | ./schweikh3
```
Maybe you will not need to buy another 10 Gb disk to store them. :-)
## Try: ## Try:
If you're in this winning entry's directory: If you're in this winning entry's directory:
```sh
find . -type f -print | ./schweikh3 find . -type f -print | ./schweikh3
```
Notice that the tool finds a duplicate file, that of the man page. Notice that the tool finds a duplicate file, that of the man page.
## Judges' remarks: ## Judges' remarks:
Maybe you will not need to buy another 10 Gb disk to store them. :-)
Also Try: Try this:
```sh
cp schweikh3 file1 cp schweikh3 file1
cp schweikh3 file2 cp schweikh3 file2
cp schweikh3.c file3 cp schweikh3.c file3
ln file3 file4 ln file3 file4
ls -1 file? | ./schweikh3 find . -type f -name 'file?' | ./schweikh3
```
NOTE: this program is not as smart as to detect identity of JPEG This program is not as smart as to detect identity of JPEG files with the same
files with the same picture but with different compression levels. picture but with different compression levels. On the other hand compression can
make it thing look/sound/feel different so these probably aren't duplicates
anyway. :-)
NOTE: Some non-gcc compilers that are not fully ANSI standard do NOTE: Some non-gcc compilers that are not fully ANSI standard do not compile
not compile this entry correctly. this entry correctly.
## Author's remarks ## Author's remarks
The source is expected to conform to IEEE Std 1003.1-1990 ("POSIX"). The source is expected to conform to IEEE Std 1003.1-1990 ("POSIX").
Thank God the IEEE does not standardize a coding style... Thank God the IEEE does not standardize a coding style...
What this program does ### What this program does
----------------------
This is best explained in the man page. The troff source for this This is best explained in the man page. The troff source for this
man page can be found in the file samefile.1. man page can be found in the file samefile.1. To render try: `man
./samefile.1`. Otherwise:
SAMEFILE(1) User Manuals SAMEFILE(1)
NAME **SAMEFILE(1) User Manuals SAMEFILE(1)**
samefile - find identical files
SYNOPSIS
samefile
DESCRIPTION **NAME**
samefile reads a list of file names (one file name per
**samefile** - find identical files
**SYNOPSIS**
**./samefile**
**DESCRIPTION**
**samefile** reads a list of file names (one file name per
line) from stdin. For each file name pair with identical line) from stdin. For each file name pair with identical
contents, a line consisting of six tab separated fields is contents, a line consisting of six tab separated fields is
output: The size in bytes, two file names, the character output: The size in bytes, two file names, the character
``='' if the two files are on the same device, ``X'' oth- `=` if the two files are on the same device, `X` otherwise,
erwise, and the link counts of the two files. The output and the link counts of the two files. The output
is sorted in reverse order by size. is sorted in reverse order by size.
samefile uses two stages to give optimum performance. samefile uses two stages to give optimum performance.
In the first stage, all non-plain files are silently In the first stage, all non-plain files are silently
ignored (directories, devices, FIFOs, sockets, symbolic ignored (directories, devices, FIFOs, sockets, symbolic
links) as well as files for which stat(2) or open(2) links) as well as files for which `stat(2)` or `open(2)`
fails. The result of the first stage (the file names) is fails. The result of the first stage (the file names) is
written into a binary tree with one node for every file written into a binary tree with one node for every file
size. Each node is in turn a linked list of file names size. Each node is in turn a linked list of file names
@@ -104,26 +118,29 @@ Notice that the tool finds a duplicate file, that of the man page.
identical files of size 100 in an inode group consisting identical files of size 100 in an inode group consisting
of the three inodes with numbers 10, 20 and 30: of the three inodes with numbers 10, 20 and 30:
```sh
$ ls -i # output edited for readability: $ ls -i # output edited for readability:
10 file1 20 file4 30 file6 10 file1 20 file4 30 file6
10 file2 20 file5 10 file2 20 file5
10 file3 10 file3
$ ls | samefile $ ls | ./samefile
100 file1 file4 = 3 2 100 file1 file4 = 3 2
100 file1 file6 = 3 1 100 file1 file6 = 3 1
```
The sum of the sizes in the first column is the amount of The sum of the sizes in the first column is the amount of
disk space you could gain by making all 6 files links to disk space you could gain by making all 6 files links to
only one file or remove all but one of the files. To be only one file or remove all but one of the files. To be
precise, disk space is allocated in blocks - you will precise, disk space is allocated in blocks - you will
probably gain two blocks here, rather than 200 bytes. probably gain two blocks here, rather than 200 bytes.
Note that it is not enough to just remove file4 and file6 Note that it is not enough to just remove `file4` and `file6`
(you would gain only 100 bytes because file5 still (you would gain only 100 bytes because `file5` still
exists.) exists.)
LIMITATIONS **LIMITATIONS**
Samefile was written with no limits in mind. The number of
`samefile` was written with no limits in mind. The number of
input lines is unlimited. The size of the actual files is input lines is unlimited. The size of the actual files is
only limited by available virtual memory needed to compare only limited by available virtual memory needed to compare
one pair of files. The only hard limit is the constraint one pair of files. The only hard limit is the constraint
@@ -131,46 +148,55 @@ Notice that the tool finds a duplicate file, that of the man page.
the same size. Experience has shown that there are rarely the same size. Experience has shown that there are rarely
more than a couple dozen files of the same size. more than a couple dozen files of the same size.
EXAMPLES **EXAMPLES**
For everybody: For everybody:
What are the duplicates under my home directory? What are the duplicates under my home directory?
$ find $HOME | samefile ```sh
$ find $HOME | ./samefile
```
For the sysadmin folks: For the sysadmin folks:
Report all duplicate files under /usr larger than 16k: Report all duplicate files under /usr larger than 16k:
$ find /usr -size +16384c -a -type f | samefile ```sh
$ find /usr -size +16384c -a -type f | ./samefile
```
For the ftp and WWW admins: For the ftp and WWW admins:
How much space is wasted below our site's /pub directory? How much space is wasted below our site's /pub directory?
```sh
$ find /pub -type f | samefile | awk '{sum += $1} END {print sum}' $ find /pub -type f | samefile | awk '{sum += $1} END {print sum}'
```
**EXIT STATUS**
EXIT STATUS If `samefile` runs out of memory the exit status is 1, zero
If samefile runs out of memory the exit status is 1, zero
otherwise. otherwise.
SEE ALSO **SEE ALSO**
find(1), ln(1), rm(1)
`find(1)`, `ln(1)`, `rm(1)`
Why I think it is obfuscated ### Why I think it is obfuscated
----------------------------
- the code is commented but this gives only an *extremely* faint clue. - The code is commented but this gives only an *extremely* faint clue.
- the layout does not reflect the program structure, or does it? - The layout does not reflect the program structure, or does it?
- The data structure the program builds is an ordered binary tree - The data structure the program builds is an ordered binary tree
where each node consists of a linked list. This is probably too where each node consists of a linked list. This is probably too
heavy to comprehend for N percent of all programmers ;-) heavy to comprehend for N percent of all programmers ;-)
- The tree is built and traversed by recursive function calls. - The tree is built and traversed by recursive function calls.
- Why does this program look at dev_t and ino_t? Heavy wizardry. - Why does this program look at `dev_t` and `ino_t`? Heavy wizardry.
- Can you find out how the algorithm works that detects identity - Can you find out how the algorithm works that detects identity
of files across arbitrarily long chains? In my private unobfuscated of files across arbitrarily long chains? In my private unobfuscated
source there are 17 lines describing just that. A classical case for source there are 17 lines describing just that. A classical case for
the well known /* You're not supposed to understand this */ comment. the well known `/* You're not supposed to understand this */` comment.
- Would Dijkstra consider this goto harmful? - Would Dijkstra consider this goto harmful?
- The only string literal is split in more than a dozen substrings - The only string literal is split in more than a dozen substrings
- For the task at hand the code is pretty terse. Some dynamic memory - For the task at hand the code is pretty terse. Some dynamic memory
@@ -180,23 +206,23 @@ Notice that the tool finds a duplicate file, that of the man page.
names. names.
Are you an algorithms guru? If so you should know why the output Are you an algorithms guru? If so you should know why the output
appears sorted without any use of qsort() or system("sort"). appears sorted without any use of `qsort()` or `system("sort")`.
Rewrite (hah!) the program to use memory mapping of files instead Rewrite (hah!) the program to use memory mapping of files instead
of read(). Does this affect performance noticeably? of `read()`. Does this affect performance noticeably?
It's easy to write a script that takes samefile's output and It's easy to write a script that takes `samefile`'s output and
hard links (or symlinks) identical files. Writing such a script is hard links (or symlinks) identical files. Writing such a script is
left as an exercise for the deobfuscator. left as an exercise for the deobfuscator.
When you run samefile on some directory tree, try to estimate When you run `samefile` on some directory tree, try to estimate
how much you think it will find. Is the actual result anywhere how much you think it will find. Is the actual result anywhere
near your estimate? How many copies of the GPL COPYING file near your estimate? How many copies of the GPL COPYING file
are there on your filesystems? :-) are there on your filesystems? :-)
In the remote event that the input has more than 8192 files with In the remote event that the input has more than 8192 files with
the same size (on systems where sizeof (char *) == 4, or 4096 when the same size (on systems where `sizeof (char *) == 4`, or 4096 when
sizeof (char *) == 8), increase the manifest constant 32767 on line `sizeof (char *) == 8`), increase the manifest constant 32767 on line
31. 31.
## Copyright and CC BY-SA 4.0 License: ## Copyright and CC BY-SA 4.0 License: