mirror/ioccc
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #966 from xexyl/sgit-sed-commands

Browse Source
This commit is contained in:
Landon Curt Noll authored and GitHub committed 2023-09-24 07:47:36 -07:00
2 parents e327c7bc92 b4aba49ae8
commit 007deba69e
2 files changed
+163 -56
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

src/sgit/README.md
+126 -35
View File
@@ -1,42 +1,85 @@
# sgit - run sed on files under git control
This was originally a quick and dirty hack to modify files under git control
via `sed`. It started out with some limitations like the inability to deal
with spaces in `sed` commands, only allowing one glob and it had terrible
efficiency because it ran git ls-files for each `sed` command.
`sgit` runs `sed(1)` on files that are under git control, based on one or more
globs (either a pattern or an exact file name). Files not under git control in
the directory and any subdirectories will not be touched. Depending on the globs
specified it might or might not recurse into subdirectories. That is the shell
at work, of course. Running it in a directory that is not a git repository is an
error.
Now it is a bit more thorough and is more efficient (though not perfect). It
allows you to specify a path to `sed`, `sed` options and multiple `sed` commands
(though this always existed). The _default_ behaviour is to use in place editing
(**WITHOUT A BACKUP EXTENSION**; use `-i` to provide a backup extension) but you
can override the saving to disk with the `-I` option.
This is extremely useful because one need not run extra commands to determine
the list of files (which might be very long) and then pass them directly to
`sed(1)`.
At least one `sed` command is required with the `-e` option (analogous to `sed
-e`) and at least one glob is required. It is an error if at least one of each
is not specified.
You can specify the path to `sed` in case you wish to use a different one. For
instance in macOS if you have GNU sed installed you might want to use that
instead. You can do that with the `-s sed` option.
If you wish to provide options to `sed` itself you can so with the `sgit -o`.
Note that you **MUST** pass the `-` to the option and for long options like
`--posix` you must pass `--`! In other words use the options like you would with
`sed` but prefixing it with `-o` first. This was a stylistic choice but it
allows one to quote the option arg to pass more than one option instead of
having to use `sgit -o` more than once. Note that not all options to `sed` have
been tested.
By default it does **in-place editing it does NOT backup files**. If you
wish to not edit the file in place (see examples later in this file) you can use
the `sgit -I` option. Note that the `-n` option (`sgit -o -n`) without `sgit -I`
can, depending on the sed commands, empty files! This is analogous to using both
`-n` and `-i` to `sed` which would do in-place editing without automatic
printing of the pattern space.
If you wish to provide a backup extension for editing files use `sgit -i`. See
example below. Note that using `-i` overwrites existing backup files and it will
also create a file for each file edited. This means that if it ends up editing
50 files there will be 50 new files created.
See the usage below or run `sgit` by itself (or with the `-h` option) to see the
rest of the options.
See the section [Script history](#script-history) for more details on the
evolution of the script.
## Usage
```sh
usage: sgit [-h] [-V] [-v level] [-x] [-I] [-i extension] [-o sed_options] [-s sed] [-e command] <glob...>
usage: sgit [-h] [-V] [-v level] [-x] [-I] [-i extension] [-o sed_option(s)] [-s sed] [-e command] <glob...>
-h print help and exit
-V print version and exit
-v level set verbosity level
-x turn on tracing (set -x)
-I disable in place editing
-i extension set backup extension (default none)
WARNING: sed -i overwrites existing files
WARNING: this will create another file for each file changed
-o sed options (NOTE: don't pass '-'!)
WARNING: use of '-o n' without '-I', can depending on command, empty files
-o append sed options to options list
WARNING: use of '-o -n' without '-I', can depending on
sed commands, empty files as if both sed -i and sed -n were
used together
NOTE: you must pass the '-' or '--' for long options!
-s sed set path to sed
-e command append sed command to list of commands to execute on globs
sgit version: 0.0.8-1 19-04-2023
sgit version: 0.0.9-1 24-04-2023
```
You **MUST** specify at least one `sed` command and one glob: the sed command by
way of the `-e` option (just like with `sed`); anything after the last option is
a glob. You may specify more than one of each. Specify `-e` for each command.
The sed commands is an array just like the sed options.
# A note about the `-i` option and backup extensions
## A note about the `-i` option and backup extensions
It works like `sed -i` except that the default behaviour of the script is to use
`-i` just with an empty backup extension (this is required for some versions of
@@ -44,51 +87,53 @@ It works like `sed -i` except that the default behaviour of the script is to use
option .. I'm looking at you, BSD/macOS `sed`).
# `-i` WARNING: each file edited will result in another file in the working directory
## `sgit -i` WARNING: each file edited will result in another file in the working directory
Of course if you have 50 files that are affected then 50 new files will be
created.
Of course if you have 50 files that are edited then 50 new files will be
created just like with `sed` itself.
# `-i` WARNING: `sed -i` **WILL OVERWRITE EXISTING FILES**!
# `sgit -i` WARNING: this **WILL OVERWRITE EXISTING BACKUP FILES**!
Please be aware that if a backup file already exists it **WILL** be overwritten!
This is because it uses `sed -i`.
I won't even try and detect this because it's not how `sed` works and it would
overly complicate the script.
# `-o n` WARNING: invalid use of -o n can empty files!
# `sgit -o -n` WARNING: invalid use of `sgit -o -n` can empty files!
Because `sed -n` does not print output if you do not use `-I` and you use
substitute you can empty the entire file. For instance **DO NOT** do this:
As already noted: because `sed -n` does not print output if you do not use `-I`
and you use substitute or do not print everything you can empty the entire file
or remove much of the file for two examples. For instance **DO NOT** do this:
```sh
sgit -o n -e 's/.//g' sgit
sgit -o -n -e 's/.//g' sgit
```
because it would empty `sgit`!
# Examples
## Examples
## Change in this file (_IN MEMORY ONLY_) `\<sed\>` to `used` but only show lines changed:
### Change in this file (_IN MEMORY ONLY_) `\<sed\>` to `used` but only show lines changed:
```sh
sgit -I -o n -e 's/\<sed\>/used/p' README.md
sgit -I -n -e 's/\<sed\>/used/p' README.md
sgit -I -o -n -e 's/\<sed\>/used/p' README.md
```
## Print out matches of `\<sed\>` in all files under git control
### Print out matches of `\<sed\>` in all files under git control
This is a simpler way of running `git --no-pager grep -h -E '\<sed\>'|sed
's/^[0-9]*://g'` which itself might or might not be more complicated than it
needs to be:
```sh
sgit -I -o n -e '/\<sed\>/p' .
sgit -I -o -n -e '/\<sed\>/p' .
```
## With tracing enabled, change references of `\<sed\>` to `used` in this file and save it:
### With tracing enabled, change references of `\<sed\>` to `used` in this file and save it:
```sh
sgit -e 's/\<sed\>/used/g' -x README.md
@@ -96,7 +141,7 @@ sgit -e 's/\<sed\>/used/g' -x README.md
```
## With tracing enabled, change references of `\<sed\>` to `used`, duplicating the lines, in this file and save it:
### With tracing enabled, change references of `\<sed\>` to `used`, duplicating the lines, in this file and save it:
```sh
sgit -e 's/\<sed\>/used/p' -x README.md
@@ -107,6 +152,7 @@ sgit -e 's/\<sed\>/used/p' -x README.md
```sh
sgit -e 's/\<sed\>/used/g' -v1 README.md
sgit -e 's/\<sed\>/used/g' -v 1 README.md
```
@@ -114,6 +160,7 @@ sgit -e 's/\<sed\>/used/g' -v1 README.md
```sh
sgit -i.bak -e 's/\<sed\>/used/g' -v3 README.md
sgit -i.bak -e 's/\<sed\>/used/g' -v 3 README.md
```
@@ -131,6 +178,32 @@ debug[2]: 0 remaining globs
Level 2 would not show how many globs remain after each operation.
### Verbosely (level 3) change references of `\<used\>` back to `sed` in this file with a backup as `README.md.bak`
```sh
sgit -i.bak -e 's/\<used\>/sed/g' -v3 README.md
sgit -i.bak -e 's/\<used\>/sed/g' -v 3 README.md
```
With that you would see something like:
```sh
sgit -i.bak -e 's/\<used\>/sed/g' -v3 README.md
debug[2]: sed commands: -e s/\<used\>/sed/g
debug[2]: looping through all globs
debug[1]: using backup extension: .bak
debug[2]: found glob: 0
debug[1]: about to run: git ls-files README.md | xargs /opt/local/libexec/gnubin/sed -i".bak" -e s/\<used\>/sed/g
debug[2]: 0 remaining globs
```
and the example title would look like:
```markdown
### Verbosely (level 3) change references of `\<sed\>` back to `sed` in this file with a backup as `README.md.bak`
```
## Change `\<sed\>` to `used` but only if it's on the first line
```sh
@@ -145,10 +218,10 @@ sgit -e '1s/\<sed\>/u&/g' README.md
sgit -e '1s/\(\<sed\>\)/u\1/g' README.md
```
## Change `\<hack\>` to `crack` but only if it's on the third line
### Change `\<sgit\>` to `gits` but only if it's on the third line
```sh
sgit -e '3s/\<hack\>/crack/g' README.md
sgit -e '3s/\<sgit\>/gits/g' README.md
```
@@ -183,7 +256,7 @@ This would likely greatly complicate the script and is I feel unneeded.
If you specify invalid `sed` commands obviously there will be problems. If you
specify invalid `sed` options there will possibly be problems as well.
Again this was originally a hack.
As below this was originally a hack.
## History
@@ -194,8 +267,26 @@ control and `sed` is the obvious way to go about it.
Working on the [IOCCC mkiocccentry](https://github.com/ioccc-src/mkiocccentry)
and the [IOCCC temporary website](https://github.com/ioccc-src/temp-test-ioccc)
and the [IOCCC temporary website](https://github.com/ioccc-src/temp-test-ioccc)
repos (and in particular the latter) is what inspired me to do something about
it.
repos (and in particular the latter) is what inspired me to finally do something
about it. The latter repo will eventually be merged into the [IOCCC winner
repo](https://github.com/ioccc-src/winner) which is the actual [IOCCC
website](https://www.ioccc.org).
### Script history
This was originally a quick and dirty hack to modify files under git control
via `sed`. It started out with some limitations like the inability to deal
with spaces in `sed` commands, only allowing one glob and it had terrible
efficiency because it ran git ls-files for each `sed` command.
Now it is much more thorough and is more efficient (though not perfect). It
allows you to specify a path to `sed`, `sed` options and multiple `sed` commands
(though the latter always existed), it can disable in-place editing, you can
backup files if you wish and much more.
`sgit` version `0.0.9-1 24-04-2023` added support to use the `-o` option more
than once. Not allowing this was an oversight.
## Other thoughts
src/sgit/sgit.sh
+37 -21
View File
@@ -1,38 +1,47 @@
#!/usr/bin/env bash
#
# sgit - run sed (by default with -i) on all files (or one or more globs) under
# git control
# sgit - run sed (by default with -i) on one or more globs under git control
#
# This script allows one to run sed under git control without having to come up
# with a list of files first and then passing those to the command.
# This script allows one to run sed on files under git control, based on globs
# (pattern or exact file name) without having to come up with a list of files
# first and then passing those directly to sed.
#
# It allows for multiple sed commands and multiple globs. By default it uses -i
# to edit the file in place (WITHOUT BACKUP EXTENSION; use -i option if you want
# a backup!) but in place editing can be overridden with the -I option. See -h
# option for all the options.
# to edit the file in place (WITHOUT BACKUP EXTENSION; use sgit -i option if you
# want a backup!) but in place editing can be overridden with the -I option. Use
# -h option to see all the options. The README.md provides much more information
# on the usage of the script and I recommend you look at that.
#
# DISCLAIMER AND WARNING: this is something of a hack and might be error prone.
# It works for what is needed but possibly could be improved. Even so I like it
# how it is and I don't think it would do much good to add many more additional
# features.
# For a list of warnings see the usage string and more so the README.md file as
# there are some options that when used together (or an option not used with
# another) can in some cases empty files (depends on command) or one can end up
# creating many additional files (if backing up the files for instance with the
# -i option).
#
# - Cody Boone Ferguson (@xexyl)
#
export SGIT_VERSION="0.0.8-1 19-04-2023" # format: major.minor.patch-release DD-MM-YYYY
export SGIT_VERSION="0.0.9-1 24-04-2023" # format: major.minor.patch-release DD-MM-YYYY
USAGE="usage: $(basename "$0") [-h] [-V] [-v level] [-x] [-I] [-i extension] [-o sed_options] [-s sed] [-e command] <glob...>
USAGE="usage: $(basename "$0") [-h] [-V] [-v level] [-x] [-I] [-i extension] [-o sed_option(s)] [-s sed] [-e command] <glob...>
-h print help and exit
-V print version and exit
-v level set verbosity level
-x turn on tracing (set -x)
-I disable in place editing
-i extension set backup extension (default none)
WARNING: sed -i overwrites existing files
WARNING: this will create another file for each file changed
-o sed options (NOTE: don't pass '-'!)
WARNING: use of '-o n' without '-I', can depending on command, empty files
-o append sed options to options list
WARNING: use of '-o -n' without '-I', can depending on
sed commands, empty files as if both sed -i and sed -n were
used together
NOTE: you must pass the '-' or '--' for long options!
-s sed set path to sed
-e command append sed command to list of commands to execute on globs
@@ -40,13 +49,14 @@ sgit version: $SGIT_VERSION"
SED="$(type -P sed)"
export SED
export SED_OPTIONS=""
export I_FLAG=
export VERBOSITY=0
export EXTENSION=""
# we need an array for sed commands
declare -a SED_COMMANDS
# we need an array for sed options
declare -a SED_OPTIONS
# we need to remove ' ' from the IFS!
IFS=$'\t\n'
@@ -69,7 +79,7 @@ while getopts :hVv:xIi:o:s:e: flag; do
;;
i) EXTENSION="$OPTARG"
;;
o) SED_OPTIONS="-$OPTARG"
o) SED_OPTIONS+=("$OPTARG")
;;
s) SED="$OPTARG"
;;
@@ -172,14 +182,20 @@ while [[ "$i" -lt "$GLOBS" ]]; do
if [[ "$VERBOSITY" -ge 1 ]]; then
if [[ -z "$I_FLAG" ]]; then
# Determine if any sed options were specified. We only need to know
# if the array is empty or not; we do not need to expand it in the
# check.
#
# SC2128 (warning): Expanding an array without an index only gives the first element.
# shellcheck disable=SC2128
if [[ -n "$SED_OPTIONS" ]]; then
echo "debug[1]: about to run: git ls-files $1 | xargs $SED -i\"$EXTENSION\" $SED_OPTIONS ${SED_COMMANDS[*]}" 1>&2
echo "debug[1]: about to run: git ls-files $1 | xargs $SED -i\"$EXTENSION\" ${SED_OPTIONS[*]} ${SED_COMMANDS[*]}" 1>&2
else
echo "debug[1]: about to run: git ls-files $1 | xargs $SED -i\"$EXTENSION\" ${SED_COMMANDS[*]}" 1>&2
fi
else
if [[ -n "$SED_OPTIONS" ]]; then
echo "debug[1]: about to run: git ls-files $1 | xargs $SED $SED_OPTIONS ${SED_COMMANDS[*]}" 1>&2
echo "debug[1]: about to run: git ls-files $1 | xargs $SED ${SED_OPTIONS[*]} ${SED_COMMANDS[*]}" 1>&2
else
echo "debug[1]: about to run: git ls-files $1 | xargs $SED ${SED_COMMANDS[*]}" 1>&2
fi
@@ -188,13 +204,13 @@ while [[ "$i" -lt "$GLOBS" ]]; do
if [[ -z "$I_FLAG" ]]; then
if [[ -n "$SED_OPTIONS" ]]; then
git ls-files "$1" | xargs "$SED" -i"$EXTENSION" "$SED_OPTIONS" "${SED_COMMANDS[@]}"
git ls-files "$1" | xargs "$SED" -i"$EXTENSION" "${SED_OPTIONS[*]}" "${SED_COMMANDS[@]}"
else
git ls-files "$1" | xargs "$SED" -i"$EXTENSION" "${SED_COMMANDS[@]}"
fi
else
if [[ -n "$SED_OPTIONS" ]]; then
git ls-files "$1" | xargs "$SED" "$SED_OPTIONS" "${SED_COMMANDS[@]}"
git ls-files "$1" | xargs "$SED" "${SED_OPTIONS[*]}" "${SED_COMMANDS[@]}"
else
git ls-files "$1" | xargs "$SED" "${SED_COMMANDS[@]}"
fi