diff --git a/src/sgit/README.md b/src/sgit/README.md index 488d7a236..4355dec23 100644 --- a/src/sgit/README.md +++ b/src/sgit/README.md @@ -1,42 +1,85 @@ # sgit - run sed on files under git control -This was originally a quick and dirty hack to modify files under git control -via `sed`. It started out with some limitations like the inability to deal -with spaces in `sed` commands, only allowing one glob and it had terrible -efficiency because it ran git ls-files for each `sed` command. +`sgit` runs `sed(1)` on files that are under git control, based on one or more +globs (either a pattern or an exact file name). Files not under git control in +the directory and any subdirectories will not be touched. Depending on the globs +specified it might or might not recurse into subdirectories. That is the shell +at work, of course. Running it in a directory that is not a git repository is an +error. -Now it is a bit more thorough and is more efficient (though not perfect). It -allows you to specify a path to `sed`, `sed` options and multiple `sed` commands -(though this always existed). The _default_ behaviour is to use in place editing -(**WITHOUT A BACKUP EXTENSION**; use `-i` to provide a backup extension) but you -can override the saving to disk with the `-I` option. +This is extremely useful because one need not run extra commands to determine +the list of files (which might be very long) and then pass them directly to +`sed(1)`. + +At least one `sed` command is required with the `-e` option (analogous to `sed +-e`) and at least one glob is required. It is an error if at least one of each +is not specified. + +You can specify the path to `sed` in case you wish to use a different one. For +instance in macOS if you have GNU sed installed you might want to use that +instead. You can do that with the `-s sed` option. + +If you wish to provide options to `sed` itself you can so with the `sgit -o`. +Note that you **MUST** pass the `-` to the option and for long options like +`--posix` you must pass `--`! In other words use the options like you would with +`sed` but prefixing it with `-o` first. This was a stylistic choice but it +allows one to quote the option arg to pass more than one option instead of +having to use `sgit -o` more than once. Note that not all options to `sed` have +been tested. + +By default it does **in-place editing it does NOT backup files**. If you +wish to not edit the file in place (see examples later in this file) you can use +the `sgit -I` option. Note that the `-n` option (`sgit -o -n`) without `sgit -I` +can, depending on the sed commands, empty files! This is analogous to using both +`-n` and `-i` to `sed` which would do in-place editing without automatic +printing of the pattern space. + +If you wish to provide a backup extension for editing files use `sgit -i`. See +example below. Note that using `-i` overwrites existing backup files and it will +also create a file for each file edited. This means that if it ends up editing +50 files there will be 50 new files created. + +See the usage below or run `sgit` by itself (or with the `-h` option) to see the +rest of the options. + +See the section [Script history](#script-history) for more details on the +evolution of the script. ## Usage ```sh -usage: sgit [-h] [-V] [-v level] [-x] [-I] [-i extension] [-o sed_options] [-s sed] [-e command] +usage: sgit [-h] [-V] [-v level] [-x] [-I] [-i extension] [-o sed_option(s)] [-s sed] [-e command] -h print help and exit -V print version and exit -v level set verbosity level -x turn on tracing (set -x) -I disable in place editing + -i extension set backup extension (default none) WARNING: sed -i overwrites existing files WARNING: this will create another file for each file changed - -o sed options (NOTE: don't pass '-'!) - WARNING: use of '-o n' without '-I', can depending on command, empty files + + -o append sed options to options list + WARNING: use of '-o -n' without '-I', can depending on + sed commands, empty files as if both sed -i and sed -n were + used together + + NOTE: you must pass the '-' or '--' for long options! + -s sed set path to sed -e command append sed command to list of commands to execute on globs -sgit version: 0.0.8-1 19-04-2023 +sgit version: 0.0.9-1 24-04-2023 ``` You **MUST** specify at least one `sed` command and one glob: the sed command by way of the `-e` option (just like with `sed`); anything after the last option is a glob. You may specify more than one of each. Specify `-e` for each command. +The sed commands is an array just like the sed options. -# A note about the `-i` option and backup extensions + +## A note about the `-i` option and backup extensions It works like `sed -i` except that the default behaviour of the script is to use `-i` just with an empty backup extension (this is required for some versions of @@ -44,51 +87,53 @@ It works like `sed -i` except that the default behaviour of the script is to use option .. I'm looking at you, BSD/macOS `sed`). -# `-i` WARNING: each file edited will result in another file in the working directory +## `sgit -i` WARNING: each file edited will result in another file in the working directory -Of course if you have 50 files that are affected then 50 new files will be -created. +Of course if you have 50 files that are edited then 50 new files will be +created just like with `sed` itself. -# `-i` WARNING: `sed -i` **WILL OVERWRITE EXISTING FILES**! +# `sgit -i` WARNING: this **WILL OVERWRITE EXISTING BACKUP FILES**! Please be aware that if a backup file already exists it **WILL** be overwritten! +This is because it uses `sed -i`. + I won't even try and detect this because it's not how `sed` works and it would overly complicate the script. -# `-o n` WARNING: invalid use of -o n can empty files! +# `sgit -o -n` WARNING: invalid use of `sgit -o -n` can empty files! -Because `sed -n` does not print output if you do not use `-I` and you use -substitute you can empty the entire file. For instance **DO NOT** do this: +As already noted: because `sed -n` does not print output if you do not use `-I` +and you use substitute or do not print everything you can empty the entire file +or remove much of the file for two examples. For instance **DO NOT** do this: ```sh -sgit -o n -e 's/.//g' sgit +sgit -o -n -e 's/.//g' sgit ``` because it would empty `sgit`! -# Examples +## Examples -## Change in this file (_IN MEMORY ONLY_) `\` to `used` but only show lines changed: +### Change in this file (_IN MEMORY ONLY_) `\` to `used` but only show lines changed: ```sh -sgit -I -o n -e 's/\/used/p' README.md -sgit -I -n -e 's/\/used/p' README.md +sgit -I -o -n -e 's/\/used/p' README.md ``` -## Print out matches of `\` in all files under git control +### Print out matches of `\` in all files under git control This is a simpler way of running `git --no-pager grep -h -E '\'|sed 's/^[0-9]*://g'` which itself might or might not be more complicated than it needs to be: ```sh -sgit -I -o n -e '/\/p' . +sgit -I -o -n -e '/\/p' . ``` -## With tracing enabled, change references of `\` to `used` in this file and save it: +### With tracing enabled, change references of `\` to `used` in this file and save it: ```sh sgit -e 's/\/used/g' -x README.md @@ -96,7 +141,7 @@ sgit -e 's/\/used/g' -x README.md ``` -## With tracing enabled, change references of `\` to `used`, duplicating the lines, in this file and save it: +### With tracing enabled, change references of `\` to `used`, duplicating the lines, in this file and save it: ```sh sgit -e 's/\/used/p' -x README.md @@ -107,6 +152,7 @@ sgit -e 's/\/used/p' -x README.md ```sh sgit -e 's/\/used/g' -v1 README.md +sgit -e 's/\/used/g' -v 1 README.md ``` @@ -114,6 +160,7 @@ sgit -e 's/\/used/g' -v1 README.md ```sh sgit -i.bak -e 's/\/used/g' -v3 README.md +sgit -i.bak -e 's/\/used/g' -v 3 README.md ``` @@ -131,6 +178,32 @@ debug[2]: 0 remaining globs Level 2 would not show how many globs remain after each operation. +### Verbosely (level 3) change references of `\` back to `sed` in this file with a backup as `README.md.bak` + +```sh +sgit -i.bak -e 's/\/sed/g' -v3 README.md +sgit -i.bak -e 's/\/sed/g' -v 3 README.md + +``` + +With that you would see something like: + +```sh +sgit -i.bak -e 's/\/sed/g' -v3 README.md +debug[2]: sed commands: -e s/\/sed/g +debug[2]: looping through all globs +debug[1]: using backup extension: .bak +debug[2]: found glob: 0 +debug[1]: about to run: git ls-files README.md | xargs /opt/local/libexec/gnubin/sed -i".bak" -e s/\/sed/g +debug[2]: 0 remaining globs +``` + +and the example title would look like: + +```markdown +### Verbosely (level 3) change references of `\` back to `sed` in this file with a backup as `README.md.bak` +``` + ## Change `\` to `used` but only if it's on the first line ```sh @@ -145,10 +218,10 @@ sgit -e '1s/\/u&/g' README.md sgit -e '1s/\(\\)/u\1/g' README.md ``` -## Change `\` to `crack` but only if it's on the third line +### Change `\` to `gits` but only if it's on the third line ```sh -sgit -e '3s/\/crack/g' README.md +sgit -e '3s/\/gits/g' README.md ``` @@ -183,7 +256,7 @@ This would likely greatly complicate the script and is I feel unneeded. If you specify invalid `sed` commands obviously there will be problems. If you specify invalid `sed` options there will possibly be problems as well. -Again this was originally a hack. +As below this was originally a hack. ## History @@ -194,8 +267,26 @@ control and `sed` is the obvious way to go about it. Working on the [IOCCC mkiocccentry](https://github.com/ioccc-src/mkiocccentry) and the [IOCCC temporary website](https://github.com/ioccc-src/temp-test-ioccc) and the [IOCCC temporary website](https://github.com/ioccc-src/temp-test-ioccc) -repos (and in particular the latter) is what inspired me to do something about -it. +repos (and in particular the latter) is what inspired me to finally do something +about it. The latter repo will eventually be merged into the [IOCCC winner +repo](https://github.com/ioccc-src/winner) which is the actual [IOCCC +website](https://www.ioccc.org). + +### Script history + +This was originally a quick and dirty hack to modify files under git control +via `sed`. It started out with some limitations like the inability to deal +with spaces in `sed` commands, only allowing one glob and it had terrible +efficiency because it ran git ls-files for each `sed` command. + +Now it is much more thorough and is more efficient (though not perfect). It +allows you to specify a path to `sed`, `sed` options and multiple `sed` commands +(though the latter always existed), it can disable in-place editing, you can +backup files if you wish and much more. + +`sgit` version `0.0.9-1 24-04-2023` added support to use the `-o` option more +than once. Not allowing this was an oversight. + ## Other thoughts diff --git a/src/sgit/sgit.sh b/src/sgit/sgit.sh index 135bd005e..599c0ab9a 100755 --- a/src/sgit/sgit.sh +++ b/src/sgit/sgit.sh @@ -1,38 +1,47 @@ #!/usr/bin/env bash # -# sgit - run sed (by default with -i) on all files (or one or more globs) under -# git control +# sgit - run sed (by default with -i) on one or more globs under git control # -# This script allows one to run sed under git control without having to come up -# with a list of files first and then passing those to the command. +# This script allows one to run sed on files under git control, based on globs +# (pattern or exact file name) without having to come up with a list of files +# first and then passing those directly to sed. # # It allows for multiple sed commands and multiple globs. By default it uses -i -# to edit the file in place (WITHOUT BACKUP EXTENSION; use -i option if you want -# a backup!) but in place editing can be overridden with the -I option. See -h -# option for all the options. +# to edit the file in place (WITHOUT BACKUP EXTENSION; use sgit -i option if you +# want a backup!) but in place editing can be overridden with the -I option. Use +# -h option to see all the options. The README.md provides much more information +# on the usage of the script and I recommend you look at that. # -# DISCLAIMER AND WARNING: this is something of a hack and might be error prone. -# It works for what is needed but possibly could be improved. Even so I like it -# how it is and I don't think it would do much good to add many more additional -# features. +# For a list of warnings see the usage string and more so the README.md file as +# there are some options that when used together (or an option not used with +# another) can in some cases empty files (depends on command) or one can end up +# creating many additional files (if backing up the files for instance with the +# -i option). # # - Cody Boone Ferguson (@xexyl) # -export SGIT_VERSION="0.0.8-1 19-04-2023" # format: major.minor.patch-release DD-MM-YYYY +export SGIT_VERSION="0.0.9-1 24-04-2023" # format: major.minor.patch-release DD-MM-YYYY -USAGE="usage: $(basename "$0") [-h] [-V] [-v level] [-x] [-I] [-i extension] [-o sed_options] [-s sed] [-e command] +USAGE="usage: $(basename "$0") [-h] [-V] [-v level] [-x] [-I] [-i extension] [-o sed_option(s)] [-s sed] [-e command] -h print help and exit -V print version and exit -v level set verbosity level -x turn on tracing (set -x) -I disable in place editing + -i extension set backup extension (default none) WARNING: sed -i overwrites existing files WARNING: this will create another file for each file changed - -o sed options (NOTE: don't pass '-'!) - WARNING: use of '-o n' without '-I', can depending on command, empty files + + -o append sed options to options list + WARNING: use of '-o -n' without '-I', can depending on + sed commands, empty files as if both sed -i and sed -n were + used together + + NOTE: you must pass the '-' or '--' for long options! + -s sed set path to sed -e command append sed command to list of commands to execute on globs @@ -40,13 +49,14 @@ sgit version: $SGIT_VERSION" SED="$(type -P sed)" export SED -export SED_OPTIONS="" export I_FLAG= export VERBOSITY=0 export EXTENSION="" # we need an array for sed commands declare -a SED_COMMANDS +# we need an array for sed options +declare -a SED_OPTIONS # we need to remove ' ' from the IFS! IFS=$'\t\n' @@ -69,7 +79,7 @@ while getopts :hVv:xIi:o:s:e: flag; do ;; i) EXTENSION="$OPTARG" ;; - o) SED_OPTIONS="-$OPTARG" + o) SED_OPTIONS+=("$OPTARG") ;; s) SED="$OPTARG" ;; @@ -172,14 +182,20 @@ while [[ "$i" -lt "$GLOBS" ]]; do if [[ "$VERBOSITY" -ge 1 ]]; then if [[ -z "$I_FLAG" ]]; then + # Determine if any sed options were specified. We only need to know + # if the array is empty or not; we do not need to expand it in the + # check. + # + # SC2128 (warning): Expanding an array without an index only gives the first element. + # shellcheck disable=SC2128 if [[ -n "$SED_OPTIONS" ]]; then - echo "debug[1]: about to run: git ls-files $1 | xargs $SED -i\"$EXTENSION\" $SED_OPTIONS ${SED_COMMANDS[*]}" 1>&2 + echo "debug[1]: about to run: git ls-files $1 | xargs $SED -i\"$EXTENSION\" ${SED_OPTIONS[*]} ${SED_COMMANDS[*]}" 1>&2 else echo "debug[1]: about to run: git ls-files $1 | xargs $SED -i\"$EXTENSION\" ${SED_COMMANDS[*]}" 1>&2 fi else if [[ -n "$SED_OPTIONS" ]]; then - echo "debug[1]: about to run: git ls-files $1 | xargs $SED $SED_OPTIONS ${SED_COMMANDS[*]}" 1>&2 + echo "debug[1]: about to run: git ls-files $1 | xargs $SED ${SED_OPTIONS[*]} ${SED_COMMANDS[*]}" 1>&2 else echo "debug[1]: about to run: git ls-files $1 | xargs $SED ${SED_COMMANDS[*]}" 1>&2 fi @@ -188,13 +204,13 @@ while [[ "$i" -lt "$GLOBS" ]]; do if [[ -z "$I_FLAG" ]]; then if [[ -n "$SED_OPTIONS" ]]; then - git ls-files "$1" | xargs "$SED" -i"$EXTENSION" "$SED_OPTIONS" "${SED_COMMANDS[@]}" + git ls-files "$1" | xargs "$SED" -i"$EXTENSION" "${SED_OPTIONS[*]}" "${SED_COMMANDS[@]}" else git ls-files "$1" | xargs "$SED" -i"$EXTENSION" "${SED_COMMANDS[@]}" fi else if [[ -n "$SED_OPTIONS" ]]; then - git ls-files "$1" | xargs "$SED" "$SED_OPTIONS" "${SED_COMMANDS[@]}" + git ls-files "$1" | xargs "$SED" "${SED_OPTIONS[*]}" "${SED_COMMANDS[@]}" else git ls-files "$1" | xargs "$SED" "${SED_COMMANDS[@]}" fi