Grep flags – the good stuff(zwischenzugs.com) |
Grep flags – the good stuff(zwischenzugs.com) |
-P, --perl-regexp
Interpret I<PATTERNS> as Perl-compatible regular
expressions (PCREs). This option is experimental when
combined with the -z (--null-data) option, and grep -P
may warn of unimplemented features.
grep -Po "Name:\K\w+"
Lets assume we have a log file with a bunch of relevant stuff, I want to highlight my search term, BUT I also want to keep all the other lines around for context:
$ dmesg
...SNIP...
[2334597.539661] sd 1:0:0:0: [sdb] Attached SCSI removable disk
[2334597.548919] sd 1:0:0:0: [sdb] 57280429 512-byte logical blocks: (29.3 GB/27.3 GiB)
[2334597.761895] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[2334597.761900] sdb: detected capacity change from 0 to 57280429
[2334597.772736] sdb:
[2334631.115664] sdb: detected capacity change from 57280429 to 0
...SNIP...
$ dmesg | grep capacity
[2334597.761900] sdb: detected capacity change from 0 to 57280429
[2334631.115664] sdb: detected capacity change from 57280429 to 0
$ dmesg | grep --color -e capacity -e ''
...SNIP...
[2334597.539661] sd 1:0:0:0: [sdb] Attached SCSI removable disk
[2334597.548919] sd 1:0:0:0: [sdb] 57280429 512-byte logical blocks: (29.3 GB/27.3 GiB)
[2334597.761895] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
*[2334597.761900] sdb: detected capacity change from 0 to 57280429*
[2334597.772736] sdb:
*[2334631.115664] sdb: detected capacity change from 57280429 to 0*
...SNIP...
$ grep -R '' /sys/module/iwlwifi/parameters/
/sys/module/iwlwifi/parameters/nvm_file:(null)
/sys/module/iwlwifi/parameters/debug:0
/sys/module/iwlwifi/parameters/swcrypto:0
/sys/module/iwlwifi/parameters/power_save:N
/sys/module/iwlwifi/parameters/lar_disable:N
...SNIP... grep --color "$1"'\|$'I use ripgrep when I need better speed. I've pretty much switched to ripgrep these days, but still use GNU grep when I'm answering questions on stackoverflow, reddit, etc.
>ABC flags
Good to also know about `--group-separator` and `--no-group-separator` when there are multiple non-contiguous matches. Helps to customize the separator or remove them altogether. Sadly, these options are still not explained in `man grep` on Ubuntu. You'll have to use `info grep` or the online manual to find them.
Options I use often that is not mentioned in the article:
* `-c` to count the number of matches
* `-F` for fixed string matching
* `-x` to match whole lines
* `-P` for PCRE (as mentioned in many comments here)
* `--color=auto` this is part of command name alias, so it is always used
I wrote a book as well on "GNU grep and ripgrep": https://github.com/learnbyexample/learn_gnugrep_ripgrep Free to read online.
function fvi { grep -rl $1 . | xargs nvim +/$1 }
#!/bin/sh
usage () {
cat >&2 <<'__USAGE__'
usage: git gsr [-P | --perl-regexp] <old> <new> [paths...]
replace all occurrances of <old> with <new> optionally limited to
<paths...> (as interpreted by git grep)
-P, --perl-regexp interpret <old> as perl regular expression;
default is to treat it as a fixed string.
__USAGE__
exit 1
}
pattern='-F'
perl='BEGIN {($old, $new) = (shift, shift)} s/\Q$old\E/$new/g'
case "$1" in
-P|--perl-regexp)
shift
pattern='-P'
perl='BEGIN {($old, $new) = (shift, shift)} s/$old/$new/g'
;;
-*) usage
;;
esac
test $# -lt 2 && usage
old=$1; new=$2; shift; shift
git grep -l -z $pattern "$old" -- "$@" |
xargs -0 perl -pi -e "$perl" "$old" "$new" sed -n 's/.*\(pattern\).*/\1/p'
grep -o 'pattern'
[0] - https://github.com/BurntSushi/ripgrep#why-should-i-use-ripgr...
LC_ALL=C grep -larP '\x1A\x2B\x3C\xFF' grep -larF "$(printf '\032\053\074\377')"
Caveats:
1) POSIX only requires printf to recognize octal escapes (\nnn, or \0nnn if using %b specifier), not hexadecimal escapes. Many implementations recognize the latter, but not Debian dash.
2) Shell command substitution strips trailing new lines from the output, so if your binary string ends in a newline you'll need to use extra tricks. E.g. S="$(printf '\032\053\074\nX')"; grep -larF "${S%X}"
3) It's probably a good idea to still specify LC_ALL=C, but because the binary string is now being passed through the shell's innards it might need to be set in the environment of the shell itself, not simply the environments of the printf and grep subcommands. (Also, technically I'm not sure if the C/POSIX locale is required to be 8-bit clean, yet, but in practice it will be.)
Bash and some other shells support an extension ($') for expanding escape sequences inline:
grep -larF $'\x1A\x2B\x3C\xFF'
find . -type f -name \*txt \; | xargs -I{} -P24 bash -c "grep -Hi foo '{}' ; :"(joking)
grep "foo.*bar" hello
grep -e "foo.*bar" -e "bar.*foo" hello
cat hello | sed -n '/foo/{ /bar/{ /baz/ p }}' grep -nRI foo ./
Often I will add `-P` and encase the regex with single-quotes of course
I also like --include and --exclude, especially since it allows regex for which files to look at
Living without those few "extensions" feels... empty
-l/-h/-v/-o show up every now and then
Years ago--I don't if it's still the case--but the fish shell used ^ to redirect stderr.
EDIT: I remembered right:
https://web.archive.org/web/20111111003423/http://fishshell....
but it seems it's no longer the case:
fish> echo foo ^ bar
foo ^ bar
Supporting ignore files in the root of the search is trivial.
However supporting files deeper is much harder, since find does not support this.
For the case where few directories have ignore files, it's best to find all the ignore files first and generate one find command for each.
If most directories have them then it's faster to exec find for each directory, and use maxdepth 1
In terms of me just using find by itself, I don't bother, since I find silently ignoring files to be a misfeature, and I don't use git at work anyways
In any case, that's all fine and good, but your tool is very clearly not a clone of ag or ripgrep IMO. Whether you consider it a misfeature or not, the "smart filtering" aspect is probably the defining quality of things like ack, ag and ripgrep. So if you don't have that, I don't think you can call it a clone IMO. The smart filtering feature is right up there next to performance in terms what things users tend to like about these tools.
The other thing you're probably missing from a perf perspective, I think, is parallel directory traversal. Neither ack nor ag have this, but ripgrep does. ;-)
It is indeed. I can't say that I have it 100% correct yet, if you are aware of a good set of tests, I could incorporate it.
> In any case, that's all fine and good, but your tool is very clearly not a clone of ag or ripgrep IMO. ... the "smart filtering" aspect is probably the defining quality of things like ack, ag and ripgrep. So if you don't have that, I don't think you can call it a clone IMO. The smart filtering feature is right up there next to performance in terms what things users tend to like about these tools.
I did implement nested gitignore files; it just murders performance (not quite by a factor of 2 with warm caches in the best case; some pathological cases can be much worse). This is actually something that logically belongs in find, as it relates to directory traversal, and a generic "walk this directory tree ignoring files in gitignore" is probably useful for many other things.
This was strictly a clone of the silver searcher because that was what was popular at the time (2016ish I think?). It started out as a weekend project just for fun, then I started adding more ag options. There are a few other unimplemented features such as --vimgrep --ackmate --column and --stats. Also -g takes a glob rather than a pcre; I could reasonably make it take a posix regex, but I think that would fool people into thinking it worked like ag, while a glob is rarely confused for a pcre.