See the History of a Method with Git log -L

See the History of a Method with Git log -L(calebhearth.com)

116 points by caleb_thompson 2 years ago | 25 comments

simonw 2 years ago |

If you're curious how Git knows the syntax of different languages in order to support this kind of feature, take a look in https://github.com/git/git/blob/master/userdiff.c

Here's how support for Python and Ruby are defined:

    PATTERNS("python",
        "^[ \t]*((class|(async[ \t]+)?def)[ \t].*)$",
        /* -- */
        "[a-zA-Z_][a-zA-Z0-9_]*"
        "|[-+0-9.e]+[jJlL]?|0[xX]?[0-9a-fA-F]+[lL]?"
        "|[-+*/<>%&^|=!]=|//=?|<<=?|>>=?|\\*\\*=?"),
        /* -- */
    PATTERNS("ruby",
        "^[ \t]*((class|module|def)[ \t].*)$",
        /* -- */
        "(@|@@|\\$)?[a-zA-Z_][a-zA-Z0-9_]*"
        "|[-+0-9.e]+|0[xXbB]?[0-9a-fA-F]+|\\?(\\\\C-)?(\\\\M-)?."
        "|//=?|[-+*/<>%&^|=!]=|<<=?|>>=?|===|\\.{1,3}|::|[!=]~"),

tessierashpool 2 years ago | |

it's a fantastic feature in theory. in practice, it's imprecise and error-prone, and I believe these regular expressions are probably why. I hadn't looked at the implementation before, but I approached it from the other end: I set up a bunch of test cases, and I was pretty disappointed.

there were two disappointments. first, `git log -L` seems to prioritize tracking blocks of code over lines of code. that's just a design choice I disagree with, so it wasn't a big deal. but it also lost track of lines of code for me quite often, and produced a number of false positives to boot.

to be fair, I haven't tried using `diff=LANG` (per a comment below), and that might get more reliable results.

Groxx 2 years ago | | |

Yeah, this has been my experience too. Easily confused by common constructions in some codebases, and that can make it almost completely useless. I would happily sacrifice a lot of speed to get a difftastic level of precision.

anitil 2 years ago | |

I've attempted something similar to your ast-search tool, but it instead iterates through git history, pulls out the relevant text and then provides the diff to the user.

It's a tricky problem because it sits somewhere between text, where a function name could get renamed and it's obvious because it is textually similar, and an AST where 'similarity' is a difficult concept.

I struggled to make it usable, but of course there's a module to do half of it that I didn't find initially - https://pypi.org/project/pyastsim/

lloydatkinson 2 years ago | |

Interesting. Also I am surprised at how short the list is!

nneonneo 2 years ago |

Oh wow, this is very cool!

The way this works boils down to the following: by default, Git has a heuristic for determining the "context" of a diff hunk by looking for lines that start with certain non-whitespace characters. This context is printed out after the "@@" marker in the hunk header. Within git, this context is referred to as the "function name", but that's a bit inaccurate as the patterns will usually match other scopes like namespaces and classes.

Setting "diff=LANG" activates a different (regular expression) pattern which is used to identify context; for example, in Python, this will look for "class" and "def" keywords. Git ships with a bunch of built-in patterns (defined in https://github.com/git/git/blob/master/userdiff.c), and the "diff.LANG.xfuncname" config option can be used to specify a custom pattern.

-L can then be used to look for hunks which have context matching a certain pattern. For example, if you want to look for function "foo", you could use -L ':\bfoo\b:file.py' (note that if you don't use \b you'll get every function that contains the word foo). Also related is the -W flag, which will show the entire function/class/scope in the diff, again based on context.

Note some limitations: the matching is line-by-line, so it will pick up "context" from things like string literals and comments, and you will only get the first line of the context (so multi-line signatures will be truncated). Also, since -L takes a regular expression to match against the context line, you'll want to take care to use an appropriate pattern to avoid matching unwanted functions (e.g. use \b to avoid substring matches, or even "def foo(" to ensure you only match to methods and not to classes or parameter names).

See also https://stackoverflow.com/questions/28111035/where-does-the-... for a very comprehensive overview of this feature.

sundarurfriend 2 years ago | |

> Within git, this context is referred to as the "function name", but that's a bit inaccurate as the patterns will usually match other scopes like namespaces and classes.

Thank you for mentioning this (and the other details). The userdiff.c file was mentioned elsewhere in the thread, but I was doubting it since its regexes also matched classes, Perl POD blocks, etc. Good to have it clarified that it's the Git man pages that are inaccurate, helps understand this file (userdiff.c) and this feature better.

Nezteb 2 years ago |

I love using the `-G` flag for tracking the history of any occurrence of a given regex across all directories/files. It feels more flexible than `-L`. As an example:

  git log \
    -G "$some_regex" \
    --patch \
    --stat \
    --source \
    --all \
    --decorate=full \
    --pretty=fuller \
    -- . ":(exclude)\*.lock"

penguin_booze 2 years ago | |

I often have to remind myself that the pathspec is defined here in the glossary: https://git-scm.com/docs/gitglossary#Documentation/gitglossa....

g0xA52A2A 2 years ago |

Huh TIL Git can do this for non C-like languages out of the box. Is there a reason these custom hunk handlers are not defined by default for the languages Git ships support with?

baggy_trough 2 years ago |

This is a great capability. Wonder how many more are lurking within tools I use every day but never read the man pages.

forrestthewoods 2 years ago |

Git really needs an equivalent to Perforce's Timelapse View.

n_sd 2 years ago |

How can we handle function overloading case with this?

gigatexal 2 years ago |

I had no idea about this. This is amazing

chiefalchemist 2 years ago |

How can this be uses with PHP?

mschuster91 2 years ago | |

According to `man gitattributes` [1] a `*.php diff=php` should be enough.

[1] https://git-scm.com/docs/gitattributes

OJFord 2 years ago | |

It's just a regex to match the line you want to see the history of. Function would be a common case, but it's completely arbitrary.

For example, `git log -L:members:Cargo.toml` will show the history of constituent rust projects in a Cargo workspace.

staunton 2 years ago | | |

It's a regex? Is it even guaranteed to work then? I would think you have to parse non-regular languages to always find the end of a function in a some languages?

Indeed, it does not always work correctly for Julia, as an arbitrary example I tried. Seems like it goes by indentation? Still nice though and worked out of the box!

PATTERNS("python", "^[ \t]*((class|(async[ \t]+)?def)[ \t].*)$", /* -- */ "[a-zA-Z_][a-zA-Z0-9_]*" "|[-+0-9.e]+[jJlL]?|0[xX]?[0-9a-fA-F]+[lL]?" "|[-+*/<>%&^|=!]=|//=?|<<=?|>>=?|\\*\\*=?"), /* -- */ PATTERNS("ruby", "^[ \t]*((class|module|def)[ \t].*)$", /* -- */ "(@|@@|\\$)?[a-zA-Z_][a-zA-Z0-9_]*" "|[-+0-9.e]+|0[xXbB]?[0-9a-fA-F]+|\\?(\\\\C-)?(\\\\M-)?." "|//=?|[-+*/<>%&^|=!]=|<<=?|>>=?|===|\\.{1,3}|::|[!=]~"),

When writing or updating patterns, assume that the contents these patterns are applied to are syntactically correct. The patterns can be simple without implementing all syntactical corner cases, as long as they are sufficiently permissive.