Debugging tricks in the browser(alan.norbauer.com) |
Debugging tricks in the browser(alan.norbauer.com) |
Whenever I go to a different zone of development, like backend or a different language, I miss this ecosystem of debugging tools that modern browsers have by default.
I've done a lot of debugging on Python, Java and C++ over the years and never felt like I was missing tools (although I certainly met plenty of people that were ignorant of their options in this regard...)
This is clever; after all, the only way to beat the recursive turtle stack of chrome debuggers debugging themselves is with the debugger statement.
sam.pl, of the infamous myspace Sammy worm, used debugging gotcha's to prevent visitors from de-mystifying his obfuscated html homepage.
Also both Chrome and Firefox have a "disable breakpoints" button that would instantly defeat this issue.
This will return even functions contained in some module that are “private”.
They do seem to have some strange restrictions on this. E.g. When you evaluate it the function returns undefined but it also outputs the array underneath. You can right click and save it to variable.
I think the point of this so that you cannot assign the output programmatically, there has to be a person who saves it to a variable by right clicking.
I've thought for years the console should add Data.gui [1] style UI for viewing/testing variable and settings values. You can see it action on this CodePen [2].
Everything seems to work ok when the code is unrolled but as soon as it gets bundled, even if not minified, it seems that a lot of debugger features get broken, at least that's been my experience.
Note: I'm not a front-end engineer and I'm probably doing something wrong.
1) Go to Network panel, start recording network requests
2) Open left sidebar and invoke search to type in the code/ui string you want to find
3) It'll usually find it in some weird bundled js chunk file, click on the result
4) It opens the network request for that file, now right click anywhere in file and pick "Open in Sources" or something along that line, that jumps to debugger
5) Now place your debugger statement, this will probably load sourcemaps too
Lately, I've inherited a very messy NodeJS backend and have been pulling my remaining hair out working without proper debugging tools. I've gone back to 'console.log' debugging, but it makes me feel like a caveman.
I can't believe that this whole popular ecosystem doesn't have a proper debugging REPL - can anybody point me in the right direction?
Its debugging REPL is just the javascript console itself.
I guess I'll try again when I find the time. Was hoping there was some 3rd party package I could use to just drop in and set a trace. Thanks.
Works pretty great, JavaScript is pretty nice to debug since everything is incapsulated inside an object for the most part.
In VsCode you can also use the “open JavaScript debug terminal” command: it opens a terminal in which any node command automatically starts with a debugger attached
https://pypi.org/project/snoop/
deep tracer decorator
Take a look at disable-devtool [1], it surprises me just how many methods can be used to detect usage of an invaluable tool that should be a user's right to use. These "exploits" should really be patched browser-side, but I don't see any active efforts by browsers to fix this.
I've created a simple anti-anti-debug extension [2] that monkey-patches my way around these anti-debug scripts. It works fine for now, but I can't imagine it working consistently in the long term once the inevitable arms race begins.
How can we get Google, Mozilla, etc... to care about dev tool accessibility?
That being said, I am pleasantly surprised at the debugging and development tooling that is built right into most modern browsers. It really does make the UI development experience very powerful. I wish more back-end languages had this experience.
I work at Replay.io, and we're building a true "time traveling debugger" for JS. Our app is meant to help simplify debugging scenarios by making it easy to record, reproduce and investigate your code.
The basic idea of Replay: Use our fork of Firefox or Chrome to make a recording of your app, load the recording in our debugger UI, and you can pause at _any_ point in the recording. In fact, you can add print statements to any line of code, and it will show you what it _would_ have printed _every time that line of code ran_!
From there, you can jump to any of those print statement hits, and do typical step debugging and inspection of variables. So, it's the best of both worlds - you can use print statements and step debugging, together, at any point in time in the recording. It also lets you inspect the DOM and the React component tree at any point as well.
I honestly wish I'd had Replay available much earlier in my career. I can think of quite a few bugs that I spent hours on that would have been _much_ easier to solve with a Replay recording. And as an OSS maintainer for Redux, there's been a number of bugs that I was _only_ able to solve myself in the last year because I was able to make a recording of a repro and investigate it further (like a tough subscription timing issue in RTK Query, or a transpilation issue in the RTK listener middleware).
If anyone would like to try it out, see https://replay.io/record-bugs for the getting started steps to use Replay (although FYI we're in the middle of a transition from Firefox to Chromium as our primary recording browser fork).
I also did a "Learn with Jason" episode where we talked about debugging concepts in general, looked at browser devtools UI features specifically, and then did an example of recording and debugging with Replay: https://www.learnwithjason.dev/travel-through-time-to-debug-...
If you've got any questions, please come by our Discord and ask! https://replay.io/discord
A few years ago, I was hacking a web bookreader.
It has a function that is used to decode images (they're encrypted in some way) into canvas, and I want to find it so I can call it directly in my user script to batch download decoded images.
So I monkey patched `CanvasRenderingContext2D` function, added breakpoint, and found where the the function is defined in the 100k lines of obfuscated JS source code, easily.
The problem is.. once the page is rendered, the function would be nested in some objects, something like `window.abd.fdsfsd.r2323.fsdfs.fasf.xyy.myfunc`.
I don't know how exactly I can find the full "path" of the function so I can call it, despite I'm literally pausing inside of it. I eventually got it done, but it was manual and painful.
So I'm wandering: is there a better way to do it? The browser obviously knows it, it just lacks of a way to tell.
https://en.m.wikipedia.org/wiki/Immediately_invoked_function...
You could (ab)use `var` to initialize the variable outside the IIFE scope so you can see the values produced by the last IIFE call.
https://github.com/kristopolous/_inject
You can basically wander around any function context at any arbitrary time and see what happened. It exploits the reference counter to keep the contexts from being destroyed. It was really great back when I did a lot of client side js
The killer app version of this would be to open a repl at any context. As it stands it requires a good bit of competency to do it well.
https://firefox-source-docs.mozilla.org/devtools-user/debugg...
Edit: just realized this is literally the first thing mentioned in the linked article.
Sure, it'd be trivial to circumvent such a block, but it'd easily inconvenience most of the low-hanging fruit enough that things like community maintained ad blockers could become ineffective. Surely simply never serving ads to people who open the dev tools would prevent the 99.9% of normal users from using effective ad blockers in their browsers.
It's pretty easy to circumvent this method, but honestly the user shouldn't have to configure their browsers to be resistant to anti-debugging. From the side of the website, it should be impossible to know if the debug tools are just open.
I know what that feels like. I have that same feeling about a whole bunch of languages that has been available in .NET for at least 15 years now.
When I got into programming, I moved from VB6 to VB.NET to C#, the latter two using a cracked version of MS Visual Studio I downloaded at school. I only started seriously using the debugger somewhere around the time I taught myself C#.
Because of this, I got used to the idea that in a normal debugger, you could just click and drag back the point of execution, modify the code, and hit resume. The VS time traveling debugger will actually revert variable assignments, recompile the modified code, insert it in its place, and continue execution. This wouldn't work with P/Invoke calls, but it worked great for what I was trying to do. Debugging and fixing mistakes was absolutely trivial.
Imagine my betrayal when I found out that there was nothing like this in C++, or Java, or Javascript. Java has gotten a similar feature, but I've never worked with an IDE that integrated it even close to how well VS 2008 integrated it.
Even today, this seemingly basic feature that I took for granted just isn't available in many modern IDEs. I understand natively compiled languages like C/C++/Rust/Go not supporting the inline code replacement feature, but the hot reload in languages like Kotlin, Java, and Javascript just don't work like it did for me back in 2008.
As for all these tricks: I think most of them are available in most IDEs. Anything involving conditional breakpoints will work regardless of the language you use (although languages like Rust can have some restrictions). Tracing callers should work in any decent debugger. Timing individual functions may require writing out long package names to get to the right methods, but they should be available. The monitor() trick can be replaced by a logging method breakpoint.
How well DOM-related debugging works, will depend on whether or not your GUI framework uses a DOM and how it's constructed. If you're composing a DOM from multiple individual functions (like in React/Flutter/Kotlin Compose) then you can add breakpoints to all the relevant state alterations. If your tooling uses a more C-inspired windowing mechanism, you're probably going to have to get creative.
When you get down into the weeds of it, I think you'll find that more of these features are available for backend languages than you might expect. I would recommend any developer to occasionally read the changelogs of their IDEs (or to grab an IDE if you normally don't use any) and take a few moments to explore the possibilities of modern debuggers.
https://www.jetbrains.com/help/idea/altering-the-program-s-e...
For compiled languages (C, C++, Go, Rust etc.), check out https://rr-project.org/
the jvm debugging experience is pretty good imho.
It's the compiled languages that have a poor experience with debugging - try injecting code to execute in a c debugger! It's hard as hell to do!
I mean, theoretically, not really? Allocate a page RW, put in some compiled code, remap as R+X, redirect execution there, return code execution to where it was. Jumping through those hoops will be more expensive when you use the debugger like that (you'd need to hot patch some kind of jump statement to circumvent that) but it's not exactly impossible.
Things become difficult when the compiler starts optimizing out code, because the debugger would need to keep track of everything, but I don't see why it'd be technically impossible to do so.
Executing code in a C debugger is basically how modern reverse engineering works, it's a lot harder than in other languages but it's certainly possible.
I am working on something similar. I am building an IDE that allows to jump from the line printed by a console.log call to a corresponding code location, and observe variables and intermediate expressions.
It also displays a dynamic calltree of a program, allowing to navigate it in a time-travel manner.
Currently it only supports pure functional subset of JavaScript, but I am working on support for imperative programming (mutating data in place).
Harder is reproducing a bug based on what I’ve got to go on in Sentry. Mostly it’s easy enough to figure out but the traces you get in JS are very light in information compared to what you get in Python for example.
Anyone have any tips on getting more information in the initial report?
We've got some articles linked in our docs that talk about how the recording/replay mechanism uses "effective determinism" that's close enough to the original to be accurate:
https://docs.replay.io/learn-more/contribute/how-replay-work...
What's the goal of this transition?
But, Chromium is the dominant browser today. It's what consumers use, it's devs use for daily development, and it's what testing tools like Cypress and Playwright default to running your tests in. So, we're in the process of getting our Chromium fork up to parity with Firefox.
Currently, our Chromium for Linux fork is fully stable in terms of actual recording capability, and we use it extensively for recording E2E tests for ourselves and for customers. (in fact, if you want to, all the E2E recordings for our own PRs are public - you could pop open any of the recordings from this PR I merged yesterday [0] and debug how the tests ran in CI.)
But, our Chromium fork does not yet have the UI in place to let a user manually log in and hit "Record" themselves, the way the Firefox fork does. It actually automatically records each tab you open, saves the recordings locally, and then you use our CLI tool to upload them to your account. We're actually working on this "Record" button _right now_ and hope to have that available in the next few weeks.
Meanwhile, our Chrome for Mac and Windows forks are in early alpha, and the runtime team is focusing on stability and performance.
Our goal is to get the manual recording capabilities in place ASAP so we can switch over and make Chromium the default browser you'd download to make recordings as an individual developer. It's already the default for configuring E2E test setups to record replays, since the interactive UI piece isn't necessary there.
Also, many of the new time-travel-powered features that we're building rely on capabilities exposed by our Chromium fork, which the Firefox fork doesn't have. That includes the improved React DevTools support I've built over the last year, which relies on our time-travel backend API to extract React component tree data, and then does post-processing to enable nifty things like sourcemapping original component names even if you recorded a production app. I did a talk just a couple weeks ago at React Advanced about how I built that feature [1]. Meanwhile, my teammate Brian Vaughn, who was formerly on the React core team and built most of the current React DevTools browser extension UI, has just rebuilt our React DevTools UI components and started to integrate time-travel capabilities. He just got a working example of highlighting which props/hooks/state changed for a selected component, and we've got some other neat features like jumping between each time a component rendered coming soon. All that relies on data extracted from Chromium-based recordings.
[0] https://github.com/replayio/devtools/pull/9885#issuecomment-...
[1] https://blog.isquaredsoftware.com/2023/10/presentations-reac...
What is the reason/need for this?
If you're sure the function is reachable from window you can search for it recursively:
(function () {
function search(prefix, obj, fn, seen = null) {
if (!seen) seen = new Set();
// Prevent cycles.
if (seen.has(obj)) return false;
seen.add(obj);
console.log('Looking in ' + prefix);
for (let key of Object.keys(obj)) {
let child = obj[key];
if (child === null || child === undefined) {
} else if (child === fn) {
console.log('Found it! ' + prefix + '.' + key);
return prefix + '.' + key;
} else if (typeof child === 'object') {
// Search this child.
let res = search(prefix + '.' + key, child, fn, seen);
if (res) {
return res;
}
}
}
return false;
}
// For example:
let fn = function() { alert('hi'); }
window.a = {};
window.a.b = {};
window.a.b.c = {};
window.a.b.c.f = fn;
return search('window', window, fn);
})();In your example, you already has a reference of `fn` to use, which isn't the case for my userscript (if I have a reference, I would just use it!).
I have to search based on the features in plain text of the function (using some regexes on `fn.toString()` to check how the arguments look like).
A tree overview of all nested functions and objects with a search function that quickly jumps to that place, and maybe even let you call that function with data from state?
[0]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/arguments/calleeI guess it technically can be done by modifying the existing JS in-place, but doing so with a very large, minified JS would be a nightmare in term of maintenance.
Since on a more open platform like the CLR, JVM Python interpreter or bare metal (as opposed to a controlled sandbox like the browser) there are many different network traffic libraries dealing with all kinds of different ways of communicating (everything from basic HTTP to binary protocols for communicating with hardware like DNP3).
How would the standard debugger hook into all these different libraries? I'm assuming they'd have to implement hooks and maintain them too, nevermind if there's more than one debugger available to be supported.
Seems like a lot of effort compared to just using a tool like Wireshark or, for HTTP stuff, a MITM proxy...
For the same reason it doesn't seem that useful to bring in very environment-specific stuff like Redux.
I had an example for Redux, because one of my pain is that there is no larger view framework for Java, which can’t access JNDI. In other words, nothing stops anybody to create something like Redux, just the view layer shouldn’t be able to access anything else except the model. Even now, it is an antipattern in my opinion to reach anything from the view which has a state and it’s not in the model. Jakarta MVC, Spring MVC, and Struts are close to this, JSF’s model is the JNDI itself, so it’s even easier to write way worse spaghetti code.
You can put any expression into a conditional breakpoint, so anything you can do in the console you can do in a conditional breakpoint.
So, if you're doing this sort of thing once, you can just type it into the console and you're golden. But if you want to modify a stack local variable over and over again every time it is initialized, it's much easier to do in a conditional breakpoint because then it will happen every time that line of code runs, and your debugger never has to pause. (see https://alan.norbauer.com/articles/browser-debugging-tricks#...)
I don't get automatically reloaded classes in IntelliJ, though I think there's a button to manually reload code manually? The whole ease of development came from not having to manually pick what code to reload while debugging. Java also seems to have a bunch of limitations that C# doesn't have (and didn't have back when I tried it on Windows). For example, you can't add a method, or alter the return type of a method.
As for time travel debugging, I'm not sure how you do that in IntelliJ? I can't even see any way to undo the previous line of execution, let alone step through code in two directions. It looks like there are plugins for a bunch of languages, though, but those are all external tools built by third parties.
Rolling back executed code seems to work in Rider, but I don't seem to get any Edit and Continue functionality on Linux, which is probably a Microsoft thing.
It looks like I was mistaken though, "IntelliTrace" was a VS2010 feature, not 2008: https://learn.microsoft.com/en-us/archive/msdn-magazine/2010...
So TLDR don't monkey-patch a giant minified source file with a bunch of your own code. Monkey-patch the function just enough to get it to be exposed globally with its context, and then have your code separately reference it.
This is still fragile (unless you have a stable entry point with regex or something, which is not completely certain to exist, it'll likely break whenever the script changes), and is still not ideal, and I'm still thinking a bit to see if I can do anything better, but I suspect it's more stable than trying to access the function directly by coding the path and should be a great deal simpler as well since it will work even if the function/context is in a closure.
----
Of course, you might be lucky and there might actually be a way to get at the function. For example, I am separately looking into seeing if I can figure out a clean way to reliably look for imports exposed through a webpack bundle, since those are exposed globally and if you can find the paths reliably you should be able to get access to any of the imports (although you still won't have access to the closures). I haven't made much progress on it though, mostly because it's kind of frustrating to work on.
----
There is also the possibility (although I suspect it would get very messy very quickly, and I'm not sure of current browser support) that you could theoretically maybe rig something together with `function.caller` (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...), possibly patching the minified source file merely to remove "use strict".
I have not messed with `caller` in probably close to a decade at this point, so I make no promises at to whether it still works at all :) but... in its heyday before we realized how wildly insecure it was, it did allow reconstructing the stack from a function in-code in some situations.
Great work Replay team! Awesome tech.
We're not currently doing active dev work on the FF fork due to the focus on Chromium, and I think the goal is to fully try to move over to Chromium as the user recording browser. So, we probably _won't_ be maintaining the FF fork going forward due to limited resources. I think our next runtime focus after Chromium parity will be Node. We've already got an alpha-level Node for Linux fork that actually works decently - I've used it myself to record Jest or Vitest tests or Node scripts, and debug them. But that does need a lot of work to flesh it out.
I can’t see any reason for preventing assigning the output programmatically.
The reason is absolutely security concerns. You don't want to leak a function that can expose a list of all objects on the JS heap.