An example would be the surface of a swimming pool - reflection of light off a swimming pool will project a pattern like this [0] onto the nearest wall, and if that's where your camera positions are they are effectively sampling that image - there is no data for what happens in between. The resulting NeRF reconstruction will look dull and flat, with the water taking on a matte or satin texture.

Another example would be a laser beam - all the cameras see is a weak translucent line, if they see anything at all, and you cannot infer that you will receive a blinding amount of light if you put your eye right in the beam unless you happen to know about lasers. Even if you do know about lasers, if the laser itself is not in shot, you cannot determine which way the beam is going.

Assuming that nothing in the scene is emitting perversely high-frequency signals like a laser, you could in theory make a guess at some of this information by inferring a lot more about the scene - calculating the position of lights, computing normal maps, guessing that certain sets of points are made from the same material and therefore likely have the same BSDF. But current NeRFs don't do any of that, they just run volume rendering on a set of emissive points.

[0] https://www.google.com/search?q=swimming+pool+reflection

That said, the geometric functions (i.e. the positions of rays w.r.t. other objects) are probably non-linear in the object position parameters indeed, and intensities certainly are (example: moving an object in front of a screen by X amount changes the illumination in front of the screen non-linearly): it's important to keep in mind what we mean by linearity (it's in terms of light field intensities for a fixed geometry scene) -- the scene are the transfer function coefficients.

What really makes things complicated in real life is (1) the presence of noise; (2) imperfections (and unknowns) in your physical/geometric apparatus. Even if you know the system perfectly, noise generally disallows reverting (or easily reverting) transfer functions, i.e. undoing blurs and arbitrarily refocusing images with simple detectors. The imperfections and even lack of rigidity of lenses and your system add even more difficulty. That's why making a simple computational lens is not so easy.

In other words, one does not obviate the other.

For example, we happily draw a linear scale on a mercury thermometer, even though we know that not to be correct, even if we incorrectly assume that the coefficient of expansion of mercury is independent of its temperature. Also, try explaining why the conversions between Celsius, Fahrenheit and Kelvin are all linear (answer for Kelvin and Celsius: they technically only are since 2019, when we redefined them (https://en.wikipedia.org/wiki/2019_redefinition_of_the_SI_ba...). I think the Fahrenheit scale was redefined as a shift on the Kelvin one around the same time)