Improve Stable Diffusion Quality with Skip Connection Rescaling (No Training)

Improve Stable Diffusion Quality with Skip Connection Rescaling (No Training)(arxiv.org)

2 points by erwannmillon 2 years ago | 4 comments

Improve SD image quality and reduce artefacts without any additional training, simply by reweighting skip connections in the decoder stage of a diffusion Unet decoder

dvrp 2 years ago | |

i know i should read the paper but... what does reweighting the skip connections mean in this case? like... how?

erwannmillon 2 years ago | | |

In the decoder, the features from the unet blocks get concatenated with features from the encoder layer through 'skip connections'. The paper discusses how rescaling the backbone features (element-wise multiplication by some scalar) before concatenation improves image quality.