In the ‘Perceptual and Semantic Compression in Latent Diffusion Models’ slide, the Rate–Distortion curve suggests that below a certain rate the distortion increases sharply, which I interpret as the region where semantic information starts to be lost. In the context of Latent Diffusion Models, when tuning the auto-encoder’s compression strength (downsampling factor, number of latent channels, strength of KL/VQ regularization ...), is there any way to quantitatively estimate or exploit this boundary between the ‘semantic’ and ‘perceptual’ regions on the curve? For example, are there procedural guidelines or experimental results that measure the R–D curve on a dataset and then choose the latent dimensionality near the critical point where semantic distortion begins to rise?
In the ‘Perceptual and Semantic Compression in Latent Diffusion Models’ slide, the Rate–Distortion curve suggests that below a certain rate the distortion increases sharply, which I interpret as the region where semantic information starts to be lost. In the context of Latent Diffusion Models, when tuning the auto-encoder’s compression strength (downsampling factor, number of latent channels, strength of KL/VQ regularization ...), is there any way to quantitatively estimate or exploit this boundary between the ‘semantic’ and ‘perceptual’ regions on the curve? For example, are there procedural guidelines or experimental results that measure the R–D curve on a dataset and then choose the latent dimensionality near the critical point where semantic distortion begins to rise?