Recherche avancée

Sur d’autres sites (7797)

• Stop doing this in your encoder comparisons

  14 juin 2010, par Dark Shikari — Uncategorized

  I’ll do a more detailed post later on how to properly compare encoders, but lately I’ve seen a lot of people doing something in particular that demonstrates they have no idea what they’re doing.

  PSNR is not a very good metric. But it’s useful for one thing : if every encoder optimizes for it, you can effectively measure how good those encoders are at optimizing for PSNR. Certainly this doesn’t tell you everything you want to know, but it can give you a good approximation of “how good the encoder is at optimizing for SOMETHING“. The hope is that this is decently close to the visual results. This of course can fail to be the case if one encoder has psy optimizations and the other does not.

  But it only works to begin with if both encoders are optimized for PSNR. If one optimizes for, say, SSIM, and one optimizes for PSNR, comparing PSNR numbers is completely meaningless. If anything, it’s worse than meaningless — it will bias enormously towards the encoder that is tuned towards PSNR, for obvious reasons.

  And yet people keep doing this.

  They keep comparing x264 against other encoders which are tuned against PSNR. But they don’t tell x264 to also tune for PSNR (–tune psnr, it’s not hard !), and surprise surprise, x264 loses. Of course, these people never bother to actually look at the output ; if they did, they’d notice that x264 usually looks quite a bit better despite having lower PSNR.

  This happens so often that I suspect this is largely being done intentionally in order to cheat in encoder comparisons. Or perhaps it’s because tons of people who know absolutely nothing about video coding insist on doing comparisons without checking their methodology. Whatever it is, it clearly demonstrates that the person doing the test doesn’t understand what PSNR is or why it is used.

  Another victim of this is Theora Ptalarbvorm, which optimizes for SSIM at the expense of PSNR — an absolutely great decision for visual quality. And of course if you just blindly compare Ptalarbvorm (1.2) and Thusnelda (1.1), you’ll notice Ptalarbvorm has much lower PSNR ! Clearly, it must be a worse encoder, right ?

  Stop doing this. And call out the people who insist on cheating.

• Revision 35436 : Petites pétouilles en passant par là (écriture aux dernières normes ...

  22 février 2010, par marcimat@… — Log

  Petites pétouilles en passant par là (écriture aux dernières normes ISO)…

• Revision 30118 : protéger des erreurs si pas configuré

  23 juillet 2009, par kent1@… — Log

  protéger des erreurs si pas configuré