In this video, Matt Cutts discusses how you can tell if your site has an algorithm penalty from Google. The answer Matt gives is very long and complicated. He first states that if you have a manual action you will know by logging into webmaster tools. But an algorithm penalty is very hard to determine, because in some cases, you might not even have a penalty, you might just not be ranking as well as the other guy. This is a fairly interesting video, but it does lack any real core substance.