ANSWER:
Yes, these two rakas are called tahiatul masjid. One is not allowed to sit before praying it. However, if one joins the congregation immediately without sitting or offers sunnah of a prayer first, this exempts him from these two Rakah of tahiyatul masjid.