If he can finish his prayer before the first takbeer, he should continue. But if he can’t, he should break his prayer and join the congregation

If one comes to the masjid late and prays 2 rakaat and the salah begins and everybody starts getting ready for congregational prayer and he still hasn’t finished tahyat al masjid should he stop his salah and pray with them or should he continue until he is done with the two rak’aat?