ANSWER:
There are two types of rewards, one for praying with the imam and this is 27 times better than praying alone. The second type is praying with the congregation and this is greater in reward the bigger the congregation is.
In your case, you won’t get the first reward unless being late was for a legitimate reason but will get the second one.