He has no right in beating his sister like this! Only her father has the right to discipline his daughter and not by using excessive force like this.
He has the right to force her to wear hijab by not allowing her to leave the house. As for prayers, if she oversleeps, he should advise her, wake her up on time and remind her of Allah.