As long as it is voluntary, he can break it whenever he wishes. The problem is that if he breaks it with the intention that people would not think he is fasting, this is a greater sin by itself because he is fearful of the people and this is one of the steps of Satan where he plays with him and manipulate him as he wishes.