There are a couple of factors here. First, if your employer is asking you to exceed the restrictions then you should politely state what your restrictions are and clarify which portion of the job you feel you cannot do. You may be wondering if a doctor can force you to go back to work? We recommend that this be done in writing so that you will always have proof that you did it and that you did it politely and in a professional manner.
Second, if the job is just too much then you should report this to the doctor. As always it would be helpful if you could explain what aspects of the job are too much for you and what your symptoms are when this happens. All of this gives you more credibility with doctors. In a situation like this where it’s often a matter of pain, conditioning, etc., having credibility is critical.