What is wrong with my son?

My five year old son came down with a fever Saturday morning. He has cold symptoms, Diarrhea, and no appetite. I took him to the Doctor on Monday, his temp was around 101-102 she said it was just a virus and plenty of rest and liquids would get rid of it. Its Wednesday night and he still has a fever and all the above symptoms. He sleeps most of the day and drinks lots, but he doesnt seem to be getting any better, worse maybe. He is cranky and miserable. I have hardly left the house all week, Im starting to go crazy. Should I take him back to the doctor or just wait it out?