When keeping a food diary, do you add up all the calories from food you have eaten and then minus from that the calories you used up from exercising?

I think I am doing something wrong as I had no more than a total (food-ex) of 1200 cals a day last week and lost 2 lbs but this week on average I had 1300 cals and have put on 2 lbs! Although this week some days I did have up to 1900 but then some days I had 900 so it averaged out to 1300 - could this be the problem - I should not average them out?

Thanks!