The reason for that is because of energy employed for farming, harvesting, processing, and distribution. From what I remember, we need something like 8 to 10 calories of energy to produce 1 calorie of food:
"10 Calories in, 1 Calorie Out – The Energy We Spend on Food"...