The aphelion is usually considered to be the point of greatest actual separation, so this would include the (very minor!) gravitational effects of other solar system bodies. It's not the slight movement of the Sun however that is most important, the gravity of our very own Moon causes significantly more wobbliness in the Earth's orbit. But all in all, the list of increasingly small effect grows as you consider more and more complications, and the aphelion is the eventual sum of all of them...
On the other hand, the aphelion is also often considered the furthest point in the mathematically perfect 2D conic section approximation of an orbit, which is usually good enough. This would give you a subtly different value.
Both of these meanings are common, and without further context one shouldn't assume either.
As a sidenote about terminology, the generic term is apoapsis, and the word changes depending of the parent body. So apogee would be the furthest point from Earth, relevant to the orbit of the Moon, ISS, etc. "apogee of Earth" would technically not make sense, hence the use of "aphelion" (furthest point from the Sun) in this answer.