Definitely - it could be ejected. But Earth would only play a minor role.
Starman now counts as a Near Earth Object, being any object crossing Earth's orbit. Any such object is occasionally in Earth's vicinity, when they cross our orbit while we are nearby. The orbits of such objects have now been modeled over time periods of millions of years. From Asteroid Mining 101 by John Lewis, of Deep Space Industries:
These studies revealed that the mean lifetime of an NEA* is about 30
million years, and the main mechanisms for loss of these bodies were
impact on a terrestrial planet or perturbations into Jupiter
approaching orbits, from which the body could strike Jupiter, or could
be ejected from the solar system, or even fall into the Sun.
So, it is by interaction with Jupiter, which has by far the strongest gravitational pull of all the planets, that an object can acquire escape velocity from the solar system.
Over time periods this long, all the orbital perturbations acting on an object, from all the planets, and to a much lesser extent moons and asteroids, causes their orbits to be completely unpredictable. That is to say, the gravitational interactions are so many and so complex, uncertainty accumulates to the point where it is impossible to say where something will be - not just you'll be off by a few percent, it could be anywhere, including no longer in the solar system. So, though Starman's orbit right now will keep looping around nicely in a safe orbit that passes close to Earth and close to Mars now and then, it is entirely possible it will drift into the kind of resonant orbit with Jupiter that will slowly make its orbit more and more elliptical until it just ejects it from the solar system altogether.
There is also the matter of the Yarkovsky effect. Astronomer Jonathan McDowell has been quoted in many places recently talking about the likely outcome. The following is from an article in SpaceFlight Now:
...two forces acting on the car will limit its lifetime. One is
Jupiter. The giant planet’s gravity perturbs bodies in the asteroid
belt and, over time, will have an effect on the Tesla’s trajectory.
The other effect is a subtle acceleration produced by tiny
temperature-related forces over extremely long periods that also would
act to change the orbit. “It’s tiny, but over timescales of millions
of years it’s enough to shrink the orbit and make the thing fall into
the sun,” McDowell said. “So it’s a race between does that happen
before some Jupiter perturbation ejects it from the system.
*NEA is Near-Earth Asteroid, but of course the orbital mechanics for any object is the same.