The practical upper velocity that can be gained by gravitational assists is limited by the number of times a spacecraft can reach another gravitational body to increase its velocity as well as the strength of the gravity well of the body. If the spacecraft is moving slowly it can alternate between two planets, increasing its velocity during each pass (and decreasing ever so slightly the orbital momentum of the planets it passes). As it moves faster and faster it will need to make passes closer closer to the planets to allow a return to the other planet. Eventually, passing closer to the planets leads to passing through an atmosphere (slowing the craft), grazing the planet (destroying the craft), or flying off away from both planets (the last pass causes a minor path deflection and not a half-orbit return to the other planet).
The limits of gravitational assists is also discussed at https://en.wikipedia.org/wiki/Gravity_assist#Limits_to_slingshot_use, where it mentions the "Grand Tour" made by Voyager, where the planets Jupiter, Saturn, Uranus and Neptune were lined up in such a way that allowed Voyager to increase its speed during the flyby of each planetary mass. If the solar system had another dozen planets that Voyager could have passed close to during its journey, it might have used them all to end up going even faster.
If you had two black holes orbiting each other, due to their extreme gravity you might, in theory, be able alternate back and forth between the two for gravitational assists until you approached the speed of light.
Another real-life limit to unlimited numbers of gravitational assists is that in order to make a pass occur along the precise path required to reach the next gravity well some reaction mass will be expended for course correction. The catch-22 here is that if you had unlimited on-board thrust you wouldn't need gravitational assists, but if you have limited thrust, after you run out you can't course correct to do the next gravitational assist.