After a considerable amount of reading on the matter, I am convinced that as Houston's influence began to rise, it was inevitable that von Braun's influence and visibility was destined to fall.
After being brought to the United States in Operation Paperclip, von Braun became technical director of the Army Ballistic Missile Agency (ABMA) in Huntsville, Alabama. After NASA was formed, ABMA was transferred to the civilian space agency as the Marshall Space Flight Center (MSFC), with von Braun as director. ABMA and MSFC developed the Jupiter and Redstone launch vehicles used by NASA's early unmanned and Mercury flights. This placed von Braun in a position of great influence in the fledgling space agency, just at the time that President Kennedy challenged in May 1961 to NASA to send a man to the moon.
Later Mercury flights and the Gemini missions used the Atlas and Titan rockets designed by the Air Force, rather than by MSFC. However, MSFC was kept busy during this period developing the Saturn launch vehicles which would be needed to go to the moon. The first Saturn I flew in October 1961.
By 1965, Saturn design was finished and the vehicle was well into testing. Von Braun realized he was in peril of losing his workforce:
Saturn development was proceeding well in 1965. The last test flights of the Saturn I were run off that year and preparations were under way for a series of Saturn IB shots. In August each of the three stages of the Saturn V was successfully static-fired at full thrust and duration. Not only that, but the third stage was fired, shut down, and restarted, successfully simulating its role of injecting the Apollo spacecraft into its lunar trajectory. Flight testing remained to be done, but Saturn V had taken a long stride.
Confident though they were of ultimate success, Marshall's 7300 employees could have felt apprehensive about their future that summer. After Saturn V there was nothing on the drawing boards. Apollo still had a long way to go, but most of the remaining work would take place in Houston.
Von Braun felt his influence, budget, and personnel being drained away by the Manned Spacecraft Center (now called Johnson Space Center) in Houston, Texas. NASA broke ground in April 1962. Construction finished in September 1963, ready for the Gemini missions 1965-1966. Houston quickly gobbled up many aspects of the Apollo program: design of the command module, design of the service module, design of the lunar lander, training of astronauts, mission planning, and mission control. Von Braun responded by asking headquarters in August 1965 to assign lunar science to his center:
Besides developing all standard and modified spacecraft, MSC would direct astronaut training, mission control, and flight operations. In addition to its launch vehicle responsibilities, Marshall would integrate experiments into the lunar module.
Since lunar-module development was under Houston's purview, the decision represented a significant step away from Apollo assignments and upset some people in Texas. On 14 October 1965 the Houston Post reported, "Marshall May Take 2nd Apollo Control." [...] The article caused a minor tempest. Rep. Olin Teague, the Texas Democrat chairing NASA's oversight subcommittee, looked into the matter.
Another frustration for von Braun was Apollo's use of contractors. Since his V-2 days, von Braun was used to having people on-site build the launch vehicles:
From von Braun all the way down, Huntsville's rocket builders were dirty-hands engineers, and they had produced many Redstone and Jupiter missiles. In 1962 von Braun remarked in an article written for a management magazine, "we can still carry an idea for a space vehicle . . . from the concept through the entire development cycle of design, development, fabrication, and testing." That was the way he felt his organization should operate, and so it did; of 10 first stages built for the Saturn I, 8 were turned out at Marshall.
To keep MSFC open, in December 1965 NASA authorized it to begin the Apollo Applications Program, to study using Apollo hardware for scientific purposes after the lunar missions. Many of the projects were little more than concepts, but after the Apollo 11 landing, others began looking for post-lunar missions for NASA. Skylab and the Apollo Telescope Mount were launched together in 1973. However, each year after 1965 the budget, personnel, and influence of MSFC decreased -- first by the growing lunar budget, and later by decreased overall funding of NASA.
So the reason why von Braun was not visible much after 1965 was that his influence was eclipsed by what was happening in Houston. Frustrated, he retired in 1972.