I know that ESA is working on a new reusable first stage booster. Does NASA have similar plans?
As a general rule, NASA tries to stay out of the launch business and focus on the science business. SLS is a weird holdover partially imposed upon NASA by Congress.
Companies like Rocket Lab, SpaceX, Blue Origin, ULA, Northrop Grumman, Virgin Orbit, ABL, Astra, Firefly, Relativity Space, and others either have shown or are in the process of showing that you don't need NASA to design rockets, even Super Heavy Lift ones (e.g. SpaceX Starship).
In 2014, then NASA Administrator Charlie Bolden said: "The Falcon Heavy may some day come about. It’s on the drawing board right now. SLS is real." And in 2016: "If you talk about launch vehicles, we believe our responsibility to the nation is to take care of things that normal people cannot do, or don’t want to do, like large launch vehicles."
But he has since changed his mind and said in 2020 (at that time, he was no longer NASA Administrator, having been replaced by Jim Bridenstine): "SLS will go away. It could go away during a Biden administration or a next Trump administration … because at some point commercial entities are going to catch up. They are really going to build a heavy lift launch vehicle sort of like SLS that they will be able to fly for a much cheaper price than NASA can do SLS. That's just the way it works."
And that seems to be the general view at NASA and in Congress as well, even if not everybody says it out loud. Since then, SpaceX has successfully launched Falcon Heavy and both SpaceX and Blue Origin are well underway on their next-generation Super Heavy Lift Vehicles.
NASA is good for tackling problems that are extremely expensive and high-risk, such that a commercial entity could never afford to tackle them. Or, projects that are expensive and have no commercial benefit. So, if NASA ever builds a new rocket again, it would be something like a nuclear-powered rocket for deep space exploration. But even then, they'd probably only build a prototype to prove the concept and analyze the risks, then hand it over to the industry.
Just a couple of weeks ago, NASA and DARPA announced that they are planning to test a nuclear-thermal engine in space by 2027.
Congress mandated SLS for a range of political reasons largely concerning aerospace jobs. SLS is likely to remain in service while these needs remain and NASA is not at liberty to start any new rocket development program without being specifically tasked to do so by Congress.
It is entirely possible that there will be no further “NASA” rocket to replace SLS when the time eventually comes. Congress may decide that there are sufficient commercial rockets available to meet its launch requirements. The next mandate might be for NASA to achieve a particular goal such as landing people on Mars, perhaps with a range of non rocket support requirements (e,g nuclear reactors for space based electrical power generation and rovers).