Seems Like It
Absurd hypotheticals aside, the Intergovernmental Agreement (IGA) specifies in Article V:
[E]ach Partner shall retain jurisdiction and control over the elements it registers...and over personnel in or on the Space Station who are its nationals.
This ESA resource summarizes:
The Intergovernmental Agreement allows the Space Station Partners States to extend their national jurisdiction in outer space, so the elements they provide (e.g. laboratories) are assimilated to the territories of the Partners States.
For ESA astronauts/modules, I believe this means property/jurisdiction of the EU.
This article interviewing a "space lawyer" explains it essentially as:
multiple embassies floating right over our heads
I think it's fair to say that if a Russian woman gave birth in, say, the Destiny module it would probably be treated the same as if she had walked into the U.S. and given birth. Of course, the U.S. and Russia could just as easily say that they are making an exception to the IGA for an unorthodox situation and that would probably be the end of discussion.