While the solar corona is very hot, it also has very low density: Wikipedia gives a ballpark figure of about 1015 particles per cubic meter, which, at 1 million Kelvins, translates to a pressure of about 0.01 Pa. That's a pretty good vacuum, comparable to that in low Earth orbit.
The low pressure means that the coronal plasma doesn't hold much heat that it could transfer to the spacecraft. In practice, a much bigger issue for heat management so close to the Sun is the intense sunlight that transfers a lot of heat to anything that absorbs it. As Gerald notes in his answer, the main way in which the Solar Probe Plus deals with that issue is by placing a highly reflective and heat-resistant shield on the Sun-facing side of the probe to shield it from sunlight, combined with an efficient cooling system and an elliptical orbit that only takes the probe close to the Sun for relatively short periods at a time.
Spacecraft Overview: "preliminary designs include an 8-foot-diameter, 4.5-inch-thick, carbon-carbon carbon foam solar shield atop the spacecraft body..., radiators for the solar array cooling system, ... actively cooled solar arrays". Low albedo for the heat shield isn't mentioned explicitely.
A highly elliptical orbit leads to relatively short periods of very high heat exposure, details in this paper.