It turns out that scientists thought the same thing very recently. According to this Anatomy of a Comet article at NASA Jet Propulsion Laboratory's website, related to the Rosetta mission,
"Scientists used to think that it was solid and firm, but NASA’s Deep Impact mission (2005), in which Rosetta participated, surprised them.
They found that the nucleus of comet Tempel-1 was more silicate or
dusty than they expected."
So, it was only about a decade ago that scientists were able to see that the surface of a comet was dusty.
Following the Deep Impact mission, JAXA also had an asteroid-related mission in 2005 called Hayabusa. Hayabusa landed on the surface of the asteroid, which disturbed some of the dust on the surface so that it could be collected and returned to Earth.
When comets are farther from the Sun, they tend to be completely frozen. When the comet moves in closer, the heat can cause nucleus to start shooting streams of gas and dust from within the comet, creating the coma and eventually the dust tail. It makes sense that some of the particles that aren't lost to space and remain in the coma would eventually succumb to the comet's gravity and settle back down on the surface.
In a somewhat related note, ESA are interested in the dust on/around the comet. They included a device on Rosetta called GIADA (Grain Impact Analyser and Dust Accumulator) to collect and analyze dust from 67P's coma.
GIADA on the Rosetta probe. Image source: Rosetta Blog: GIADA Investigates Comet's “Fluffy” Dust Grains