What exactly is a comet made of?
A comet is mostly made of water ice. Other chemicals may be present as well, mostly amonia (which will behave in a similar way to ice, as it will vaporize on approach to the sun). A comet will also contain dust and may contain more or less amounts or rocky material. The distinction between comets and asteroids may be more blurry than initially thought. At least one asteroid as been seen to develop a comet-like tail. This means that asteroids that come close enough to the Sun may experience vaporization of volatile materials (water and amonia ice) just like comets do.
For a comet to exhibit a tail when it approaches the sun, it must have some volatiles (easily-evaporated compounds called ices). Comets consist of different mixes of ices (water ice, ammonia ice, methane, etc.) as well as rock, dust, and non-volatile material. The exact composition of any comet is unknown, and it appears that each comet is different in composition. But they are all basically rock, ice, and dirt.
Spectroscopic measurements and visits to a few comets by spacecraft have shown that different comets are made of different stuff. A lot of it appears to be evaporative (e.g. water and dust particles) which are given off when the comet warms as it approaches the sun. In fact, we don’t know the answer to your question generally, and only know it partially for a couple comets spacecraft have visited.
