Is there any light source without the stars in the Universe?
All objects that don’t have a temperature of absolute zero kelvin emits electromagnetic radiation, we just can’t see most of it. Only objects with very high temperatures emit electromagnetic radiation within the visible spectrum. If no fusion took place in the universe there would be no heat except for whatever energy existed at the moment of the Big Bang. Today after several generations of stars radioactive decay of some elements produce a lot of heat without any more influence from fusion, but they were created by the fusion process. Fusion has created all elements above hydrogen.