Christians of all kinds: Would you listen to the Christian music of an artist who had later de-converted?
The most important thing is why you listen to the music. If you are listening because the songs are heartfelt expressions of love and devotion to Jesus then because at a later date that devotion wavers, it does no invalidate the truth and love of the previous songs. Elvis Presley was a devoted Christian and singer before his “king” status and these songs and the way they are sung still inspire love and joy when they are listened to. Merely because he changed his beliefs does not mean his earliest beliefs were not right and firm, nor does it mean his old songs can impact on us less. So yes I would as it does not change the song only the creator. But if it bothers you that the sentiment is no longer evident then perhaps you shouldn’t. It is up to you and depends purely on the way you feel about it. No one else can give you an answer that will fit what you need.