Does the non-custodial parent have more rights than the grandparent?
Seems like the child would have to be determined by a court-ordered study to be living permanently with the grandparent, rather than the custodial parent, for you to have any shot at gaining custody. You may be able to implement an investigation through the court that awarded custody initially, but you will probably also end up paying the costs involved in having everyone evaluated. The court will award a child advocate to investigate each party and their home environment thoroughly. If a custody hearing is granted, you would have to show that you are the more fit parent in every way, and that includes financially. Be prepared for a long, expensive battle, as this can drag out for years. But if the child is in harm with the grandparent or custodial parent, it will have been worth it.