Answer :

No that sentence isn't correct because , "He is elder to me". Doesn't make sense, moreover elder means old saying to the person that your elder then them makes no sense at all.
Hope this helps:)
well elder is a noun or adj. but the correct word would be he is old to me