When you write “However, on the plat map, the original right of way goes through where they want to build. The right of way was moved about 20 years ago, and the other neighbor has always used it.” that the original predial servitude (and I’m assuming that this is a predial servitude and not a personal servitude) was abandoned or that the servitude was terminated and another servitude established (this would have been done by the signing and recordation of documents)?
Because of the details concerning the possible existence or non-existence of the servitudes, this is NOT a DIY project; this requires the attention of an attorney.