You can theoretically do that, it may circumvent some of the input protection though, or rather make it less effective, since you are coming from the wrong side of the input filter... There is no -5V anyhwere on the board, the Rock64 only uses 5V as input, so only connect the 5V and GND pins.
Come have a chat in the Pine IRC channel >>