If the existing hedge is sparse, then a hole large enough for the roots of the bare-rooted tree to spread out in will be sufficient. In a denser hedge there is the issue of root copmetition from other trees and a larger hole may be required.
Yes plant at least 1m from fence line. You might need access along that 1m to replace fence and even after a growing season there will be less than 1m of clear access.
Sea buckthorn is mainly insect pollinated so plants can be a fair way away from each other although fruiting will be better is no more than 10m apart.