This paper presents a robust orbit–attitude control method for spacecraft body-fixed hovering over a captured asteroid in the earth–moon system. First, relative orbit–attitude motion dynamics model of spacecraft with respect to an asteroid is derived in the context of circular restricted three-body problem. As opposed to previous works focusing on point-mass translational models, this paper proposes a new feature point-based model which is established in a local frame originated in a selected spot attached to the asteroid surface. In such a case, the kinematic coupling effect is explicitly included, and the desired relative motion trajectories are thus to be time-invariant. Next, an adaptive sliding mode controller is devised for spacecraft robust hovering as well as sensor pointing in the presence of external disturbances and parameter uncertainties. Finally, a simulation scenario of spacecraft hovering over an asteroid which is assumed to be captured in a stable lunar distant retrograde orbit is illustrated. Results demonstrate that both orbital and attitude relative motion track the reference trajectories well under the proposed control law.