The electro-optical properties of most semiconductors and insulators of technological interest are dominated by the presence of electron-hole quasi-particles, called excitons. The manipulation of excitons in dielectrics has recently received great attention, with possible applications in different fields including optoelectronics and photonics. Here, we apply attosecond transient reflection spectroscopy in a sequential two-foci geometry and observe sub-femtosecond dynamics of a core-level exciton in bulk MgF2 single crystals. Furthermore, we access absolute phase delays, which allow for an unambiguous comparison with theoretical calculations. Our results show that excitons surprisingly exhibit a dual atomic- and solid-like character, which manifests itself on different time scales. While the former is responsible for a femtosecond optical Stark effect, the latter dominates the attosecond excitonic response. Further theoretical investigation reveals a link with the exciton sub-femtosecond nanometric motion and allows us to envision a new route to control exciton dynamics in the close-to-petahertz regime. The capability to follow electron motion in solids is necessary to explore the ultimate speed limits of optical charge manipulation and signal processing in optoelectronic devices. Here, the authors reveal the sub-femtosecond dynamics of core excitons in MgF2 and find the dual atomic-solid nature of the exciton quasi-particle to deeply affect its ultrafast dynamics.