Electron dynamics in metallic clusters are examined using a time-dependent density functional theory that includes a “memory term,” i.e., attempts to describe temporal nonlocal correlations. Using the Iwamoto, Gross, and Kohn exchange-correlation XC kernel, we construct a translationally invariant memory action from which an XC potential is derived that is translationally covariant and exerts zero net force on the electrons. An efficient and stable numerical method to solve the resulting Kohn-Sham equations is presented. Using this framework, we study memory effects on electron dynamics in spherical jellium gold clusters. We find memory significantly broadens the surface plasmon absorption line, yet considerably less than measured in real gold clusters, attributed to the inadequacy of the jellium model. Memory effects on nonlinear spectroscopy are studied as well: a real-time pump-probe setup is used to study the temporal decay profile of the plasmon, finding a fast decay followed by slower tail; and in high harmonic generation, we show that memory narrows and redshifts emission lines.