Sub-cycle electron control in the photoionization of xenon using a few-cycle laser pulse in the mid-infrared