Trump has lost his core voters over the years