Carlos Valdes and Tom Cavanagh are both leaving the Flash CW Series