"Worst team in the league," Smith said, shaking his head