The well-liked manager, who got the club up to the Premier League last season, has not been able to replicate that form.
As they struggle in the Championship, Luton have taken the decision to change things up.
CEO Gary Sweet said: “As with all decisions to part company with a manager, this has been a tough decision to arrive at, together with Rob, who has conducted himself with great professionalism and dignity throughout his two years at the Club, and again this week as we have genuinely come to a mutual conclusion.
“Rob has given us some of our greatest days to celebrate as Hatters. Those two barnstorming derby victories at Kenilworth Road, the play-off final at Wembley and our season in the Premier League will live with us forever and in doing so Rob has earned himself a legendary status.
“Results this season, especially away from home, have clearly not been acceptable and while we have supported Rob and his team in every possible way, we all felt that action was required to rectify this with 20 league matches still to play.
“It is typical of Rob’s humility that he also recognised this himself and leaves not just as a former manager, but as a trusted friend who has left his mark on every player and member of staff.
“Rob and his family will always be welcome visitors to Kenilworth Road. The door is always open. On behalf of the board, I would like to give him our heartfelt thanks for two years of tireless work and so many memories, wishing him every success for his future career.”