Saints are still searching for a first win in the Premier League this season and face title holders Manchester City today.
Martin told the Daily Echo: "I just think if you look back at the... I think whenever clubs change I'm just not sure how productive it is always.
"Especially when you have a group of players that are trying so hard and are fighting for something and are giving everything.
"Maybe if the players are not giving everything they can it feels a bit different but the team is fighting for them, for the fans, for the club and the owners.
"We've drawn a line of sand trying to be the team we want to be in the Premier League - to not just try and hang on, to try and be the team we want to be.
"There would have been fans after the Stoke game and the end of the season that didn't think I was going to get the team promoted through the play-offs.
"Of course, things change quickly. My job is just to focus on what I can control which is how we train, how we work, the process we have and the rest honestly...
"I care about what the fans feel about their team and I want to win for them, they're the most important people at the club but I can't control it.
"They don't see the work we do, they don't see the care that goes into it so if they feel that way because of the results I understand.
"But ultimately, I can't let it really affect how I feel or how I work because then my life would be absolute chaos."