Ryan Flamingo and Malik Tillman scored for PSV before a second booking for Martinez 10 minutes after halftime left Girona facing an uphill battle.
Their hosts scored twice more via Johan Bakayoko and a Ladislav Krejci own goal.
"The first half of both teams was very good in terms of the pace of play, looking for the opponent's goal," Michel told reporters after the match.
"The expulsion was decisive. From there, PSV had chances and deserved to win. In this competition, the small details make a difference."
Girona's Dutch defender Daley Blind, a former stalwart at PSV’s archrivals Ajax, admitted his Spanish side were second best but echoed his coach’s view that the red card was decisive.
"The first chance was for us, but we conceded a goal from a throw-in, which cannot happen," Blind said.
"After that, we tried to play. It was an open game. We had a lot of movement when we had possession, but overall PSV had the upper hand.
"We missed a few players through injury. It's a difficult situation. But I don't want to give that as an excuse. We need to do it with the players we have.
"We fought hard. We came out of the dressing room with good spirit. But after the red card, the game was over."
Girona have three points from four games and next travel to Sturm Graz on November 27th, after which they have home fixtures against Liverpool and Arsenal, and an away trip to AC Milan in an incredibly difficult finish to their group campaign.