Olsen kept a clean sheet as Villa won 1-0 thanks to Ollie Watkins' second-half goal.
The Swede has stepped in for injured No1 Emi Martinez and told Villa TV: "I’m so happy.
“I’m so proud of the boys and the team, we knew that it’s a really tough away game.
“We knew that we had to play at our best, so we did everything we spoke about before the game.
“I’m just so pleased and proud of the team.”
Olsen added: “It’s never easy to have a one-nil lead, I knew they would push us a little bit more.
“We were struggling a little bit but we defended our box brilliantly.
“We deserved this clean sheet and those three points.”
The Swede was celebrated by teammates and the away support after the final whistle.
Olsen admits he was left lost for words by the gesture. He said: “I don’t know what to say, I’m not used to that.
“Of course, it’s always nice to celebrate with the fans who have travelled down here and supported us for 90 minutes.
“It’s just nice to celebrate with them.”