The centre forward came on in the second half, with four minutes of regular time to go, when England needed a goal to get back in the game.
After they turned around the contest and won 2-1 in extra time, Southgate admits Toney may have changed his mind about that change.
“I had belief we would get a goal. The boys that came on, Ivan (Toney) played a big part – he was disgusted at me putting him on at that moment. There's no question he was pretty disgusted when I put him on with a minute to go," the England boss said.
"I think we've made up now. He's had a big impact in the second goal as well. You put a sub on at that time, it is a final throw of the dice. I completely understand it. I don't like putting a player in that position.
"But I had the feel that he'd be able to cause the chaos that happened. I have to say the courage shown by the players was fantastic tonight. We haven't come to get to a quarter-final.
"But to get through a night like tonight was fantastic character. We now play a Swiss team that have been very good. We have a couple of days to recover and get ready for them. We had enough attacking players on the pitch."