Veltman's hand-ball handed Iliman Ndiaye the chance to convert his matchwinning penalty just before halftime.
But the Dutchman said afterwards: "It was a long ball forward from them and I was sandwiched between two guys.
“I got a little push from the right hand side that moved my hand forward and the ball was there.
“When you fall, your automatic reaction is to try to get your hands forward on to the ground before you hit your head, it was more a reaction than a purpose handball.
"I would 100% have headed it or cleared it with my feet without the push, like a corner kick or something.”
Veltman also said: "I think the fans were booing because we were 1-0 behind and then they were unhappy with some of the referee's decisions. I understand their frustrations. At home against Everton we have to do better, we need to look in the mirror and not blame other things.
"Everton have those qualities to how do you say it, 'park the bus' and wait for a counter-attack and at times hope for some breaks.
“We did well second half but we could not make a 100% chance and that is what we have to improve.
"You need to not go 1-0 behind against these types of teams but, if you do, then you need to play better and create more - use the sides and cross more when we have Danny (Welbeck), Joao (Pedro) and Georginio (Rutter).”