Spence admits his confidence hit rock bottom during Conte's time in charge.
"It wasn’t a nice feeling," Spence told the Rio Ferdinand Presents podcast. "Especially as I was coming to the club on a high, I was confident, I was buzzing, had just won promotion.
"Then it was like running into a brick wall. It shattered my confidence a bit, obviously I’m young. It’s not nice to hear.
"I knew it was rubbish at the time. It wasn’t a nice feeling. I was even going into the club like (puffs out cheeks) and, ‘What am I doing?’”
“I feel like whatever I did, that man wasn’t happy about anything.
"Even if you did the right things, you were like, ‘Did I do the right thing?’ Cos he’s not really a complimenting guy.
"It gets to a point where you don’t really need validation from a coach. You just go, ‘Oh’.
"I probably had one conversation with him."
Spence admits his belief needed building up once Conte had departed.
"I think I’ve been a victim of being too humble," he said. "Thinking things are just going to happen by not doing nothing and not saying nothing. But sometimes you have to express yourself.
"In my first year at Tottenham, I feel I gave a bit too much respect to the players and the surroundings, because it’s Tottenham. I came from Forest, I’m going to the Premier League, a top-six club, and I thought, 'Phew, this is big'.
"I feel I gave them too much respect in regards I didn’t really believe in myself."
Spence added: "I’d say this season (that's changed). I’ve got no time to waste now. I’ve got to go get it, it’s time."