Joselu left Real after helping coach Carlo Ancelotti's team win the Champions League and LaLiga titles.
He told La Sexta: "Saying goodbye to Madridwas one of the most difficult moments of my life. I couldn't make that decision with my heart because I would have stayed at Madrid for life, it gives me goosebumps just thinking about it.
"I thought about my family, it's my children and I think it's positive in the long run, but at the time it was negative."
He also said: "When I decided to leave, I was unable to speak to my wife. Lucas Vázquez called me and I was unable to speak to him, just like with (Luka) Modric, I couldn't find the words.
"Telling Ancelotti was difficult, he was upset because he wanted me to stay."
Joselu added, "I can die in peace because I have played and won titles with Real Madrid, the club of my life."