Rayo won 2-1, with Celta having Marcos Alonso sent off in injury-time. Adri Embarba and Jorge de Frutos struck for Rayo, with Borja Iglesias scoring for the visitors.
"I can't speak. It's a Madrid referee who whistled for us today in Madrid. And from there I think he wasn't right. He tried to do the best he could. I think this kind of thing shouldn't be allowed. That's my opinion," said Giraldez.
"These are decisions that the referee has to make on the fly. I think we have to use the VAR, which is what it is there for, to review actions when there are doubts. In my opinion, there are two glaring penalties on Borja who was grabbed and shot in the area that were not seen.
"Vicente's play, apart from Vicente's action, there is a foul that he is going to call and he doesn't... well, situations that are a little difficult to understand and in my opinion it hasn't helped us and neither the result nor the refereeing has been fair. That's my opinion.
"It's very difficult to explain that we left empty-handed. I think it's difficult to be more compact and evolve as much during the match as we did today. I'm leaving angry because we didn't deserve not to win."