The Cottagers lost all three points despite taking the lead through Raul Jimenez and dominating possession for most of the game.
Silva was left disappointed by the result as he feels his team were dangerous enough to punish the Seagulls, particularly in the second-half.
“It was a difficult one to take,” he told the club website. “It’s very hard to lose a game like we did this afternoon.
“But that is the reality and that is football. We gave them those two moments.
“I really believe in the first half we were at a very good level. The second half we were not.
“In the first half we were able to settle into our game. They normally like to come and press man on man but the way we set up we were able to calm down the game, to set or game and to play with our identity and stick to our plan.
“It was a great offensive transition for us for the first goal.”
Silva added: “We had things completely under control for the first fourty minutes of the game. Us on the ball, no chances for Brighton. They were not able to find the gaps and not create what they wanted. We blocked them well.
“The reality is from a free kick we gave them a chance. The game was then balanced.
“Our second half was not at our best level. We gave the ball away many times and we lost momentum in the game.
“It’s clearly our fault the way the game went this afternoon. I will not complain about the penalty, a clever striker touched the ball and felt the contact there.”