Saints were beaten 2-1 at St Mary's, with Silva accusing Rusk's players of not trying to score.
Rusk said afterwards: "Yeah, you get in front and work so hard to get in front, and get to the stage of the game, 73 minutes, the first goal against really top opposition.
"When you look at the investment into the squad they've got and they've the players, there's a reason they're challenging for European competition.
"To see the game slip away from you like that is really difficult, quite painful, but I still think there's a lot that we can take from that moving forward."
On Silva's claim, Rusk continued: "He's a great manager with lots of experience, isn't he? And I think, I take that with a pinch of salt at the end of the day. However you get in front, we got in front and we deserve to be in front and we had other moments in the game.
"I'm pretty sure that if he was in our situation he'd be finding the best possible way for his players to try and win a game of football. I'm not too concerned about other people's assessment of us, what matters is our assessment of us."
Stephens goal was special
On Jack Stephens' goal, Rusk added: "It is still a special goal, any goal in the Premier League is special and he's been a fantastic captain since I've inherited the interim role.
"It was a nice moment for him. I know, of course, our overriding feeling is not one of joy around the good moments in the game for us here.
"We're hurt and we're disappointed with the way the game has ended but I'm sure as the week goes on we'll be able to reflect on some things that were good in there and that will be one of them."