Villa started well in South London, but the hosts took the lead just before the half-hour mark.
Emery acknowledged his team struggled to cope in the final stages, despite Morgan Rogers’ second-half equalizer.
Post-game, he said: “We played the first 30 minutes, controlling of the game and more or less dominating. We were creating chances, and we were not conceding the chances of them.
“The first corner we conceded, the first chance they had, they scored. It was tough for us. Then, progressively, we were getting worse.
“We drew (level in) the second half, and in that moment we (thought) maybe it is the moment we can again react, dominating the match again, keeping ball possession better and trying to stop them.
“But they scored the second goal, the third goal, and it was progressively, of course, more difficult for us. We were not dominating a lot of duels as well with them.
“It’s tough to lose here. We have to accept it. We are frustrated and we have to try to move on.”