Chelsea and Colwill next meet Ipswich Town after the Cottagers setback.
"We have to keep moving," he said. "December has been a tough month, with a lot of games, and we have another game in three days, so we have to keep going. We’re not where everyone expects to be, realistically. We still have a lot to improve on.
"We have to keep pushing. We don’t have time to let our heads drop. We have to keep moving and keep improving together."
Colwill added, "It’s going to hurt the momentum, but we can build it back up.
"We weren’t lying when we said we weren’t in the race. This is a realisation we’re not there yet. We will just try to keep picking up three points.
"We’re a very young team with a lot of confidence. We will be there again, don’t worry. We will keep going for the next game and get three points there."