Pochettino's side have won just once in six league games under Pochettino after Ollie Watkins' second-half goal sealed the points for Villa at Stamford Bridge.
The Blues are languishing in 14th place, just four points above the relegation zone, after failing to score in their last three games.
Chelsea's latest flop was sparked by young defender Malo Gusto's dismissal after 58 minutes for a challenge on Lucas Digne.
After a pitchside VAR review, referee Jarred Gillett upgraded the initial yellow card to a red for dangerous play.
Adding to Pochettino's woes, Chelsea striker Nicolas Jackson, who has scored just once since his move from Villarreal, picked up his fifth booking for attempting to block a Villa free-kick, earning a one-game ban.
Pochettino refused to criticise the officials and instead said he and his players must take the blame for Chelsea's woes.
"It's our responsibility and the players' responsibility. We can't blame the VAR or the referee," Pochettino said.
"The situation, we need to act different, in a different way. I'm not going to blame or say anything against Malo Gusto.
"Situations happen in football and they affect the game and the team in a negative way.
"We need to grow up like a team, not only in an individual way. A player like Nico (Jackson) that is so young, feeling the Premier League and he's learning, he needs time."
Despite spending over £350 million on new signings since his arrival in July, Pochettino has been unable to improve a Chelsea side that struggled badly last term.
Chelsea's policy under co-owner Todd Boehly has been to sign younger players with high potential and sell-on value, but that has left Pochettino with a largely inexperienced line-up.
The former Paris Saint-Germain and Tottenham boss has also battled with a lengthy injury list.
Chelsea's raw team hasn't been up to the task so far and Pochettino is looking for a response as the pressure mounts.
"In this type of game, we're competing and we want to win. But players, when they are young, need to learn with experience," he said.
"That's why we feel disappointed because we are playing too many situations like this. Another small detail and in the end we are losing the game. We are in a situation we need to change as soon as possible."
