Why would bees leave a swarm trap?

We finally had decent weather in my area this week after an unusually cold and rainy April. I had three swarm traps set and Sunday through Tuesday I had three swarms move in.

I planned on leaving them for a week before moving them into my apiary and an actual hive box. However, today (Wednesday) I went to watch the traps and two of them no longer had swarms, just a few bees that were left behind. I’m really sad I lost them. What kinds of things would make a swarm leave a trap? It’s been in the high 90s this week, would weather and poor ventilation from the small swarm trap opening play a role?