Our docker service was a bit slowly loading...
$ docker ps -aq | wc -l
Spinning up a container seems to be much quicker than removing an old container. I started to delete these containers this morning. After 7 hours there are still around 45000 containers left.