TFS Builds stacked in queue
I have experienced strange issue.
I have started build using check-in.
We know that this will add the build in queue and agent after then pick this up and start processing the build.
In my case this behavior did not happen, and multiple builds did queue up.
Conclusion was simple,
agents are not picking up the jobs(builds) from the queue.
The important is that everything else still worked and no-one complain about anything else.
I have looked on the agents and for good measure restarted them. This did not helped. and therefore the issue is elsewhere.
Next I have attempted to trigger another build manually and this has failed with error:
Please select build controller
When I went to select build controller I have noticed that on build definition it was de-selected with information that is loading.
Ok, issue bust be with build controller in some way.
As far I understand it, build controller controls build agents.
Now i have checked the controller in tfs administration console.
The controller was shutting down. I have no idea why as there were no logs for this I could find.
Simple click on restart and once the command has finished its operation, agents started to pick up the jobs and building.