We have applied mitigation steps in our backend system to re-enable the jobs for all the Hosts that have been active in last 24Hrs Build Queue times are current and back to normal. We are working a hotfix to fix the underlying problem. We will keep you posted on the progress of this issue and we apologize for the inconvenience caused.
------------------------------
Update-Hosted Build platform upgrade deployment on 7/11 had a bug where Jobs remaining in dormant state even after check-in causing impact to both scheduled Hosted Builds and scheduled On-prem builds. We have identified the issue and the hotfix is under testing and it will be rolled out to Production ASAP and in the mean time we have implemented a mitigation process to address this issue and scheduled builds should start working again.
The mitigation step opened up the Queue and we are seeing Queue violations right now and it should be catching up soon.
We have high percent of scheduled Hosted Builds and scheduled On-prem builds impacted with this incident, We will keep you posted on the progress of this issue and we apologize for the inconvenience caused.
-------------------------------
Initial update - Thu, Jul 18 2013 2:43 PM -UTC
We are currently investigating an issue with scheduled builds on Hosted Build service where scheduled builds are not working since our last Hosted Build platform upgrade. We will keep you posted on the progress of this issue and we apologize for the inconvenience caused.