We have to be careful about sending logs with elevated level. Provisionerd logs them directly at the level we get from the provisioner, which then will end up in operators log streams. That's especially problematic if provisionerd is embedded in Coderd.

I get that, arguably, if there is drift while you are claiming a prebuild then you are doing something wrong. But that isn't true of builds generally, and it's a bad idea for us to emit a bunch of WARN logs to core Coder operators based on stuff happening down in templates and builds, as template authorship might be delegated out to teams that are clients of the Coder platform.

The quickest way forward is to just not emit at WARN for this kind of stuff.

A more flexible and general solution would be to introduce some sort of log scope, to distinguish warnings and errors with builds and templates from warnings and errors with the provisioner. Only the latter should be emitted at WARN/ERROR level into the provisioner's log stream. The former can be emitted into thebuild logs at the given level so we display them appropriately in the UI.