We might be losing here output fromcoderd, which can be really useful during backend debugging. In the legacy version, it is logged like this:

[stderr] [unknown]: [WebServer] 2023-10-17 06:54:03.412 [info]  provisionerd.runner: apply successful  job_id=89020454-1bd8-4eed-a0c9-eb12fa532a92  template_name=68ac3052  template_version=hardcore_moser9  workspace_build_id=465c8cad-3fcb-42fb-b9bb-56007b4a1db3  workspace_id=74006ba8-82a8-4693-b971-f3da61ad02e0  workspace_name=d93260bd  workspace_owner=admin  workspace_transition=start  resource_count=1  resources="[name:\"example\" type:\"echo\"]"  state_len=0

It looks like the new reporter simply skips these log records.

that was sort of intentional. it adds a lot of noise when trying to debug a frontend failure, and since it's not tied to a specific test case it's hard to buffer and display later in a way that makes sense.

I also added some logic forpreserveOutput, but I just read the docs and it turns out it does something entirely different from what I expected, and can't even be set from the command line like I hoped 😵‍💫

the idea was supposed to be that backend peeps could set--preserveOutput=always when they ran the tests... I'd really like to find a compromise here to keep the output clean when possible. it's just always such a pain to find theactual error in all of the mountains of logs.

example run, which now has 41 occurrences of the word "error" but succeededhttps://github.com/coder/coder/actions/runs/6549912328/job/17787879014?pr=10304

A good compromise would be only filtering outaudit_log entries, which do not bring a lot of value. 👍

I would leave coderd output as it is helpful with debugging the request flow or provisioner behavior, especially while dealing with flaky issues.

if we can also ignore lines likeecho: recv done on Session session_id= error=EOF then I'll be happy. :)

Agree 👍