Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat: add logSender for sending logs on agent v2 API#12046

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
spikecurtis merged 4 commits intomainfromspike/10534-log-sender
Feb 15, 2024

Conversation

spikecurtis
Copy link
Contributor

@spikecurtisspikecurtis commentedFeb 7, 2024
edited
Loading

Adds a new subcomponent of the agent for queueing up logs until they can be sent over the Agent API.

Subsequent PR will change the agent to use this instead of the HTTP API for posting logs.

Relates to#10534

@spikecurtisGraphite App
Copy link
ContributorAuthor

spikecurtis commentedFeb 7, 2024
edited
Loading

@spikecurtisspikecurtis marked this pull request as ready for reviewFebruary 7, 2024 11:13
@spikecurtisspikecurtisforce-pushed thespike/10534-log-sender branch 2 times, most recently from25c4047 tod71e92eCompareFebruary 7, 2024 11:17
Base automatically changed fromspike/10534-report-stats tomainFebruary 7, 2024 11:26
// sendLoop sends any pending logs until it hits an error or the context is canceled. It does not
// retry as it is expected that a higher layer retries establishing connection to the agent API and
// calls sendLoop again.
func (l *logSender) sendLoop(ctx context.Context, dest logDest) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Please take into consideration what context is passed here when this code is used. On shutdown the context will likely be cancelled but we still want to try to send all the logs.

spikecurtis reacted with thumbs up emoji
@spikecurtisspikecurtis changed the base branch frommain tospike/log-limit-exceededFebruary 9, 2024 11:54
@spikecurtisspikecurtisforce-pushed thespike/log-limit-exceeded branch 2 times, most recently from04d7016 toed748d4CompareFebruary 9, 2024 12:05
Base automatically changed fromspike/log-limit-exceeded tomainFebruary 9, 2024 12:17
Copy link
Member

@mafredrimafredri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

A few remaining nits, but otherwise this is looking good! Nice work.

I wonder if it would make sense to move more of the logic toenqueue vssendLoop, though? Pre-processing allows us to avoid the temporary memory consumption for data we don't want.

q.logs[i] = nil
}
q.logs = q.logs[n:]
l.outputLen -= outputToRemove
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Do we actually need to decrement this?

I mean, it's probably fine, but in theory I supposed we could end up queueingmaxBytesQueued even after the database is near-full. Let's say we sentmaxBytesQueued-1, then the connection is interrupted, and we queue up anothermaxBytesQueued only to be discarded after connection is re-established.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I didn't want to reimplement the server side limit in the agent. So, if the server changes the limit the agents would still work with the new limit.

I get that we want to havesome agent side limit since the logs are stored in memory, and so the current server limit feels appropriate. But, as long as the agent is keeping pace relatively well sending logs to the server, I don't want to stop it sending until the server says we've reached the limit.

Copy link
Member

@mafredrimafredri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Just one minor nit, but other than that, looks good 👍

pl, err := agentsdk.ProtoFromLog(log)
if err != nil {
logger.Critical(context.Background(), "failed to convert log", slog.Error(err))
return
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Since we don't give an error, perhapscontinue is better here so we don't discard the entire batch?

@spikecurtisspikecurtis merged commit2aff014 intomainFeb 15, 2024
@spikecurtisspikecurtis deleted the spike/10534-log-sender branchFebruary 15, 2024 12:57
@spikecurtisGraphite App
Copy link
ContributorAuthor

Merge activity

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@mafredrimafredrimafredri approved these changes

Assignees

@spikecurtisspikecurtis

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

2 participants
@spikecurtis@mafredri

[8]ページ先頭

©2009-2025 Movatter.jp