Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

consumer: fix consumeNum to respect the consume timeout#1052

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Closed
katreniak wants to merge3 commits intoBlizzard:masterfromkatreniak:master

Conversation

@katreniak
Copy link

@katreniakkatreniak commentedNov 6, 2023
edited
Loading

Steps to reproduce the problem:

  • configure consumer withsetDefaultConsumeTimeout(1000)
  • produce 1 message every 500ms
  • callconsumer.consume(128, cb)

Actual outcome:

  • consume returns batch of 128 messages after 64 seconds

Expected outcome:

  • consumer returns batch of ~2 messages after 1 second

KafkaConsumerConsumeNum call underlaying c++m_consumer->Consume
in cycle until

  • either the accumulated batch is full
  • or the call to c++ m_consumer->Consume times out on the total timeout

KafkaConsumerConsumeNum must enforce its timeout
over allm_consumer->Consume invocations altogether.

One round poll to broker is needed to read the message.Second round poll to broker is needed to detect EOF.The test is asserting that we get message and EOF within 2s.But the consumer is configured with 'fetch.wait.max.ms': 1000.We don't fit 2 timout out polls within 2s.We need to increase the consume timeout to 3s.
One round poll to broker is needed to read the message.Second round poll to broker is needed to detect EOF.The test is asserting that we get EOF, data and EOFwhen data is produced at 2nd second.But the consumer is configured with 'fetch.wait.max.ms': 1000.We read the data in poll to broker slightly after 3rd second.We detect the 2nd EOF slightly after 4th second.We need to increase the consume timeout to 5s.
Steps to reproduce:* configure consumer with setDefaultConsumeTimeout(1000)* produce 1 message every 500ms* call consumer.consume(128, cb)Actual outcome:* consume returns 128 messages after 64 secondsExpected outcome:* consumer returns ~2 messages after 1 secondKafkaConsumerConsumeNum call underlaying c++ m_consumer->Consumein cycle until* either the accumulated batch is full* or the call to c++ m_consumer->Consume times out on the total timeoutKafkaConsumerConsumeNum must enforce its timeoutover all m_consumer->Consume invocations altogether.
@katreniak
Copy link
Author

Closing in favour of#1053

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@katreniak

[8]ページ先頭

©2009-2025 Movatter.jp