Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

[FLINK-38689] [flink-protobuf] Add support for protobuf 4.x field name conflict resolution#27245

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
khaledh wants to merge1 commit intoapache:master
base:master
Choose a base branch
Loading
fromkhaledh:protobuf-name-conflict-resolution

Conversation

@khaledh
Copy link
Contributor

What is the purpose of the change

Flink's protobuf format fails when processing messages compiled with protobuf-java's field naming conflict resolution. When proto schemas contain certain field patterns, protoc appends field number suffixes to accessor methods (e.g.,getStatus1(),getTags4List()), but Flink's codegen assumes canonical names (getStatus(),getTagsList()). Conflict scenarios:

  • Repeated field vs*_list/*_count fields -> numbered suffixes
  • Enum field +*_value field -> both get numbered suffixes (protobuf-java 4.30.0+)

This affects users with conflict-prone schemas on protobuf-java 3.x+ (repeated field conflicts) and 4.30.0+ (enum field conflicts).

Brief change log

  • AddedPbFieldConflictResolver with caching for conflict-aware accessor resolution
  • Refactored deserializer/serializer codegen to use conflict-aware accessors

Verifying this change

This change added tests and can be verified as follows:

  • Addedtest_field_conflict.proto with field conflict scenarios
  • Generated Java classes with protoc 4.32.1 to verify accessor name matching
  • AddedFieldConflictProtoToRowTest (14 tests) andFieldConflictRowToProtoTest (4 tests)
  • All tests pass with protobuf-java 4.32.1

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency):no
  • The public API, i.e., is any changed class annotated with@Public(Evolving):no
  • The serializers:no (the changes only affect which accessor methods are called during codegen, not the serialization format or behavior)
  • The runtime per-record code paths (performance sensitive):no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper:no
  • The S3 file system connector:no

Documentation

  • Does this pull request introduce a new feature?no
  • If yes, how is the feature documented?not applicable

@flinkbot
Copy link
Collaborator

flinkbot commentedNov 17, 2025
edited
Loading

CI report:

Bot commands The@flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@khaledh@flinkbot

[8]ページ先頭

©2009-2025 Movatter.jp