Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

docs: Add Spark 4.0, StructType, and VariantType support to connector…#4798

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
ShimonSte wants to merge3 commits intomain
base:main
Choose a base branch
Loading
fromspark-4.0-struct-variant-support

Conversation

@ShimonSte
Copy link

Add Spark 4.0, StructType, and VariantType support documentation

This PR updates the Spark Native Connector documentation to reflect new features introduced in version 0.9.0:

Changes

  • Spark 4.0 Support: Added Spark 4.0 to compatibility matrix with Java 17+ requirement and Scala 2.13 only
  • StructType (Tuple) Mapping: Documented bidirectional support for ClickHouse Tuple and Spark StructType
  • VariantType (JSON) Support: Added Spark 4.0+ VariantType support for ClickHouse JSON type
  • Boolean Type Update: Changed BooleanType mapping fromUInt8 toBool (version 0.9.0+)
  • Version Updates: Updated ClickHouse JDBC version to 0.9.4

Related Commits

… documentation- Add Spark 4.0 support with Java 17+ requirement and Scala 2.13 only- Update compatibility matrix to include version 0.9.0 with Spark 4.0- Add StructType (Tuple) bidirectional mapping support  - Support for named and unnamed tuples  - Nested structs and nullable fields- Add VariantType (JSON) support for Spark 4.0+  - New spark.clickhouse.read.jsonAs configuration option  - Default to VariantType, optional StringType mode- Update Boolean type mapping from UInt8 to Bool (version 0.9.0)- Update ClickHouse JDBC version to 0.9.4 for main and 0.9.0
@vercel
Copy link

vercelbot commentedNov 26, 2025•
edited
Loading

The latest updates on your projects. Learn more aboutVercel for GitHub.

ProjectDeploymentPreviewCommentsUpdated (UTC)
clickhouse-docsReadyReadyPreviewCommentNov 26, 2025 4:14pm
3 Skipped Deployments
ProjectDeploymentPreviewCommentsUpdated (UTC)
clickhouse-docs-jpIgnoredIgnoredNov 26, 2025 4:14pm
clickhouse-docs-ruIgnoredIgnoredPreviewNov 26, 2025 4:14pm
clickhouse-docs-zhIgnoredIgnoredPreviewNov 26, 2025 4:14pm

VariantType (JSON) requires JSON write format. Arrow format is not supported for this data type.
VariantType currently only supports objects, not primitives. Changed status to partial support (⚠️) to reflect this limitation.
Copy link
Contributor

@BentsiLeviavBentsiLeviav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

PLEASE DON'T MERGE THIS.

The content here is pending release inhttps://github.com/ClickHouse/spark-clickhouse-connector/releases

@BlargianBlargian added the Don't MergeDon't merge yet labelNov 27, 2025
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@BentsiLeviavBentsiLeviavBentsiLeviav approved these changes

@mzitnikmzitnikAwaiting requested review from mzitnik

Assignees

No one assigned

Labels

Don't MergeDon't merge yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ShimonSte@BentsiLeviav@Blargian
[8]ページ先頭
©2009-2025 Movatter.jp