googleapis/google-cloud-pythonPublic

NotificationsYou must be signed in to change notification settings
Fork1.6k
Star5.2k

BigQuery: Use configurable bucket name for GCS samples data in systems tests.#8783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Merged

tswast merged 3 commits intogoogleapis:masterfromtswast:issue8550-bq-vpc-sc-tests

Jul 29, 2019

Merged

BigQuery: Use configurable bucket name for GCS samples data in systems tests.#8783

tswast merged 3 commits intogoogleapis:masterfromtswast:issue8550-bq-vpc-sc-tests

Jul 29, 2019

Conversation

Copy link

Contributor

tswast commentedJul 25, 2019

This allows the VPC-SC team to use their private mirror which is within
the security boundary when testing BigQuery VPC-SC support.

Closes#8550.

Use configurable bucket name for GCS samples data in systems tests.

cdc063b

This allows the VPC-SC team to use their private mirror which is withinthe security boundary when testing BigQuery VPC-SC support.

tswast added testing api: bigquery

Issues related to the BigQuery API.

type: processA process-related concern. May include testing, release, or the like. labels

Jul 25, 2019

tswast requested a review froma team

July 25, 2019 18:32

googlebot added the cla: yesThis human has signed the Contributor License Agreement. label

Jul 25, 2019

tswast requested a review fromshollyman

July 25, 2019 18:33

tseaver reviewed

Jul 25, 2019

View reviewed changes

bigquery/tests/system.py

		# The VPC-SC team maintains a mirror of the GCS bucket used for code
		# samples. The public bucket crosses the configured security boundary.
		# See: https://github.com/googleapis/google-cloud-python/issues/8550
		SAMPLES_BUCKET=os.environ.get("GCLOUD_TEST_SAMPLES_BUCKET","cloud-samples-data")

Copy link

Contributor

tseaverJul 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Do we need to updatenoxfile.py to set this value in the environment?

Copy link

ContributorAuthor

tswastJul 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

No, the default ofcloud-samples-data is fine for our purposes. The VPC-SC team has a separate Kokoro (or maybe something similar?) config where they'll set the necessary environment variable.

Copy link

ContributorAuthor

tswast commentedJul 25, 2019

___________________ test_bq_to_arrow_array_w_special_floats ____________________module_under_test = <module 'google.cloud.bigquery._pandas_helpers' from '/tmpfs/src/github/google-cloud-python/bigquery/google/cloud/bigquery/_pandas_helpers.py'>    @pytest.mark.skipif(pandas is None, reason="Requires `pandas`")    @pytest.mark.skipif(pyarrow is None, reason="Requires `pyarrow`")    def test_bq_to_arrow_array_w_special_floats(module_under_test):        bq_field = schema.SchemaField("field_name", "FLOAT64")        rows = [float("-inf"), float("nan"), float("inf"), None]        series = pandas.Series(rows, dtype="object")        arrow_array = module_under_test.bq_to_arrow_array(series, bq_field)        roundtrip = arrow_array.to_pylist()        assert len(rows) == len(roundtrip)        assert roundtrip[0] == float("-inf")>       assert roundtrip[1] != roundtrip[1]  # NaN doesn't equal itself.E       assert None != None

failure is unrelated. I think it's a behavior change / regression frompyarrow.