This is an incremental step to test out running exttests.
But hopefully soon want we do is go multi-pod/multi-stage
and split off exttests separately - we may not always
want to gate on them, etc.

ci: Use the exttests container

56c4909

This is an incremental step to test out running exttests.But hopefully soon want we do is go multi-pod/multi-stageand split off exttests separately - we may not alwayswant to gate on them, etc.

cgwalters mentioned this pull request

Oct 9, 2020

exttests: New container image with upstream testscoreos/coreos-assembler#1745

Merged

Copy link

MemberAuthor

cgwalters commentedOct 9, 2020

From Jenkins logs looks like it worked!

--- PASS: ext.ostree.destructive-rs.transactionality (397.98s)

Copy link

MemberAuthor

cgwalters commentedOct 9, 2020

To recap with this we have a flow where:

Change to tests (or new test for new feature) in upstream ostree/ignition/etc git repo is submitted
Change is tested in CI on that repo
Change merges
(Later, or maybe sooner) Submit PR to bump the tag onhttps://github.com/coreos/coreos-assembler/blob/master/exttests/build.sh (could be automated)
(hmm, that change should be covered by CI but isn't today)
New exttests container builds
New test is run on our main builds too

Or in other words, we run upstream tests on both upstream builds and our pipeline without duplicating the tests.

jlebon reviewed

Oct 13, 2020

View reviewed changes

.cci.jenkinsfile

		cosaPod {
		// Use the exttests container here in CI to get extra coverage; xref
		// https://github.com/coreos/coreos-assembler/pull/1745
		cosaPod(image: "registry.svc.ci.openshift.org/coreos/cosa-exttests:latest") {

Copy link

Member

jlebonOct 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I have a strong preference for having CI keep using the canonical cosa image from quay.io for all the main steps. It's what devs use locally. We've also had multiple encounters with OpenShift CI having stale versions of cosa or rebuilds of layered images like this not triggering. It's a major pain to debug.

How about instead we add sugar to coreos-ci-lib to have the exttest container be added to the pod and then for the kola run, it knows to use the exttest container just for the installed tests? (I.e. usingcontainer("exttests") { ... }).

Hmm, or maybe it is indeed easier to just fold this into cosa for now until we do something fancier as discussed incoreos/coreos-assembler#1745.