Add notation replication test case (#19738)

Fix #19548

Signed-off-by: Yang Jiao <yang.jiao@broadcom.com>
Co-authored-by: Yang Jiao <yang.jiao@broadcom.com>
This commit is contained in:
Yang Jiao 2023-12-20 15:51:01 +08:00 committed by GitHub
parent 9a084ec4dd
commit 38d3b3512b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -591,7 +591,6 @@ Test Case - Enable Replication Of Cosign Deployment Security Policy
Check Latest Replication Job Status Succeeded
# check project_pull_dest
Go Into Project project_pull_dest${d}
Switch To Project Repo
Repo Exist project_pull_dest${d} ${image1}
Repo Exist project_pull_dest${d} ${image2}
Repo Exist project_pull_dest${d} ${index}
@ -607,7 +606,6 @@ Test Case - Enable Replication Of Cosign Deployment Security Policy
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed ${image2_short_sha256}
# check project_push_dest
Go Into Project project_push_dest${d}
Switch To Project Repo
Repo Exist project_push_dest${d} ${image1}
Repo Exist project_push_dest${d} ${image2}
Repo Exist project_push_dest${d} ${index}
@ -623,6 +621,204 @@ Test Case - Enable Replication Of Cosign Deployment Security Policy
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed ${image2_short_sha256}
Close Browser
Test Case - Enable Replication Of Notation Deployment Security Policy
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
${image1}= Set Variable hello-world
${tag1}= Set Variable latest
${image2}= Set Variable busybox
${tag2}= Set Variable latest
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Project And Go Into Project project_push_dest${d}
Create An New Project And Go Into Project project_pull_dest${d}
Switch To Registries
Create A New Endpoint harbor e${d} https://${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Create A Rule With Existing Endpoint rule_pull_${d} pull project${d}/* image e${d} project_pull_dest${d}
Logout Harbor
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Project And Go Into Project project${d}
# push images
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} ${image1}:${tag1}
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} ${image2}:${tag2}
# enable notation deployment security policy
Goto Project Config
Click Notation Deployment Security
Save Project Config
Content Notation Deployment security Be Selected
# push mode replication should fail
Switch To Registries
Create A New Endpoint harbor e${d} https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Create A Rule With Existing Endpoint rule_push_${d} push project${d}/* image e${d} project_push_dest${d}
Select Rule And Replicate rule_push_${d}
Check Latest Replication Job Status Failed
# pull mode replication should fail
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Select Rule And Replicate rule_pull_${d}
Check Latest Replication Job Status Failed
# sign
Docker Login ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Notation Generate Cert
Notation Sign ${ip}/project${d}/${image1}:${tag1}
Notation Sign ${ip}/project${d}/${image2}:${tag2}
Docker Logout ${ip}
# push mode replication should success
Logout Harbor
Sign In Harbor https://${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Select Rule And Replicate rule_push_${d}
Check Latest Replication Job Status Succeeded
# pull mode replication should success
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Select Rule And Replicate rule_pull_${d}
Check Latest Replication Job Status Succeeded
# check project_pull_dest
Go Into Project project_pull_dest${d}
Repo Exist project_pull_dest${d} ${image1}
Repo Exist project_pull_dest${d} ${image2}
# check project_push_dest
Go Into Project project_push_dest${d}
Repo Exist project_push_dest${d} ${image1}
Repo Exist project_push_dest${d} ${image2}
Close Browser
Test Case - Enable Replication Of Cosign And Notation Deployment Security Policy
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
${image1}= Set Variable hello-world
${tag1}= Set Variable latest
${image2}= Set Variable busybox
${tag2}= Set Variable latest
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Project And Go Into Project project_push_dest${d}
Create An New Project And Go Into Project project_pull_dest${d}
Switch To Registries
Create A New Endpoint harbor e${d} https://${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Create A Rule With Existing Endpoint rule_pull_${d} pull project${d}/* image e${d} project_pull_dest${d}
Logout Harbor
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Project And Go Into Project project${d}
# push images
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} ${image1}:${tag1}
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} ${image2}:${tag2}
# enable cosign deployment security policy
Goto Project Config
Click Cosign Deployment Security
Save Project Config
Content Cosign Deployment security Be Selected
# enable notation deployment security policy
Goto Project Config
Click Notation Deployment Security
Save Project Config
Content Notation Deployment security Be Selected
# push mode replication should fail
Switch To Registries
Create A New Endpoint harbor e${d} https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Create A Rule With Existing Endpoint rule_push_${d} push project${d}/* image e${d} project_push_dest${d}
Select Rule And Replicate rule_push_${d}
Check Latest Replication Job Status Failed
# pull mode replication should fail
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Select Rule And Replicate rule_pull_${d}
Check Latest Replication Job Status Failed
# cosign sign
Cosign Generate Key Pair
Docker Login ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Cosign Sign ${ip}/project${d}/${image1}:${tag1}
Cosign Sign ${ip}/project${d}/${image2}:${tag2}
Docker Logout ${ip}
# push mode replication should fail
Switch To Replication Manage
Select Rule And Replicate rule_push_${d}
Check Latest Replication Job Status Failed
# pull mode replication should fail
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Select Rule And Replicate rule_pull_${d}
Check Latest Replication Job Status Failed
# notation sign
Docker Login ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Notation Generate Cert
Notation Sign ${ip}/project${d}/${image1}:${tag1}
Notation Sign ${ip}/project${d}/${image2}:${tag2}
Docker Logout ${ip}
# delete cosign accessory
Logout Harbor
Sign In Harbor https://${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Go Into Project project${d}
Go Into Repo project${d} ${image1}
Retry Double Keywords When Error Retry Button Click ${artifact_list_accessory_btn} Delete Accessory By Aeecssory XPath ${artifact_cosign_accessory_action_btn}
Retry Double Keywords When Error Retry Button Click ${artifact_list_accessory_btn} Retry Wait Element Not Visible ${artifact_cosign_accessory_action_btn}
Go Into Repo project${d} ${image2}
Retry Double Keywords When Error Retry Button Click ${artifact_list_accessory_btn} Delete Accessory By Aeecssory XPath ${artifact_cosign_accessory_action_btn}
Retry Double Keywords When Error Retry Button Click ${artifact_list_accessory_btn} Retry Wait Element Not Visible ${artifact_cosign_accessory_action_btn}
# push mode replication should fail
Switch To Replication Manage
Select Rule And Replicate rule_push_${d}
Check Latest Replication Job Status Failed
# pull mode replication should fail
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Select Rule And Replicate rule_pull_${d}
Check Latest Replication Job Status Failed
# cosign sign
Docker Login ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Cosign Sign ${ip}/project${d}/${image1}:${tag1}
Cosign Sign ${ip}/project${d}/${image2}:${tag2}
Docker Logout ${ip}
# push mode replication should success
Logout Harbor
Sign In Harbor https://${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Select Rule And Replicate rule_push_${d}
Check Latest Replication Job Status Succeeded
# pull mode replication should success
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Select Rule And Replicate rule_pull_${d}
Check Latest Replication Job Status Succeeded
# check project_pull_dest
Go Into Project project_pull_dest${d}
Repo Exist project_pull_dest${d} ${image1}
Repo Exist project_pull_dest${d} ${image2}
Go Into Repo project_pull_dest${d} ${image1}
Retry Button Click ${artifact_list_accessory_btn}
Should Be Signed By Cosign ${tag1}
Should Be Signed By Notation ${tag1}
Go Into Repo project_pull_dest${d} ${image2}
Retry Button Click ${artifact_list_accessory_btn}
Should Be Signed By Cosign ${tag2}
Should Be Signed By Notation ${tag2}
# check project_push_dest
Go Into Project project_push_dest${d}
Repo Exist project_push_dest${d} ${image1}
Repo Exist project_push_dest${d} ${image2}
Go Into Repo project_push_dest${d} ${image1}
Retry Button Click ${artifact_list_accessory_btn}
Should Be Signed By Cosign ${tag1}
Should Be Signed By Notation ${tag1}
Go Into Repo project_push_dest${d} ${image2}
Retry Button Click ${artifact_list_accessory_btn}
Should Be Signed By Cosign ${tag2}
Should Be Signed By Notation ${tag2}
Close Browser
Test Case - Carvel Imgpkg Copy To Harbor
[Tags] imgpkg_copy
Init Chrome Driver