lom-pipeline-kev ua tiav-II

Kev Deep Dive rau hauv CI/CD Pipelines Qhov Tsis Zoo (II): Tshuaj Lom Tsis Ncaj Qha Pipeline Kev Ua Haujlwm (I-PPE)

Hauv peb cov ntawv tshaj tawm dhau los, peb tau pom yuav ua li cas nrhiav thiab tiv thaiv kev lom los ntawm Direct Poisoning Pipeline Kev Ua Haujlwm (D-PPE). Peb kuj tau pom yuav ua li cas nrhiav tau qhov tsis muaj zog ntawd siv Xygeni Scanner, nrog rau qee cov txheej txheem tiv thaiv. 

 Lom Pipeline Kev Ua Haujlwm (PPE) yog tsim tawm thaum tus neeg tawm tsam tuaj yeem hloov kho qhov pipeline logic nyob rau hauv ib qho ntawm ob txoj kev:

  • Los ntawm kev hloov kho cov ntaub ntawv teeb tsa CI (tus pipeline) -> Cov Khoom Siv Tiv Thaiv Kab Mob Ncaj Qha (D-PPE)
  • Los ntawm kev hloov kho cov ntaub ntawv uas tau hais los ntawm pipeline (piv txwv li: cov ntawv sau uas tau hais los ntawm sab hauv pipeline cov ntaub ntawv teeb tsa) -> Cov Khoom Siv Tiv Thaiv Kab Mob Tsis Ncaj Qha (I-PPE)
pp2

Hauv tsab ntawv no, peb yuav kawm tob txog Indirect PPE. Tab sis, ua ntej ntawd, thiab ua ib qho kev ntxiv rau kuv tsab ntawv dhau los, cia peb saib ua ntej seb GitHub tswj hwm kev ua haujlwm ntawm li cas pipelines thiab cov txheej txheem tiv thaiv tiv thaiv D-PPE yog dab tsi.

GitHub tiv thaiv kev ua haujlwm li cas pipelines los ntawm PRs?

GitHub ua haujlwm li cas txog kev ua tiav ntawm kev hloov kho pipelines?

hloov pipelines tuaj yeem los ntawm Pushes lossis Pull Requests (PR). Ua ib qho kev coj ua zoo tshaj plaws, nws raug pom zoo kom tsis txhob muaj kev "thawb" ncaj qha mus rau ceg uas muaj kev tiv thaiv thiab siv Pull Requests ua ib txoj hauv kev los ua kom muaj kev tshuaj xyuas ua ntej lees txais cov lej uas tau muab los. 

Pull Requests tej zaum yuav los ntawm ob qhov chaw sib txawv:

  • Cov PRs los ntawm rab phom
  • Cov PRs los ntawm ceg ntoo

Cov PR los ntawm rab phom tuaj yeem los ntawm pej xeem or private lub tsev khaws ntaub ntawv.

Thaum peb tab tom cuam tshuam nrog PPE (Luam tawm) Pipeline Kev Ua Haujlwm), peb lub ntsiab lus tseem ceeb tsis yog "kev lees txais" ntawm PR tab sis kev ua tiav ntawm kev hloov kho pipeline thaum lub sijhawm PR lees txais/pom zoo. Hauv plawv ntawm kev tawm tsam PPE, muaj kev ua tsis tau xav tseg ntawm "kev phem" hloov kho pipeline. 

Hauv ob peb lo lus, Lom zem Pipeline Kev ua tiav (PPE) yog tsim tawm thaum tus neeg tawm tsam tuaj yeem hloov kho qhov pipeline logic.

Muaj ob txawv:

  • Cov Khoom Siv Tiv Thaiv Kab Mob Ncaj Qha (D-PPE): Hauv qhov xwm txheej D-PPE, tus neeg tawm tsam hloov kho cov ntaub ntawv teeb tsa CI hauv ib lub chaw cia khoom uas lawv muaj kev nkag mus rau, los ntawm kev thawb qhov kev hloov pauv ncaj qha mus rau ib ceg chaw nyob deb uas tsis muaj kev tiv thaiv ntawm lub repo, lossis los ntawm kev xa PR nrog qhov kev hloov pauv los ntawm ceg lossis rab rawg. Vim yog CI pipeline kev ua tiav yog txhais los ntawm cov lus txib hauv cov ntaub ntawv teeb tsa CI hloov kho, tus neeg tawm tsam cov lus txib phem thaum kawg khiav hauv lub node tsim thaum lub tsev tsim pipeline yog ua rau muaj teeb meem.
  • Cov Khoom Siv Tiv Thaiv Kab Mob Tsis Ncaj Qha (I-PPE): Hauv qee kis, qhov ua tau ntawm D-PPE tsis muaj rau tus yeeb ncuab uas nkag mus rau SCM lub chaw khaws ntaub ntawv (piv txwv li yog tias pipeline tau teeb tsa los rub cov ntaub ntawv teeb tsa CI los ntawm ib ceg sib cais, tiv thaiv hauv tib lub chaw khaws ntaub ntawv). Hauv qhov xwm txheej zoo li no, es tsis txhob lom cov tshuaj lom pipeline nws tus kheej, tus neeg tawm tsam txhaj cov lej phem rau hauv cov ntaub ntawv uas tau hais txog pipeline (piv txwv li: cov ntawv sau uas tau hais los ntawm sab hauv pipeline cov ntaub ntawv teeb tsa)

Hauv ob qho xwm txheej, GitHub yuav ua qhov kev hloov kho tshiab pipeline tsis tas yuav tsum tau kev tshuaj xyuas lossis kev pom zoo ua ntej.

PRs los ntawm cov forks rau pej xeem so

GitHub tso cai rau kev teeb tsa tus cwj pwm thaum ua tiav PRs los ntawm cov forks hauv pej xeem repos.

Thaum PR los ntawm ib rab diav rawg, GitHub yeej ib txwm yuam qee qib ntawm "kev pom zoo" ua ntej ua tiav qhov pipeline cuam ​​tshuam nrog PRQib kev pom zoo no pauv ntawm qib tsis muaj zog mus rau qib kev pom zoo nruj.

At Qib koom haum (Org>>Settings>>Actions>>General), koj tuaj yeem txiav txim siab ntawm ntau txoj kev xaiv "kev pom zoo":

ppe3

Qhov nruj tshaj plaws yog qhov kawg ("Yuav tsum tau kev pom zoo los ntawm txhua tus neeg koom tes sab nraud") vim tias GitHub yuav ib txwm xav tau kev pom zoo thaum PR los ntawm cov forks los ntawm cov neeg koom tes sab nraud. 

Tab sis txawm tias nyob rau hauv rooj plaub nruj no, muaj qhov sib txawv ntawm cov neeg koom tes nrog kev tso cai nyeem thiab sau.

  • Thaum PR los ntawm ib qho nyeem tus neeg siv, tus kev ua tiav ntawm pipeline raug nres kom txog thaum muaj kev pom zoo rau kev hloov pauv. Yog tias kev pom zoo zoo, ces qhov kev hloov kho pipeline raug tua pov tseg. 
  • Thaum PR los ntawm ib qho sau ntawv tus neeg siv, tus tsis tas yuav tsum tau kev pom zoo thiab qhov hloov kho pipeline yog ib txwm raug tua pov tseg !! 
pp4

Xaus lus, PRs los ntawm cov forks ntawm cov chaw khaws ntaub ntawv pej xeem raug tiv thaiv me ntsis los ntawm PPE. Muaj qee qhov kev tiv thaiv tawm tsam cov neeg siv sab nraud (nyeem), tab sis tsis muaj dab tsi cuam tshuam rau cov neeg siv sab hauv (sau).

Yuav ua li cas txog PRs los ntawm cov forks los ntawm cov chaw khaws ntaub ntawv ntiag tug?

PRs los ntawm cov forks rau private so

Hauv qhov xwm txheej no, GitHub muab qee qhov chaw teeb tsa muaj txiaj ntsig.

ppe9

Cov chaw teeb tsa saum toj no tuaj yeem teeb tsa ntawm hloov khoom nruab nrog Lossis ntawm Repo theem.

thaum tsis muaj kev xaiv twg raug kuaj xyuas, GitHub yuav thov kom tau kev pom zoo thiab nws yuav tsis ua qhov kev hloov pauv uas tau hloov kho pipelineQhov no yog qhov kev teeb tsa zoo tshaj plaws!!

cov kev teeb tsa tsis muaj kev nyab xeeb tshaj plaws yog thaum twg "Ua cov haujlwm los ntawm cov cuab yeej pull request"tau kuaj xyuasHauv qhov no, tib yam rau ob tus neeg nyeem thiab sau ntawv, Github yuav ua qhov hloov kho pipeline!! Thiab qhov xwm txheej no tuaj yeem ua tau txawm tias zuj zus yog tias "Xa cov cim sau ntawv mus rau cov workflows los ntawm fork pull requests"Thiab"Xa cov lus zais thiab cov hloov pauv mus rau cov haujlwm ua haujlwm los ntawm rab rawg pull requests"Tau kuaj xyuas lawm. Tsis txhob ua qhov no tshwj tsis yog tias muaj laj thawj tseeb!!

Yog hais tias “Yuav tsum tau kev pom zoo rau rab rawg pull request workflows"yog kuaj xyuas, qhov xwm txheej saum toj no tau txhim kho me ntsis: GitHub yuav thov kev pom zoo thiab tsis ua qhov hloov kho pipeline rau tus neeg siv nyeem, tab sis nws tseem yuav ua nws rau tus neeg siv sau.

ppe6

Cov diav rawg pom, ua li cas txog Cov PRs tuaj ntawm cov ceg ntoo?

Cov PR los ntawm ceg ntoo

Yuav kom tiv thaiv tau qhov xwm txheej no koj yuav tsum vam khom Cov Cai Tiv Thaiv Ceg. 

Ntawm theem repo, koj tuaj yeem tsim cov cai tiv thaiv ceg rau txhua ceg. Cov cai no ntxiv qee qhov kev txwv rau kev hloov kho ntawm cov ceg tiv thaiv.

Txawm hais tias koj teeb tsa txoj cai rau "Xav tau ib pull request ua ntej sib koom ua ke"Thiab"Yuav tsum tau kev pom zoo", qhov hloov kho pipeline yuav raug ua tiav thaum PR tsim.Qhov "kev pom zoo" tsuas yog siv rau qhov kev txiav txim sib koom ua ke xwb.

ppe7

Yuav ua li cas txog Indirect Poisoned Pipeline tiav

Raws li peb tau pom saum toj no, D-PPE tuaj yeem txo qis los ntawm kev siv rub_qhov_thov_lub_target, tab sis nws tsis siv rau I-PPE.

Yog tias koj siv pull_request_target, qhov kev kuaj xyuas ua ntej yuav yog tus lej pib. Tab sis yog tias koj xav lees paub qee qhov kev kuaj xyuas ntawm tus lej pab txhawb (PR code) koj yuav tsum kuaj xyuas tus lej PR kom meej. Yog li ntawd, yog tias tus lej PR tau hloov kho txhua tsab ntawv plhaub uas tau thov los ntawm pipeline, lub "hauv paus" (muaj kev nyab xeeb) pipeline yuav siv cov ntawv sau "hloov kho" → Indirect PPE!!

Cov kev daws teeb meem no nyuaj me ntsis (tsis muaj lub mos txwv khawv koob zoo li pull_request_target). 

peb pipeline tam sim no muaj kev nyab xeeb rau D-PPE vim peb siv pull_request_target. Tab sis nws tseem muaj kev phom sij rau I-PPE. 

Hauv peb qhov piv txwv sim, peb yuav tsum tau kuaj xyuas PR code kom ua qhov kev tsim kho, tab sis cov kev sim raug ua tiav ntawm cov khoom cuav uas tsim los ntawm kev tsim kho. 

Yog li .. vim li cas ho tsis xyuas ob lub codebases? 

  • Tshawb xyuas PR code vim yog qhov code uas peb xav tsim thiab sim.
  • Checkout Base code los khiav cov version thawj ntawm pipeline thiab cov ntawv sau tsim / sim 

Qhov no yuav ua tau los ntawm kuaj xyuas cov codebases rau cov folders sib txawv: cov lej pib yuav raug kuaj xyuas rau hauv cov nplaub tshev hauv paus, thiab PR rau lwm cov nplaub tshev. Hauv qhov no peb yuav ua qhov kev tsim kho thiab cov ntawv xeem los ntawm cov nplaub tshev hauv paus tawm tsam cov lej muab tso rau hauv cov nplaub tshev tshiab.

Qhov no yog ib qho kev daws teeb meem yooj yim, tau kawg!! Tab sis, rau lub hom phiaj kev kawm kuv xav qhia txog ib qho kev hloov pauv nthuav heev (...) 

GitHub workflow_run qhov xwm txheej ua rau muaj teeb meem

Dhau li rub_qhov_thov_lub_target, GitHub muab lwm qhov xwm txheej trigger: workflow_runQhov kev tshwm sim no tso cai rau kev ua tiav ntawm ib qho pipeline ua tsaug rau lwm tus pipelinekev ua tiav. 

workflow_run thiab rub_qhov_thov_lub_target cov teeb meem zoo sib xws hauv ib qho: ob qho tib si yuav raug ua tiav hauv hom cai thiab, txawm tias muaj kev hloov pauv PR, lub hauv paus pipeline yuav raug tua pov tseg!! 

Cia peb saib peb qhov tam sim no pipeline:

				
					name: PR TARGET CI


on:
  pull_request_target:
    branches: [ main ]


env:
  MY_SECRET: ${{ secrets.MY_SECRET }}
 
jobs:
  prt_build_test_and_merge:
    runs-on: ubuntu-latest


    steps:
      # checkout PR code
      - name: Checkout repository
        uses: actions/checkout@v4
        with:


          # This is to get the PR code instead of the repo code
          ref: ${{ github.event.pull_request.head.sha }}


      # Simulation of a compilation
      - name: Building ...
        run: |
          mkdir ./bin
          touch ./bin/mybin.exe
          ls -lR
     
      # Simulation of running tests
      - name: Running tests ...
        id : run_tests
        run: |
          echo Running tests..
          chmod +x runtests.sh
          ./runtests.sh 
          echo Tests executed.
         
      #
      # Let’s omit the check conditions at this moment …
      #
      - name: pr_check_conditions_to_merge
        [...]
				
			

Qhov seem tsim kho muaj kev nyab xeeb rau D-PPE, tab sis qhov seem sim tseem muaj kev phom sij rau I-PPE.

cov pipeline nws tus kheej muaj kev nyab xeeb rau D-PPE vim yog rub_qhov_thov_lub_target ua rau muaj teeb meem. Tab sis cov kauj ruam sim tseem muaj kev phom sij rau I-PPE vim yog siv cov ntawv sau sab nraud.

Tsis txhob siv I-PPE 

Lub hom phiaj ntawm qhov saum toj no pipeline yog los tsim thiab sim cov lej uas tau muab pab txhawb, ua kom muaj kev nyab xeeb rau PPE. 

Yog li .. Vim li cas ho tsis faib qhov pipeline mus rau hauv ob? Ib qho rau kev tsim kho thiab lwm qhov rau kev sim..

  • Qhov 1 pipeline (Tsim CI) yuav them tus lej PR (los tsim nws), ua qhov kev tsim kho thiab tsim ib qho khoom cuav.
  • Lub 2nd pipeline (Kev Ntsuas CI) yuav checkout lub Base code (kom tsis txhob hloov kho shell script) thiab ua cov ntawv sau thawj zaug tawm tsam cov khoom cuav. 
  • Txhawm rau sib dhos qhov Kev Ntsuas CI pipeline kom khiav TOM QAB Tsim CI pipeline, peb yuav siv cov workflow_run lub cuab yeej ua kom tawg. 
ppe8

Ua li no:

  • pipeline Tsim CI is muaj kev nyab xeeb rau ob qho tib si D-PPE (vim rub_qhov_thov_lub_target) thiab I-PPE (vim nws tsis ua tiav cov ntawv sau plhaub lawm).
  • pipeline Kev Ntsuas CI yog tseem muaj kev nyab xeeb rau ob qho tib si D-PPE (vim workflow_run) thiab I-PPE (vim nws checkout lub hauv paus code kom tau txais thawj lub plhaub script) 

Cia peb saib cov code ntawm ob qho tib si pipelines raws li cov kev hloov kho no ...

1st pipeline (Tsim CI):

				
					name: Build CI


on:
  pull_request_target:
    branches: [ main ]


env:
  MY_SECRET: ${{ secrets.MY_SECRET }}
  GITHUB_PAT: ${{ secrets.GH_PAT }}
 
jobs:
               
  prt_build_and_upload:
    runs-on: ubuntu-latest
    steps:
      - name: Checking out PR code
        uses: actions/checkout@v4
        if: ${{ github.event_name == 'pull_request_target' }}
        with:
          # This is to get the PR code instead of the repo code
          ref: ${{ github.event.pull_request.head.sha }}


      - name: Building ...
        run: |
          mkdir ./bin
          touch ./bin/mybin.exe
	    # Save some PR info for later use by the 2nd pipeline
          echo "${{github.event.pull_request.title}}" > ./bin/PR_TITLE.txt
          echo "${{github.event.number}}" > ./bin/PR_ID.txt
 
	# Upload the binary as a pipeline artifact
      - name: Archive building artifacts
        uses: actions/upload-artifact@v3
        with:
          name: archive-bin
          path: |
            bin
				
			

2nd pipeline (Kev Xeem CI):

				
					ame: Test CI


on:
  workflow_run:
    workflows: [ 'PR TARGET CI' ]
    types: [completed]
   
env:
  MY_SECRET: ${{ secrets.MY_SECRET }}
  GITHUB_PAT: ${{ secrets.GH_PAT }}




jobs:
  deploy:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion == 'success' }}
    steps:
 


      # By default, checks out base code (not PR code)
      - name: Checkout repository
        uses: actions/checkout@v4


	# Download the artifact
      - name: 'Download artifact'
        uses: actions/github-script@v6
        with:
          script: |
            let allArtifacts = await github.rest.actions.listWorkflowRunArtifacts({
               owner: context.repo.owner,
               repo: context.repo.repo,
               run_id: context.payload.workflow_run.id,
            });
            let matchArtifact = allArtifacts.data.artifacts.filter((artifact) => {
              return artifact.name == "archive-bin"
            })[0];
            let download = await github.rest.actions.downloadArtifact({
               owner: context.repo.owner,
               repo: context.repo.repo,
               artifact_id: matchArtifact.id,
               archive_format: 'zip',
            });
            let fs = require('fs');
            fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/myartifact.zip`, Buffer.from(download.data));


	# Unzip the artifact
      - name: 'Unzip artifact'
        run: |
          unzip -o myartifact.zip


      # Runs tests
      - name: Running tests ...
        id : run_tests
        run: |
          echo Running tests..
          chmod +x runtests.sh
          ./runtests.sh
          echo Tests executed.


#
      # Let’s omit the check conditions at this moment …
      #
      - name: pr_check_conditions_to_merge
        [...]

				
			

Auj... kev daws teeb meem zoo kawg!! Tab sis ….. Peb puas muaj kev nyab xeeb? Kuv ntshai tias tsis muaj 😭

Muaj tseeb tiag, peb tau qhia txog qhov tsis muaj zog tshiab!! Qhov twg? Qhov no yuav yog lub ntsiab lus ntawm peb cov ntawv tom ntej 🙂 ... Nyob twj ywm mloog!! 

PS: Thov txim, kuv nyob ntsiag to tsis tau 🤐 ..Koj puas tau hnov ​​txog Kev lom khoom cuav ? 😂 ib

Kev Lom Khoom Siv thiab Kev Txhaj Tshuaj Code

Kev Deep Dive rau hauv CI/CD Pipelines Qhov Tsis Muaj Zog (III)

Tiv thaiv kev lom khoom cuav los ntawm kev lees paub software

Kev Deep Dive rau hauv CI/CD PipelineKev Tsis Muaj Zog (IV)

Lom Pipeline Kev Ua Haujlwm (PPE)

Kev Deep Dive rau hauv CI/CD Pipelines Qhov Tsis Muaj Zog (I)
cov cuab yeej sca-software-composition-analysis-cuab yeej
Tsom xyuas qhov tseem ceeb, kho, thiab tiv thaiv koj cov kev pheej hmoo software
Tau txais koj tus Account Dawb.
Tsis muaj credit card yuav tsum tau.

Ruaj ntseg koj txoj kev tsim kho software thiab kev xa khoom

nrog Xygeni Product Suite