In our previous post, we saw how to detect and protect against Direct Poisoned Pipeline Execution (D-PPE). We also saw how to detect that vulnerability using Mea Nānā Xygeni, as well as some protection mechanisms.
Poisoned Pipeline Hoʻokō (PPE) is produced when the attacker can modify the pipeline logic in either of two ways:
- By modifying the CI config file (the pipeline) -> Direct PPE (D-PPE)
- By modifying files referenced by the pipeline (no ka laʻana: nā palapala i kuhikuhi ʻia mai loko mai o ka pipeline configuration file) -> Indirect PPE (I-PPE)
In this post, we will deep dive into Indirect PPE . But, before that, and as a complement to my previous post, let’s see first how GitHub manages the execution of pipelines and what are the protection mechanisms against D-PPE.
How does GitHub protect the execution of pipelines coming from PRs?
How does GitHub work regarding the execution of modified pipelines?
mau ala pipelines can come from Pushes or Pull Requests (PR). As a major best practice, it’s strongly recommended to avoid any direct “push” to a protected branch and use Pull Requests as a mechanism to enforce some review before accepting any contributed code.
Pull Requests may arrive from two different sources:
- PRs coming from pua
- PRs coming from nā lālā
PRs from pua can come either from aupuni or uku hale kūʻai.
As we are dealing with PPE (Poisoned Pipeline Execution), our main point is not the “acceptance” of a PR but the execution of a modified pipeline during the PR’s acceptance/approval process. At the core of a PPE attack, there is an unintended execution of a “malicious” modified pipeline.
I loko o kekahi mau huaʻōlelo, ua make i ka lāʻau make Pipeline Hoʻopuka ʻia ka hoʻokō ʻana (PPE) i ka wā hiki i ka mea hoʻouka ke hoʻololi i ka pipeline kūpili.
ʻElua nā mea likeʻole:
- PPE pololei (D-PPE): Ma kahi hiʻohiʻona D-PPE, hoʻololi ka mea hoʻouka i ka faila hoʻonohonoho CI i loko o kahi waihona i loaʻa iā lākou ke komo, ma ke kaomi pololei ʻana i ka hoʻololi i kahi lālā mamao i pale ʻole ʻia ma ka repo, a i ʻole ma ka waiho ʻana i kahi PR me ka hoʻololi mai kahi lālā a i ʻole kahi ʻō. Mai ka wā o ka CI pipeline Ua wehewehe ʻia ka hoʻokō ʻana e nā kauoha i loko o ka faila hoʻonohonoho CI i hoʻololi ʻia, holo hope nā kauoha ʻino a ka mea hoʻouka i loko o ka node kūkulu ke kūkulu ʻia pipeline hoʻomaka ʻia.
- PPE ʻaʻole pololei (ʻO I-PPE): I kekahi mau hihia, ʻaʻole i loaʻa ka hiki ke loaʻa iā D-PPE i kahi ʻenemi me ke komo ʻana i kahi SCM waihona (e laʻa inā ʻo ka pipeline ua hoʻonohonoho ʻia e huki i ka faila hoʻonohonoho CI mai kahi lālā ʻokoʻa a pale ʻia ma ka waihona like). Ma ia ʻano kūlana, ma kahi o ka hoʻomake ʻana i ka pipeline ʻo ia iho, hoʻokomo kahi mea hoʻouka i ke code ʻino i loko o nā faila i kuhikuhi ʻia e ka pipeline (no ka laʻana: nā palapala i kuhikuhi ʻia mai loko mai o ka pipeline faila hoʻonohonoho)
Ma na hihia elua, E hoʻokō ʻo GitHub i ka mea i hoʻololi ʻia pipeline me ka ʻole o ka pono no ka loiloi mua a i ʻole ka ʻae ʻia.
PRs from forks on aupuni hoʻomaha
GitHub allows configuring the behaviour when processing PRs coming from forks in public repos.
When a PR is coming from a fork, GitHub always forces some level of “approval” before executing the pipeline associated with the PR. This level of approval trades off from a weak to a strict approval.
At Org level (Org>>Settings>>Actions>>General), you can decide among several “approval” options:
The strictest is the last one (“Require approval from all outside collaborators”) because GitHub will always require approval when the PR is coming from forks from outside collaborators.
But even in this strict case, there are differences between collaborators with read and write permissions.
- When the PR comes from a heluhelu user, the hooko o ka pipeline is STOPPED until there is an approval of changes. If the approval is ok, then the modified pipeline ua hookoia.
- When the PR comes from a kakau user, the approval is not needed and the modified pipeline is always executed !!
As a conclusion, PRs coming from forks on public repositories are lightly protected against PPE. There is some protection against external (read) users, but nothing related to internal (write) users.
Pehea PRs coming from forks from private repos?
PRs from forks on uku hoʻomaha
In this scenario, GitHub provides some useful configuration settings.
Above settings can be configured either at ʻOihana paha ma repo Papa.
i ka wa no option is checked, GitHub will ask for approval a it will not execute the modified pipeline. This is the safest configuration!!
ka unsafest configuration Aia a "Run workflows from fork pull request” is checked. In this case, same for both read and write users, Github will automatically execute the modified pipeline!! And this situation can be even inoino inā "E hoʻouna i nā hōʻailona kākau i nā kahe hana mai ka fork pull requests"A me"Send secrets and variables to workflows from fork pull requests” are checked. Do not do this unless clearly justified!!
Inā "Require approval for fork pull request workflows” is checked, the above situation is somewhat enhanced: GitHub will ask for approval and not execute the modified pipeline for the read user, but it will still execute it for a write user.
Forks seen, what about PRs coming from branches?
PRs from nā lālā
To protect this scenario you must rely on Branch Protection Rules.
At repo level, you can create branch protection rules for any branch. These rules add some constraints to modification of protected branches.
Although you configure a rule to “Koi a pull request ma mua o ka hui ʻana"A me"Require approvals", the modified pipeline will be automatically executed upon PR creation.The “approval” will only apply to the merge action.
What about Indirect Poisoned Pipeline hooko
As we saw above, D-PPE can be mitigated by using pahuhopu_noi_pull, akā ʻo ia does not apply to I-PPE.
If you use pull_request_target, the default checkout will be the base code. But if you want to validate some checks on the contributed code (PR code) you need to explicitly checkout the PR code. Therefore, if the PR code has modified any shell script invoked by the pipeline, the “base” (safe) pipeline will invoke the “modified” shell script → Indirect PPE!!
The solution to this is a bit more complicated (there is not a magic bullet like pull_request_target).
ko kakou pipeline is now safe to D-PPE because we are using pull_request_target. But it is still vulnerable to I-PPE.
In our test example, we need to checkout the PR code basically to make the build, but the tests are executed on the artifact generated by the build.
No laila .. why don’t check out both codebases?
- Checkout PR code because is the contributed code what we want to build and test
- Checkout Base code to run the original version of the pipeline and the build/tests scripts
This might be done by checking out those codebases to different folders: the base code might be checked out to the root folder, and the PR to a different folder. In this case we would execute the build and the test script from the root folder against the code placed into the new folder.
This is an easy solution, of course!! But, for learning purposes I would like to introduce a quite interesting variant (…)
GitHub kaʻina hana_holo hanana hoʻāla
Ma waho pahuhopu_noi_pull, GitHub provides another trigger event: kaʻina hana_holo. This event allows execution of a pipeline conditioned to another pipeline’s execution.
kaʻina hana_holo a pahuhopu_noi_pull triggers are similar in one aspect : both will be executed in privileged mode and, despite the PR modifications, the base pipeline will be executed !!
Let’s see our current pipeline:
name: PR TARGET CI on: pull_request_target: branches: [ main ] env: MY_SECRET: ${{ secrets.MY_SECRET }} jobs: prt_build_test_and_merge: runs-on: ubuntu-latest steps: # checkout PR code - name: Checkout repository uses: actions/checkout@v4 with: # This is to get the PR code instead of the repo code ref: ${{ github.event.pull_request.head.sha }} # Simulation of a compilation - name: Building ... run: | mkdir ./bin touch ./bin/mybin.exe ls -lR # Simulation of running tests - name: Running tests ... id : run_tests run: | echo Running tests.. chmod +x runtests.sh ./runtests.sh echo Tests executed. # # Let’s omit the check conditions at this moment … # - name: pr_check_conditions_to_merge [...]The build section is safe to D-PPE, but the test section is still vulnerable to I-PPE.
ka pipeline itself is safe to D-PPE due to the pahuhopu_noi_pull trigger. But the test step is still vulnerable to I-PPE due to invoking an external shell script.
Avoiding I-PPE
The purpose of the above pipeline is to build and test the contributed code, being safe to PPE.
No laila .. Why don’t split the pipeline into two ? One for building and another for testing..
- ʻO ka 1st pipeline (Build CI) would checkout the PR code (to build it), make the build and generate an artifact.
- ʻO ka 2 pipeline (Test CI) would checkout the Base code (to avoid shell script modification) and execute the original scripts against the artifact.
- No ka hoʻonohonoho ʻana i ka Test CI pipeline e holo MA HOPE o ka Build CI pipeline, we will use the kaʻina hana_holo hoʻāla.
In this way:
- pipeline Build CI is maluhia i nā mea ʻelua D-PPE (ma muli o pahuhopu_noi_pull) a me ka ʻO I-PPE (because it no longer executes the shell script).
- pipeline Test CI ia maluhia i nā mea ʻelua D-PPE (ma muli o kaʻina hana_holo) a me ka ʻO I-PPE (because it checkout the base code to get the original shell script)
Let’s see the code of both pipelines according to these modifications …
1st pipeline (Build CI):
name: Build CI on: pull_request_target: branches: [ main ] env: MY_SECRET: ${{ secrets.MY_SECRET }} GITHUB_PAT: ${{ secrets.GH_PAT }} jobs: prt_build_and_upload: runs-on: ubuntu-latest steps: - name: Checking out PR code uses: actions/checkout@v4 if: ${{ github.event_name == 'pull_request_target' }} with: # This is to get the PR code instead of the repo code ref: ${{ github.event.pull_request.head.sha }} - name: Building ... run: | mkdir ./bin touch ./bin/mybin.exe # Save some PR info for later use by the 2nd pipeline echo "${{github.event.pull_request.title}}" > ./bin/PR_TITLE.txt echo "${{github.event.number}}" > ./bin/PR_ID.txt # Upload the binary as a pipeline artifact - name: Archive building artifacts uses: actions/upload-artifact@v3 with: name: archive-bin path: | bin2nd pipeline (Test CI):
ame: Test CI on: workflow_run: workflows: [ 'PR TARGET CI' ] types: [completed] env: MY_SECRET: ${{ secrets.MY_SECRET }} GITHUB_PAT: ${{ secrets.GH_PAT }} jobs: deploy: runs-on: ubuntu-latest if: ${{ github.event.workflow_run.conclusion == 'success' }} steps: # By default, checks out base code (not PR code) - name: Checkout repository uses: actions/checkout@v4 # Download the artifact - name: 'Download artifact' uses: actions/github-script@v6 with: script: | let allArtifacts = await github.rest.actions.listWorkflowRunArtifacts({ owner: context.repo.owner, repo: context.repo.repo, run_id: context.payload.workflow_run.id, }); let matchArtifact = allArtifacts.data.artifacts.filter((artifact) => { return artifact.name == "archive-bin" })[0]; let download = await github.rest.actions.downloadArtifact({ owner: context.repo.owner, repo: context.repo.repo, artifact_id: matchArtifact.id, archive_format: 'zip', }); let fs = require('fs'); fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/myartifact.zip`, Buffer.from(download.data)); # Unzip the artifact - name: 'Unzip artifact' run: | unzip -o myartifact.zip # Runs tests - name: Running tests ... id : run_tests run: | echo Running tests.. chmod +x runtests.sh ./runtests.sh echo Tests executed. # # Let’s omit the check conditions at this moment … # - name: pr_check_conditions_to_merge [...] Wow… nice solution!! But ….. Are we safe? I’m afraid that no 😭
Indeed, we have introduced a new vulnerability!! Which one? This will be the subject of our next post 🙂 … Stay tuned!!
PS: Sorry, I can’t keep quiet 🤐 ..Have you heard about Ka lāʻau make o nā mea hana lima ? 😂
