Abin da zai iya faruwa da CI/CD pipelines?

Ci gaba da haɗa kai da kuma ci gaba da isarwa (CI/CD) pipelines sune ginshiƙin kowace ƙungiyar software da ke gina software ta hanyar "zamani". Atomatik yana ba da babban iko, amma yawancin masu haɓakawa ba su da alhakin da ke tattare da shi.

developer: Haka ne, mun yarda CI/CD tsaro da gaske kuma suna da iko mai ƙarfi akan masu kula da lambar, sake dubawa commitkafin haɗuwa; ayyuka da pipelinemanyan ma'aikata ne ke kula da su, suna kula da kada su tona asirin a cikin pipelines. Kuma ma'aikatan da suka san abin ne suka sanya kayan aikin. Me zai iya faruwa?

Ya ku masu haɓaka fasaha, CI/CD Tsarin yana da rikitarwa. Faɗin harinsa ya jawo hankalin masu aikata mugunta. Ya kamata a yi taka tsantsan kuma kada a taɓa yin ƙarfin gwiwa fiye da kima.

A wasu lokutan ana kiyaye tsarin tsoho kuma yana zama babban aboki ga masu kutse. Manyan kurakurai na iya kasancewa a ciki CI/CD pipeline tushe, a cikin tsarin, ko kuma a kusa da tsari da mahallin pipeline da kuma yadda ake tayar da shi.

A cikin wannan rubutun za mu saka kanmu a cikin halin miyagun 'yan wasan kwaikwayo. Ka yi tunanin muna karanta tunanin M3M3N70 (Memento Mori?) da kuma Fushin fadama wani wuri a cikin yanar gizo mai duhu, wataƙila a cikin harshen da ba na yamma ba, amma kada ku manta cewa mugunta tana yaɗuwa a duk duniya.

 

A zamanin da ya kasance mai sauƙi sosai…

M3M3N70: Komawa ga zamanin da kasuwancinmu ya kasance mai sauƙi sosai… Kwanaki sifili ba su da amfani, manhajoji suna buɗewa tare da sauƙin amfani da ƙwayoyin cuta, kuma za mu iya motsawa a gefe cikin sauƙi.

Fushin fadama: Fu#@Jahannama! Wasu sun yi tunanin zomo a can tukuna, amma abubuwa sun canza. Manyan mutane sun yi shakka sosai kan wannan abin da AppSec ya yi.

M3M3N70: Eh. Amma sabbin wawaye su ne masu haɓaka. A gare mu, ya fi sauƙi mu zaɓi kayan aikin da waɗannan mutanen ke amfani da su. Musamman CI, ma'adinan zinare ne! Alamun shiga girgije, SCM takardun shaida, kalmomin shiga na bayanai na samarwa, maɓallan sirri na SSH, takardun shaidar sauran masu amfani da CI… Tsallakewa daga abubuwan ci gaba masu ban sha'awa zuwa ainihin abin ba shi da mahimmanci.

Tsarin aiki ta atomatik don ginawa, gwaji da kuma tura software ta amfani da CI/CD Sau da yawa kayan aiki yana buƙatar aika sirrin umarni zuwa matakai. Kuma sau da yawa ana fallasa su, tare da mummunan sakamako.

Pipelines suna buƙatar sirrin da wani lokacin ake fallasa su

M3M3N70: The code monkeys slipped their AWS keys in a pipeline on a GitHub commit, later they removed the thing but not changed git history. For our scripts it was trivial to scan the history, grab the key and wreak havoc.

Wataƙila lokutan da suka yi kyau a tarihi an same su a cikin Git .env fayil (mai haɓaka ya manta ya ƙara shi zuwa .gitignore):

AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=wJalrXUtn...
AWS_REGION=us-east-1
APP_FOLDER=...
S3BUCKET=...

wanda aka yi amfani da shi a cikin tsarin aiki na GitHub .github/deploy.yaml wanda ya ƙunshi wani abu kamar haka:

jobs:
  build:
    name: Automated build and deployment into AWS
    runs-on: 	ubuntu-22.04
    steps:
      # Load environment from .env file
      - id: dotenv
        uses: falti/dotenv-action@v1.0.2
        
      - name: Configure AWS Credentials
        uses:
        with:
          args: --acl public-read --follow-symlinks --delete 
          aws-access-key-id: $
          aws-secret-access-key: $
          aws-region: $
      
      # ... build steps skipped ...
      
      -  name: Upload compiled code for deployment
         working-directory: $/packaged_app
         run: aws s3 cp my-app.zip s3://$ 
         
      - name: Deploy the app
        run: aws deploy create-deployment ...

M3M3N70: Kai! Waɗannan maɓallan aws suna aiki! Mun fara gwada canjin inocuos a cikin manhajar, sannan muka ƙara ƙarar domin waɗannan mutanen ba su sani ba. Bingo! Wannan kamfen ɗin ya yi kyau…

Mugun mutumin ya yi amfani da maɓallan AWS kawai don loda wani aikace-aikacen da aka gyara tare da malware sannan ya gudanar da umarnin tura kayan aiki tare da irin waɗannan takaddun shaida. Sirrin da aka fallasa, tare da bayanan da ke cikin pipeline"Wannan kamfen ɗin!" wataƙila yana nufin cewa Memento ya lalata wa talakan da abin ya shafa rai.

Abin da Memento ya gaya mana a nan shi ne cewa da zarar wani ɓoye sirri ya faru, kamar maɓallan shiga AWS a cikin misalin, dole ne ka soke sirrin (juya maɓallan da ke sama) Nan da nanAkwai koyaushe taga mai fallasa tsakanin zubar da ruwa commit da kuma rashin sirrin da aka yi; sake rubuta tarihin Git yana da wahala (har ma da mafi tsananin mulkin kama-karya ya gwada sake rubuta tarihin, amma bai yi nasara ba) kuma wataƙila bai yi tasiri ba (abokanmu na iya yin kwafin bayanai kafin a adana bayanan sirrin da ke ɓoye a cikin rumbun adana bayanai. commit) Juya maɓallan nan da nan, kuma yi addu'a yayin karanta rajistan ayyukan asusun da aka yi niyya a lokacin taga fallasa!

Wataƙila ƙungiyoyi ya kamata haramta amfani da sirrin dogon lokaci a cikin CI/CD pipelines, kuma maye gurbinsu da takardun shaidar lokaci. A cikin misalin da ya gabata tare da maɓallan AWS a cikin ayyukan GitHub, ya fi aminci a yi amfani da Mai bada sabis na OpenID Connect (OIDC) don samun takardun shaidar da ake buƙata na ɗan gajeren lokaci don ayyuka.

Fushin fadama: Kun yi sa'a sosai! Zubar da rubutun da maɓallan da aka yi wa lamba abu ne da aka saba gani a da, har ma a kan bokitin S3 da jama'a za su iya shiga. Abin da kawai za ku yi shi ne ku ratsa abubuwan da ke cikin bokitin ku yi wasu bincike don nemo abubuwa masu ban sha'awa.

Wani lokaci yankin da aka yi amfani da shi don tura (bokitin AWS S3 a cikin wannan misalin) yana buɗe don karantawa daga mutanen waje, saboda lahani na tsari (wanda ba a gano shi ba). Fushin fadama an yi amfani da wani abu kamar haka:

aws s3 ls --recursive s3://<bucket_name>/<path>/ | \
awk '{print $4}' | \
xargs -I FNAME sh -c "echo FNAME; \
  aws s3 cp s3://<bucket_name>/FNAME - | \
  grep '<regex_pattern>'"

Wataƙila an ƙirƙiri bokitin a cikin samfurin samar da kayayyaki wanda za a iya duba shi ta atomatik don gano kurakuran tsaro.

Tsarin kayan aikin tsoho ya kasance abin wasa a gare mu

Domin bayar da misalai na zahiri bari mu yi magana a kai Jenkins, ɗaya daga cikin kayan aikin CI mafi shahara.

Fushin fadama: Shin kuna tuna da akwatin "Enable Security" a cikin Jenkins, da kuma nawa ne ƙungiyoyin da suka zaɓi kada su kunna shi don jin daɗi? Kuma waɗannan "Duk wanda zai iya yin wani abu"Haɗuwa da izini a matsayin tsoho? Kuma waɗannan abubuwan haɗin Jenkins masu ban haushi, kamar Mai ƙara GitHub OAuthMutumin da ya tsara shi ya zaɓi duka "Ba wa duk Masu Amfani da Aka Tabbatar izini" da kuma "Yi amfani da izinin ma'ajiyar GitHub", wanda ke ba mu damar shiga duk ayyukan su.

(Yi haƙuri, Jenkins, don misalta ka 😉

Ka kasance mai ƙwarewa (har ma da mai jaraba) ga ƙa'idodin tsaro. Ɗaya daga cikinsu shine An tsare ta hanyar tsoho ƙa'ida: sarrafawa ya kamata ya zama na asali zuwa saitunan da suka fi tsaro. Ya kamata a gina tsaro a ciki CI/CD kayan aiki da pipelinedaga tushe, maimakon zama tunani na baya. Amma sauƙin amfani da sauƙin amfani sau da yawa yana karo da tsaro.

Ga batun Jenkins, ingantaccen ingantaccen tsari yana da rauni sosai: Kada a taɓa amfani da hanyoyin tantancewa da aka gina a cikin Jenkins. Zai fi kyau a zaɓi tsarin ɓangare na uku (SAML, LDAP, Google ...), tare da plugin ɗin Tsarin Izini na Tushen Aiki (“RBAC”). Kuma a yi taka tsantsan sosai game da admin asusu.

Kula da yadda ake aiki da kuma yadda ake pipeline Ana sarrafa fayiloli a Jenkins. Haka kuma da Saitin Saita-kamar-Lambar plugin da fayilolin saitinsa, waɗanda suka shafi tsarin Jenkins.

Canjawa daga mai masaukin baki kai tsaye CI/CD tsarin zuwa ga waɗanda ke amfani da SaaS na girgije yana kawar da wasu haɗarin da ke ba da damar motsi a gefe a cikin hanyar sadarwar ƙungiya, amma ya ƙara wasu, kamar buɗe haɗin waje tsakanin tsarin ciki na yanzu da na waje CI/CD kayan aiki.

Ya kamata ƙungiyoyi su yi aikicise kulawa mai kyau wajen taurarewa CI/CD tsarin, farawa daga saitunan da suka fi ƙuntatawa kuma a hankali yana buɗewa tare da ƙarancin izini da ake buƙata don pipeline matakai.

Saita tsaro a cikin CI/CD Kayan aiki na iya zama aiki mai rikitarwa. Da yawa suna da plugins ko kari waɗanda ke da mafi yawan raunin da ke buƙatar sabuntawa.

Na'urorin duba bayanai marasa tsari na tsaro don irin waɗannan kayan aiki masu rikitarwa, ko ma'auni na iya taimakawa.

Shigar da lambar a ciki pipeline umarni don nishaɗi da riba

M3M3N70: Shin kun taɓa amfani da Untrusted Code Checks, cewa ayyuka masu rauni da rubutun da ke fuskantar barazanar allurar umarni?

Wannan rukuni yana nuna cewa pipeline kanta na iya samun kurakuran coding waɗanda ke ba wa miyagun 'yan wasa damar shigar da aiwatar da lambar ba bisa ƙa'ida ba a cikin pipeline ba tare da canza yanayin ba pipeline tushen kantaMisali ta amfani da PR

Misali na farko na wani rashin aikin GitHub:

# INSECURE. Provided as an example only.
on:
  pull_request_target #1

jobs:
  build:
    name: Build and test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          ref: $ #2

      - uses: actions/setup-node@v1
      - run: |
          npm install #3
          npm build
      # ... more steps ...

Haɗa pull_request_target abin da ke haifar da aiki tare da duba bayanan PR mara aminci aiki ne mai haɗari wanda zai iya haifar da sulhu a wurin ajiya. A cikin misalin, haɗin da ba shi da daɗi na:

• pull_request_target event, wanda ta hanyar tsoho yana da izinin rubutawa ga ma'ajiyar da aka nufa da sirrin ma'ajiyar da aka nufa, har ma daga forks na waje, kuma yana gudana a cikin mahallin ma'ajiyar da aka nufa ta PR,
• duba lambar PR daga tushen, wurin ajiya mara aminci,
• fara kowane rubutun da zai iya aiki akan abubuwan da PR ke sarrafawa, kamar a yanayin npm install, Da kuma
• ba a yi amfani da wani yanayi ba wajen haifar da pull_request_target za a gudanar da taron ne kawai idan an sanya wani nau'in alamar 'an tantance wannan PR' ga PR (masu amfani na waje ba za su iya sanya lakabi ga PR ba).

Misali na biyu yana ɗaukar bayanai marasa aminci (daga wata matsala, sharhi ko pull request) a matsayin tushen hujjojin da aka mika wa wani pipeline umarni ta hanyar maganganu. Wannan shine pipeline sigar raunin shigarwar umarnin OS.

- name: Check title
  run: |
    title="$"
    if [[ ! $title =~ ^.*:\ .*$ ]]; then
      echo "Bad issue title"
      exit 1
    fi

Aikin gudu yana samar da rubutun harsashi na ɗan lokaci bisa ga samfurin, tare da $ an maye gurbinsa, wanda hakan ya sa ya zama mai sauƙin shigar da umarnin harsashi. Mai kai hari da asusun GitHub na bogi zai iya haifar da matsala da take a"; bad_code_goes_here;#, kuma bulb! 

Fushin fadama: Kai, waɗannan mutanen suna buɗe ƙofar allurar umarni ta hanyar buɗe wata matsala kawai…

Akwai raunin aiwatar da lambar a cikin ayyukan GitHub, kamar gajira-comment, yanzu an gyara. Don Allah a karanta "Ba a amince da shigarwar da aka bayar a cikin ayyukan GitHub ba" don cikakkun bayanai.

Dabi'ar labarin: Kada ka taɓa duba da kuma gina PR daga majiyoyin da ba a amince da su ba ba tare da fara duba PR ba. 'Ba a amince da shi ba' a nan, sai dai idan an tabbatar da asalinsa sosai, yana iya nufin duk wani asusun mai haɓaka da za a iya sacewa.

 

An fara amfani da malware ba tare da niyya ba a nan!

Ci gaba da turawa Wannan shine ƙarshen atomatik, amma wannan ƙarshen zai iya zama takaici saboda rashin ingantattun iko na amincewa akan pipeline gudana.

Haɗarin shigar da bayanai ta atomatik daga tushe commit Tsarin samarwa ya haɗa da yuwuwar tura lambar mugunta zuwa yanayin samarwa ba tare da an gano ta ba, da kuma yuwuwar kurakurai a cikin tsarin aikawa don haifar da katsewa ko dakatarwa.

Domin rage waɗannan haɗarin, sau da yawa ana ba da shawarar ƙungiyoyi su aiwatar da "karya mai wahala" a cikin tsarin tura su, wanda ke buƙatar yardar mutum kafin a tura kayan aiki zuwa wuraren da aka keɓe.

Suna rufe ƙofofin

Fushin fadama: Waɗannan kalmomin sirri masu farin ciki a cikin CI/CD Ana goge kayan aiki. Samun dama ga /var/lib/jenkins/secrets/initialAdminPassword yanzu hanya ce mara kyau. Kayan aiki da yawa yanzu suna samar da 2FA, wanda Covid ya sa ya shahara, har ma biri mafi lalaci a can yana amfani da shi!

M3M3N70: Muna fafatawa da 2FA, amma ba abu ne mai sauƙi ba. Yana da wuya a yi wa waɗannan mutanen zagon ƙasa, kamar yadda "Scatter Swine" ya yi da TwilioDa maɓallan WebAuthn, yana da wahala sosai. Aƙalla, za mu iya ƙoƙarin yin hakan satar kukis don kauce wa MFA, amma kuna buƙatar shiga cikin akwatin mai haɓaka.

Tabbatar da Tabbatar da Tabbatar da Abubuwa da yawa mataki ne mai kyau a kan hanya madaidaiciya don iyakance haɗarin ɓoye sirrin tantancewa. Yawancin kayan aikin DevOps na zamani suna goyan bayan MFA. Kuma maɓallan tantancewa a ƙarƙashin WebAuthn / U2F (duba Aikin FIDO2) wataƙila su ne mafi kyawun zaɓi ga MFA a cikin DevOps, idan an sarrafa su yadda ya kamata.

Fushin fadama: Mutanen DevOps suna farkawa. Suna da abin da ba shi da wani amfani a jininsu. Kuma ba su zama birai na code ba. Yanzu muna fuskantar kama-karya daga masu bita.

A gaskiya ma, pipelines yanzu sun fi ƙarfi fiye da shekaru biyu da suka gabata, tare da cire ayyuka marasa ƙarfi da rubutun, kuma tare da ƙarin matakan gwajin tsaro waɗanda har ma sun gano droppers ɗinmu da aka ɓoye a ɓoye. commits da fakitin da muka yi fashi.

Tambaya ga mai karatu: shin tsarin gina manhajar daga tushe da kuma tura ta zuwa samarwa kasuwanci ne mai haɗari? Shin za ku iya ganin DevOps ɗinku a matakin lokaci mai kyau ga miyagun mutane?

Shawarwarin karshe

Inda za a fara da CI/CD pipelines?

Shawarar farko tana da sauƙi a nan: A hankali review pipelines (su ne m albarkatu) don matsalolin tsaro. Sharhi suna da tsada amma dole ne, kuma ya kamata a yi su yadda ya kamata. Masu bita ya kamata su san abin da za su duba. Dole ne a duba kowane mataki don ganin ko akwai kurakurai.

Wataƙila haɗakar ƙwararrun masu bita da ke ɗauke da na'urorin duba lambobi masu cutarwa ta atomatik na iya taimakawa.

Shawara ta biyu ita ce horar da masu haɓaka waɗanda ke rubutu pipelineda kuma kiyaye su a kan tsaroAbubuwan da za a yi la'akari da su:

• Yadda ake sarrafa tantancewa yadda ya kamata ta hanyar amfani da ayyukan ciki da na girgije, tare da guje wa matsalolin sarrafa takaddun shaida na dogon lokaci.
• Yadda ake iyakancewa pipelinezuwa ga ainihin albarkatun da yake buƙatar samun dama. Ka'idar mafi ƙarancin gata ta sake haskakawa.
• Yadda ake rubuta matakan da za a yi pipelines mai sake bugawa kamar sigar pinning, da kuma guje wa raunin allurar umarni.
• Yadda ake amincewa da tura sojoji daga mahangar tsaro (wasu ne!): wace tsaro? standardya kamata a daidaita s da kuma yadda ake ƙara cak/ƙofofi masu dacewa a cikin pipelines.

Shawara ta uku ita ce saita CI/CD tsarin da kulawa ta dace. Ingantaccen tantancewa, babu kalmomin shiga na asali ko saitunan rashin tsaro, ƙarancin gata… Kula da raunin da ke tattare da plugins da kari da aka shigar. Wannan na iya zama babban abin da sakonnin da ke biye za su mayar da hankali a kai, don Allah a ci gaba da kasancewa tare da mu.

Shawara ta huɗu ita ce amfani da CI/CD pipelines don sarrafa kansa ta tsaroBinciken lambar tushe (SAST), nazarin abubuwan da aka samo asali (SCA), ana iya gudanar da binciken ɓoye sirri, kayan aikin hana malware, na'urorin duba tsaro na kwantena, ko na'urorin gano lokacin gudu ta atomatik (DAST da malware) akai-akai akan na'urorin. pipelineKuma ƙungiyar ku za ta iya aiwatar da standardgame da ɗaukar hoto kan na'urar daukar hoto ta tsaro a cikin CI/CD.

Ka tuna, waɗannan kayan aikin ba sa cire bita na ƙwararru daga lissafin, in ba haka ba za ka iya samun rashin tsaro.

Idan kun ƙware a fannin manyan ayyukan OWASP, wani kyakkyawan aiki na baya-bayan nan shine OWASP Mafi Girma 10 CI/CD Hadarin Tsaro.

Bayanin Bayanin Gargaɗi

(1) Misalan da ke cikin wannan rubutun suna amfani da GitHub kamar SCM, AWS a matsayin mai samar da girgije, da kuma GitHub Actions ko Jenkins a matsayin CI/CD Kayan aiki. Ba su da rauni/mafi aminci fiye da madadinsu. Ba a yi niyyar yin amfani da mummunan ra'ayi ba! Waɗannan kayan aikin suna da ƙarfi kuma suna buƙatar a yi amfani da su yadda ya kamata.

(2) M3M3N70 da kuma Fushin fadama haruffa ne na almara. Duk wani kama da mutane ko ƙungiyoyi, rayuwa ko mutuwa, kawai dai abin da ya faru ne… ko kuwa?

Domin karantawa

• Haymore, A. da sauransu. "Labarai 10 na gaske game da yadda muka yi sulhu" CI/CD pipelines ". Ƙungiyar NCC, Janairu 2022.
• configure-aws-credentials Aikin GitHub da kuma Saita Haɗin OpenID a cikin Ayyukan Yanar Gizo na Amazon don cikakkun bayanai kan yadda ake gudanar da umarnin AWS don turawa a cikin ayyukan aikin GitHub.
• Lobacevski J. "Tsayar da Ayyukan GitHub da ayyukan aikinku amintattu Kashi na 1: Hana buƙatun pwn"GitLab Security Lab, Disamba 2020.
• Lobacevski J. "Tsayar da Ayyukan GitHub ɗinku da hanyoyin aiki lafiya Kashi na 2: Shigarwar da ba a amince da ita ba"GitLab Security Lab, Janairu 2021.
• OWASP. "Manyan 10 na OWASP CI/CD Hadarin Tsaro”Yuli 2022.
• Saltzer J. da Schroeder M. "Kare Bayanai a Tsarin Kwamfuta"Afrilu 1975. Ka'idojin tsaro sun samo asali ne daga fasaha, amma bayan shekaru 47, yawancin ra'ayoyin S&S suna aiki.
• Hukumar NCSC ta Burtaniya. "Tabbatar da ginawa da kuma tura sojoji pipeline"Cibiyar Tsaron Yanar Gizo ta Ƙasa ta Burtaniya, Fabrairu 2019.