Compute4PUNCH & Storage4PUNCH: Tsarin Kayayyakin Hadin Kai don Kimiyyar Barbashi, Taurari, da Nukiliya

1. Gabatarwa

Particles, Universe, NuClei and Hadrons for the National Research Data Infrastructure (PUNCH4NFDI) ƙungiya ce ta Jamus da DFG (Deutsche Forschungsgemeinschaft) ke tallafawa. Tana wakiltar kimanin masana kimiyya 9,000 daga fagagen barbashi, taurari, barbashin taurari, hadron, da kimiyyar nukiliya. Babban manufar ƙungiyar ita ce kafa dandalin bayanan kimiyya na haɗin gwiwa da FAIR (Findable, Accessible, Interoperable, Reusable). Wannan dandalin yana da nufin samar da damar haɗin kai ga nau'ikan albarkatun kwamfuta da ajiya daban-daban waɗanda cibiyoyin membobinta suka ba da gudummawa a duk faɗin Jamus, tare da magance ƙalubalen gama gari na nazarin girman bayanai masu girma da ƙima tare da ƙa'idodi masu rikitarwa.

2. Tsarin Kwamfuta na Hadin Kai (Federated) – Compute4PUNCH

Ra'ayin Compute4PUNCH yana magance ƙalubalen samar da damar shiga cikin sauƙi ga ɗimbin albarkatun High-Throughput Compute (HTC), High-Performance Compute (HPC), da na Girgije waɗanda aka ba da gudummawa. Waɗannan albarkatun sun bambanta a tsarin gini, OS, software, da tabbatar da ainihi, kuma suna aiki kuma ana raba su, suna buƙatar hanyar haɗawa marar tsangwama.

3. Tsarin Ajiya na Hadin Kai – Storage4PUNCH

A layi daya da kwamfuta, ra'ayin Storage4PUNCH yana haɗa tsarin ajiya da al'umma ke samarwa, galibi bisa dCache da XRootD fasahohin, waɗanda suka kafu sosai a cikin Kimiyyar Makamashi Mai Girma (HEP).

4. Cikakkun Bayanai na Fasaha & Tsarin Lissafi

Babban ƙalubalen tsarawa ana iya ƙirƙira shi azaman matsalar inganta albarkatu. Bari $R = \{r_1, r_2, ..., r_n\}$ ya wakilci saitin albarkatu daban-daban, kowannensu yana da halaye kamar tsarin gini, cores masu samuwa $c_i$, ƙwaƙwalwar ajiya $m_i$, da lokacin jira na layi $w_i$. Bari $J = \{j_1, j_2, ..., j_m\}$ ya wakilci ayyuka tare da buƙatun $\hat{c}_j, \hat{m}_j$.

Na'urar tsarawa (COBalD/TARDIS) tana da nufin haɓaka amfani gabaɗaya ko kuma yawan aiki. Aikin da aka sauƙaƙa don sanya aikin zai iya zama don rage girman lokacin kammalawa ko haɓaka amfani da albarkatu, la'akari da ƙuntatawa:

$\text{Minimize } \max_{r \in R} (\text{completionTime}(r))$

mai ƙuntatawa: $\sum_{j \in J_r} \hat{c}_j \leq c_r \quad \text{da} \quad \sum_{j \in J_r} \hat{m}_j \leq m_r \quad \forall r \in R$

inda $J_r$ shine saitin ayyukan da aka sanya wa albarkatu $r$. Yanayin motsi TARDIS ne ke sarrafa shi, wanda ke "yaudarar" HTCondor don ganin albarkatun nesa a matsayin wani ɓangare na tafkin gida.

5. Sakamakon Gwaji & Matsayin Samfuri

Takardar ta ba da rahoto game da matsayin yanzu da kuma abubuwan farko tare da aikace-aikacen kimiyya akan samfuran da ake da su. Duk da yake ba a bayyana takamaiman lambobin ma'auni a cikin abin da aka ba da shawarar ba, an nuna nasarar aiwatar da ainihin ayyukan kimiyya. An nuna haɗin HTCondor tare da COBalD/TARDIS don haɗa albarkatu daga yankuna daban-daban na gudanarwa ta hanyar motsi. An gwada shigar mai amfani ta farko ta hanyar JupyterHub da token-based AAI, yana ba da tabbacin ra'ayi don shiga ɗaya. An tabbatar da amfani da CVMFS don isar da yanayin software da ake buƙata a cikin tsarin haɗin gwiwa.

6. Tsarin Bincike: Nazarin Halin da ake ciki na Ra'ayi

Yi la'akari da binciken taurari mai yawan saƙo da ke neman abokan hannu na neutrino ga fashewar gamma-ray. Aikin yana ƙunshe da:

1. Gano Bayanai: Mai bincike yana amfani da kundin metadata na haɗin gwiwa (ana kimantawa a cikin Storage4PUNCH) don gano bayanan abubuwan da suka faru na neutrino daga IceCube da bayanan gamma-ray daga Fermi-LAT, waɗanda aka adana a cikin misalan dCache a DESY da Bielefeld.
2. Ƙaddamar da Aikin: Ta hanyar fuskar JupyterHub, mai bincike ya ayyana binciken share sigogi. An ƙayyade buƙatun aikin (software: Python, IceCube software suite ta hanyar CVMFS; kwamfuta: sa'o'i 1000 na CPU).
3. Tsarawa: Overlay na HTCondor, wanda COBalD/TARDIS ke jagoranta, yana daidaitawa da aika ɗaruruwan ayyuka zuwa ramuka masu samuwa a cikin HPC na KIT, HTC na Bonn, da albarkatun girgije. Token AAI yana sarrafa tabbatar da ainihi cikin sauƙi.
4. Aiwatarwa & Samun Bayanai: Ayyuka suna jawo software daga CVMFS, suna karanta bayanan shigarwa kai tsaye daga ajiyar haɗin gwiwa ta hanyar ƙofofin XRootD, kuma suna rubuta sakamakon tsaka-tsaki zuwa wurin ajiya na wucin gadi.
5. Haɗa Sakamako: An haɗa sakamakon ƙarshe kuma an sake rubuta su zuwa ma'ajiyar dindindin, mai bin ka'idodin FAIR a cikin haɗin gwiwar Storage4PUNCH.

Wannan lamarin yana nuna ƙimar ra'ayi: masanin kimiyya yana hulɗa da tsari ɗaya, mai daidaituwa don amfani da albarkatun da aka watsar a ƙasa, daban-daban ba tare da sarrafa rikitarwar da ke ƙasa ba.

7. Hangar Aikace-aikace & Hanyoyin Gaba

Haɗin tsarin Compute4PUNCH da Storage4PUNCH yana da babban yuwuwar fiye da al'ummomin PUNCH na farko:

• Hadin Kai na Yankuna Daban-daban: Za a iya ƙaddamar da samfurin zuwa wasu ƙungiyoyin NFDI ko shirye-shiryen European Open Science Cloud (EOSC), ƙirƙirar ainihin tsarin haɗin gwiwa na Turai.
• Haɗa Kwamfuta na Geɓe: Don fagage kamar ilmin taurari na rediyo ko sa ido kan na'urar ganowa, haɗa albarkatun kwamfuta na gefe kusa da na'urori masu auna firikwensin na iya zama mataki na gaba na ma'ana.
• Tallafin Ayyukan AI/ML: Haɓaka mai tsarawa don tallafawa albarkatun GPU/accelerator na asali da tsarin kamar Kubernetes don manyan ayyukan horar da ML.
• Gudanar da Bayanai na Ci gaba: Haɗawa mai zurfi na sanya bayanai mai hankali, sarrafa rayuwa, da kundin metadata masu aiki don inganta ayyukan aiki masu yawan bayanai.
• Haɗin Kwamfuta na Quantum: Yayin da kwamfutar quantum ta girma, haɗin gwiwar na iya haɗa na'urori masu sarrafa quantum azaman albarkatu na musamman don matakan algorithm na musamman.

Nasarar wannan haɗin gwiwar za ta dogara da tallafin dorewa, ƙarfin aiki, da ci gaba da amincewar al'umma ga samfurin haɗin gwiwa akan inganta gida.

8. Nassoshi

1. Ƙungiyar PUNCH4NFDI. "PUNCH4NFDI – Particles, Universe, NuClei and Hadrons for the NFDI." White Paper, 2021.
2. Thain, D., Tannenbaum, T., & Livny, M. "Distributed computing in practice: the Condor experience." Concurrency and Computation: Practice and Experience, 17(2-4), 323-356, 2005.
3. Blomer, J., et al. "CernVM-FS: delivering scientific software to globally distributed computing resources." Journal of Physics: Conference Series, 396(5), 052018, 2012.
4. Fuhrmann, P., & Gulzow, V. "dCache, storage system for the future." In European Conference on Parallel Processing (pp. 1106-1113). Springer, Berlin, Heidelberg, 2006.
5. XRootD Collaboration. "XRootD – A highly scalable architecture for data access." WSEAS Transactions on Computers, 10(11), 2011.
6. Isard, M., et al. "Quincy: fair scheduling for distributed computing clusters." In Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles (pp. 261-276), 2009. (Don mahallin ka'idar tsarawa).
7. Wilkinson, M. D., et al. "The FAIR Guiding Principles for scientific data management and stewardship." Scientific data, 3(1), 1-9, 2016.

9. Bincike na Asali: Fahimta ta Asali, Tsarin Ma'ana, Ƙarfi & Kurakurai, Abubuwan da za a iya aiwatarwa

Fahimta ta Asali: PUNCH4NFDI ba ta gina sabon babban kwamfuta ba; tana injiniyan Layer na haɗin gwiwa na ƙaramin kutsawa mai yuwuwa. Wannan amsa ce mai hankali, mai basira a siyasance ga ƙuntatawa na ainihin duniya na rarrabuwar kwamfutar bincike na Jamus, mallakar al'umma. Ainihin ƙirƙira ba ya cikin fasahohin ɗaiɗaiku—HTCondor, dCache, CVMFS an gwada su—amma a cikin tsarinsu zuwa tsarin ƙasa mai daidaituwa tare da token-based AAI a matsayin manne. Dabarar "overlay network" ce ta gargajiya da aka yi amfani da ita akan tsarin sadarwa na yanar gizo, mai tunawa da yadda aka gina intanet kanta a kan hanyoyin sadarwa na jiki daban-daban. Yayin da European Open Science Cloud (EOSC) ke fuskantar irin wannan ƙalubalen haɗin gwiwa, hanyar PUNCH tana ba da takamaiman tsari, na aiki.

Tsarin Ma'ana: Ma'ana tana da sauƙi mai ban sha'awa: 1) Karɓi bambance-bambance a matsayin yanayi na dindindin, ba matsalar da za a kawar da ita ba. 2) Yi amfani da tsarawa mai sauƙi (COBalD/TARDIS) don ƙirƙirar tafki na zahiri, guje wa buƙatar gyara masu tsarawa na gida (SLURM, PBS, da sauransu). 3) Raba ainihi da sarrafa samun shiga ta hanyar alamomi, tare da kauce wa mafarkin daidaita asusun cibiyoyi. 4) Raba software daga tsarin kayayyaki ta hanyar CVMFS/kwantena. 5) Aiwatar da irin wannan ma'anar haɗin gwiwa ga ajiya. Gudun yana daga sauƙin fuskar mai amfani (JupyterHub) ƙasa ta hanyoyin ɓoyayyen Layer zuwa rikitarwar da ke ƙasa.

Ƙarfi & Kurakurai: Ƙarfin da ya mamaye shi ne iyawar aiwatarwa. Ta hanyar buƙatar ƙananan canje-canje daga masu samar da albarkatu, yana rage shingen shiga, wanda ke da mahimmanci don fara ƙungiya. Yin amfani da kayan aikin HEP masu girma yana tabbatar da dogaro da rage haɗarin haɓakawa. Duk da haka, kurakurai suna cikin ciniki. Samfurin overlay na iya haifar da ƙarin aiki a cikin aikawa da aiki da samun bayanai idan aka kwatanta da tsarin haɗin kai mai ƙarfi. Ƙirar "mafi ƙanƙanta na gama gari" na iya iyakance samun dama ga fasalulluka na musamman na takamaiman tsarin HPC. Mafi mahimmanci, samfurin dorewa na dogon lokaci ba a tabbatar da shi ba—wa ke biyan kuɗin daidaitawa na tsakiya, kula da mai tsarawa, da tallafin mai amfani? Aikin yana haɗarin gina samfuri mai haske wanda zai bushe bayan tallafin DFG na farko na shekaru 5.

Abubuwan da za a iya aiwatarwa: Ga sauran ƙungiyoyi, abin da ya fi mahimmanci shi ne fara da mulki da haɗin kai mai sauƙi, ba babban sake fasalin fasaha ba. 1) Nan da nan karɓi AAI na alama; shine mai ba da dama na tushe. 2) Ba da fifiko ga ƙwarewar mai amfani (JupyterHub) don haɓaka amfani; masana kimiyya ba za su yi amfani da tsari mai wahala ba. 3) Kafa kowane abu tun daga ranar ɗaya. Don tabbatar da tallafin gaba, dole ne su samar da ma'auni masu ban sha'awa game da ƙarin amfani da albarkatu, haɗin gwiwa tsakanin cibiyoyi, da kuma yawan aikin kimiyya. 4) Shirya don "haɗin gwiwa na biyu"—yadda za a haɗa kai da sauran ƙungiyoyin NFDI ko EOSC. Ya kamata a ƙera tsarin ginin fasaha a fili don haɗin gwiwa da aka haɗa. A ƙarshe, dole ne su haɓaka samfurin raba farashi mai haske don ayyukan tsakiya, suna matsawa daga tallafin aikin zuwa samfurin tallafin aiki na haɗin gwiwa kamar WLCG (Worldwide LHC Computing Grid). Fasahar ta shirya; ƙalubalen dindindin na zamantakewa-fasaha ne.