IDAN AI YA KWAFI AI, DOLE WANI YA CI GABA DA KOYO
Horar da samfuri mai samarwa (generative model) a kan abin da wani samfuri na baya ya samar, sai ya lalace. A cikin zuriyoyi (generations) masu bi da juna, abubuwan da suka fi ƙaranci suna ɓacewa kafin waɗanda suka zama ruwan dare, kuma abin da samfurin ke samarwa yana ƙara takura. An yi nazari sosai a kan wannan rushewar samfuri (model collapse) a yanzu — haka ma magungunansa. Duk magungunan da ke aiki suna shigo da wani abu daga wajen da’irar (loop): sababbin bayanan ɗan adam, bayanan gaske da aka ajiye, masu tantancewa (verifiers), zaɓen ɗan adam (human curation).
Nam Le, masanin kimiyyar kwamfuta a Jami’ar Vermont, na nuna abin da waɗannan nazarce-nazarce ke ɗauka a matsayin tabbas. Suna ɗaukar sababbin bayanai a matsayin tsayayyen ma’ajiya (archive), da ake samu a kowane adadi da ake buƙata, kuma suna ɗauka cewa duniyar da ake bayyanawa ba ta motsi. Babu ɗayansu da yake gaskiya: mutane ne ke rubuta sabon rubutu, kuma suna ƙara tambayar samfuran maimakon su bincika abubuwa da kansu; kuma duniya na ci gaba da canzawa.
Wani tsohon saɓani
Duk batutuwan biyu suna a zuciyar wata tsohuwar ka’ida (theory). Bayanin Rogers na 1988 ya kwatanta abin da yanzu ake kira saɓanin Rogers (Rogers’ paradox): a cikin al’umma (population) inda kwafa ke yaɗuwa saboda, ga kowane mutum, ya fi koyo arha, kowa yana ƙarewa ba tare da halinsa ya fi na al’ummar da ba ta taɓa kwafa ba. Kwafe-kwafe suna tsufa, kuma dole wani ya ci gaba da biyan kuɗin koyo.
Ayyukan da suka biyo baya a juyin halittar al’adu (cultural evolution) sun inganta ra’ayin. Mai koyo mai bincike sosai (critical) yana kwafa, ya duba kwafin da duniya, kuma ya koya da kansa idan kwafin ya gaza; irin waɗannan masu koyo suna ɗaga ribar (payoff) kowa. Mai koyo mai bin yawancin jama’a (conformist) yana kwafar abin da ya zama ruwan dare, wanda ke da arha amma yana zaizaye nau’o’i masu ƙaranci (rare variants).
Hujjar Le ita ce horar da samfuri a kan abin da wani samfuri ya samar koyo ne na zamantakewa (social learning) daidai da wannan ma’ana, don haka ka’idar ya kamata ta yi aiki. A mafi sauƙin siffarta, tana ba da wata doka ta layi ɗaya (one-line rule) ta yawan ilimin da jerin zuriya (lineage) ke riƙewa yayin da wani kaso na bayanan horarwarsa sababbi ne.
Darwin da Moby-Dick, an sake kwafa su
Don gwada ta, aikin Le ya horar da jerin zuriyoyin (lineages) ƙananan samfuran harshe (language models) a kan littattafai biyu daga Project Gutenberg, On the Origin of Species na Darwin da Moby-Dick na Melville. Kowace zuriya ta koya daga rubutun da zuriyar da ta gabace ta ta samar, haɗe da wani sabon rubutu daga littafin.
Da aka ciyar da su da abin da suka samar kaɗai, samfura masu aiki a matakin kalma (word-level) sun rasa kalmomi huɗu cikin biyar na kalmomi masu ƙaranci a cikin zuriyoyi goma sha biyu. Wani misalin rubutu (sample) daga zuriyar farko har yanzu yana karantuwa “i cannot see that a genus become fitted for most justly celebrated elder huber”; a ta goma sha biyu, “struggle of the on that may inheritable detected i hybrids, kept.”
Ɓangaren mai ban mamaki shi ne hasashen. Aikin Le ya daidaita ma’auni (parameter) ɗaya tilo na dokar — damar da wani ɓangaren ilimi da aka kwafa ke ɓacewa a kowace zuriya — a kan jerin zuriyar da aka ciyar da abin da ta samar gaba ɗaya kaɗai, sannan ya yi amfani da shi, ba tare da canji ba, don hasashen jerin zuriyoyi masu sababbin bayanai na 10%, 30% ko 50%. Ya yi daidai da yawan kalmomi masu ƙaranci da suka riƙe a cikin maki 3 na kashi (percentage points). Da ƙarin wani rukuni na biyu na ilimi, mai rauni (fragile) da mai ɗorewa (durable), dokar ta kuma yi aiki ga ƙaramin transformer da kuma wani samfurin harshe da aka yi wa horon farko (pretrained), OPT-125m.
Idan duniya ta motsa
Na gaba, an sa “duniya” kanta ta canza: wata taga (window) da ke zamewa cikin littattafan biyu, ta yadda sababbin kalmomi ke ci gaba da isowa, tsofaffi kuma ke ci gaba da barin wurin. Yayin da take ƙara saurin motsi, haka jerin zuriyar da ke kwafar kanta ke ƙara buƙatar sababbin bayanai. Don a riƙe kashi 80% na abin da cikakken tarin rubutu sabo (fully fresh corpus) zai ƙunsa, dole kason sababbin bayanai ya tashi daga kusan 30% zuwa kusan 45%. Fannoni masu saurin canzawa, a hasashen takardar, su ne za su fara lalacewa yayin da rubutun roba (synthetic text) ke taruwa.
Samfurin da aka yi wa horon farko ya nuna hali daban: iliminsa ya ɗore ko duniya ta motsa ko ba ta motsa ba. Fassarar Le ita ce ma’aunan nauyinsa na horon farko (pretrained weights), waɗanda ake gadarwa ba tare da canji ba a kowace zuriya, ilimi ne da ba ya taɓa ratsa sarkar kwafa. Da wani sharaɗi: rubutun da ke motsi a nan rubutu ne da samfurin ya riga ya gani a lokacin horon farko.
Mutane a cikin da’irar
Sannan sai ɓangaren da ya ba takardar sunanta. A cikinta, Le na gina wani samfuri (model) inda mutane za su iya ko dai su gano wata hujja (fact) da kansu, da tsada, ko kuma su tambayi AI kyauta kuma su yi daidai kawai gwargwadon yadda AI ke yin daidai. Abin da mutane suka rubuta yana zama bayanan horarwa na samfuri na gaba, tare da wani rubutu da samfurin ya rubuta da kansa.
Daidaiton (equilibrium) da ake samu yana da tsauri:
- AI yana ƙarewa yana dacewa sau isassu kawai ta yadda tambayarsa ke da riba daidai da ta bincike da kai, idan aka lissafa tsadar bincike da kai — ko mene ne girmansa ko tsarinsa (architecture);
- mutane gaba ɗaya ba su fi kyau ba fiye da ba tare da AI ba: saɓanin Rogers, ga mutane da na’urori tare;
- yawan rubutun horarwa da samfuran ke rubutawa da kansu, haka yawan mutanen da dole ne su ci gaba da koyo da kansu — ninki biyu idan rabin rubutun samfura ne suka rubuta. Sababbin bayanai, kamar yadda Le ke kammalawa, ana biyan kuɗinsu da ƙoƙarin ɗan adam.
Akwai hanyar fita. Idan mutane suna duba amsar AI da duniya kuma suna sake koyo idan ba daidai ba ce, saɓanin yana karyewa: a cikin kwaikwayon (simulation), matsakaicin riba (payoff) yana tsayawa a 0.93, idan aka kwatanta da 0.70 ba tare da AI ba. Kayan aikin da ke sa tantancewa (verification) ya yi arha, a wannan hangen, ba kayan jin daɗi ba ne, amma su ne abin da ke hana dukan tsarin rushewa.
Matatun da ke ƙara ɓata lamarin
Wannan hangen yana kuma rarrabe matatun (filters) da ake amfani da su don tsabtace bayanan roba. Riƙe misalan (samples) da samfuri ke ganin sun fi yiwuwa (most plausible) kwafa ne na bin yawancin jama’a (conformist), kuma a cikin gwaje-gwajen ya hanzarta ɓacewar kalmomi masu ƙaranci: samfurin da aka yi wa horon farko ya riƙe kashi 5% nasu, idan aka kwatanta da 57% lokacin da aka riƙe rabin misalan bisa ga sa’a (at random). Matatun da ke duba rubutu da duniya kaɗai ne ke aiki kamar masu koyo masu bincike sosai (critical). An nuna duk wannan da ƙananan samfura, mafi girmansu yana da ma’aunai (parameters) miliyan 125, kuma Le na ambaton gwada shi a babban mataki (at scale) a matsayin babbar tambayar da har yanzu ke buɗe.
