KA TAMBAYI CHATBOT YA ZAƁI LAMBA TA BAZATA

Mutane suna roƙon chatbots su raba kunnen doki, su yi zaɓi ba tare da wani dalili ba, ko su maye gurbin jefa tsabar kuɗi. A wajen Samar Ansari, wannan batun daidaiton yiwuwa (calibration) ne: samfurin da ba shi da dalilin fifita wani zaɓi ya kamata ya raba yiwuwarsa a kan dukkansu. Abin da samfuri ke yi idan babu abin da ke fifita wata amsa, a kalaman takardar, “wata siffa ce da za a iya aunawa, ba wasan dabo ba” (a measurable property, not a parlour trick).

Nazarce-nazarcen baya sun ƙirga amsoshi: a yi tambaya ɗaya kusan sau ɗari, sannan a zana jadawalin mitoci (histogram). Wannan yana cakuɗa abin da samfurin da kansa ya fi so da abin da saitunan fitar da rubutu (decoding settings) ke yi masa, kuma zaɓe ɗari suna ɓoye ƙananan bayanai.

Ainihin yiwuwa, ba tare da jefa ɗan caca ba

Ga umarni (prompt) ɗaya tabbatacce, samfurin harshe (language model) yana ƙayyade yiwuwa ɗaya ga kowane ci gaban rubutu mai yiwuwa. Saboda haka maimakon ɗebo amsoshi, marubucin yana lissafa yiwuwar cewa samfurin zai rubuta kowace cikakkiyar lamba daga 1 zuwa N — sannan ya tsaya. Wannan mataki na ƙarshe yana da muhimmanci: idan babu shi, rubuta “1” da rubuta “17” ba za su zama amsoshi daban-daban ba, tunda ɗayan shi ne farkon ɗayan.

Umarnin koyaushe iri ɗaya ne: “Ba ni cikakkiyar lamba ta bazata tsakanin 1 da [N], har da su biyun. Amsa da lambar kawai.” Binciken ya ƙunshi iyalai 18 na samfurori a buɗe (Qwen, Gemma, Llama, OLMo da wasu, babu wanda ya wuce parameters biliyan 8), kewayo biyar daga 1–5 zuwa 1–100, saitunan fitar da rubutu 29 da kwamfutoci biyu daban-daban. Wani gwaji da amsoshi na gaske 5,000 da aka ɗebo daga samfuri ɗaya ya ba da lamba ɗaya da aka fi so, 15, da yiwuwar 0.579 a lissafi da 0.587 a ɗebo.

Daidaitawa tana matse yaɗuwar

Ma’aunin shi ne tazarar jimillar bambanci (total variation distance) daga zaɓe na adalci: kason yiwuwar da ke kan lambobin “da ba daidai ba”. Sifili yana nufin daidaito cikakke; tsakanin 1 da 30, mafi girma shi ne 0.967.

Ga samfurori 15, binciken yana kwatanta ɗanyar sigar “tushe” (base) da sigar da aka ƙara daidaitawa (fine-tuned) don bin umarni. A dukkan nau’i-nau’i 15, daidaitawar ta sa amsoshin su rasa daidaito. Gemma-3-4b tana tashi daga 0.135 zuwa 0.966, gashi ɗaya kawai daga mafi girma: idan masu binciken suka riga suka rubuta farkon amsarta a matsayin “Answer: “, tana ba da 17 da yiwuwa 1.000 (0.997 idan ta amsa cikin ’yanci). Wasu samfuran tushe kuwa, sun kusan zama na adalci: OLMo-2-7B yana kan 0.063.

Ɗabi’u uku

• Guje wa iyakoki. Samfuran tushe tuni suna ba da 1 ko lamba mafi girma sau 0.30 zuwa 0.60 kawai na yawan da suke ba da kowace lamba da ke tsakaninsu. Samfuran da aka daidaita kusan ba sa yi kwata-kwata.
• Guje wa lambobi masu zagaye — amma bayan daidaitawa kawai. Idan aka ajiye iyakokin a gefe, samfuran tushe suna ba ninkin 10 kasonsu na adalci. Samfuran da aka daidaita suna ba su 0.00 zuwa 0.12 na wannan kaso; tsakanin 1 da 30, raguwa ce ta ninki goma sha tara. Gwargwadon zagayewar lamba, gwargwadon ƙarfin gujewar.
• Karkata ga lambobi mara cika (odd), a fili a cikin samfuran da aka daidaita kawai tsakanin 1 da 100, inda 27 cikin tsare-tsare 33 da aka daidaita ke fifita su.

Tsakiyar kewayon kuwa, da ƙyar ake guje mata kwata-kwata.

Babu lambar da kowa ke so

Daidaitawa ba ta dasa lamba ɗaya ta sihiri a kowane samfuri. Lambobin da samfuran da aka daidaita suka fi so akai-akai su ne 3 (1 zuwa 5), 7 (1 zuwa 10), 14 (1 zuwa 20), 17 (1 zuwa 30) da 42 (1 zuwa 100) — daga nan ne sunan takardar ya fito — amma sama da 10, babu lambar da aka fi so da fiye da 42% na tsare-tsaren ke tarayya a kanta. Yiwuwar 17 tana hawa bayan daidaitawa a cikin nau’i-nau’i 10 kuma tana sauka a cikin 5. Qwen3-1.7B, wanda sigarsa ta tushe ke fifita 17, ya ƙare yana ɗora 0.993 a kan 15.

Kuma ba aikin mai fitar da rubutu (decoder) ba ne. Yanke amsoshi marasa yiwuwa, wani saiti da aka saba, yana ƙara tsaka-tsakin ƙima (median) na 0.035 kawai ga tazarar daga zaɓe na adalci: son kai (bias) yana cikin samfurin kansa.

An aro ɗabi’u, ba lambobi ba

A wani shahararren nazari kan zaɓin ɗan adam, Kubovy da Psotka sun tambayi masu wucewa 558 su faɗi lamba guda ɗaya (digit) daga 0 zuwa 9; idan aka haɗa da wasu nazarce-nazarce uku na baya, 28.4% na mutane 1,770 sun zaɓi 7. Bayaninsu, wanda suka gabatar a matsayin hasashe (conjecture), yana nuna mutane suna guje wa iyakoki, tsakiya, lambobi cikakku (even) da ninkin 3. Samfuran da aka daidaita suna tarayya cikakke a ɗabi’a ta farko da kuma a karkata ga lambobi mara cika a kewayo mafi girma, amma ba a tsakiya ba — kuma ɗabi’arsu mafi ƙarfi, guje wa lambobi masu zagaye, ba ta cikin bayanin na ɗan adam. Lambobin kansu sun dace da na ɗan adam sau ɗaya kawai: 7, tsakanin 1 da 10.

Kuskuren, marubucin ya jaddada, yana da tsari ne maimakon hayaniya (noise): samfurin da aka daidaita da ke zaɓi ba da dalili ba tsakanin lambobi yana kauce wa zaɓi na farko, na ƙarshe da masu zagaye. Shawarar aiki ta biyo baya — kada a yi amfani da samfurin harshe a matsayin na’urar samar da lambobin bazata; a zaɓi lambar a wajensa, sannan a miƙa sakamakon ga samfurin.