差别

这里会显示出您选择的修订版和当前版本之间的差别。

--- zh:notes:math_softwares [2026/06/26 19:14]
pzczxs [Count One- and Two- Grams for PMI Calculation]
+++ zh:notes:math_softwares [2026/07/07 20:29] (当前版本)
pzczxs [Extract Themes]
@@ 行 61: / 行 61: @@
 </code>
-Then, one- and two-grams statistics can be obtained by running the following code (cf. Project <color red>WikipediaTool</color>).
+Then, the statistics for single terms and term pairs can be obtained by running the following code (cf. Project <color red>WikipediaTool</color>).
 <code bash>
-> java -jar .\WikipediaTool.jar -t 0 -i ..\Wikipedia\ -o ..\Wikipedia-token\ > tokenization.log
+> java -jar WikipediaTool.jar -t 0 -i ..\Wikipedia\ -o ..\Wikipedia-token\ > tokenization.log
-> java -jar .\WikipediaTool.jar -t 1 -i ..\Wikipedia-token\ -o ..\Wikipedia-token-cleaned\ > cleaning.log
+> java -jar WikipediaTool.jar -t 1 -i ..\Wikipedia-token\ -o ..\Wikipedia-token-cleaned\ > cleaning.log
-> java -jar .\WikipediaTool.jar -t 2 -d ..\data\math_software.word.vocab -i ..\Wikipedia-token-cleaned\ -o ..\data\Wikipedia\ > extraction.log
+> java -jar WikipediaTool.jar -t 2 -d ..\data\math_software.word.vocab -w 10 -i ..\Wikipedia-token-cleaned\ -o ..\data\Wikipedia\ > extraction-terms.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 0 -u 1000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting0-1000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 0 -u 5000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting0-5000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 1000 -u 2000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting1000-2000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 5000 -u 10000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting5000-10000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 2000 -u 3000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting2000-3000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 10000 -u 20000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting10000-20000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 3000 -u 4000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting3000-4000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 20000 -u 30000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting20000-30000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 4000 -u 4000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting4000-4000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 30000 -u 40000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting30000-40000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 5000 -u 6000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting5000-6000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 40000 -u 86000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting40000-86000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 6000 -u 7000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting6000-7000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 7000 -u 8000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting7000-8000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 8000 -u 9000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting8000-9000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 9000 -u 10000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting9000-10000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 10000 -u 11000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting10000-11000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 11000 -u 12000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting11000-12000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 12000 -u 13000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting12000-13000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 13000 -u 14000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting13000-14000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 14000 -u 15000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting14000-15000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 15000 -u 20000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting15000-20000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 20000 -u 25000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting20000-25000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 25000 -u 30000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting25000-30000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 30000 -u 35000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting30000-35000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 35000 -u 40000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting35000-40000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 40000 -u 45000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting40000-45000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 45000 -u 50000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting45000-50000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 50000 -u 55000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting50000-55000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 55000 -u 60000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting55000-60000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 60000 -u 65000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting60000-65000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 65000 -u 70000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting65000-70000.log
-> java -Xmx25g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -l 70000 -u 86000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting70000-86000.log
->
+> java -jar WikipediaTool.jar -t 4 -d ..\Wikipedia-statistics\ -o ..\data\Wikipedia\ > merging.log
 </code>
@@ 行 104: / 行 83: @@
 <code bash>
-> java -jar .\GaussianATModelWithCredit.jar -D 50 -n 2000 -s 7 -fb math_software/math_software -K 200
+> java -Xmx26g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb math_software/math_software -K 100
 </code>
 ===== Calculate Diversity Indicators =====
 The following three diversity indicators are calculated: (1) Rao-Stirling ([[https://www.jstor.org/stable/25050293|Rao, 1982]]; [[https://doi.org/10.1098/rsif.2007.0213|Stirling, 2007]]), DIV ([[https://doi.org/10.1016/j.joi.2019.03.016|Leydesdorff et al., 2019]]), and Diversity ([[https://doi.org/10.1007/s11192-022-04336-3|Mutz, 2022]]).

硕风徐徐

用户工具

站点工具

差别

页面工具