zh:notes:math_softwares
差别
这里会显示出您选择的修订版和当前版本之间的差别。
| 两侧同时换到之前的修订记录 前一修订版 后一修订版 | 前一修订版 | ||
|
zh:notes:math_softwares [2026/04/10 11:21] pzczxs [Count One- and Two- Grams for PMI Calculation] |
zh:notes:math_softwares [2026/07/07 20:29] (当前版本) pzczxs [Extract Themes] |
||
|---|---|---|---|
| 行 61: | 行 61: | ||
| </code> | </code> | ||
| - | Then, one- and two-grams statistics can be obtained by running the following code (cf. Project <color red>WikipediaTool</color>). | + | Then, the statistics for single terms and term pairs can be obtained by running the following code (cf. Project <color red>WikipediaTool</color>). |
| <code bash> | <code bash> | ||
| - | > java -jar .\WikipediaTool.jar -t 0 -i ..\Wikipedia\ -o ..\Wikipedia-token\ > tokenization.log | + | > java -jar WikipediaTool.jar -t 0 -i ..\Wikipedia\ -o ..\Wikipedia-token\ > tokenization.log |
| - | > java -jar .\WikipediaTool.jar -t 1 -i ..\Wikipedia-token\ -o ..\Wikipedia-token-cleaned\ > cleaning.log | + | > java -jar WikipediaTool.jar -t 1 -i ..\Wikipedia-token\ -o ..\Wikipedia-token-cleaned\ > cleaning.log |
| - | > java -Xmx25g -jar .\WikipediaTool.jar -t 2 -d ..\data\math_software.word.vocab -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting.log | + | > java -jar WikipediaTool.jar -t 2 -d ..\data\math_software.word.vocab -w 10 -i ..\Wikipedia-token-cleaned\ -o ..\data\Wikipedia\ > extraction-terms.log |
| + | |||
| + | > java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 0 -u 5000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting0-5000.log | ||
| + | > java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 5000 -u 10000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting5000-10000.log | ||
| + | > java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 10000 -u 20000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting10000-20000.log | ||
| + | > java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 20000 -u 30000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting20000-30000.log | ||
| + | > java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 30000 -u 40000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting30000-40000.log | ||
| + | > java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 40000 -u 86000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting40000-86000.log | ||
| + | |||
| + | > java -jar WikipediaTool.jar -t 4 -d ..\Wikipedia-statistics\ -o ..\data\Wikipedia\ > merging.log | ||
| </code> | </code> | ||
| 行 74: | 行 83: | ||
| <code bash> | <code bash> | ||
| - | > java -jar .\GaussianATModelWithCredit.jar -D 50 -n 2000 -s 7 -fb math_software/math_software -K 200 | + | > java -Xmx26g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb math_software/math_software -K 100 |
| </code> | </code> | ||
| ===== Calculate Diversity Indicators ===== | ===== Calculate Diversity Indicators ===== | ||
| The following three diversity indicators are calculated: (1) Rao-Stirling ([[https://www.jstor.org/stable/25050293|Rao, 1982]]; [[https://doi.org/10.1098/rsif.2007.0213|Stirling, 2007]]), DIV ([[https://doi.org/10.1016/j.joi.2019.03.016|Leydesdorff et al., 2019]]), and Diversity ([[https://doi.org/10.1007/s11192-022-04336-3|Mutz, 2022]]). | The following three diversity indicators are calculated: (1) Rao-Stirling ([[https://www.jstor.org/stable/25050293|Rao, 1982]]; [[https://doi.org/10.1098/rsif.2007.0213|Stirling, 2007]]), DIV ([[https://doi.org/10.1016/j.joi.2019.03.016|Leydesdorff et al., 2019]]), and Diversity ([[https://doi.org/10.1007/s11192-022-04336-3|Mutz, 2022]]). | ||
zh/notes/math_softwares.1775791277.txt.gz · 最后更改: 2026/04/10 11:21 由 pzczxs