差别

这里会显示出您选择的修订版和当前版本之间的差别。

--- zh:notes:math_softwares [2026/07/07 10:07]
pzczxs [Extract Themes]
+++ zh:notes:math_softwares [2026/08/24 10:15] (当前版本)
pzczxs [Obtain Embedding from a Vector Model]
@@ 行 34: / 行 34: @@
 The citing articles can be exported for Gaussian AT<sup>credit</sup> model by running <color red>ToGaussianATCreditConverter.java</color> in the pakcage <color red>cn.edu.bjut.converter</color>.
+==== Obtain Embedding from a Vector Model ====
+The //text-embedding-v4// model is employed to produce a 64-demensional embedding vector for each word.
+<code bash>
+> python .\extract_embedding.py --model text-embedding-v4 --dimension 64 --batch-size 10 --output text-embedding-v4-64.txt --resume .\math_software.word.vocab
+</code>
 ==== Count One- and Two- Grams for PMI Calculation ====
 [[https://dumps.wikimedia.org/backup-index.html|Wikepedia dumps]] (version: 2026-03-01) should be downloaded in advance.
@@ 行 63: / 行 69: @@
 Then, the statistics for single terms and term pairs can be obtained by running the following code (cf. Project <color red>WikipediaTool</color>).
 <code bash>
-> java -jar .\WikipediaTool.jar -t 0 -i ..\Wikipedia\ -o ..\Wikipedia-token\ > tokenization.log
+> java -jar WikipediaTool.jar -t 0 -i ..\Wikipedia\ -o ..\Wikipedia-token\ > tokenization.log
-> java -jar .\WikipediaTool.jar -t 1 -i ..\Wikipedia-token\ -o ..\Wikipedia-token-cleaned\ > cleaning.log
+> java -jar WikipediaTool.jar -t 1 -i ..\Wikipedia-token\ -o ..\Wikipedia-token-cleaned\ > cleaning.log
-> java -jar .\WikipediaTool.jar -t 2 -d ..\data\math_software.word.vocab -w 10 -i ..\Wikipedia-token-cleaned\ -o ..\data\Wikipedia\ > extraction-terms.log
+> java -jar WikipediaTool.jar -t 2 -d ..\data\math_software.word.vocab -w 10 -i ..\Wikipedia-token-cleaned\ -o ..\data\Wikipedia\ > extraction-terms.log
-> java -Xmx26g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 0 -u 5000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting0-5000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 0 -u 5000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting0-5000.log
-> java -Xmx26g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 5000 -u 10000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting5000-10000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 5000 -u 10000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting5000-10000.log
-> java -Xmx26g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 10000 -u 20000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting10000-20000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 10000 -u 20000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting10000-20000.log
-> java -Xmx26g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 20000 -u 30000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting20000-30000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 20000 -u 30000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting20000-30000.log
-> java -Xmx26g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 30000 -u 40000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting30000-40000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 30000 -u 40000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting30000-40000.log
-> java -Xmx26g -jar .\WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 40000 -u 86000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting40000-86000.log
+> java -Xmx26g -jar WikipediaTool.jar -t 3 -d ..\data\math_software.word.vocab -w 10 -l 40000 -u 86000 -i ..\Wikipedia-token-cleaned\ -o ..\Wikipedia-statistics\ > counting40000-86000.log
-> java -jar .\WikipediaTool.jar -t 4 -d ..\Wikipedia-statistics\ -o ..\data\Wikipedia\ > merging.log
+> java -jar WikipediaTool.jar -t 4 -d ..\Wikipedia-statistics\ -o ..\data\Wikipedia\ > merging.log
 </code>
@@ 行 82: / 行 88: @@
 Project: <color red>GaussianATModelWithCredit</color>
+==== GloVe Embedding ====
+<code bash>
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb data/math_software/10/math_software -K 10
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb data/math_software/20/math_software -K 20
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb data/math_software/30/math_software -K 30
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb data/math_software/40/math_software -K 40
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb data/math_software/50/math_software -K 50
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb data/math_software/60/math_software -K 60
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb data/math_software/70/math_software -K 70
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb data/math_software/80/math_software -K 80
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb data/math_software/90/math_software -K 90
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -n 50 -sc 7 -fb data/math_software/100/math_software -K 100
+</code>
+==== text-embedding-v4 ====
 <code bash>
-> java -Xmx26g -jar GaussianATCreditModel.jar -sf -D 50 -e resources/glove.2024.wikigiga.50d.zip -i 500 -n 50 -sc 7 -fb math_software/math_software -K 100
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 256 -e resources/qwen3.7-embeddings.zip -n 50 -sc 7 -fb data/math_software/10/math_software -K 10
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 256 -e resources/qwen3.7-embeddings.zip -n 50 -sc 7 -fb data/math_software/20/math_software -K 20
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 256 -e resources/qwen3.7-embeddings.zip -n 50 -sc 7 -fb data/math_software/30/math_software -K 30
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 256 -e resources/qwen3.7-embeddings.zip -n 50 -sc 7 -fb data/math_software/40/math_software -K 40
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 256 -e resources/qwen3.7-embeddings.zip -n 50 -sc 7 -fb data/math_software/50/math_software -K 50
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 256 -e resources/qwen3.7-embeddings.zip -n 50 -sc 7 -fb data/math_software/60/math_software -K 60
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 256 -e resources/qwen3.7-embeddings.zip -n 50 -sc 7 -fb data/math_software/70/math_software -K 70
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 256 -e resources/qwen3.7-embeddings.zip -n 50 -sc 7 -fb data/math_software/80/math_software -K 80
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 256 -e resources/qwen3.7-embeddings.zip -n 50 -sc 7 -fb data/math_software/90/math_software -K 90
+> java -Xmx20g -jar GaussianATCreditModel.jar -sf -D 256 -e resources/qwen3.7-embeddings.zip -n 50 -sc 7 -fb data/math_software/100/math_software -K 100
 </code>
 ===== Calculate Diversity Indicators =====
 The following three diversity indicators are calculated: (1) Rao-Stirling ([[https://www.jstor.org/stable/25050293|Rao, 1982]]; [[https://doi.org/10.1098/rsif.2007.0213|Stirling, 2007]]), DIV ([[https://doi.org/10.1016/j.joi.2019.03.016|Leydesdorff et al., 2019]]), and Diversity ([[https://doi.org/10.1007/s11192-022-04336-3|Mutz, 2022]]).

硕风徐徐

用户工具

站点工具

差别

页面工具