这里会显示出您选择的修订版和当前版本之间的差别。
两侧同时换到之前的修订记录 前一修订版 后一修订版 | 前一修订版 | ||
zh:notes:disruptive_index [2025/01/17 22:08] pzczxs [Update Publication Year] |
zh:notes:disruptive_index [2025/03/11 09:00] (当前版本) pzczxs [Download Data] |
||
---|---|---|---|
行 52: | 行 52: | ||
==== Import Citations ==== | ==== Import Citations ==== | ||
The citations can be imported to the database by running <color red>CitationImporter.java</color> in the package <color red>cn.edu.bjut.open.citations</color>. | The citations can be imported to the database by running <color red>CitationImporter.java</color> in the package <color red>cn.edu.bjut.open.citations</color>. | ||
+ | |||
+ | <code bash> | ||
+ | > nohup ./import-citations-opencitations.sh 1 OpenCitations/1/target_articles > opencitations-target-1.log 2>&1 | ||
+ | > nohup ./import-citations-opencitations.sh 2 OpenCitations/2/target_articles > opencitations-target-2.log 2>&1 | ||
+ | > nohup ./import-citations-opencitations.sh 3 OpenCitations/3/target_articles > opencitations-target-3.log 2>&1 | ||
+ | > nohup ./import-citations-opencitations.sh 4 OpenCitations/4/target_articles > opencitations-target-4.log 2>&1 | ||
+ | > nohup ./import-citations-opencitations.sh 1 OpenCitations/1/cited_articles > opencitations-cited-1.log 2>&1 | ||
+ | > nohup ./import-citations-opencitations.sh 2 OpenCitations/2/cited_articles > opencitations-cited-2.log 2>&1 | ||
+ | > nohup ./import-citations-opencitations.sh 3 OpenCitations/3/cited_articles > opencitations-cited-3.log 2>&1 | ||
+ | > nohup ./import-citations-opencitations.sh 4 OpenCitations/4/cited_articles > opencitations-cited-4.log 2>&1 | ||
+ | </code> | ||
==== Import Metadata ==== | ==== Import Metadata ==== | ||
The metadata can be imported to the database by running <color red>MetadataImporter.java</color> in the package <color red>cn.edu.bjut.open.citations</color>. | The metadata can be imported to the database by running <color red>MetadataImporter.java</color> in the package <color red>cn.edu.bjut.open.citations</color>. | ||
+ | |||
+ | <code bash> | ||
+ | > nohup ./import-metadata-opencitations.sh 1 OpenCitations/1/citing_articles > opencitations-citing-1.log 2>&1 | ||
+ | > nohup ./import-metadata-opencitations.sh 2 OpenCitations/2/citing_articles > opencitations-citing-2.log 2>&1 | ||
+ | > nohup ./import-metadata-opencitations.sh 3 OpenCitations/3/citing_articles > opencitations-citing-3.log 2>&1 | ||
+ | > nohup ./import-metadata-opencitations.sh 4 OpenCitations/4/citing_articles > opencitations-citing-4.log 2>&1 | ||
+ | </code> | ||
===== Dimensions ===== | ===== Dimensions ===== | ||
==== Download Data ==== | ==== Download Data ==== | ||
- | TODO | + | For each domain (e.g., SYNTHETIC BIOLOGY), the procedure begins by querying the Dimensions API with DOIs from a specified list (e.g., doi_list1.csv) to retrieve target articles’ metadata (ID, title, authors, year, journal) and their referenced publications (referenced_pubs), saving results as JSON files in the target_articles folder by running <color red>dimensions_retrieve.ipynb</color>. Next, reference IDs extracted from the referenced_pubs field are used to fetch metadata for cited articles, stored in the cited_articles folder by running <color red>dimensions_references.ipynb</color>. Finally, a combined list of IDs from target and cited articles is queried to identify citing articles (those whose reference_ids include any of the input IDs), with outputs saved to the citing_articles folder by running <color red>dimensions_citations.ipynb</color>. |
==== Import Target Articles with Backward Citations ==== | ==== Import Target Articles with Backward Citations ==== | ||
Import the target articles with the resulting backward citations to the database by running <color red>TargetArticleImporter.java</color> in the package <color red>cn.edu.bjut.dimensions</color>. | Import the target articles with the resulting backward citations to the database by running <color red>TargetArticleImporter.java</color> in the package <color red>cn.edu.bjut.dimensions</color>. | ||
+ | |||
+ | <code bash> | ||
+ | > nohup ./import-dimensions.sh 1 Dimensions/1/target_articles 1 > dimensions-target-1.log 2>&1 | ||
+ | > nohup ./import-dimensions.sh 2 Dimensions/2/target_articles 1 > dimensions-target-2.log 2>&1 | ||
+ | > nohup ./import-dimensions.sh 3 Dimensions/3/target_articles 1 > dimensions-target-3.log 2>&1 | ||
+ | > nohup ./import-dimensions.sh 4 Dimensions/4/target_articles 1 > dimensions-target-4.log 2>&1 | ||
+ | </code> | ||
==== Import Cited Articles ==== | ==== Import Cited Articles ==== | ||
Import the cited articles to the database by running <color red>CitedArticleImporter.java</color> in the package <color red>cn.edu.bjut.dimensions</color>. | Import the cited articles to the database by running <color red>CitedArticleImporter.java</color> in the package <color red>cn.edu.bjut.dimensions</color>. | ||
+ | |||
+ | <code bash> | ||
+ | > nohup ./import-dimensions.sh 1 Dimensions/1/cited_articles 2 > dimensions-cited-1.log 2>&1 | ||
+ | > nohup ./import-dimensions.sh 2 Dimensions/2/cited_articles 2 > dimensions-cited-2.log 2>&1 | ||
+ | > nohup ./import-dimensions.sh 3 Dimensions/3/cited_articles 2 > dimensions-cited-3.log 2>&1 | ||
+ | > nohup ./import-dimensions.sh 4 Dimensions/4/cited_articles 2 > dimensions-cited-4.log 2>&1 | ||
+ | </code> | ||
==== Import Citing Articles with Backward Citations ==== | ==== Import Citing Articles with Backward Citations ==== | ||
Import the citing articles with the resulting backward citations to the database by running <color red>CitingArticleImporter.java</color> in the package <color red>cn.edu.bjut.dimensions</color>. | Import the citing articles with the resulting backward citations to the database by running <color red>CitingArticleImporter.java</color> in the package <color red>cn.edu.bjut.dimensions</color>. | ||
+ | |||
+ | <code bash> | ||
+ | > nohup ./import-dimensions.sh 1 Dimensions/1/citing_articles 3 > dimensions-citing-1.log 2>&1 | ||
+ | > nohup ./import-dimensions.sh 2 Dimensions/2/citing_articles 3 > dimensions-citing-2.log 2>&1 | ||
+ | > nohup ./import-dimensions.sh 3 Dimensions/3/citing_articles 3 > dimensions-citing-3.log 2>&1 | ||
+ | > nohup ./import-dimensions.sh 4 Dimensions/4/citing_articles 3 > dimensions-citing-4.log 2>&1 | ||
+ | </code> | ||
<!-- | <!-- | ||
行 111: | 行 150: | ||
The citing articles with the resulting backward citations can be imported to the database by running <color red>CitingArticleImporter.java</color> in the package <color red>cn.edu.bjut.wos</color>. | The citing articles with the resulting backward citations can be imported to the database by running <color red>CitingArticleImporter.java</color> in the package <color red>cn.edu.bjut.wos</color>. | ||
<code bash> | <code bash> | ||
- | > nohup ./import-wos.sh 1 WoS/1/citing_articles > 1.log 2>&1 | + | > nohup ./import-wos.sh 1 WoS/1/citing_articles > wos-citing-1.log 2>&1 |
- | > nohup ./import-wos.sh 2 WoS/2/citing_articles > 2.log 2>&1 | + | > nohup ./import-wos.sh 2 WoS/2/citing_articles > wos-citing-2.log 2>&1 |
- | > nohup ./import-wos.sh 3 WoS/3/citing_articles > 3.log 2>&1 | + | > nohup ./import-wos.sh 3 WoS/3/citing_articles > wos-citing-3.log 2>&1 |
- | > nohup ./import-wos.sh 4 WoS/4/citing_articles > 4.log 2>&1 | + | > nohup ./import-wos.sh 4 WoS/4/citing_articles > wos-citing-4.log 2>&1 |
</code> | </code> | ||
行 129: | 行 168: | ||
> UPDATE article SET doi = "10.3389/FPLS.2016.00706" WHERE doi = "10.3389/F,OLS.2016.00706"; | > UPDATE article SET doi = "10.3389/FPLS.2016.00706" WHERE doi = "10.3389/F,OLS.2016.00706"; | ||
> UPDATE article SET doi = "10.1016/J.GEB.2019.07.003" WHERE doi = "10.1016/J,GEB.2019.07.003"; | > UPDATE article SET doi = "10.1016/J.GEB.2019.07.003" WHERE doi = "10.1016/J,GEB.2019.07.003"; | ||
+ | > UPDATE article SET doi = "10.1017/S0140525X21001370" WHERE doi = "10.1017/S0140525X21001370,E120"; | ||
+ | > UPDATE article SET doi = "10.1007/978-3-030-68386-3_18" WHERE doi = "10.1007/978-3-030-68,38,6-3_18"; | ||
+ | > UPDATE article SET doi = "10.1016/J.MULFIN.2018.06.001" WHERE doi = "10.1016/J.MULFIN,2018.06.001"; | ||
+ | > UPDATE article SET doi = "10.1007/978-3-030-68386-3_8" WHERE doi = "10.1007/978-3-030-68,38,6-3_8"; | ||
+ | > UPDATE article SET doi = "10.1002/CPT.1619" WHERE doi = "10.1002/CPT.1619MASSACHUSETTS,USA.*"; | ||
</code> | </code> | ||
+ | <!-- | ||
<code bash> | <code bash> | ||
> ./merge-article-doi-wos.sh > merge-article-doi.log | > ./merge-article-doi-wos.sh > merge-article-doi.log | ||
</code> | </code> | ||
+ | --> | ||
Several cited articles are attached with multiple DOI numbers. The cited articles with multiple DOI names can be resolved by running <color red>CitedArticleMultipleDoiResolver.java</color> in the package <color red>cn.edu.bjut.wos</color>. Note that this operation needs to access the [[https://www.doi.org/the-identifier/resources/factsheets/doi-resolution-documentation|DOI parser]]. | Several cited articles are attached with multiple DOI numbers. The cited articles with multiple DOI names can be resolved by running <color red>CitedArticleMultipleDoiResolver.java</color> in the package <color red>cn.edu.bjut.wos</color>. Note that this operation needs to access the [[https://www.doi.org/the-identifier/resources/factsheets/doi-resolution-documentation|DOI parser]]. | ||
行 207: | 行 253: | ||
<code bash> | <code bash> | ||
> nohup ./update-doi-publication-year.sh > update-doi-publication-year.log 2>&1 | > nohup ./update-doi-publication-year.sh > update-doi-publication-year.log 2>&1 | ||
- | > nohup ./check-doi-publication-year.sh > check-doi-publication-year-2.log 2>&1 | + | > nohup ./check-doi-publication-year.sh > check-doi-publication-year.log 2>&1 |
</code> | </code> | ||
+ | About 1500 citing articles are not attached any publication year at all. In this case, we can supplement the resulting publication years of these citing articles, and save them in the Excel file <color red>doi-publication-year20250110.xlsx</color>. | ||
<code bash> | <code bash> | ||
> nohup ./update-doi-publication-year-with-excel.sh > update-doi-publication-year-with-excel.log 2>&1 | > nohup ./update-doi-publication-year-with-excel.sh > update-doi-publication-year-with-excel.log 2>&1 | ||
> nohup ./check-doi-publication-year.sh > check-doi-publication-year.log 2>&1 | > nohup ./check-doi-publication-year.sh > check-doi-publication-year.log 2>&1 | ||
</code> | </code> | ||
+ | |||
+ | Note that 23 citing DOI names actually point to a journal, not a publication. It is very difficult to assign a publication year to these DOI names. Hence, we fix the resulting publication year of these DOIs to null. | ||
==== Normalization ==== | ==== Normalization ==== | ||
Before normalization, a global DOI set is generated by running <color red>DoiUpdater.java</color> in the package <color red>cn.edu.bjut.indices</color>. Then, the resulting publication years are updated by running <color red>DoiPublicationYearUpdater.java</color> in the package <color red>cn.edu.bjut.indices</color>. | Before normalization, a global DOI set is generated by running <color red>DoiUpdater.java</color> in the package <color red>cn.edu.bjut.indices</color>. Then, the resulting publication years are updated by running <color red>DoiPublicationYearUpdater.java</color> in the package <color red>cn.edu.bjut.indices</color>. | ||
The citations from Web of Science, Dimensions, and OpenCitations can be normalized by running <color red>CitatoinUpdater.java</color>, <color red>CitatoinDimensionsUpdater.java</color>, and <color red>CitatoinOpenCitationsUpdater.java</color> respectively in the package <color red>cn.edu.bjut.indices</color>. | The citations from Web of Science, Dimensions, and OpenCitations can be normalized by running <color red>CitatoinUpdater.java</color>, <color red>CitatoinDimensionsUpdater.java</color>, and <color red>CitatoinOpenCitationsUpdater.java</color> respectively in the package <color red>cn.edu.bjut.indices</color>. | ||
+ | |||
+ | <code base> | ||
+ | > ./update-doi-with-excel.sh > update-doi-with-excel.log | ||
+ | > ./summary.sh > summary.log | ||
+ | </code> | ||
==== Calculate Disruptive Index ==== | ==== Calculate Disruptive Index ==== | ||
+ | Before calculation, the related data can be exported by running <color red>CitationExporter.java</color> in the package <color red>cn.edu.bjut.ui</color>. | ||
+ | <code bash> | ||
+ | > ./export-citations.sh | ||
+ | </code> | ||
~~DISCUSSION:closed~~ | ~~DISCUSSION:closed~~ |