S · O NFDI4DS Ontology (nfdi4dso)

S · O dct:abstract

· O The NFDI4DS ontology (nfdi4dso) is an ontology describing various resources all resources (datasets, data providers, persons, projects and other entities) within the domain of NFDI4DataScience. nfdi4dso is a module that builds upon the NFDIcore Ontology (https://ise-fizkarlsruhe.github.io/nfdicore/2.0.0/) and maintains alignment with the Basic Formal Ontology (BFO).

S · O Shared Task SOTA

S · O dct:abstract

· O The central activity around empirical AI research includes automated tasks defined via a task dataset for which machine learning models are developed whose performance can be evaluated by a standard set of evaluation metrics. Pushing the state-of-the-art boundaries in empirical AI research means optimizing the models developed for the tasks in terms of speed, accuracy, or storage. As such researchers in this domain often seem to ask the central question “What’s the state-of-the-art result for task XYZ right now?” Instead of seeking out the answer buried in the ranked list of documents via a search query made on traditional search engines, researchers instead look for the answer on community-curated leaderboards such as https://paperswithcode.com/ or https://orkg.org/benchmarks. These leaderboards are websites specifically designed to showcase the performance of all introduced machine learning models on a machine learning task dataset. As such researchers seeking to find out the best model performance on a task dataset can easily obtain this information on these websites via their performance trendline overviews showcasing various model performances over a task dataset over time. In this Shared Task, we hope to go beyond the community curation of leaderboards and instead realize the vision of obtaining the most efficient machine learning model capable of automatically detecting leaderboards. The efficiency of the submitted machine learning models as a solution to the shared task will be tested based on speed, model parameters, and leaderboard detection F1 measure.

S · O Shared Task SOMD

S · O dct:abstract

· O Data-driven scientific processes strongly rely on the use of software to collect and prepare data and to generate insights via automated analysis. Hence, tracking the provenance of software artifacts is becoming an essential aspect of transparency and reproducibility. Additionally, aggregated observations of software citations can help to measure their usage and impact in the long run. While the referencing of scientific articles is handled according to well-established patterns, the citation practices of code bases and software programs are less coherent. Therefore, we invite participants of our shared task to develop robust supervised information extraction models that facilitate the disambiguation of software mentions and relevant metadata in scholarly publications. The task utilizes the Software Mentions in Science - SoMeSci knowledge graph of software mentions (Schindler et al., 2022). As a novelty presented with this task, SoMeSci will be extended to include more publications in the fields of Artificial Intelligence (AI) and Computer Science.

S · O Shared Task FORC

S · O dct:abstract

· O In recent years we have witnessed an explosion of published research across different fields of research. This brought forth an increasing difficulty for researchers to discover relevant literature that specifically caters to their needs, interests, and current research results. To find appropriate literature, researchers have to manually filter out many unrelated papers that are still being suggested by different scientific search engines. This shared task tackles this issue by introducing two subtasks. The first subtask aims to foster the development of single-label multi-class research field classifiers for about 50 general research fields; we will provide a benchmark dataset with different levels of label granularities based on the ORKG research fields taxonomy, as well as metadata and abstracts. The second subtask will specifically focus on different fields of research within data science and artificial intelligence. This subtask will introduce a different dataset and aims to model multi-label classifiers for more fine-grained labels in those research areas.