Il 16/07/25 16:19, Daniele Gambetta ha scritto:
non so se era già passato, ma in questi giorni tra colleghi se ne parla molto.. https://aaai.org/aaai-launches-ai-powered-peer-review-assessment-system/ lì si legge
The pilot program will thoughtfully integrate LLM technology at two specific points in the established review process:
1. Supplementary First-Stage Reviews: LLM-generated reviews will be included as one component of the initial review stage, providing an additional perspective alongside traditional human expert evaluations. 2. Discussion Summary Assistance: LLMs will assist the Senior Program Committee (SPC) members by summarizing reviewer discussions, helping to highlight key points of consensus and disagreement among human reviewers.
riflessioni su questi temi erano già presenti nel 2019 in Beta Writer. /Lithium-ion batteries. A machine-generated summary of current research/. New York, NY: Springer, 2019. https://doi.org/10.1007/978-3-030-16800-1. nell'introduzione, scritta da Henning Schoenenberger:
machine-generated content makes it even more necessary to re-emphasize the crucial role of peer-review itself. Though peer-review is also in the course of being continuously re-defined (and in the future we expect to see substantial progress in machine-support also in this regard) we still think that for the foreseeable future we will need a robust human review process for machine-generated text. Especially in the area of Deep Learning it becomes increasingly difficult to understand how a result has been actually derived. While concepts such as Explainable Artificial Intelligence (XAI) become more and more crucial, also the review process on machine-generated research content needs refinement, if not a complete re-definition. The term peer itself indicates a certain inadequacy for machine-generated research content. Who are the peers in this context? Would you as a human reader consider yourself as peer to a machine? And should an expert in a specific research field become an expert of neural networks and Natural Language Processing as well in order to be able to evaluate the quality of a text and the related research? In the field of machine-summarization of texts this might not be an issue yet, especially since the underlying sources are peer-reviewed. However, soon enough we will see machine-generated texts from unstructured knowledge bases that will lead to more complex evaluation processes. Also in this area, we have to work together to find answers and define common standards related to machine-generated content. come Shönenberger credo che l'attività di review prodotta da un LLM non sia una peer review (l'LLM non è un peer dello studioso). ma credo che un sistema di IA possa compiere su input del reviewer delle operazioni di analisi testuale di supporto M
------------------------------------------------------------------------ non credo a nessuna liberazione né individuale né collettiva che si ottenga senza il costo di un’autodisciplina, di un’autocostruzione, di uno sforzo Italo Calvino ------------------------------------------------------------------------ Maurizio Lana Università del Piemonte Orientale Dipartimento di Studi Umanistici Piazza Roma 36 - 13100 Vercelli