If science is diverse, and no doubt it is, then the evaluation of scientific performance must also be diverse. Reply to Janus Toth’s comments.
Written by Attila Zoldos, Research Professor (Institute of History, Center for Research in the Humanities, Eötvös Loránd Research Network)
I read with interest on the surface of mandiner.hu on June 14, 2021 Will this really be the “golden team” of Hungarian science? – Janus Toth is a communications researcher at Mandiner Published an interview.
Much of the discussion revolved around the findings of research conducted by Janus Toth and Marton Demeter examining the publishing habits of researchers at several ‘institutes of the social sciences and humanities’. During the interview, there were also several other issues that were certainly very interesting as well, but my attention was mainly drawn to the parts related to the said investigation. Reading the text, I found myself again in the summer of 2018, when a series of similar writings on research decisions of the Hungarian Academy of Sciences was released a year later, and it is difficult to assess whether the current article is somewhat lagging behind whether it is a member or likely to be a member. be another pioneer.
The said study was done with science metrics tools, so the interview is about Scopus, MTMT, Q1 and the like, but Mandiner.hu readers don’t get any help interpreting it. This is not surprising, as it is very scientifically drawn up and is therefore well placed to give the uninformed and therefore unskeptical reader the impression that researchers at the respective institutes have little “international value performance” independent of regional informal aspects, most of whom “remain invisible world”, So the idea of ”don’t spend public money” to “maintain” a stock with such poor performance may justifiably arise. This last note is missing from the article presenting the research dedicated to science, but it is included in the interview published on mandiner.hu, which is understandable, as it is likely that more people read mandiner.hu than the newspaper that publishes the results of the research. You can mature, sure.
It’s a big problem because the research starts from a wrong basic assumption, uses wrong methodology, and uses data of highly questionable value, so its claimed results are useless. anything.
Before the above discussion, I would like to point out that I do not intend to deal with the affairs of all the institutes involved. I only turn to the Institute of History of the Research Center for the Humanities, which is justified by two circumstances. On the one hand, I know best unsurprisingly, since I work there, and on the other hand, this institute is often mentioned by commentator, either by name or by name, but it is still well known. This phenomenon is so amazing that one almost thinks of personal involvement.
wrong basic assumption أساسي
In the research of Janus Toth and Marton Demeter, he used Hungarian (Hungarian Library of Scientific Works, MTMT) and an international bibliographic and scientific database (Scopus), all in the belief that the data obtained from them could be used to infer an “internationally valuable performance”. “.
MTMT covers Hungarian scientific researchers relatively well, whether they work at a research institute, university or elsewhere, but Scopus, Elsevier’s service for publishing scientific books and journals based in the Netherlands, primarily treats the publisher’s journals from a scientific measurement point of view. Journals are mainly in English, which you consider worthwhile in light of commercial considerations.
Journals with a majority of Hungarian history do not usually belong to these journals. It is therefore not possible to form a reliable picture of the performance of Hungarian historians, including those working at the Institute of History, on the basis of Scopus. The situation would not change much, as there is a strong belief in historical science that a scholarly publication has value in itself (or has no value), regardless of the place of publication, which is not usually chosen in my profession according to the supposed prestige of a journal. Historical science is quite averse to highlighting a journal’s presumed status over a study published in it simply because statistical probabilities provide an opportunity to do so. Because basically this is the case: the Q1 rating already belongs to the journal, but the writing published in it receives that recognition once it is published, after which it either justifies the expected confidence or not, is no longer of much interest to anyone.
The vast majority of Hungarian historians deal with Hungarian history. This fact can, of course, be seen as a sign of self-containment, the inability to go beyond national frameworks, in short, the provinces of Hungarian-Ugric, but it is not worth it. For this, the simple truth must be ignored: there is nothing more obvious than that of a galactic historian dealing with galactic history. Who else would do that? Although there are always some fellows in some remote part of the world who can’t resist the temptation of the galactic world that seems foreign to them, there is little to be expected of them other than crafting the outer aspects that are always worth keeping in mind. In our narrow geographical environment, of course, it is more common for galactic history to be approached by a non-Hungarian scholar, but reading their work it is often difficult to realize that it is in fact galactic history if they make visible efforts to survive. Within the framework of historical sciences.
It is also advisable to keep in mind that for Hungarian historians, the Carpathian Basin and its immediate surroundings are international terrain. Why, let’s see, a successful course of history may suffice, and then it becomes clear why you and my colleague at the Institute worked on a series of scientific books in Oradea, volumes of which have already been published in Romanian. The fact that history is a national flag is not some kind of excuse to hide readiness, but a reality, and not only in Central or Eastern Europe. It follows that the international language for the study of Hungarian history, as strange as it may seem, is Hungarian. It is customary to suspect this only at home, in Slovakia, for example, Hungarian has always been a mandatory foreign language for college students majoring in history (and probably still is today).
The main burden of scientific standards, as I see it, is that while the quality of the engraving can be read in the flaming red letters on its flag, in practice it is still forced to account for the quantities produced by various imaginative methods.
The contradiction is not resolvable, but its edge can be faint. Scientific standard analysis can only be taken seriously if it uses as many types of data as possible to be able to capture performance in its diverse realities and thus draw sound conclusions. An “analysis” which is limited to one scientific measurement element always raises the suspicion that its only role is to support the conclusion already reached, especially if the element least relevant to the object of study can be identified, as in our case.
The mention of Q1 essays in the interview invokes a publication strategy typical of the life sciences and natural sciences for historians, completely ignoring fundamental differences between disciplines. As a result, when the Demeter-Tóth authors balance my science performance between 2014 and 2018 just by looking at the supposed standing values of journals, they simply ignore 90.8% of my published work in those years, while still evaluating my overall performance. It is an unfounded opinion. However, there are also a large number of references to my writings in Hungarian in the works of foreign (not Hungarian) scholars, which is sufficient evidence that the question of “international vision” is somewhat more complex than the authors suppose.
Very questionable data
In the interview, Janus Toth stated that although they received “subtle criticism in social media” for their communications dedicated to science, “that’s it, no one questioned the data.”
I certainly don’t argue, but that situation will change now. At the end of the article published in their own magazine, in response to a question from a journalist, the raw data “Breaked by person, MTA Institute, position” was also published. This could be a mistake, it was worth consulting a lawyer about this, anyway to give me the opportunity to check the accuracy of my data.
In my case, two publications that could be evaluated in a regional context in Central and Eastern Europe were taken into account, nothing else (Q1-4 article or book chapter). I tried to identify my lucky post but it didn’t work. It could be one of my studies published in Historický Časopis in 2015, but at the time of publication, watch the miracle! , got a Q2 rating, so the study should have been in that column, but no trace of it. If one, after all, is the journal of the Institute of History of the Slovak Academy of Sciences, that is, the territory of Central Europe is indisputable, then it can still be the other. Obviously, these are not my studies published in Romania – in study volumes and in Romanian – because three of them were published during the study period. Nor can I write my article in the 2017 Melanges de l’Ecole Francaise de Rome, because it would be difficult to consider the journal of the French Institute in Rome of Central European importance, although it turns out that I am starting to become uncertain. By then my article in a Slovak study volume might be more appropriate, but the blank in the book chapter still awaits an answer, even though a chapter I wrote in 2016 has been published in the online version of one of the German language handbooks. Leibniz-Institut für Ost- und Südosteuropaforschung.
mandiner.hu readers are tired of my own posts because I control my own data, however, each of my institutional colleagues can give similar comments on the data used, because “that’s the way it is now, that’s gas. Until it freezes.”
It is certainly a fun hobby to play with science metrology, but playing as it is well known is worth it, so it is not advisable to ignore professionalism in the field of science metrology either. The lesson could be at most: If science is diverse, and it is no doubt so, then the evaluation of scientific performance must also be diverse. All other cheap games…