Conference Agenda

What is it that software sees, hears and perceives when technologies for pattern recognition are applied to media historical sources? All historical work requires interpretation, but what kind of algorithmic interpretations of modernity does software yield from historical archives? Modern Times 1936 is a research project funded by Sweden's second largest funding body (Riksbankens jubileumsfond) running between 2022 and 2025. The project involves four researchers, one developer and one master student (http://modernatider1936.se/en/). The project is empirically committed to everyday experiences and sets out to study how machines interpret symbols of modernity in media from the 1930s. By utilizing primarily photographic and audiovisual collections, the project seeks to analyze how modern Sweden was, while also exploring how computational methods can help us understand modernity in new ways.

We would like to propose a one-hour panel around some aspects of the research we have so far done within Modern Times 1936—with a focus on audiovisual media, foremost film and photography. Within our project we have collaborated with the heritage sector, assembling a dataset of some 80,000 photographs from the 1930s, scraped from the heritage portal DigitaltMuseum. We have also initiated a joint venture with the Nordic Museum in Stockholm. Regarding moving images we have used the media historical portal, filmarkivet.se—run by the National Library of Sweden and the Swedish Film Institute. We have used a number of speech-to-text-models on newsreels from the 1930s, but foremost upscaling algorithms for explorative film restoration. The latter work stems from previous film historical projects (Snickars 2015), as well as research done within, European History Reloaded: Curation and Appropriation of Digital Audiovisual Heritage, a EU-funded project that examined algorithmic ways of examining archival film reuse, and introducing a method for mapping video reuse with the help of AI and convolutional neural nets (Eriksson, Skotare & Snickars 2022). Hence, given the special theme of DHNB 2024 of collaboration with the heritage sector and involvement of ALM professionals alongside digital humanities research, we do think that a panel about Modern Times 1936 is a perfect match.

In our panel we will first make a short general presentation of Modern Times 1936—and yes, the Chaplin pun is intended. In short, our project explores how artificial intelligence and machine learning methods can foster new knowledge about the history of Swedish modernity—while at the same time critically scrutinizing algorithmic toolboxes for the study of the past. Within the panel we suggest to emphasize three particular strands that we have been working with: (1.) upscaling algorithms for film restoration, (2.) generative AI and pattern exploration within a major photographic dataset, and (3.) photographic super-resolution fantasies and the production of synthetic media.

(1.) Following a boom of user-friendly artificial intelligence tools in recent years, AI-enhanced (or manipulated) films have been framed as a serious threat to film archives. Film archivists are usually conservative; following their métier they are in the business of safeguarding film heritage. Today, however, the film archive—understood in a wide sense—is also elsewhere, most prominently online where each media asset becomes "at the instant of its release, an archive to be plundered, an original to be memorized, copied, and manipulated" (de Kosnik 2016). To explore these matters, trace and critically evaluate how algorithmic upscaling can modify older films, within our project we initiated a collaboration with Swedish AI artist ColorByCarl. He has been working with silent films from filmarkivet.se, and drawing on this collaboration we were able to study the procedures within the AI enhancement community, highlighting generative AI's potential to encourage reuse, remix and rediscovery of the filmic past.

(2.) Within our project we have been using pattern exploration within a major photographic dataset of some 80,000 photographs from the 1930s (taken from DigitaltMuseum). We have tagged each image with metadata categories using Doccano, and the general idea has been to train models to automatically find different types of images. Gender has been a case in point; two preliminary models can detect images with (or without) men and women with approximately 95 percent accuracy. We have also dealt with different kinds of object recognition models to track symbols of modernity such as factories, vehicles or cinemas. It has proven significantly more difficult, but we believe the work can be improved by using more older imagery as training data. The ambition of this work has been to develop models that can automatically annotate and sort larger visual heritage collections—and consequently we have initiated a collaboration with the Nordic Museum using their new collection 100,000 Bildminnen (image memories) as a case. On the one hand, we are developing models that can help the Nordic Museum to search and sort this collection in new ways, and on the other we are also interested in producing new images based on the same collection. Since Stable Diffusion is open source it can be trained on a specific dataset, such as 100,000 Bildminnen, and hence generate new historical photographs. Our aim is of course not to demonstrate that generative AI can picture the past in a better way. Rather, we believe that such a collaboration will open up new ways of understanding historical image collections.

(3.) Generative AI can indeed prove a useful tool to trace tropes and patterns in historical datasets (Offert & Bell 2020), and recent scholarship has also suggested that generative AI can offer new opportunities, not least in media history (Wilde 2023). Super-resolution technologies describe a set of computational methods for enhancing the resolution and/or sharpness of low-resolution digital visual content. Designed to fix what Hito Steyerl once referred to as poor images (Steyerl 2009), image upscaling is frequently promoted as a tool for improving visual content, as for example poorly digitized historical photographs. Yet what do super-resolution technologies actually do to visual and audiovisual content and make us see? Focusing on resolution technologies in real and imagined ways of enriching visual imagery, within our project we have explored how machine learning models are increasingly shaping ways of seeing, interpreting, and caring for historic photographs. Drawing from a series of experiments aimed at studying if/how super resolution technologies hallucinate and introduce new visual elements to historic photographs, we have explored how super resolution technologies unsettle boundaries between reality and fiction, hence both clarifying and occluding visions of the past.

Forthcoming publications:

Eriksson, Maria (2024) "On the Meaning of Scale in Image Upscaling", MontageAV (forthcoming)

Eriksson, Maria (2024), "Truthful Pixels: Synthetic Images the Measurement of Photorealism", Transbordeur (forthcoming)

Stjernholm, Emil & Snickars, Pelle (2024), "Upscaling Swedish Biograph", Journal of Scandinavian Cinema (forthcoming)

[Purpose and Methodology]

The purpose of this study is to construct a network of "people" and "data" that connects diverse digital cultural heritage with children's enriched learning.

To this aim, we propose "S×UKILAM (School × University, Kominkan (Regional Community Center), Industry, Library, Archives, Museum) collaboration" as a scheme for co-creating educational use of digital cultural heritage through collaboration between schools and various institutions that hold and release digital cultural resources.

[Workshop on "Creating Learning Materials" by utilizing digital cultural heritage]

At first, as a practical method of S×UKILAM collaboration, a workshop was designed to co-create "learning materialization" of diverse cultural heritage and assign metadata, and more than 10 workshops were held from the national scale to various local governments, universities, and libraries. As a result, 396 institutions from over 95% of the prefectures participated in these workshops.

The scheme has been scaled up and an international version of the S×UKILAM collaboration was held as a workshop to develop learning materials together with members of Europeana.

The international version of the workshop utilized the digital cultural heritage of Japan and Europe to create learning materials that would contribute to the learning of children in various countries from a global perspective.

[Questionnaire results]

The results of the questionnaire survey suggested the effectiveness of the workshop, as well as the issues that were revealed through the dialogue between participants from different backgrounds in utilizing cultural heritage. The importance of open licensing and the co-creation of "educational metadata" based on the perspective of the educational field were also indicated.

[Development archive of learning materials]

Over 100 unique learning materials co-created based on the workshops were licensed for secondary use and given back to society and the future by using the internationally interoperable IIIF (International Image Interoperability Framework). This has created a cycle in which new knowledge and information created by using the digital archive are returned to the digital archive again and has realized the flow of cultural resources that had been stocked and the construction of a scheme in which cultural resources are given new value and passed on to the future.

[Constructing Linked Open Data]

An LOD (Linked Open Data) model was also developed to connect and structure the learning materials based on digital cultural heritage created through this inter-organizational and inter-disciplinary collaboration with existing diverse information, digital collections, and curricula in a more machine-readable form. Specifically, we developed a Resource Description Framework (RDF) dataset and SPARQL endpoints using the archive of educational materials co-created through the "S×UKILAM collaboration. As a result, it was possible to construct a findable LOD model by connecting educational information such as curriculum guidebook codes and video contents by NHK and Japan Search in a highly machine-readable form. The model contributes to the informatization of education and the promotion of the use of digital cultural heritage by providing easy access to learning materials and a diverse educational information.

[Development of curation application for end users]

In addition, considering the difficulty of handling SPARQL, we also developed an application for end users. This application has succeeded in automatically curating and providing not only metadata assigned to learning materials but also related information and contents from the Web, with simple operations based on an easily understandable UI.

[Implications and Contributions]

The workflow provided by this research is highly versatile internationally and suggests a way to construct a network of "people" and "data" to advance the educational use of digital cultural heritage through the collaboration of DH research institutions, GLAM, and school education.

The Helsinki Term Bank for the Arts and Sciences (HTB) is an internationally unique Semantic MediaWiki-based platform for interdisciplinary cooperation and a discussion forum for society as a whole. It utilizes nichesourcing in which teams of experts from different disciplines are responsible for content creation with the aim of bringing all disciplines together in the same online service. This presentation focuses on the educational and societal goals of the HTB, lessons learned from collaborative projects between various disciplines, aspects of multilingualism, pedagogical applications and implications, technical challenges, and the potential for international cooperation.

Objectives

The objectives of the HTB serve both the scientific community and society as a whole. Thus, the aim is to bring together the current terminology of different disciplines in a single online service, thereby supporting the development of national and minority languages as languages of science and facilitating multilingualism in the academic world. The online service based on Semantic MediaWiki also enables a new kind of collaboration and co-authorship that can be particularly useful in multidisciplinary and interdisciplinary research. At the same time, it makes all scientific knowledge creation available to all interested parties, thus promoting fairness and equality in education and open science.

Current situation

The HTB has been in existence for ten years and its contents are still under construction. At present, it covers over 50 disciplines or specialty fields and over a thousand experts from different fields have contributed to it. There are over 45,000 concept pages or articles and over 300,000 terms with translation equivalents.

As a wiki, the HTB is a work in progress due to its continuous progress and evolution. Bringing in active experts is a prerequisite for expanding the content. At present, over a thousand experts are already involved in the work, but their activity varies. The incompleteness of the database requires patience from both users and experts.

The HTB is used by people of all ages from school children to retirees, although the largest user group consists of students. The HTB has a total of almost two million page views per year.

Utilizing the terminology in society

The opportunities to use the HTB affect society more broadly than just research and university education. As can be uncovered from user statistics, school students are an important user group. The HTB is important for the education system in the national languages. It involves developing specialized vocabulary in the national languages and, in general, opening up how scientific knowledge is constructed. Although not all terms are needed in educational materials, a broader background knowledge is important for authors of educational materials. The same applies to non-fiction and the popularization of science in general.

Materials at the HTB can be used freely and therefore different applications can use it and improve, for example, indirectly the quality of different language models. Societal activities are increasingly based on different electronic applications, and it is essential that they are compatible. It is essential that the terminology used for interoperability can be interpreted in the same way. HTB can serve as an aid to this.

It is also worth noting that the importance of reliable information, source criticism, and media literacy is emphasized at a time when attempts are being made to influence people's minds in a variety of ways: with fake news, inaccurate or misleading information, or with different beliefs. Knowledge that is based on research and is openly accessible meets a great need, and just as importantly, it opens up how knowledge is formed, and concepts emerge.

It is also worth noticing that there is a continuum of knowledge utilization between experts and laymen. The researcher is an expert in his or her own specialties. The further researchers move away from their discipline, the more they also lack the same kind of clear explanatory knowledge that shows how the knowledge has been generated. Scientific activity and the societal impact of science thus go hand in hand at the HTB.

Challenges

Scientific terminology can only be created by experts in different disciplines. However, experts are rarely terminologists and the limited voluntary input results in somewhat uneven and heterogeneous content. Furthermore, the Mediawiki experts need to understand what scientific experts are doing while the scientific experts need to understand what can be done on the platform.

The best situation is reached when terminologists and discipline experts work together. Wikis can be used as a database, content management systems, and publishing platforms, but the unique aspect is facilitating asynchronous collaboration via simple editing, discussion pages, and change histories.

One of the reasons Semantic MediaWiki was chosen for the HTB was that it allows for loose and dynamic data modeling that is accessible to people who are not programmers. In a way, we apply the iterative progress idea also to the data modeling, not only to the primary content.

Furthermore, nichesourcing has a self-correcting effect that works if a critical mass of researchers participates in the terminology work. However, in today’s academic world, everything ultimately depends on resources. Besides funding, insufficient working time is an obstacle or a delaying factor. Nonetheless, there is a solution to this: carrying out terminology work together with PhD students would integrate terminology work as part of teaching at the university. The cooperation has already begun with various doctoral programs in Finland.

Finally, terminology work corresponds fully to the three main tasks of higher learning at universities: research, teaching, and societal impact. The limited voluntary contributions are necessary for the terminology work, but they also present a challenge.

This paper discusses best practices in the development and operation of the network of Knowledge centres within the framework of the transnational and cross-disciplinary CLARIN Knowledge Infrastructure. A Knowledge centre is an institution or a collective of institutions which “have agreed to share their knowledge and expertise with others” (Branco et al. 2023), which has been certified by Common Language Resources and Technology Infrastructure CLARIN, and which has an established website providing the users with information on the offered services and contact forms. Currently, there are 28 Knowledge centres from different countries covering a wide range of topics, offering various research and training related services and organised as either a virtual centre of one or several institutional partners in one country or distributed across multiple countries. This paper aims to present this rich diversity of the research infrastructure and its state-of-the-art by reviewing existing Knowledge centres and their areas of expertise, describing the activities initiated by CLARIN to strengthen K-centre networking, showcasing different types of K-centres, and presenting the operation and collaboration practices shared by K-centres in their annual report for 2022-2023.

Introduction

The labour movement has facilitated the development of Swedish democracy, and has significantly shaped the Swedish labour market and welfare state system (e.g., Jansson, 2017). Correspondingly, the archives of the labour movement hold great value in many professional and scholarly areas where knowledge of historical events and processes is of importance – including but not limited to history, economics, political science, and trade union work. The main deliverable of the Labour's Memory (LM) project (2020–2023, with prolongation) is to digitise and make accessible annual and financial reports from the Swedish trade union organisations on a local, regional, national, and international level in a 1880-2020 timeframe.

The LM project team consists of archivists, librarians, and researchers from history and economic history, information studies, computational linguistics, and computerised image processing that work towards making the digitised documents available via a portal developed with a basis in the needs and preferences of key user groups. The portal will include features like robust document metadata, capable search features based on applications of computational linguistics including spelling normalization of historical text and named entity recognition (NER), transcriptions produced by the application of handwritten text recognition (HTR) and optical character recognition (OCR) techniques, and high-quality digital reproductions of selected archival holdings from the Swedish Labour Movement’s Archive and Library (ARAB), the Popular Movements’ Archive in Uppsala (FAC), the International Institute of Social History (IISH) and the Archive of Social Democracy (AdSD).

The purpose of this paper is to trace the main domains of knowledge and expertise represented in LM and to reflect on how they intersect in different elements of project work. With its focus on digitization and heritage dissemination driven by the collaborative efforts of multiple archival stakeholders and researchers from different domains, LM epitomises many of the characteristics and challenges of large-scale heritage projects in the digital humanities (Allington et al., 2016; Luhmann and Burghardt, 2021). Reflections and lessons learned from LM project experience have broader relevance in the digital humanities, specifically so for projects and other collaborative efforts where desired outcomes are dependent on co-work and communication across scholarly and GLAM communities.

Mapping the Labour’s Memory knowledge domains

Work in LM draws on expertise from several different research areas and organisations and is characterised by interdisciplinary, international, and inter-institutional exchanges. The project is distributed across four knowledge domains that are mapped and numbered below, alongside their respective contributions.

(1) At the core of LM lies the handling and digitisation of the annual and financial reports. This essential expertise is provided by the archival project partners, who implement the long-term storage and digitisation in accordance with the respective material’s requirements and limitations. Digitisation approaches range from in-house image acquisition, employing digital cameras and dedicated book scanners, to outsourcing via digitisation providers.

(2) The digital image acquisition is followed by OCR for printed and typewritten documents and implemented via off-the-shelf solutions and HTR for manuscripts. The HTR methods are developed in-house at FAC in cooperation with the Department of Information Technology, Uppsala University. OCR and HTR application yields machine-readable texts that enable full-text searches and subsequent processing steps, and make the contents more accessible to interested readers by removing the barrier of deciphering old handwritings or faded prints.

(3) The LM portal’s search capabilities and document interaction opportunities are further enriched through the efforts of computational linguists from the Department of Linguistics and Philology, Uppsala University. Firstly, spelling normalisation of historical texts allows users to search for words in their modern, standardised spelling and receive results that include older variations. And secondly, metadata, such as the names of persons and organisations, are extracted from the text via NER, making it possible for users to perform faceted searches and filterings, for example limiting a query to documents containing references to a specific organisation (Tudor and Pettersson, 2024).

(4) Development work in LM is supplemented by user-driven design methodologies. Researchers from the Department of ALM, Uppsala University, elicit feedback from representatives of vital user groups via a set of user studies (Sköld and Huvila, 2024). Collected insights, for example regarding potential user motivations and goals, inform e.g., the development and design of the portal and its search functionalities and underpinning metadata structures .

Discussion

LM is presently in its final phase, preparing the platform, local infrastructures and data for publication. At this point two intersections between the domains of knowledge and expertise represented in the project emerge as most impactful on LM work processes and results. The first key intersection is between archival and technical expertise, two areas that exist at the very core of LM and many other DH projects in the heritage data domain. Key lessons learnt include the importance of working deliberately not only towards un-siloing these knowledge domains within the project framework, but also to create the conditions for mutual cross-domain learning, inquiry, and exploration of issues to circumvent or solve.

The second intersection of knowledge domains with major impact on the course of LM work is that of user knowledge and the aggregate knowledge domain represented by the LM project team. The user studies conducted in LM confirm that representatives of pivotal user groups — researchers in history, literary studies, and the digital humanities and trade-union professionals — possess significant experience and insights of a practical and theoretical nature that can immensely assist in creating a portal that is successful in helping future users accomplish their goals and ambitions. Experience from LM shows that while it is easier to elicit and make use of user feedback at the early and late stages of development work, it is notably more difficult to do so at the midpoint of the development cycle. The main take-away with regards to user-driven design in DH projects with teams of multiple academic and GLAM stakeholders is to jointly identify areas where user feedback is most beneficial at an early stage, and then determine what tasks need to be accomplished to make the feedback possible to engender and use.

Labour's Memory is funded by Riksbankens Jubileumsfond under grant agreement IN20-0040.