Conference Agenda

Applying AI methods to digitised map collections around the world has emerged as a major area of interest, supporting library and archive goals of improving collection discovery and reuse (McDonough, 2024; McDonough and Vitale, 2024). The timely intersection of web mapping, imaging, and computer vision technologies with maturing IIIF infrastructure and digital humanities expertise created the foundation for projects like Allmaps (https://allmaps.org/), Machines Reading Maps (MRM, https://machines-reading-maps.github.io/), and MapReader (https://github.com/maps-as-data/MapReader, Hosseini et al, 2022). In particular, MRM’s efforts to automatically transcribe text on maps were a key demonstration of how AI experiments in libraries are also useful to scholarly communities in other disciplines: text on maps is a major new way of exploring the history of cartography, landscapes, and societies (McDonough et al, 2024; Nelson, 2024). Allmaps and MapReader are now linking up the data, software, and underlying infrastructures of these projects to further support open map research and collection discovery.

Using the mapKurator text spotting pipeline (now PALETTE, Lin and Chiang, 2024), MRM created data for all the text printed on 56,628 georeferenced maps in the David Rumsey Historical Map Collection, radically transforming how students, researchers, and the general public can search the collection (Vitale, 2023; Larsen et al, 2024). More recently, the open-source software MapReader has implemented text spotting as well as support for IIIF input (https://mapreader.readthedocs.io/en/latest/using-mapreader/step-by-step-guide/1-download.html#downloading-maps-from-iiif-servers), aiming to create accessible, and humanities-oriented computer vision research tools (Wood et al, 2024; Coleman, 2024). For Fantastic Futures 2025, we will share recent work to lower barriers to working with the output of text spotting, best practices in data interoperability, and examples of research enabled by these efforts.

Text spotting outputs represent visual, textual, and spatial data simultaneously, but with new kinds of errors that curators and scholars are just now beginning to understand: viewing these outputs on a map is an essential part of devising evaluation metrics, and formulating research questions based on imperfect data. We are therefore developing a new IIIF-friendly approach to formatting text spotting output and simplifying it for browser-based visualisation. This turns the inferred outputs from AI methods into accessible, explorable data useful in both library discovery interfaces and in research applications.

Text spotting output from mapKurator for the Rumsey collection was delivered as 56,628 geojson files (1 per map) collectively containing 111,072,386 instances of text (zipped 51.65 GB). (3 versions of the data exist: versions 1 and 2 are openly available via the Stanford Digital Repository (Machines Reading Maps Collection, 2023); version 3 will be released alongside a forthcoming publication. Work here is based on version 3.) Each filename refers to a UID in the Rumsey metadata, but no metadata was contained in these output files. This data was added alongside the Rumsey catalog to enable “Text on Maps”-based searching of the Rumsey collection. However, because of the size of this data, and the absence of metadata, the data was very difficult to evaluate or analyze.

If the Rumsey site facilitates initial exploration and map-by-map viewing, we now need more robust tools for viewing,reading, filtering, and exporting across maps based on metadata and other data-driven criteria. To work towards this goal, we used Allmaps tools to transform each instance of text in the original geojson mapKurator output into IIIF Georeference Annotations (https://iiif.io/api/extension/georef/). Next, we combined annotations from all sheets into a single composite JSON file which also includes selected metadata (e.g. title, creator, publication date). We then transformed the JSON data into vector tiles. With Allmaps, it is now possible to plot and explore this data on a web map, but also to plot the text annotations in their correct pixel locations on top of the original scanned map image: pixel coordinates can be transformed into geospatial coordinates, and vice versa. Examples of this interoperability are viewable at https://observablehq.com/d/f5b83904983ce90b and https://observablehq.com/@allmaps/rumsey (van Wissen et al, 2025). A significant advantage of this approach is that if the georeferencing metadata is updated in the future, the text spotting data can also be easily updated. All new data will be published shortly on Zenodo and code is available at https://github.com/allmaps/text-on-maps-viz.

One aim of this work is that the results of this data curation work will serve as a model for future text spotting output formatting: going forward we plan to implement IIIF georeference annotations as a standard output option for the text spotting task in MapReader. We are embedding open standards and IIIF interoperability across the emerging ‘Open Maps’ landscape (Schoonman and Baptist, 2024), in particular in Allmaps and MapReader functionality. Simplifying the ability to move between JSON/GeoJSON and Web Annotations/IIIF Georeference Annotations is powerful, reducing file sizes and enabling work with both pixel or geospatial data.

In this presentation, we will discuss these workflows and visualisation interfaces through examples of research in progress that is enabled by this work. After using the Rumsey data as the initial use case for the approach to structuring text spotting output as IIIF annotations, we are developing approaches to visualising change over time in both the Rumsey data and in two editions of large-scale Ordnance Survey (OS) sheets from the National Library of Scotland. While the Rumsey collection’s global scope over 5 centuries presents certain visualisation and research opportunities, the OS is more circumspect both geographically (to Britain) and temporally (to the nineteenth and early twentieth centuries). With each collection, the team is learning how to interact with text spotting output given both cartographic and AI biases. Through such research, the broader community can become more adept at interrogating big data derived from digital image collections as well as better prepared to develop approaches to presenting this data to the library and archive publics via search interfaces. We hope that from these initial case studies, and the tools and lessons emerging from our work, colleagues will have models for creating, sharing, and exploring text on maps data for their own collections.

References

Coleman, C. N. “Art, Cartography, and Machines: AI Tool Design for Historians.” Imago Mundi 76, no. 2 (2024): 307–313. https://doi.org/10.1080/03085694.2024.2453339

Ducatteeuw, V., Danniau, F. & Verbruggen, C. “Mapping Ghent’s cultural heritage: a place-based approach with web GIS.” International Journal of Digital Humanities (2025). https://doi.org/10.1007/s42803-025-00099-4

Hosseini, K., Wilson, D. C. S., Beelen, K., and McDonough, K. “MapReader: A Computer Vision Pipeline for the Semantic Exploration of Maps at Scale.” In Proceedings of the 6th ACM SIGSPATIAL International Workshop on Geospatial Humanities. GeoHumanities ’22. New York, NY, USA: Association for Computing Machinery: 8–19 (2022). https://doi.org/10.1145/3557919.3565812

Larsen, K. L., Thornberry, E., & Vitale, V. “Teaching and Learning with Text on Maps: A David Rumsey Collection Case Study.” Imago Mundi 76, no. 2 (2024): 290–296. https://doi.org/10.1080/03085694.2024.2453335

Lin, Y. and Chiang, Y.-Y. “Hyper-Local Deformable Transformers for Text Spotting on Historical Maps.” In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '24). Association for Computing Machinery, New York, NY, USA (2024): 5387–5397. https://doi.org/10.1145/3637528.3671589

Machines Reading Maps Collection, Stanford Digital Repository (2023) https://searchworks.stanford.edu/view/rc349kh8402

McDonough, K. “Maps as Data.” In Computational Humanities, ed. by J. M. Johnson, D. Mimno, & L. Tilton. University of Minnesota Press, 2024.

McDonough, K., Beelen, K., Wilson, D. C. S., & Wood, R. “Reading Maps at a Distance: Texts on Maps as New Historical Data.” Imago Mundi 76, no. 2 (2024): 296–307. https://doi.org/10.1080/03085694.2024.2453336

McDonough, K., and Vitale, V. “Introduction.” Imago Mundi 76 (2): 279–84. https://doi.org/10.1080/03085694.2025.2453329

Nelson, R. K. “Mapping Environmental Inequalities: Using MapReader to Uncover Mid-Century Industrial Burdens.” Imago Mundi 76, no. 2 (2024): 284–290. https://doi.org/10.1080/03085694.2024.2453333

Schoonman, J. and Baptist, V. Open Maps Meeting, November 5-6 2024, TU Delft, https://openmapsmeeting.nl/.

van Wissen, L., Kuruppath, M. and Petram, L. “Unlocking the Research Potential of Early Modern Dutch Maps.” European Journal of Geography 16, no. 1 (2025): s12-s17. https://doi.org/10.48088/ejg.si.spat.hum.l.wis.12.17

Vitale, V. “Searching Maps by Words: How Machine Learning Changes the Way We Explore Map Collections.” Journal of Cultural Analytics 8, no. 1 (Apr. 2023). https:/​/​doi.org/​10.22148/​001c.74293.

Wood, R., Hosseini, K., Westerling, K., Smith, A., Beelen, K., Wilson. D. C. S., & McDonough, K. “MapReader: Open software for the visual analysis of maps.” Journal of Open Source Software 9, no. 101 (2024): 6434. https://doi.org/10.21105/joss.06434

Background

Government departments are required to appraise and select records of historical interest, and transfer them to The National Archives (TNA). In the pre-digital era, when the Civil Service operated primarily by pen and paper, this was no mean feat. But with small, specialised teams, departments were able to remain on top of managing their physical documents, and transfer appropriate records to TNA.

Times have changed.

The arrival of the digital age, with most office work now being done on computers, has brought an entirely new scale of complexity. Civil Servants now generate roughly 1,000 times more documents than they did previously. This has left many organisations struggling with a “digital heap” – vast quantities of files and folders, with some dating back to the 90s and 00s. Some of these will be of historical interest; many will be redundant, outdated or trivial. Sorting out the valuable from the irrelevant has become a monumental challenge.

Research carried out by TNA in 2018 indicated that manually working through the full digital heap would take ~990 people 100 years to complete. Since then, the heap has only continued to grow.

Further complicating the issue is that, for many organisations, their Knowledge and Information Management (KIM) professionals did not work at their organisation 20+ years ago, when many of these records were created. They may not have the necessary organisational knowledge to be able to identify who the key decision-makers were, or what material is of greatest historical interest.

In short, manual review of the digital heap at scale is not viable. Therefore, we at TNA have been investigating the potential use of AI to assist. Our work has focused on two key areas: (1) using AI to reconstruct lost organisational memory, and (2) using AI to assist KIM teams in appraising and selecting from the digital heap.

Using AI to Reconstruct Lost Organisational Memory

Some appraisal and selection decisions can be made easily, without any pre-requisite knowledge of the organisation’s historical context. (For example, old timesheets, or paystubs, or “dave_s_christmas_card_list_2002_PERSONAL.txt” are likely candidates for disposal, regardless of which department you work for.)

But understanding who the key decision-makers were, what the key policies, projects, and pieces of legislation were, in a given historical period, is crucial to being able to identify documents of high importance, and can significantly speed up the appraisal and selection process.

Fortunately, much of this information can already be found in TNA’s own holdings – including the UK Government Web Archives, and legislation.gov.uk, among others. These sources are publicly available, and the information can be manually assembled. However, we believe that this process can be carried out much more efficiently using AI.

Our experiments in this space have included using Large Language Models (LLMs) to take information from unstructured documents (such as departments’ published accounts, and organisational charts from the early 2000s, both in image format and HTML format), and using that information to populate pre-defined, machine-readable formats (such as RDF, or JSON).

The results have been very promising: LLMs are clearly capable of carrying out these tasks to a high degree of accuracy, much quicker than humans can, and do not appear to present a significant risk of hallucination when doing so. Utilising AI to extract specific information from archival materials, and translate them into machine-ready formats appears to be an ideal use-case for the technology.

Using AI to Appraise and Select from the Digital Heap

Because of the sheer scale of the digital heap, it is widely accepted across the KIM community that some level of computer assistance is necessary for carrying out appraisal and selection on any reasonable timescale.

Large Language Models are likely to form at least a part of this assistance, as they have pre-trained “knowledge” of the meaning of natural language. This “knowledge” is vital to understanding the potential significance of historical records.

Our most successful investigation in this space so far has been into clustering files by the meanings of their contents. We have done this using an open-source tool called BERTopic. The approach begins by making use of specialised LLMs to “vectorise” documents. This converts text into a vector (essentially a series of numbers), representing the underlying meaning of the text. Consequently, texts of similar meaning have similar vector representations (so “The cat sat on the mat” will have a similar vector to “The feline sat on the carpet”) while texts with very dissimilar meanings will have very dissimilar vectors.

Once these vectors have been generated, the approach uses a (non-AI) algorithm to cluster documents of similar meanings together. In this way, we can generate meaningful groups of documents, with each group covering a similar topic. This allows KIM professionals to see, at a glance, groups of files / folders which may be linked, even if they sit in completely different areas of the file-path.

This approach can be further enhanced, by summarising the contents of each cluster. Once the clusters are created, we use generative AI to create brief overviews of the documents held therein. This can give KIM professionals a quick, high-level understanding of a drive’s contents, without having to trawl through the folders and documents one-by-one.

We believe this information can be integrated with other, non-AI-derived data (e.g. enhanced metadata, extracted using tools like Apache Tika), and with other AI-driven approaches (e.g. semantic search, allowing users to locate documents by their contents’ meaning, rather than by specific strings of text). Such an approach would provide KIM professionals with a much richer, more powerful toolset in making appraisal and selection decisions at scale.

Next Steps

We have identified certain AI approaches which perform well at the tasks we are interested in. However, all of our research so far has been carried out on dummy data. We are currently working with a UK Government department to set up a Proof-Of-Concept project, which will test our approaches on real-life digital heap data. This work is likely to be ongoing at the time of the Fantastic Futures conference; as such, our presentation will include relevant updates, but may not yet have reached a solid conclusion.

It should be noted that the approaches we are investigating will not replace humans in the decision-making process. We are firm believers in the principle of keeping a “human in the loop”. As such, any tools we incorporate into the work would aim to assist KIM professionals in making informed decisions at scale, but would not make the decisions themselves.

We are optimistic that this work will prove successful. If it does, it will not only demonstrate TNA’s archival holdings’ ongoing value (especially now, when AI enables quicker extraction of valuable information), but it will also move us closer to resolving a significant barrier in transferring large volumes of digital information to TNA.