Development of a Classifier for the Arquivo Histórico Ultramarino Catalog: An Experiment with Natural Language Processing and Artificial Intelligence Applied to Archival Summaries

Abstract

This article describes the computational and methodological architecture of the Catálogo AHU-Brasil, a documentary triage tool that gathers, in a structured and searchable database, about 167,000 archival summaries of documents from the Arquivo Histórico Ultramarino (AHU) concerning colonial Brazil. The project began as the "AHU-South Classifier", restricted to 7,051 summaries concerning Southern and Southeastern Brazil extracted from the Projeto Resgate Barão do Rio Branco, and was later expanded to every series of the Overseas Council concerning Brazil, based on the catalogues published by the archive itself, in the Arquivo Científico Tropical Digital (ACTD), and by the Projeto Resgate. To overcome the limitations of lexical search in unstructured large-scale datasets, a Python pipeline was developed, integrating the automated collection of the catalogues, metadata cleaning, the standardization and conversion of archival call numbers (CRAV/DigitArq standard), quality control of the summaries, and the linking of the documents to their DigitArq records. This study also details the semantic vectorization of the summaries to implement a hybrid search engine, which enables both specific lexical term queries and broader semantic contextual searches. As a complementary description, it presents a sociolinguistic annotation experiment carried out with the DeepSeek v3 model on the 7,051 summaries of the AHU-South collection, which infers social categories, communication vectors, and the likelihood of scribe mediation, synthesized into the Score of Potential Sociolinguistic Relevance (SRSP), a heuristic indicator to guide researchers toward manuscripts with a higher propensity for containing syntactic innovations of colonial Brazilian Portuguese. In alignment with Open Science principles, this work presents the complete dataset, source code, and a publicly available interactive search interface. The final objective is to demonstrate how the alliance between Digital Humanities, Data Science, and Natural Language Processing can substantially optimize documentary selection for research in Diachronic Linguistics and Corpus Linguistics.