This blog post follows the previous post, XML Coding with AI, Episode 2: The Hallucinations Strike Back. In the last post, I explored some of the pitfalls of using AI to help with structured cataloguing – especially its tendency to ‘hallucinate’ plausible-sounding but entirely fictional information. I will now explain some of the ways that I have integrated it – carefully and selectively – into my everyday work, starting with XML coding.
The Architect awakens
Firstly, by adopting the strategies outlined in the last post, I have found I am able to produce consistently high quality XML faster and more easily than a manual process of typing and pasting. To reduce repetition of instructions I created a custom GPT, the EAD XML Architect, for this task; essentially an instance of ChatGPT with pre-established rules (above) to try and cut down on the paraphrasing, condensing and assumptions that come so naturally to the program. Using the Architect, I was able to generate XML code from a word document box list of the Bellot Papers, significantly speeding up the production of a working catalogue record (with contents and biographical information added by curators). The use of AI in producing this catalogue record is clearly signposted in the Archivist’s Note. For this record I worked in small sections at a time and carefully checked each output. This slows down the process, but is important to ensure accuracy. Of course, errors will also occur in code typed manually by human users. I can also see this improving as AI tools develop higher processing capacity in the future.
Reading like a machine
One particularly interesting strength of AI was its ability to ‘read’ lists of information in a different way than a human. When reading a box list from top to bottom, sometimes I would miss crucial links between items that were far apart in a list. The AI, on the other hand, was sensitive enough to context that it was able to make connections within box list that I hadn’t noticed. For example, one item in a box list noted that a named woman had married and changed her name. Many pages later, a piece of correspondence from after her marriage was described, but only her maiden name was used. When rewriting the listed data as an XML catalogue record, ChatGPT automatically updated the metadata to her married name – an improvement that a human not paying attention to this specific detail may have missed (as the original author of the list did, in fact). Curators can be particularly susceptible to missing these sort of links when working on large datasets, or working in multiple stints.
Beyond coding for XML, ChatGPT’s sensitivity to context, ability to recognise patterns and proficiency with tone have proven very useful for written work. I have had success using ChatGPT to suggest alt text and physical descriptions of collection items. It can also respond to plain text requests such as ‘reduce this down to 50 words’ or ‘include more description of the materials’. Generative AI can be wrong, just like a human, and will often sound more convincing than a human would be (although it is usually fairly easy to spot when things are not as described). In the interaction in the screenshot above, for example, ChatGPT’s description has picked out some important details, but it is wrong about the ‘bicycle wheel symbol’ – I know this from context as the snake from the University’s crest. From this the AI has inferred that it is a badge for the cycling club when in fact the ‘MC’ represents the Motor Club. However, even an inadequate physical description for an item can be helpful to have, as it sometimes highlights certain details that I hadn’t thought of. I also generally find it easier to work from a flawed description than producing one from scratch.
The on-demand editor
ChatGPT has also proven its worth as a sense, grammar and tone checker for final drafts, much like a spellchecker can check for mistyped words or phrases. Recently, I produced some text for an interpretation panel for one of our most celebrated academics, the activist and Nobel laureate Sir Arthur Lewis. I had sent a first draft out to colleagues across the University and had positive feedback plus some minor comments on style and sentence structure. I first used ChatGPT to suggest some alternative sentence layouts that could address the feedback, and then selected some improvements from one or more of its recommendations. Finally, I ran the whole text panel through the AI for any further suggestions. Some I disagreed with and ignored, but it correctly managed to flag an error in the date of Lewis’ arrival into Manchester (1948 was mistyped as 1958). Whilst this may have been caught by further final checks, the AI had provided a valuable service beyond what was possible with a simple spellchecker.
Appropriately, I have also sourced suggestions from AI for this blog. As you can see in the above example, AI was able to offer suggestions for how to better introduce or explain concepts I am personally familiar with. I could then pick and choose from the AI’s suggestions of varying quality; some could be used verbatim to improve a section, others had elements that were valid but needed some editing and some I felt did not meaningfully improve the content. The original ‘tone’ of AI chatbots can be quite obvious to people and stand out as artificial, so I prefer to work valid suggestions into the text using my own authorial voice (again, this is one of the guidelines with using AI anyway). I would be interested to know if anyone reading this had already noticed that AI suggestions were used for some of the content!
Over time, I have come to see AI as a combination of tools; a quick typist, a wordy intern and, occasionally, an overconfident editor. It is good with communication and code, but it is inconsistent and has a poor grasp on truth. It is not a replacement for expertise, but if used carefully it can be incredibly helpful. Learning to effectively use AI has been a frustrating process, at times; but where once I felt like an AI apprentice, I am now on the path to becoming a master!