Jekyll2024-02-13T12:51:20+00:00https://thesciencemuseum.github.io/heritageconnector/feed.xmlHeritage Connector BlogTransforming text into data to extract meaning and make connections {"name"=>nil, "email"=>nil}Consolidation Report: Insights from Towards a National Collection Foundation Projects2023-03-03T00:00:00+00:002023-03-03T00:00:00+00:00https://thesciencemuseum.github.io/heritageconnector/post/2023/03/03/consolidation-reportOn 24 February 2023 the Towards a National Collection Programme Directorate published a report by Carlotta Paltrinieri entitled Consolidation Report: Insights from Towards a National Collection Foundation Projects drawing together the findings of the eight Foundation Projects funded through the programme including Heritage Connector.

]]>{"name"=>nil, "email"=>nil}Webinar: Foundation Projects: Heritage Connector, Persistent Identifiers and Provisional Semantics (3 May 2022)2022-06-09T00:00:00+00:002022-06-09T00:00:00+00:00https://thesciencemuseum.github.io/heritageconnector/events/2022/06/09/tanc-findings-WebinarOn 3 May 2022 the findings from the Heritage Connector project was presented at the webinar “Foundation Projects: Heritage Connector, Persistent Identifiers and Provisional Semantics” organised by the Towards a National Collection programme. A recording of the session is available below:

]]>{"name"=>nil, "email"=>nil}Presentation: 2022 Smithsonian Digitisation Conference (24 March 2022)2022-03-26T00:00:00+00:002022-03-26T00:00:00+00:00https://thesciencemuseum.github.io/heritageconnector/events/2022/03/26/smithsonian-digitisation-conferenceOn 24 March 2022 the Heritage Connector project was presented at the 2022 Smithsonian Digitisation Conference in a talk entitled Building Links with Natural Langauge Processing as part of the session Pandemic Pivoting: What we do when we can’t digitize objects. A recording of the session is available below:

Download presentation slides

]]>{"name"=>nil, "email"=>nil}Heritage Connector: Final Report2022-02-09T00:00:00+00:002022-02-09T00:00:00+00:00https://thesciencemuseum.github.io/heritageconnector/post/2022/02/09/Final-ReportThe Heritage Connector project’s final report is now published and available at https://doi.org/10.5281/zenodo.6022678.

We would like to thank the AHRC’s Towards a National Collection programme for funding this project, and the project team (Professor Jane Winters, John Stack, Kalyan Dutia, Jamie Unwin, Rhiannon Lewis, Richard Palmer and Angela Wolff), Dr Tim Boon, Hope Miyoba, Stuart Prior, Rebecca Bailey and Colin McDowall for their work, help and support of the project.

]]>{"name"=>nil, "email"=>nil}Heritage Connector Code Repositories2022-01-24T00:00:00+00:002022-01-24T00:00:00+00:00https://thesciencemuseum.github.io/heritageconnector/post/2022/01/24/Code-RepositoriesThe Heritage Connector project is composed of multiple components and services split across multiple GitHub repos. Below we give an overview of the project’s main code repositories:

Heritage Connector Pipeline

Contains the main bootstrap / pipeline used by the Heritage Connector to import data into a graph before performing NPL and Graph analysis on it. Individual repos provide much of the actual functionality and are listed separately below.

https://github.com/TheScienceMuseum/heritage-connector

A set of tools to:

  • load tabular collection data to a knowledge graph
  • find links between collection entities and Wikidata
  • perform NLP to create more links in the graph (also see hc-nlp)
  • explore and analyse a collection graph ways that aren’t possible in existing collections systems

Heritage Connector Deployment

An easy way to deploy all the Heritage Connector API’s and Endpoints, with the exception of the main pipeline, in one step.

https://github.com/TheScienceMuseum/heritage-connector-deployment

The following services are included and all configured through environment variables:

  • fuseki - RDF triplestore
  • thor - front end for performing SPARQL queries
  • thor-cors-proxy - CORS proxy to enable thor to connect to fuseki
  • heritage-connector-vectors - an nearest neighbours on knowledge graph embeddings
  • heritage-connector-apis - API endpoints to wrap some common SPARQL queries and the nearest neighbours API.

Heritage Connector Demos

This repo contains various demos and sketches of demos for Heritage Connector.

https://github.com/TheScienceMuseum/heritage-connector-demos

  • an interactive streamlit app showing NER and entity linking which uses static data for speed (not hosted at the moment)
  • a bookmarklet to view connections from an SMG collection, blog or journal page
  • a macro visualisation of the whole collection/knowledge graph
  • a visualisation of the combined SMG and V&A collections
  • maps of all the places in the knowledge graph

Heritage Connector NLP

NLP tools for heritage collections

https://github.com/TheScienceMuseum/heritage-connector-nlp

Heritage Connector Vectors

Generates graph and language embeddings for the Heritage Connector project.

https://github.com/TheScienceMuseum/heritage-connector-vectors

Heritage Connector APIs

Various API’s for the querying Heritage Connector project https://github.com/TheScienceMuseum/heritage-connector-apis

Heritage Connector Data

Various Public data sets for the Heritage Connector Project.

Further public datasets and outputs can be found on the HC / TANC page on Zenodo

Heritage Connector Thor / Fuseki

These repos provide a SPARQL server (Fuseki), a SPARQL client (Thor) and a Proxy server (to deal with CORS headers) as Docker containers. All three component can be installed in-one-go via this repo

Elastic Wikidata

Simple CLI tools to load a subset of Wikidata into Elasticsearch.

]]>{"name"=>nil, "email"=>nil}Project Demonstrators Directory2022-01-07T00:00:00+00:002022-01-07T00:00:00+00:00https://thesciencemuseum.github.io/heritageconnector/post/2022/01/07/demonstratorsThe Heritage Connector project developed a suite of demonstrators which allow exploration of the data generated through the project.

Additional demonstrators were created through a hackathon in November 2021.

Screengrab of Heritage Connector Demonstrators webpage.

]]>{"name"=>nil, "email"=>nil}Hackathon Projects2021-12-05T00:00:00+00:002021-12-05T00:00:00+00:00https://thesciencemuseum.github.io/heritageconnector/post/2021/12/05/Hackathon-DemosOn 19 November 2021, the Heritage Connector project held a one-day hackathon in partnership with Cogapp.

The hackathon explored creative use of the project’s knowledge graph and related APIs to develop new forms of user interface and audience engagement.

The following list provides details of the projects developed at the hackathon:

The hackathon projects were presented at the project’s final webinar on 3 December 2021.

Thanks to Cogapp for hosting the day and to all the participants:

  • Alex Morrison
  • Ben Rubinstein
  • Grant Cieciura
  • Iain Bean
  • James Waterfield
  • Jamie Unwin
  • John Stack
  • Kalyan Dutia
  • Luke Watson-Davis
  • Neil Hawkins
  • Nitin Ladwa
  • Rhiannon Lewis
  • Richard Palmer
  • Scout Burghardt
  • Sophie Frost
  • Sotirios Alpanis
  • Steve Norris
  • Stuart Prior
  • Tim Boon
  • Tristan Roddis
]]>{"name"=>nil, "email"=>nil}Map Interface Demonstrator2021-12-04T00:00:00+00:002021-12-04T00:00:00+00:00https://thesciencemuseum.github.io/heritageconnector/post/2021/12/04/MapOn 19 November 2021, the Heritage Connector project held a one-day hackathon in partnership with Cogapp.

Ahead of the hackathon, a proof-of-concept map interface was developed to explore the potential of creating new entry interfaces into the knowledge graph.

Screengrab of browser window showing interactive map interface

The hc_dump_latest.csv file was wrangled in BBEdit to extract all the Wikidata QIDs.

These were then processed in OpenRefine to add the following Wikidata property P625 (coordinate location) values giving latitude and longitude.

The resulting data was then manipulated in OpenRefine to add links to the Heritage Connector Metadata Explorer and then uploaded to Google Maps resulting to two maps:

Because there is a limit on Google Maps of ten layers, each with max 2,000 pins, and there are 38,576 pins in total, the pins are split across two maps. As you explore, you may discover that there are some pins which actually relate to locations on the moon!

Download map data

]]>{"name"=>nil, "email"=>nil}Timeline Interface Demonstrator2021-12-04T00:00:00+00:002021-12-04T00:00:00+00:00https://thesciencemuseum.github.io/heritageconnector/post/2021/12/04/TimelineOn 19 November 2021, the Heritage Connector project held a one-day hackathon in partnership with Cogapp.

For our project Rhiannon Lewis and I (John Stack), worked with Timeline JS to build and interactive timelines for events and people in the knowledge graph.

Screengrab of browser window showing interactive timeline interface

The hc_dump_latest.csv file was wrangled in BBEdit to extract all the Wikidata QIDs.

These were then processed in OpenRefine to add the following Wikidata P (property) values for each of the list of QIDs:

  • Date of birth
  • Date of death
  • Start time
  • End time
  • Announcement date
  • Point in time
  • Publication date
  • Image (filename of any associated images)

The resulting data was then manipulated in OpenRefine into the required structure for Timeline JS, links to the Heritage Connector Metadata Explorer added, and then converted into Google Sheets: one for people and one for events.

These were then linked to Timeline JS to give the two timelines:

The performance of the timeline is quite slow as the maximum number of points on the timeline is recommended to be 200 and each of our timelines contains several thousand entries, which is itself only a tiny fraction of the number of people and events in the whole knowledge graph.

]]>{"name"=>nil, "email"=>nil}Open Access Data Outputs2021-12-03T00:00:00+00:002021-12-03T00:00:00+00:00https://thesciencemuseum.github.io/heritageconnector/post/2021/12/03/Heritage-Connector-DatasetThe Heritage Connector data outputs have been published on Zenodo at https://doi.org/10.5281/zenodo.5752010. The data outputs are available under the open access CC-BY 4.0 licence.

The contents of the data outputs available are:

  • triples (.nt) and csv (.csv) files - the final knowledge graph produced. Includes data from Science Museum Group (SMG) and Victoria and Albert Museum (V&A) collections and Wikidata.
  • triples (.nt) and csv (.csv) files - the knowledge graph produced from the SMG collection and Wikidata.
  • kg_embeddings.zip - knowledge graph embeddings generated by training RotatE models on the above datasets. Also includes 2D and 3D projections of these embeddings generated using UMAP.
]]>{"name"=>nil, "email"=>nil}