Reporting Metrics on GLAM-Wiki Part 1

As I mentioned in a post earlier in the week, documenting and assessing the effects of a GLAM-Wiki partnership on a institutions digital presence is a critical step to GLAM-Wiki cooperations. There are a number of ways to take metrics related to Wikipedia. In this blog I am going to highlight a few and why they matter. If you would like to see a comprehensive list of tools that GLAM professionals can use to measure content on Wikipedia, check out this list of tools used by the GLAM-Wiki community.

Metrics are an important part of any project, because it allows whoever is coordinating that project to communicate to others exactly what happened. This is particularly important when working with academic or cultural institutions: their is always to much work in that space and not enough labor. Metrics allow academics and GLAM-participants to assess the ratio of effort to public impact. For this blog post, I will cover some of the most basic types of metrics used by GLAM-Wiki assessment: quality and number of articles and external links.

Blake task force and Article quality

The 1.0 Assessment chart from https://en.wikipedia.org/wiki/Wikipedia:Blake#Article_assessment

One of the most common ways for Wikipedians to organize work and content is to create community projects where they can coordinate efforts. Projects for large swaths of material are called WikiProjects and for more specific subtopics, sometimes these WikiProjects will form task forces (check out this link for a list of WikiProjects on English Wikipedia). For this GLAM-Wiki cooperation, I created a William Blake Task force as part of WikiProject Poetry on English Wikipedia. One of the advantages of WikiProjects is that they allow users to know who is working on a content area and for participants to categorize. On Wikipedia, the community has developed an assessment system called the 1.0 assessment scheme to map out the importance and quality of articles on English Wikipedia.

When I first found out I would be doing an internship with the archive (as reported in an earlier post), I began tagging articles related to Blake within this schema. In doing so, I identified 150 pages related to Blake and his work (now 151 page because of the a new article). I assessed them according to the 1.0 criteria.  As you can see in the screenshot of the assessment graph below, many of these articles are lower in quality (starts and stubs are small starter articles on Wikipedia with little content or references while Featured articles are well referenced and thorough articles that undergo a review process) and many of the most importance articles related to Blake and his art (Blake’s biography and his main publications) could use focused improvement. By identifying these qualities, we can direct contributors that we enlist through the Education assignments, editathons and Wikipedia community content drives towards the articles that need the most work and are most important.  At the end of the internship, I will reassess how successful these activities are by documenting the changes in article quality and number.

URLs to the Institution

One of the easiest ways for an institution to report it’s reach and presence on Wikipedia, as well as the internet more generally, is to determine how many urls lead towards different parts of their website. For English Wikipedia, a contributor who works for the Library of Congress, Ed Summers, created a tool called LinkyPedia which maps out how often links get used on English Wikipedia. According to the tool, currently, the Blake Archive has 110 links to its pages across 56 Wikipedia pages (not all of them articles). This is a small number of URLs considering how important the Blake Archive is as the primary Academic source of Blake images and transcription on the web. Anyone who researches a Blake poem or publication will likely find their way to the English Wikipedia article on the topic, and, if links to the authoritative Archive records are not available, they may not realize that the archive exists or can provide them support. This is a problem for students, researchers and Blake scholarship more generally: the free authoritative materials from the Blake Archive can easily be ignored for someone unfamiliar with the project. Adding links that meaningfully help readers connect to this outside source, as long as its not simply link spam, fulfills both Wikipedia’s mission of helping individuals access free knowledge and the Blake Archive’s mission in increasing use of their reference resources produced through almost two decades of institutional funding and grants.

Links on other Wikimedia projects are also important. For the Blake Archive, the most important links will be from Wikimedia Commons. Because the high resolution scans from the Blake Archive have been available for free on the internet since 1996, Wikimedia contributors have been downloading the images and uploading them to Wikimedia Commons because of their public domain status. Many of these images have been accompanied with a URL that provides the source of these images, properly accrediting the Archive for making them available. Though the MediaWiki software allows searching for the use of URLs, the report it generates doesn’t account for multiple URLs on a page thus allowing you to count the number of pages. To create a data similar to Linkypedia’s I ran two different tools to create metrics for links on Wikimedia Commons: first, running the built in tool, I discovered that their are 710 links to the Blake Archive; next, using the tool AutoWikiBrowser, I ran another report which reported 644 distinct file pages using those urls. On Wikimedia Commons, their are 2287 images related to William Blake, thus the report suggests that the Blake Archive is the source of over 1/4 of Blake images already in use across Wikimedia Projects. Already, in my initial survey of those images, I have discovered other Blake Archive scans without proper metadata attributing their source; this suggests that a significantly larger portion of those images are from the Blake Archive.

These metrics about link data have allowed me prioritize a particular activity that should improve my internship’s public impact: adding appropriate links and metadata to English Wikipedia and Commons. By adding links to the William Blake Archive and the institutional webpages of those institutions that hold the physical Blake objects, images will be accredited to their producers and readers will be affiliated with direct channels to free academic sources.

Coming soon

In my next blog post (or two), I will talk about the tools available for tracking page views, use of images on the family of Wikimedia projects, and how the Archive’s website metrics can be compared to Wikimedia projects to help improve public access to the materials.

Meanwhile, I would like to encourage everyone interested in William Blake or GLAM-Wiki to help improve content related to Blake on English Wikipedia. For things to do, check out the list of potential projects at https://en.wikipedia.org/wiki/Wikipedia:Blake#To_do . If you need help learning how to edit, check out http://editathon.org/ .

The Common Trajectory of GLAM-Wiki Projects

Last week, I answered the questions “What exactly is GLAM-Wiki? This week I thought I would outline more of what GLAM-Wiki collaborations look like and some of the practical concerns that go into running the early stages of collaborations. I started this  discussion two days ago when I explained some of the concerns I focus on when giving outreach talks about Wikipedia. Today, I am going to outline what seems to be the common trajectory of GLAM-Wiki projects since Liam Wyatt’s inaugural residency at the British Museum. Having been involved in a number of collaborations as a volunteer, in my experience, these projects seem to take the following steps:

1. A GLAM, or individual enthusiast within a GLAM, recognizes it wants to engage Wikimedia projects or Wikipedia because of their importance within the public’s perception of the internet (Wikipedia is the go to source for quick information, tops Google searches and is integrated into Facebook and other websites). In the case of my current internship, Mark Crosby, one of the faculty at Kansas State, was very supportive of my other Wikipedia and Digital Humanities activities, is an editor of the William Blake Archive and jumped at the opportunity to have me do an internship helping the Archive.
2. A Wikimedian begins presenting ideas to the GLAM’s employees and other volunteers in order to rally support. As I reported in my last blog post, the presentation I gave at Kansas State’s Beach Museum is a good example of these outreach events. Because the Blake Archive is a digital project, I am doing mostly outreach at my current location, Kansas State University, instead of within the GLAM.
3. The Wikimedia community alongside GLAM professionals begins to facilitate a cooperation by identifying what resources would best help support Wikimedia’s mission to provide free and open-licensed educational materials. For my current internship, we knew that almost everything on the website fits into Wikimedia’s needs: the William Blake Archive has mostly public domain media content because Blake died nearly 200 years ago, and the archive is clearly an authority on Blake, so can be used to reference related content on Wikipedia.
4. Wikimedians, usually the Wikipedian in Residence, do an initial assessment of what content is already on Wikipedia, Wikimedia Commons and other Wikimedia projects and begins rallying support for the project at the GLAM and on Wikimedia sites ( I am in the process of gathering this information and support. I hope to talk about the statistics in an upcoming blogpost).
5. The GLAM donates resources to the Wikimedia community (usually either physical space for volunteer activities, staff time, or digital materials such as scans or images). For right now, the Blake Archive supports both myself, as an unpaid intern, and Mark, as an editor of the project, working on Wikimedia activities under their name. Also, the Beach Museum provided me the space for the presentation two weeks ago and for an upcoming Blake Archive event, allowing my GLAM-Wiki outreach to become inter-institutional outreach as well.
6. Wikimedia volunteers and GLAM volunteers leverage the resources to improve Wikimedia space by writing Wikipedia articles, illustrating articles with GLAM images or building other free educational content like textbooks, public domain ebooks, etc.
7. Wikimedia content provides the public with better education and direction towards authoritative institutions. Furthermore, GLAMs can show donors, grantmakers, and other decision makers metrics that support their claims to they impacting public knowledge through one of the most used families of websites in the world.
8. GLAM-Wiki volunteers and the GLAM employees assess the impact of the project and refine their approach so as to ensure better public impact of the GLAM and volunteer time.

Right now I am about to finish step 4 and ramping up steps 5 and 6 with the Blake Archive. In the next couple blog posts, I plan to outline a little bit more of what exactly I am doing for the Blake Archive to assess it’s impact on the Wikimedia movement, corral volunteer labor to improve William Blake content and generally improve public knowledge about Blake.