The Perils of Letting Sand Think For Us: Automated Decisions Using Conflicting Principles

Machine learning algorithms, colloquially referred to as AI (artificial intelligence), are demonstrating new capacities to generate text, images, and other data sets. These capabilities are marketed and hyped with breathless superlatives dedicated to how the latest generation of programs can produce text or images that, on their face, look convincingly like they originated from a human. Have a question? Chat with human-like agents empowered by data sets to get you answers! Trying to determine how a genre mashup might work, in text or in images? Plug it into a generator and see new masterpieces come forth as if from creators of wildly different time periods and genres. Build an entire set of artifacts around a movie that never existed and never will, faster than Tumblr created Goncharov!1

The creators of the bots have let them run wild on the Internet, with the express idea to let the collective us poke at them, push them to their limits, and see how well they respond to both the mundane and the bizarre. The results have been less than encouraging. While on the surface the output of the programs may appear to be error-free, when examined in detail by domain experts, there are errors that make it apparent the text or image was generated by an algorithm. Sufficiently so that some spaces have banned the use or output of the algorithms because of their persistence in error.2 Others have pointed out that the training data used for the machine learning models comes from the real world, and therefore reflects the biases and problems of humans in the real world, even when, theoretically, controls have been put in place to stop the robots from saying or doing overtly racist things.3 Similar controls supposedly stop the robots from making recommendations for illegal activities, such as shoplifting or bomb-making, but it doesn’t take long for determined people to figure out how to circumvent those restrictions as well.4 The result produces robots that can generate realistic-looking texts and images perfect for a quick disinformation campaign (or 90% of one, and all that’s needed are some final touches), or that might supply someone with the instructions they need to carry out their illegal or terroristic plans. 

A Clash Of Principal Principles

Library professionals have often touted Neil Gaiman’s remark, “Google can bring you back, you know, a hundred thousand answers. A librarian can bring you back the right one,” as the gold standard of library services and, occasionally, as the expression of a smug belief that machines and robots will never replace the trained librarian in the course of their duties.5 Librarians also point to the American Library Association (ALA) Code of Ethics, the Library Bill of Rights, and similar documents as proof that library workers are more trustworthy because they have safeguards and controls that prevent them from making the same types of mistakes robots do, and those safeguards are not so easily circumvented. 

Except when it turns out those safeguards may or may not exist in practice, or are superseded in practice by other principles. 

Robert Hauptman published the results of an experiment in 1976 where he asked thirteen reference librarians if they would help him gather information on how to create a bomb he intended to use to blow up a house. None of them refused to assist him in finding that information by claiming it was unethical to do so.6 Thirteen years later, Robert C. Dowd conducted a similar experiment, asking reference librarians for information on how to freebase cocaine and encountering no objections on ethical grounds.7 Hauptman and Dowd came to differing conclusions about their experiments: Hauptman was appalled at the lack of ethics present in the library profession, while Dowd was pleased that librarians did not restrict the free flow of information. 

The ALA Code of Ethics and Library Bill of Rights support Dowd’s position more than Hauptman’s in any situation where a user is asking an information professional to help them find material even when it could harm themselves or others. 

From the Code of Ethics:

1. We provide the highest level of service to all library users through appropriate and usefully organized resources; equitable service policies; equitable access; and accurate, unbiased, and courteous responses to all requests.

2. We uphold the principles of intellectual freedom and resist all efforts to censor library resources.

3. We protect each library user's right to privacy and confidentiality with respect to information sought or received and resources consulted, borrowed, acquired or transmitted.

[…]

7. We distinguish between our personal convictions and professional duties and do not allow our personal beliefs to interfere with fair representation of the aims of our institutions or the provision of access to their information resources.8

And the Library Bill of Rights: 

1. Books and other library resources should be provided for the interest, information, and enlightenment of all people of the community the library serves. Materials should not be excluded because of the origin, background, or views of those contributing to their creation.

2. Libraries should provide materials and information presenting all points of view on current and historical issues. Materials should not be proscribed or removed because of partisan or doctrinal disapproval.

[…]

5. A person’s right to use a library should not be denied or abridged because of origin, age, background, or views.

[…]

7. All people, regardless of origin, age, background, or views, possess a right to privacy and confidentiality in their library use. Libraries should advocate for, educate about, and protect people’s privacy, safeguarding all library use data, including personally identifiable information.9

The statements in these documents make a strong suggestion, if not a demand, that any qualms a library worker may have about providing certain pieces of information are “personal convictions” or part of “partisan or doctrinal disapproval” and must be quashed while representing the library as an employee. A library worker may have their own personal worries or concerns, but those do not rise to the level of institutional concerns, and therefore it matters little what is sought or for what purpose; it should be provided. Which leads to situations where employees of color are asked to help someone find racist material, queer employees are asked to find queerphobic material, conspiracy theorists ask for material that supports their disbelief in reality, and employees are told to honor requests to avoid queer material or characters of color in finding “good” or “clean” material for a user (or, more often, their child). 

Hauptman’s argument gets a little assistance in our current days from the newest addition to the Code of Ethics, the ninth principle, which gives libraries affirmative ethical requirements:

9. We affirm the inherent dignity and rights of every person. We work to recognize and dismantle systemic and individual biases; to confront inequity and oppression; to enhance diversity and inclusion; and to advance racial and social justice in our libraries, communities, profession, and associations through awareness, advocacy, education, collaboration, services, and allocation of resources and spaces.10

This additional principle can serve as a base to build policies that stop requiring employees to help people actively interested in harming them or other marginalized groups. In my opinion, this principle requires libraries to put those policies into place, if they haven’t already, and to examine their current policies to ensure they explicitly allow not just handing off a difficult interaction to another staff person, but terminating it or refusing to engage because it would violate the ninth principle’s requirement to “confront inequity and oppression.” Compared to the policy positions of the ALA, however, I recognize that I am, at the moment, an outlier.

ALA’s interpretations still take the position that all viewpoints need representation, even as they also talk about the need to review collections on other grounds, such as accuracy and diversity. For example, in a document created to address what libraries can do about hateful conduct and speech in their libraries, ALA still prioritizes ensuring a library has material for all opinions in the community and avoiding “censor[ship],” rather than prioritizing a metric like accuracy over other metrics: 

The “Selection & Reconsideration Policy Toolkit for Public, School, & Academic Libraries” states that an optimal library collection would be “reviewed on a consistent basis for accuracy, currency, usage, diversity, and subject area gaps.” Libraries demonstrate the value of their community by honestly reflecting on their bias and striving to serve users better with a broader range of viewpoints that aren’t censored. Library staff can show their trust in library users by allowing them to think for themselves and determine what information and reading choices are best for them and their families. Honest and frank discussions about tough topics in collections, programs, and resources ultimately benefit communities.11

“I know there’s a proverb which says, ‘To err is human’ but a human error is nothing to what a computer can do if it tries.”12

Commitment to the principle of letting the users think for themselves, coupled with automated processes with no human oversight, produced a situation where my own organization bought multiple copies of material that trafficked in conspiracy theories, from the pull quote to the pages inside, because the ratio of holds requests to available copies of the material had exceeded a pre-determined threshold. 

The entire process started with a request for purchase of this book. As a public library, we are responsive to requests for purchase, because we know our users have broad tastes and interests in a multitude of subjects and opinions on those subjects. Fulfilling user requests also allows the library to demonstrate that it spends tax dollars collecting things of interest to the community. That said, most libraries have criteria to express the boundaries of what they will and will not buy, and in many cases, those criteria reference many of the same review criteria mentioned in the ALA toolkit quote above. 

What, then, is a selector to do when a user requests a work and a fairly cursory search of the author, their organization, and some of the promotional material turns up the high likelihood that the work itself contains misinformation, disinformation, and conspiracy-theorizing from the perspective of someone who believes all those things to be true? Can the selector claim additional concerns about the material if they can’t find (positive) reviews of the book in places that would normally be quick to review books written by a person with name recognition? 

Does the selector prioritize the need for accuracy and currency and refuse the request, arguing that the need to maintain the library’s reputation for providing true and accurate information is more important than fulfilling this particular request?13 Does the selector argue that the need for diversity of viewpoint and accurate representation of the beliefs of the community is more important than the need for factual accuracy in the collection, and so some inaccurate material will find its way into nonfiction all the same?14 Or does the selector punt on the ethical question, as ALA guidance and core documents suggest they should, and purchase the material simply because it was a request from a patron, and the only official opinion the library has on any subject is: “we have no opinion, and we trust our users will think for themselves?”15

Equally important, regardless of what the selector decides, does their management support the decision, or does the management reserve the authority to make the final decision if they disagree with the selector’s choice and reasoning?

The ultimate decision made in my system was to buy the book. With at least one copy bought and available for users to request, the hold list ramped up significantly as an indicator of demand for the work in the community. I can’t say whether the interest was toward spreading and reinforcing its messages, or to debunk and dunk upon it, because we don’t ask why someone wants to read a book or use a resource once we have it. The process of determining whether to purchase additional copies of a work and place them in the “get the hold list down to a reasonable ratio” collection is mostly automated, with an additional copy purchased every time the ratio of requests to copies exceeds the threshold set. Leaving request balancing to machines speeds the process significantly and ensures that popular materials aren’t forgotten about or left behind if a human gets overwhelmed with other work. In this particular instance, the automatic request-balancing system transformed one copy into several copies, at several different locations around the system. The end result? More tax dollars spent on conspiracy theory–friendly material and our library system’s contribution, in our own small way, toward the legitimizing and platforming of that material by offering it and placing it in prominent places in our locations because of its popularity.

At this point, I expect a library philosopher to call a timeout and argue that the mere presence of material in the collection is not an endorsement of the ideas contained in the material, and that I am perilously close to letting my personal beliefs interfere with the fair provision of access to information resources. I expect this argument even in our current world where there are massive public pressure campaigns to remove any mention of queerness, any acknowledgement of sex or gender identity, or any history that does not conform to a specific hagiographical standard from school libraries and curricula, on the basis that the mere presence of such works or concepts in a library or a classroom is inherently harmful and damaging to children.16 I expect this argument in our world where library employees in at least one state are under a gag order not to provide information that might expose them or the library to civil lawsuits under laws worded with the express purpose of preventing anyone providing accurate information about medical procedures.17 I expect this argument because many of the ALA’s documents abstract reality and provide one-size-fits-all guidance from a single perspective, deifying the concepts of vocational awe and neutrality even as they exhort us to think about the specific situation a library worker might face.18 (The emphasis on neutrality in the cited work refers specifically to a prevalent anti-equity, White Supremacy–reinforcing belief that relies on pretending equal treatment is equitable treatment and only individual acts that sharply deviate from ordinary or systemic racism are problematic and require action, rather than the ordinary meaning of neutrality.)

Yes, the library has a responsibility to carry multiple opinions on topics, but the library also has standards. Published standards and policies, most obviously, but also the argument that collection development, selection, and deselection are the province of trained, credentialed professionals whose judgment we should trust based on their study and understanding of all the factors that go into building a proper library collection. If, as an institution, the library does not fundamentally believe that there have to be standards a work must meet to be included in the collection, or that these standards cannot be subordinate to user requests and demands, then we can free up significant amounts of payroll and other expenses by firing all the selectors and collection management librarians. Turning over collection duties to the population at large to do as they please will build a collection that reflects at least some segment of the community the library serves. If mob rule seems too chaotic, here’s an alternative with the same effect: Give all selection and deselection authority to a program in the lineage of these chatbots and image remixers to determine for us what the library should stock and discard. Either way, the end result is a collection that reflects the biases of the data it was trained on and draws correct-looking but ultimately wrong conclusions from that data.

If the library wishes to base some of the credibility and authority of the institution on the ability to find the right answer in the sea of possible answers, then the library cannot abdicate or disclaim responsibility for enforcing the standards it has created for what belongs in the collection and what information the library staff will provide when asked. In 1996, Haputman revisited both his and Dowd’s experiments and reiterated worries that the information profession was not acting in an ethical manner in an era when materials and instructions on antisocial acts were much more easily accessed through the Internet and the World Wide Web. Hauptman writes:

After twenty years, my perspective remains unchanged: censorship is never warranted, but it should not be confused with a refusal to aid and abet egregiously antisocial acts in the name of some higher obligation. Claiming that patrons may only be interested in reading about something is an easy way to avoid rendering a difficult judgment. Professional disseminators of information must assume responsibility for every action they take; they must make individual decisions based on a complex of principles and necessities and not merely react casuistically because of their training.19

Hauptman’s argument is still salient today. With the increasing use of networked computers, programs, and decision-making entities, it becomes significantly easier for an error in judgment, the misapplication of a principle, bias in data sets and metric formulation, or a program working exactly as intended on something that needed human oversight to magnify its own effect and create a larger problem more quickly. In addition to scrutinizing programs to ensure they do what they claim to and stay within their scope, library workers and administrators must scrutinize the decisions these programs make and the data they use to perform their tasks. If, due to algorithmic decisions, buying one copy of a conspiracy theory will mean buying forty copies before the community’s interest in it fades, the true decision before us is whether the library prefers the consequences of buying forty copies of a conspiracy theory or the consequences of refusing a user request in order to buy zero copies of a conspiracy theory. If a library services vendor proudly touts how little human touch is involved in the provision of a service, or insists that it has the right to scrape, track, and sell user data to a greater degree than the library would, the question before us is whether the benefit of the service provided to the community is worth the risk of harm to the community from inhuman actions.

Which is ultimately a question about what principle a library prioritizes: showing responsiveness to the community and prioritizing access, even if it means buying things that aren’t true or are potentially harmful to the community or the staff, or building a collection that the staff and community can be confident is accurate, true, and meant to minimize potential harms, even if it doesn’t contain everything the community wants. Whichever way a library decides, it should be a conscious and public decision, so the staff and the community know what to expect from their library.

Alex is a Youth Services Librarian with the Pierce County Library System in Western Washington. When not indulging their polymath tendencies, they’re usually engaging children with the world of stories, rhymes, and activities through recommendations, programming, and deliberately looking silly in front of children and their grownups. Spot them on Mastodon at [email protected]