The Bird Is Not the Migration Data: Insights from an Artwork Exploring Machine-Animal Collaboration

Abstract

This paper documents the process of building a Python-based musical synthesizer that generates musical scores from GPS drift, specifically related to the movement of Golden eagles. The paper walks through the design process of the synthesizer, engaging in philosophical questions that explore data gathering practices and interpretation through the lens of art, poetics, and imagination. It concludes by addressing objectivity and neutrality as forms of an imaginary which, if acknowledged and engaged as such, could create a framework for valuing greater diversity in data science.

This paper documents the process of building a Python-based musical synthesizer that generates musical scores from GPS drift, specifically related to the movement of Golden eagles. The paper walks through the design process of the synthesizer, engaging in philosophical questions that explore data gathering practices and interpretation through the lens of art, poetics, and imagination. It concludes by addressing objectivity and neutrality as forms of an imaginary which, if acknowledged and engaged as such, could create a framework for valuing greater diversity in data science.

Main Text

Introduction

I recently created a platform to explore kinship between animals and machines, a way of revealing overlapping systems of abstraction: that of machine logics, that of animal logics, and the subjective human logic of the artist bringing them together. As a student of Applied Cybernetics at the Australian National University’s 3A Institute, I was closed in by social distancing measures in the winter of 2020. My long walks in between online systems analyses and machine learning coursework were frequently interrupted by chance encounters with a variety of Australian birds with distinct sets of behaviors. This piqued my interest in the idea of birds as agents—and the possibility it created for exploring relationships between non-human actors, specifically for the sake of designing sustainable, even ecologically beneficial, cyber-physical systems.

I was interested in using data to create a space for communication between two sets of non-human agents: “the animal and the machine” of Norbert Wiener’s Cybernetics. How might we enable relationships, even collaboration, between non-human actors? This paper documents insights that emerged from an art experiment to create a synthesizer modeled on translating a bird for a machine—and resulted in a series of compositions, collectively titled Music for an Eagle and Three Satellites.

A Brief History of Birdsong

Musical notation is a dataset. Reading about the composer Olivier Messiaen’s work in the field, collecting and notating details about birdsongs in walks through forests, his process was more akin to a good data scientist than any musician. He listened for sounds to encode into musical notation, and his scores instructed performers to go for similar walks, to understand the ground conditions they would replicate through their instruments. Although Messiaen committed to producing these scores, he of course negotiated compromises between observing, preserving, and performing his paper programs to audiences in concert halls. Musical instruments would recreate those notes, just as an original mainframe could create billing functions from holes punched into notecards. The sounds of the orchestra were close enough to bird beaks and diaphragms to dazzle audiences, who celebrated the accuracy of Messiaen’s models, but the notes were ultimately coming from reeds and brass that could not replicate the structure of a bird’s throat or beak.¹

For me, birdsong is a wonderful metaphor for exploring what is lost when we render observations into communication. In Science and Sanity, Korzybski writes: “whatever we say about something is always a partial formulation of a few characteristics abstracted from the total event … that each one may abstract in his own way.”² Interpretations rendered from abstractions—be they music, or datasets, or poetry—should never be handled as truth. There are “pre-verbal … unspeakable”² elements of our world that cannot be generalized. Even when meaningful stories can be captured with emotional richness, those details dissolve when transformed into processes for machines to read and model. Finn writes that “algorithms enact theoretical ideas in pragmatic instructions, always leaving a gap between the two in the details of implementation. The implementation gap is the most important thing we need to know, and the thing we most frequently misunderstand, about algorithmic systems.”³

In his 1967 poem, “All Watched Over by Machines of Loving Grace,” Richard Brautigan envisioned a “cybernetic forest” where “mammals and computers live together in mutually programming harmony.” Rather than using sensors to collect data and render the movement or behaviors of birds into abstractions, I aimed to create a platform that would enable a relationship between the sensor and the sensed.

The emotional power of poetry, music, and art rises from similar gaps, but between the word and the experience—a space where meanings are suggested, leaving me to resolve them in personal terms. These gaps are present whenever we collect or interpret data. In data science and in art, we transform observations into new symbols, packing dense meaning into tighter spaces. Birds become numerals, and environments become spreadsheets. How might we explore the gaps between the translation and the translated?

The Cybernetic Forest Synthesizer

The resulting musical pieces, Music for an Eagle and Three Satellites, are the result of designing that platform, the Cybernetic Forest Synthesizer. The synthesizer is a small algorithm repurposed almost entirely from a Python project for data sonification created by journalist Michael Corey, MidiTime. Corey wrote the software to create music from earthquake frequency data from the 20th century. MidiTime collects data and sorts it, slicing it into seven sections—one for each of seven notes in a musical scale. The data for the score was pulled from bird migration data, culminating with a study of Golden eagle migration across New Mexico.⁴

The initial tests traced a series of GPS coordinates for lark sparrows, which had to be converted to the distances they traveled. Each of these coordinates, marked over the course of years, was then reduced and assigned to one of the seven categories of notes, which were written to a MIDI score. The early “test” pieces reflected a sequence: a bird could arrive at a point in space, and the coordinates of that space were recorded as a data point representing its location. Another program calculated the distance between these locations, which were sorted into categories of first through seventh largest distance traveled. That category produced the appropriate note on the musical score at the pace of one quarter note for each day in the study (no data, no note). In this test, the results were clearly interpretable: a listener could hear that a lark sparrow had traveled a longer distance through a rise in pitch (Box 1).

Box 1. Listen: “Migratory Behavior of Lark Sparrows”.

https://soundcloud.com/onigiripod/migratory-behavior-of-lark-sparrows

In that version of the program, the data of these GPS locations and the lark sparrows were still missing a relationship between the digital and analog. It had produced a sonification of a spreadsheet, rather than producing a collaborative work between the bird and the machine, i.e., Brautigan’s “mutually programming harmony.” This did not satisfy the question I’d set out to answer.

In searching for useful interaction points in the bird-migration tracking system, I began to explore the phenomenon of GPS drift. In the GPS system, three satellites coordinate the time between small signals from the tracking device to roughly coordinate the distance of the source from the satellites. If fewer than three satellites receive the signal, the coordinates lose precision—“GPS drift.” These errors can occur when the communication path to one or more satellites is blocked by an obstacle, such as dense trees, large stones, or mountains. The ambiguity of these measurements is delightfully bird-driven. As an eagle chases prey into a dense patch of forest, satellites might lose the bird’s position. The resulting uncertainty is recorded as a margin of error, tracked as a radius of GPS drift. The higher the uncertainty within the system, the larger the radius. The tracking devices simply travel alongside them, sending signals back to tracking towers and satellites. When the birds move into spaces that hinder the signal, they introduce “drift” to the system. The distance of this drift can be roughly approximated to a radius if signals connect to just one or two satellites instead of all three, and that radius can be measured in feet or meters.

For my work, this radius of drift became a proxy variable for uncertainty and took on the feeling of a system improvising. The radius of uncertainty, a margin of error, created a liminal space between the bird and the machine designed to track it. By having the synthesizer write notes based on GPS drift radius, rather than the distance a bird traveled, the system changed. Rather than producing music from the location of a bird within human-designed coordinates, the program was now organized around uncertainty in a broader system. One might imagine the GPS tag as an instrument being “played” by the bird as it moved through different terrain, introducing greater degree of uncertainty into the system. The score was produced by an exchange between the bird’s movements and a computer’s abstract understanding of those movements (Box 2).

Box 2. Listen: “Music for an Eagle and Three Satellites”.

https://soundcloud.com/the-organizing-committee/music-for-an-eagle-and-three

The Politics of Uncertainty

To suggest poetry as a form of rigor within data science projects might strike some as heresy. Nonetheless, a growing number of feminist and intersectional scholars in the field are exploring the symbolic and ideological values embedded into a spreadsheet. Denying the role of subjective interpretation has created a wave of harmful machine-learning applications for data—but acknowledging subjectivity, rather than denying it, can lead to safer, more socially responsible systems.

Strict interpretations that ignore emotional and cultural interactions of data and their social contexts “promote moral detachment in scientists which, reinforced by specialization and bureaucratization, allows them to work on all sorts of dangerous and harmful projects with indifference to the human consequences.”⁵ Work from Joy Buolamwini and Timnit Gebru have shown how the lack of imagination for social contexts among predominantly white researchers has produced image recognition systems that harm, ignore, or perpetuate injustices against people of color. When we acknowledge that data are inscribed and interpreted along subjective and ideological frames, we can tap this imaginary space to identify harms that might arise when subjective interpretations are mistaken for neutral, objective approaches and deployed into the real world.

But the benefits are not only in harm reduction. Rejecting this mantle of empirical objectivity could allow for “more than one interpretation of any theory to be reasonable” and provide for “the growth of science in ever new directions.”⁵ When interpretations arise from similar cultural contexts and perspectives within teams of developers, researchers, or scientists, the data immediately loses vitality and possibility. What we today call “objective” and “neutral” interpretations are just one strand of imagination—a strand that all too often tends to conform to white, Western, male, and able-bodied frameworks. The rejection of this “neutral” orthodoxy, and the embrace of diverse interpretations of data and how we use them, would center and value challenges to knowledge in ways that contribute to the decolonization of the field’s imagination.

Contemporary art or poetry can bridge the “objective” imaginary—numbers, and the logic of digital machines—with the imaginary of cultural, social, and human experience. Computational poetics remind us that our interpretations of the world are personal, to be mindful that what we pull from datasets will never represent a a universal or generalizable truths.

Conclusion

Ideally, Music for an Eagle and Three Satellites is a space to ponder relationships and boundaries. I hope the work serves as a model for navigating frictions between emotional and computational logics through computational poetics, artmaking, and the analog in digital spaces. I want listeners to re-imagine the centrality of algorithms as “an ordering logic, a superstructure or ontology for how we organize meaning in our lives.”³ There are limits to this process, including the high level of human subjectivity—from the selection of musical scale and tonality, to the concepts of agency and intent attributed to birds as actors. As an artwork and thought experiment, the associations with birds and machines are embedded within a deep network of personal and cultural symbolism. By thinking through these elements of a system as an exchange, it reminds us that columns in spreadsheets carry poetic associations within cultural contexts, just as the symbolism associated with an eagle or sparrow guides our interpretations of their behaviors.

Biography

Eryk Salvaggio is an artist and researcher exploring the effects of digital mediation on relationships between living systems and machines. He is currently completing an MSc. in Applied Cybernetics at the 3A Institute at the Australian National University and holds an MSc. in Media and Communications from the London School of Economics.