Scaling up analogical innovation with crowds and AI

Abstract

Analogy—the ability to find and apply deep structural patterns across domains—has been fundamental to human innovation in science and technology. Today there is a growing opportunity to accelerate innovation by moving analogy out of a single person’s mind and distributing it across many information processors, both human and machine. Doing so has the potential to overcome cognitive fixation, scale to large idea repositories, and support complex problems with multiple constraints. Here we lay out a perspective on the future of scalable analogical innovation and first steps using crowds and artificial intelligence (AI) to augment creativity that quantitatively demonstrate the promise of the approach, as well as core challenges critical to realizing this vision.

The ability to find and apply analogies from other domains has been fundamental to human achievement across numerous domains, including architecture, design, technology, art, and mathematics (1–4). For example, in 2013 a group of engineers partnered with a world-renowned origami expert to design a large solar array to be carried by a narrow rocket. Using an analogy to origami-folding techniques, they were able to fit the array into 1/10th of its deployed size (5). The history of innovation in science and industry is replete with similar cases of analogical transfer, in which ideas from one domain are profitably used to solve a problem in another one (1, 6, 7): Salvador Luria advanced the theory of bacterial mutation by applying an analogy between bacteria and slot machines, and the Wright brothers used an analogy to a bicycle to design a steerable aircraft.

Despite its importance, finding and applying analogies to drive innovation remains challenging. While people can find deep relational similarities between domains given the appropriate data (8), the rapid increase of information makes it harder to mine any single field for analogies, much less identify deep similarities across multiple fields. Furthermore, the sensitivity of human memory to surface similarity means that people often become fixated on surface-level details that prevent them from retrieving distant analogs or applying them (9, 10). Finally, many real-world problems are complex and have multiple subproblems. Multiple analogies at different levels of abstraction might be needed to solve the set of subproblems. These three challenges—scalability, fixation, and complexity—are fundamental barriers to analogical innovation.

On the other hand, the recent explosion of data available online together with novel machine-learning and crowdsourcing techniques represents new opportunities to develop methods for finding analogies in multiple domains. For example, there are more than 9 million patents in the US Patent database; more than 2 million product and solution ideas submitted to ideation platforms such as InnoCentive, Kickstarter, Quirky, and OpenIDEO (www.InnoCentive.com, www.Kickstarter.com, www.Quirky.com, www.OpenIDEO.com); hundreds of millions of scientific papers and legal cases searchable on Google Scholar; and billions of web pages and videos searchable on the World Wide Web. Here we describe initial steps toward a future where people, augmented by machines, can search through billions of sources based on deep structural similarity rather than simple keywords to solve important societal problems. For example, scientists or designers might find potential solutions in other fields to the problems they are trying to solve, and lawyers or legal scholars might find legal precedents sharing similar systems of relations to a contemporary case.

The key insight from this paper is that instead of considering analogy as the province of a single mind (e.g., a “lone genius”), one can disaggregate the analogical processing typically done by a single individual into discrete steps assigned to different sets of individuals and/or machine agents. This approach has several potential advantages, including leveraging each agent’s complementary strengths while ameliorating their weaknesses; scaling up the number of agents to increase the number of potential analogies found; and capturing the mental work that each agent does so that it can be built upon by others. However, the disaggregation also introduces several new challenges, including how to coordinate many diverse agents’ efforts and determine which of the many possible configurations of human and machine processors are most effective at boosting analogical innovation.

Approach

In this paper, the approach to addressing these challenges builds on a foundation of past research in cognitive psychology, engineering, management, and design that has investigated the processes that humans use to find and apply analogies (e.g., refs. 4, 6, and 11–16) as well as research on the development of methods and tools to help people design by analogy (e.g., refs. 17–22). Analogical processing within a single individual’s mind has been extensively studied, with past research suggesting that it involves three core processes: abstracting the problem into a schema (i.e., representing problems and potential solutions in a way that drops out surface features and facilitates comparison), searching for analogies (i.e., identifying potentially fruitful domains that are distant from an initial problem and finding analogies within them), and applying the found analogies to generate solutions to the original problem (12, 13).

Fig. 1 (Top) demonstrates the process. For example, consider the task of designing a kindergarten chair. First, the designer abstracts away surface-level parts of the problem and focuses on core requirements (e.g., moveable by 5-y olds; does not tip over). Next, he or she uses the abstracted problem to search for inspirations in distant domains unfettered by the surface structure of the original domain, finding analogies such as spill-proof cups or Weeble Wobble toys. Finally, the designer picks one of the stabilization mechanisms found in other domains and applies it to the original problem, resulting in designs such as a bottom-heavy, egg-shaped chair that can right itself.

Fig. 1.

Model of the analogy-driven innovation process, consisting of three core processes: abstracting the problem, searching for analogies, and applying them to generate solutions to the original problem. Traditionally the process takes place inside one person’s head. Instead, we propose distributing the process across many information processors, both human and machine. Bottom summarizes a series of studies we have conducted with crowd- and AI-powered systems and the challenges they address (36–38, 46, 48, 50, 51).

In the following sections we explore how the abstraction, search, and application processes can be distributed in different configurations across multiple humans and machines. Fig. 1 (Bottom) summarizes our studies, visually illustrating different configurations we have experimented with, leveraging human and machine capabilities in various forms to overcome the key challenges of fixation, scalability and complexity.

Fixation

A recurring challenge in the science and practice of analogical innovation is that people fail to find analogies because of fixation. Human retrieval is highly sensitive to surface similarities, favoring “near,” or within-domain, analogs that share attributes of an object over “far,” or structurally similar analogs from different domains, that share relations to the object (8, 9, 13, 23). For example, people trying to solve Duncker and Lees’ (24) radiation problem are much more likely to retrieve analogs involving cancer or radiation than an analog of an army splitting up to attack a target. A large body of work has shown that designers often fail to retrieve relevant analogs from other domains because they are fixated on surface features of their source problem (e.g., refs. 25 and 26). Several approaches have been developed to help individual designers build more abstract problem representations (17, 27–29), such as encouraging them to generate multiple abstract representations of their problem or, in the case of problems that can be formalized, using specialized representations (such as functional ontologies) to represent the relational structure of their problems (e.g., refs. 30–33).

In contrast, the approach in this paper takes advantage of recent developments in web technologies and crowdsourcing that support assembling crowds of thousands of people across the globe to engage in complex cognitive tasks (34, 35). In the case of designing a kindergarten chair (Fig. 1), one set of people could abstract the problem. Downstream, a different set of people could use the abstracted problem and search for inspirations in distant domains, unfettered by fixation on the source problem. Yet another set of people could apply the mechanisms discovered in the analogies to the original problem. Such a process separates people who develop schemas of an original problem from people who find inspirations and solve the problem, thus avoiding fixation and turning a core challenge of collaborative cognition—that each person sees only a small part of the whole—into a strength.

This process could fail in several places. Crowd workers might not be sufficiently versed in either the source domain or the process of abstracting the problem to generate a good schema. Subsequent workers given the schema might not have sufficient context without seeing the details of the source problem (e.g., without knowing that the problem is about kindergarten chairs) to find relevant and useful analogical inspirations from other domains. Finally, previous work has shown that even when an analogical inspiration is presented, people often fail to recognize the deep structural relations and transfer them to the target domain (8, 9, 13).

In a series of experiments (36, 37) we explored the value of this distributed process and of abstracted schemas (see Fig. 2 for an example of a product and its schema). These and all other studies described here were approved by the Carnegie Mellon Institutional Review Board and included informed consent from all participants. Yu et al. (36) developed a brief training procedure to help crowd workers learn to induce schemas: They were shown a rerepresentation of a concrete product (e.g., a device to separate leaves from a rake) into a problem schema specifying a purpose (e.g., detaches things) and a mechanism (e.g., uses comb-like features), omitting surface details. Given this training, crowds were able to generate schemas rated as high quality by multiple expert judges and that were subsequently used to find inspirations from diverse domains. Crowds generated high-quality schemas when they were asked to identify the common principles behind multiple related products but could not do as well when trying to generate a schema for a single product.

Fig. 2.

Example of a concrete product description and a schema representation of it.

After the abstraction stage, a second group of crowd workers found analogical inspirations based on the schemas and then a third group solved the original, concrete problem, taking advantage of the inspirations. This led to significantly more novel and useful ideas than traditional design techniques, such as brainstorming or using concrete examples for inspiration. When using an abstract schema, people explored significantly more diverse domains and avoided fixation on surface features of the concrete examples from which schemas were derived. Furthermore, breaking up the process of analogical transfer into stages allowed intermediate evaluations, enabling the selective retention of the best outputs from each stage.

We have explored two alternate configurations of distributed analogical innovation. In problem-based ideation (37), a designer already has a problem in mind and searches a data repository for mechanisms to address it. Using the crowdsourcing process described above, crowds were asked to search through a community-generated idea repository (www.Quirky.com) for inspirations given the abstract schema of a problem, concrete features of the problem, or the original problem description itself. Crowds given the abstract problem schema found significantly more analogs that matched the deep problem structure in the repository. Moreover, other participants given a sample of the schema-inspired analogs generated more creative ideas than those under the other conditions. Free ideation (36) instead starts from the identification of an interesting problem–mechanism schema from one or more examples (e.g., a power strip that supports more plugs by placing them at different heights) and uses it to first identify other domains in which the schema could be applied (e.g., storing planes in a hangar so they do not block each other) and then apply the mechanism to that domain to generate a solution (e.g., designing sections of a hangar at different heights so planes’ wings can overlap but not touch). Crowds were able to reliably generate high-quality schemas from multiple examples, use those schemas to find distant and useful domains and analogs, and generate more creative ideas than they were able to in control conditions.

Scalability

Another key challenge with analogy is scalability. The work described in the previous section looked at relatively small idea repositories, where searchers were looking through hundreds of ideas. Today, more sources of potential analogies are easily available than ever before—millions of patents, research papers, videos, products, web pages, and more. However, searching through these sources to find distant but useful analogs often becomes overwhelming. This section describes two processes for reliably finding useful analogical inspirations in very large datasets. The first one is crowd based: Crowd workers first identify types of experts who are likely to have interesting insights into solving an abstracted problem, and then new workers retrieve inspirations from those expert domains. The second system uses artificial intelligence (AI) to go through large datasets and suggest potential analogies.

With Crowds.

Finding “outside-the-box” inspirations in very large idea repositories such as the web is difficult, because without guidance, people select domains to search based on surface similarity to the problem’s domain (similar to the challenge in Fixation above). To address this challenge, we explored a two-stage process (38) in which crowds first identify domains of expertise that are distant from the initial problem but relevant enough to inspire useful and nonobvious solutions. For example, we rerepresented the problem of fitting more plugs into a power strip with its schematic representations, e.g., fitting objects of different sizes into a container. We then asked crowd workers to use the schematic representations to recommend relevant domains of expertise. For example, they were asked to “suggest three types of experts who might provide useful or interesting perspectives in solving [the schematic problem] and explain why,” resulting in relevant expert domains such as “plumber,” “magician,” and “architect.”

In the second stage, new workers searched in these domains to find inspirations that might be adapted to solve the original problem. The key insight here is that a rich set of expert-generated ideas, solutions, and skills has already been documented on the web, such that nonexperts might find and appropriate these resources and suggest promising directions despite not possessing deep expertise themselves.

Crowd workers who were given the abstract problem schema identified more distant and fruitful expert domains in which to search, compared with crowd workers who were given the original problem description. For example, those given the problem schema of “fit objects of different sizes into a container” identified distant professions such as contortionists, carpenters, or experts on Japanese aesthetics, whereas workers given the original problem description identified professions closer in domain to power strips, such as computer technicians, electricians, or interior designers. When workers searched the web for inspirations in the distant domains, they found interesting analogs: For example, individuals searching in a carpenter’s domain retrieved curved drawers as analogs, and those searching in the domain of Japanese aesthetics retrieved multilevel buildings. Adapting analogs from the abstract problem condition generated significantly more creative ideas, such as placing plugs in curved lines or at differing heights, respectively (38).

With AI.

Another approach to scalability involves using AI to winnow down large datasets to promising analogically similar items. Doing so may have complementary benefits to using crowds, with the machine able to process millions or billions of items at a speed unrivaled by people. Thus, even if the results of the AI’s search are noisy and require humans to inspect, select, and adapt the found analogs to generate creative solutions, boosting the likelihood of humans processing useful analogs could nonetheless significantly accelerate innovation.

Finding analogies is challenging for machines because doing so requires an understanding of the deep relational similarity between two entities that may be very different in terms of surface attributes (39). For example, Chrysippus’ analogy between sound waves and water waves required ignoring many surface differences between the two, such as the viscous liquid nature of water or its visibility (8). Much work addressing this in computational analogy has focused on fully structured data, often with logic-based representations. For example, Gentner and coworkers’ (40) seminal structure-mapping engine relies on predicate calculus representations. In contrast, most descriptions of products, problems, or ideas in existing online databases are short or sparse or lack consistent structure. Although logic-based representations are very expressive, they can be difficult to generate reliably from unstructured text, even for seemingly simple items, and doing so requires significant training and effort. Representations of more complex items, like biological organisms, can take tens of person hours per item (31). More concerning from our standpoint, automatic extraction of relational representations across domains remains a difficult open problem (41).

Conversely, recent advances in data mining and information retrieval rely on regularities in the surface attributes of language (e.g., word co-occurrence, parts of speech) (42) to calculate similarity measures, for example word embeddings (43), vector-space models like latent semantic indexing (44), and probabilistic topic modeling approaches like latent Dirichlet allocation (45). While these approaches scale well on raw text and excel at detecting surface similarity, they are often unable to detect deep relational similarity between documents whose word distributions are disparate. This problem is especially acute when the source and target domains are different.

We introduce two insights that together make this problem tractable for analogical innovation. First, rather than trying to solve the problem of fully structured analogical reasoning, this approach exploits the idea that for retrieving practically useful analogies, one can use coarse structural representations that can be learned and reasoned with at scale (in other words, trading off expressivity for ease of extraction). Specifically, our research investigated whether the purpose and mechanism representations, which proved useful in the experiments described above, are useful ways to represent and find useful analogies computationally. The second insight is that advances in crowdsourcing have made it possible to harvest rich signals of analogical structure that can help machine-learning models learn in ways that would not be possible with existing datasets alone.

Our approach for computationally finding analogies from unstructured text, based on these two ideas, allows for search that goes beyond surface features (46). At a high level, this approach collects behavioral traces of crowd workers as they search for analogies and label the purpose and mechanisms embedded in the analogs and then uses those traces to develop AI models and similarity metrics suited for analogy mining. Specifically, crowds annotated the parts of a product description that they considered to be about the purposes of the product (i.e., what it is good for) and the parts related to mechanisms (i.e., how it works), as shown in the example in Fig. 3. Using a corpus of purpose and mechanism label pairs, a recurrent neural network was trained to take raw product description text and output vector representations of products’ purpose and mechanism at scale.

Fig. 3.

We asked crowd workers to annotate the parts of the product description that they considered to be about the purposes of the product (i.e., What is it good for?) and the parts related to mechanisms (i.e., how it works). We used these annotations to train a recurrent neural network to take raw product text and output representations of the product’s purpose and mechanism. Republished with permission of Association for Computing Machinery, from ref. 46; permission conveyed through Copyright Clearance Center, Inc.

These representations enable a type of analogical search by finding items that closely match on one dimension (e.g., having a similar purpose) but are distant on another dimension (e.g., using a different mechanism). Since both dimensions are represented as vector similarity spaces, to provide diverse results, the system clusters the distant items and provides exemplars from different clusters that span the similarity space.

The system’s ability to help generate creative ideas was evaluated through a standard ideation task, in which participants were asked to redesign an existing product (47), for example, a cell phone case that also charges the phone. Participants were given inspirations drawn from the purpose–mechanism representation approach (i.e., products with a similar purpose and a different mechanism), a standard information retrieval baseline, or a random baseline (Fig. 4). As illustrated in Fig. 4, the standard baseline retrieved highly relevant but nondiverse results (e.g., cell phone cases and chargers); the random baseline returned highly diverse but less relevant results (e.g., shampoo pods and meetup apps); while the purpose–mechanism approach returned diverse results while maintaining structural relevance (e.g., backup batteries for other products, motion-powered generators), thus allowing the user to explore distant but relevant parts of the design space. Participants given inspirations retrieved using the purpose–mechanism approach generated approximately twice as many ideas rated as good by judges blind to condition compared with those in the baseline conditions (see ref. 46 for details).

Fig. 4.

Overview and excerpts of the ideation experiment, comparing analogical inspirations found using scalable coarse structural representations and baseline methods. (Top) Seed product. Workers were asked to solve the same problem in a different way. (Middle) Top three inspirations for each condition. The TF-IDF baseline returns results from the same domain, while our method returns a broader range of products. (Bottom) Ideas generated by users exposed to the different conditions. Republished with permission of Association for Computing Machinery, from ref. 46; permission conveyed through Copyright Clearance Center, Inc.

Extending to More Complex Domains.

While the purpose–mechanism schema approach was useful for the domain of product innovation explored above, it is possible that other domains might require different structures. For example, more complex artifacts such as research papers, patents, or legal cases have not only more text than a typical product description, but also multiple distinct purposes that are hierarchically dependent on each other. For example, a research paper might explore how to reduce the risk of harm in algorithmic systems (higher-level problem) by enabling citizens to audit the algorithms pervading their digital lives (lower-level subproblem). Furthermore, mechanisms contributed by a given paper are typically most directly and causally related to the lower-level subproblems (e.g., generating interpretable rationales for algorithmic predictions), suggesting that analogical search over research papers is likely to be more fruitful if matching is done on the lower-level subproblems.

To test this intuition, the original purpose–mechanism scheme was adapted to incorporate two new elements: the higher-level problem (“background”) and the results of what the study found (“findings”). This yielded an annotation scheme with four elements (48): (i) background [What other (higher-level) goals/questions can be furthered by this work? How might this help other research(ers)?], (ii) purpose [What specific thing(s) do the paper’s authors want to do or know?], (iii) mechanism (How did the paper’s authors do it or find out?), and (iv) findings (Did it work? What did the paper’s authors find out?). Fig. 5 shows an example of a scientific research paper’s abstract annotated with this scheme.

Fig. 5.

Example of a research paper abstract annotated with our modified four-part annotation scheme, to support analogical queries over large datasets of complex research papers. Republished with permission of Association for Computing Machinery, from ref. 48; permission conveyed through Copyright Clearance Center, Inc.

The resulting vector representations of a paper’s abstract enabled discovery of known analogies in research papers with substantially higher precision than did a baseline using vector representations of all of the words in the abstracts (49). Importantly, ranking pairs based on similarity of purpose and mechanism but explicitly ignoring background yielded even higher rates of analogy finding; this result suggests that separating the higher-level and lower-level purposes of a paper may be an important extension to the purpose–mechanism approach. In a case study involving a domain expert in mechanical engineering who had spent months seeking analogies from other domains (e.g., materials science, civil engineering, aerospace engineering), our approach provided twice as many papers that the expert identified as valuable, unexplored sources of new ideas compared with a standard machine-learning baseline. These results suggest that the strategy of obtaining intermediate structural representations holds promise for enabling AI systems to suggest useful analogies from a range of different real-world datasets, ranging from relatively simple consumer product descriptions to complex research papers.

Complexity

Exploring increasingly complex real-world problems quickly reveals the challenges with assuming that problems involve only a single analogical schema. Instead, many product design and engineering problems involve multiple, often conflicting schemas. For example, the design of a kindergarten chair requires addressing multiple constraints, such as its safety (preventing it from tipping over or pinching fingers) and flexibility (making it easy to move or store). Furthermore, each of these constraints could be represented at multiple levels of abstraction, with more abstract levels (e.g., “safety”) potentially matching more analogs but running the risk of including less relevant analogs to the target problem. Below, we discuss extending our general approach to support multiple constraints and developing a computational system to help augment people’s exploration of multiple constraints and levels of abstraction.

With Crowds.

We developed reliable crowd processes for decomposing complex multiconstraint problems into multiple-constraint schemas (e.g., safety vs. flexibility), manipulating the level of abstraction for each of those schemas (e.g., safety vs. pinching fingers), and then integrating the analogs found for each of those schemas into a solution that addressed each of the relevant constraints (50). This research found that crowd workers could transform an ill-formed, open-ended design problem (e.g., design a creative kindergarten chair) into a better-structured statement comprising well-defined constraints (e.g., design a chair that is easily movable, is stackable, will not tip over, and protects extremities). Other crowd workers could then find diverse inspirational examples in remote domains that could satisfy the constraints in novel ways, such as Weeble dolls that right themselves and flying buttress-stabilized coffee cups. Using these inspirations, other crowd workers were able to integrate all of the constraints to create designs for chairs that were judged better (i.e., more practical and original) than when the inspirations they received were generated from exposure to the original problem representation.

One important factor that affected the usefulness of the inspirations was the level of abstraction for the constraint schemas. Compared with higher-level schemas that abstracted away the specific nature of the constraint (e.g., safety), schemas that made the constraint concrete (e.g., prevent tipping over) led to more relevant inspirations that were still distant from the original problem domain. They led to significantly better chair designs.

With AI.

This approach of addressing complexity with crowds highlights the critical importance of supporting designers in both focusing on multiple, specific constraints and targeted abstraction of those constraints, allowing manipulations of the level of abstraction. Building on our AI approach to augmenting analogical innovation, in ref. 51 we introduced a system in which a designer can specify a focus for a given product description and then abstract that focus beyond its surface features in a targeted manner by specifying the key properties of the relations and entities involved that are crucial for understanding the core relational structure.

For example, consider a designer who is interested in exploring ways to adjust a soap dish to soap bars of different sizes. Fig. 6 (Top) demonstrates our interface. Designers first focus on a part of the product description (“extendable for different sizes of soap bars”). Next, they abstract the problem, keeping some properties of soap (like “personal product”) and dropping out others such as its color, water solubility, or cleaning function. Similarly, the designer can abstract the word “size” in the original product description to consider products that can adjust to different heights, weights, or other spatial quantities. Some words, like “bars,” are irrelevant to the abstraction and are thus dropped.

Fig. 6.

We built a system that allows designers to focus on a relevant aspect of a problem (Top) and then abstract this aspect of the problem using common-sense knowledge bases. The system then finds analogies (Bottom) for the focus-abstracted query, supporting an analogical search for complex problems with multiple aspects and levels of abstraction. Republished with permission of Association for Computing Machinery, from ref. 51; permission conveyed through Copyright Clearance Center, Inc.

Our system then uses this focus-abstracted query to computationally search a large database of commonsense knowledge (52) for analogically relevant matches tuned to the designer’s specific needs. In particular, the system first abstracts the corpus according to the query and then looks for matches. Fig. 6 demonstrates the abstracted query and shows example matches found for the soap dish problem—a knife rolodex with multiple slots for different sizes and a telescopic frame adjusting to different phones, which the designer might adapt to the soap dish.

The system was evaluated across a range of focused query scenarios from seeds sampled from Quirky. For example, when designing a camping coffee maker, a designer might be interested in separately exploring the inspiration space around cooking without electricity or around notifying the user when the food is done. Significant benefits were found for both the focusing aspect of the system (which provided more relevance compared with wholesale matching) and specifying the level of abstraction (which returned more distant inspirations compared with traditional machine-learning baselines). Formative studies with designers suggested that they found these aspects of the tool useful in more systematically engaging with aspects of the problem and abstractions, leading them to consider possibilities they had not previously thought of.

Discussion

In summary, this paper described ways of distributing the process of analogical innovation across many human and machine information processors, mediated through a computational system that employs each to its maximum benefit. A series of crowdsourcing processes and AI systems demonstrate the promise of scaling up serendipity in a distributed way. These processes and systems overcome three key challenges to distributing analogy: (i) fixation, allowing humans and machines to look beyond surface features to find structurally similar analogs in distant domains; (ii) scalability, scaling up the process of finding analogs in large idea repositories; and (iii) complexity, supporting multiple constraints and levels of abstraction. See Table 1 for a summary of selected findings.

Table 1.

Summary of selected results

Challenges	Findings
Fixation	Schemas help crowds find more analogies, especially far analogies (not sharing surface features). Crowds are able to reliably
	generate high-quality schemas.
Scalability	Schemas help crowds identify a more diverse set of domains to explore, which in turn results in more creative solutions. Coarse
	representations that can be learned and reasoned with at scale can enable AI systems to suggest useful analogies from very
	large, unstructured corpora.
Complexity	Crowds can generate multiple constraints at different levels of abstraction. The level of abstraction influences the quality of the
	solutions (abstract domain, concrete constraints worked best). AI can help users generate targeted abstractions. Analogies
	suggested by the system achieved higher relevance and domain distance (far analogies).

Increasing the efficiency of the methods used to use large numbers of people in the analogical innovation process through distributed analogy could have a number of important benefits. First, increasing the number of people involved increases the capacity to find and use analogies across many domains. Second, by distributing the steps to different people, alternative innovation paths could be explored effectively and in parallel. Third, disaggregating the innovation pipeline opens up participation to more types of people; one need not be an expert in a problem domain, for example, to find analog solutions in remote domains that experts have already proposed. Fourth, in contrast to innovation contests such as InnoCentive that recruit many people to innovate but reward only a few contest winners (53), the sequential method proposed here allows people to build on each other’s work and preserves the value of labor from the majority of participants.

Involving AI in the process adds complementary benefits. While people are unparalleled in their ability to induce and apply deep relational schemas from unstructured real-world data, they are limited in their ability to broadly search across huge repositories of potential analogs. Our research has shown how using coarse structural schemas (i.e., purposes and mechanisms) at scale can power AI approaches that can find more analogs in distant domains than current machine-learning approaches, which are highly influenced by surface features. This computational approach can play a valuable role by identifying inspirations for human problem solvers by performing a first pass of sifting through vast idea repositories of millions (e.g., patents) or billions (e.g., the web) of items.

We have explored only a small number of the possible configurations of people and machine processors for boosting analogical discovery. These explorations (including several unsuccessful ones not discussed here) suggest several principles about which configurations lead to successful outcomes and what challenges remain to be addressed. Specifically, we have found that humans are needed in several points in the process—vetting candidate inspirations retrieved by AI, applying them to the problem at hand, and integrating across multiple potentially conflicting inspirations. This brings up questions that require future study about where and when in a distributed analogical innovation pipeline to deploy what kinds of expertise for maximum effectiveness. Domain expertise can have positive effects, for example enabling deep understanding of problem constraints and ways to adapt new solutions. On the other hand, it can also have negative effects, including greater fixation on defining the problem, finding solutions, and evaluating their suitability. We speculate that adapting and integrating analogical inspirations to solve complex R&D problems is likely to require deep knowledge of the various constraints of a domain (e.g., business, technical, operational). In contrast, the interstitial steps of the pipeline, including abstraction and finding domains and analogies, may be less sensitive to the need for domain knowledge or in fact might benefit from including people outside of the domain to increase diversity. However, our research has not yet tackled the role of human expertise.

The role of AI in the innovation pipeline is also a rich subject for future study. In the short term, additional training data for purpose–mechanism representations could likely improve the hit rate of relevant analogies found and, with minor modifications, be useful even for complex domains such as scientific literature and patents. However, we believe there is a long way to go to support the true hierarchical, complex, and interrelated nature of real-world problems. Increasing the expressivity of representation to model hierarchies of subproblems and multiple levels of abstraction among them could have significant additional benefits. For example, being able to automatically model and explore the space of multiple subproblems could vastly expand a designer’s capabilities to explore a problem space, similar to how approaches in computer-aided design can already do so for more formalized domains such as material and shape. Furthermore, many scientific analogies rely on systems of interconnected relations rather than independent relations. For example, the Bohr model of the atom is similar to the structure of the solar system not only because the electrons orbit the nucleus but also in their relative size compared with the nucleus; meanwhile, the high speed of electrons (vs. planets) suggests that other factors such as relativity might be in play. Supporting such interrelated relations could greatly increase the expressivity and power of both crowd and computational approaches to distributed analogy. However, increases in expressivity will likely need to involve further research advances in enabling AI to learn from existing large-scale but noisy datasets and to account for any algorithmic biases in those datasets.

In summary, we have presented one path toward transforming the process of innovation toward a more on-demand, persistent, and reliable one. However, the research described here explores only a small subset of the possible configurations of distributed human and AI cognition that may be effective in boosting innovation. We hope that future researchers may be inspired by these examples and further accelerate the process of finding innovative solutions to the diverse set of challenging and important problems that our society faces.