Skip to main content
NCBI home page
Ecology and Evolution logoLink to Ecology and Evolution
. 2016 Mar 30;6(9):3026–3031. doi: 10.1002/ece3.2078

TiLIA: a software package for image analysis of firefly flash patterns

Junsuke Konno 1, Yoko Hatta‐Ohashi 1, Ryutaro Akiyoshi 1, Anchana Thancharoen 2, Somyot Silalom 3, Watana Sakchoowong 4, Vor Yiu 5, Nobuyoshi Ohba 6, Hirobumi Suzuki 1,✉
PMCID: PMC4814377  PMID: 27069594

Abstract

As flash signaling patterns of fireflies are species specific, signal‐pattern analysis is important for understanding this system of communication. Here, we present time‐lapse image analysis (TiLIA), a free open‐source software package for signal and flight pattern analyses of fireflies that uses video‐recorded image data. TiLIA enables flight path tracing of individual fireflies and provides frame‐by‐frame coordinates and light intensity data. As an example of TiLIA capabilities, we demonstrate flash pattern analysis of the fireflies Luciola cruciata and L. lateralis during courtship behavior.

Keywords: Flash communication, flight path tracing, time‐lapse image analysis

Introduction

Light production in adult fireflies is a mating adaptation (McDermott 1910; Mast 1912; Barber and McDermott 1951), and species‐specific mating‐related flash signals play a role in reproductive isolation (Lloyd 1966). Thus, signal‐pattern analysis is important for understanding flash communication systems of the firefly.

In the earliest studies, the relative intensities and pulse duration of firefly flash signals and behaviors were estimated and observed by eye (Osten‐Sacken 1861; Emery 1887; McDermott 1910; Mast 1912), and McDermott (1914, 1917) drew a diagram of male and female flash patterns of some firefly species. Since the middle 1900s, several methods for recording firefly flash patterns have been used. Barber and McDermott (1951) measured the pulse duration and frequency of flashes by stopwatch and subjectively described the intensity and color of the flashes. Lloyd (1966, 1969) developed a recording system in which the flash was detected by a photo‐sensor, and the light intensity was transduced to a frequency‐modulated audio signal, which was recorded on a magnetic tape. The recorded tone was then transduced to direct current voltage that was fed into an oscilloscope and a chart recorder. With this system, which was used for observations in both the laboratory and the field (Seliger et al. 1964; Buck and Buck 1968; Papi 1969; Lloyd 1973, 1984; Ohba 1979, 1983), the female response to male or artificial signals could be analyzed within 10 msec. However, the working distance of the photo‐sensor and the ability to analyze multiple individuals simultaneously were limited.

The recording of flash images on film was attempted (Buck and Buck 1968; Ohba 1980), but insufficient photosensitivity of the film restricted this approach to the analysis of flash intervals only. Since that time, video recording coupled with an image intensifier has allowed flash signals to be analyzed in more detail (Ohba 1984). The flight path of the firefly can be traced with a hand‐held photo‐sensor manually on a monitor screen, and the light intensity of a firefly can be transduced to direct current voltage that is fed into a chart recorder or a personal computer (Makino et al. 1994; Copeland and Moiseff 1995). With this system, multiple studies have described flash patterns of numerous firefly species together with their behavior and ecology (Copeland and Moiseff 1995, 1997; Ohba et al. 1995, 2001; Moiseff and Copeland 2000; Ohba 2000, 2001; Ohba and Wong 2004; Fu et al. 2006; Fu and Ballantyne 2008; Ohba and Shimoyama 2009; Ballantyne et al. 2013). Although flashing behavior is recorded on a videotape, this system was also initially restricted to the analysis of flash intervals.

Recently, the photosensitivity of digital video recorders has improved, and flash patterns of fireflies can be recorded without an image intensifier (Kawano 2009; Yiu 2013). This allows much easier spatiotemporal image analysis of flash patterns by a personal computer. However, commercially available software packages for spatiotemporal analyses of time‐lapse images tend to be too expensive and complicated for use in this simple analysis. Therefore, we developed time‐lapse image analysis (TiLIA), a free open‐source software package for signal and flight pattern analyses of fireflies using video‐recorded image data. TiLIA enables flight path tracing of individual fireflies and provides frame‐by‐frame coordinates and light intensity data chronologically. The method and open‐source application proposed in this study is a step forward into the understanding of flash signal communication in fireflies, species identification, and even for the study of other (marine) organisms using bioluminescent displays for interaction, communication, and/or courtship. As an example of TiLIA capabilities, we performed flash pattern analysis of the fireflies Luciola cruciata Motschulsky and Luciola lateralis Motschulsky and presented graphical male‐ and female‐specific flash patterns during courtship of L. lateralis.

Materials and Methods

The TiLIA package

TiLIA is operated with the following three steps (Fig. 1): A) trim a scene of video for analysis and convert it to audio video interleave (AVI)‐format file; B) trace the flight path of object and analyze the flight path and light intensity; and C) record the output results as image, video, and text data. Figure 2 shows the graphical user interface of TiLIA, which is composed of several spaces and areas: A) image display space, B) data analysis space, C) image property adjustment area, D) region of interest (ROI) select area, E) video operation area, and F) command area. An AVI‐format video file of a luminescent object is loaded into TiLIA, and each frame is saved as a separate tagged image file format (TIFF) file. The brightness and contrast of each image are adjusted (original image file is not rewritten), and the appropriate size of the ROI is assigned to an object in the image display space (Fig. 2A). The object is then manually (by mouse) traced frame by frame. This process is very time‐consuming; therefore, a scene selected for analysis is trimmed appropriately using a video editing software or TiLIA in advance. The light intensity value of the ROI in each frame is recorded, and the resulting time course is plotted graphically in the data analysis space (Fig. 2B). The tracing process can be confirmed by viewing the reintegrated video file, and the position of the ROI can be adjusted as necessary. The flight track of the ROI can also be displayed in the video. The graph of time course of light intensity (Fig. 2B) is saved as an image file, and the light intensity values are stored as text data. If flight paths of two fireflies coincide, the two assigned ROIs trace the two paths superpositionally. However, it is impossible to distinguish between the light intensities of two fireflies. Images of the ROI and of flight path, and the ROI coordinates data are also stored.

Figure 1.

Figure 1

Open in a new tab

Flowchart for use of TiLIA. (A) A scene of video for analysis is trimmed, and the image file is converted to AVI‐formatted video file. (B) Flight path of the object is traced, and the flight path and light intensity are analyzed. (C) Outputs of flight path, light intensity, and movie are recorded as image, text, and video data.

Figure 2.

Figure 2

Open in a new tab

The graphical user interface of TiLIA, which is composed of several spaces and areas: (A) image display space, (B) data analysis space, (C) image property adjustment area, (D) ROI select area, (E) video operation area, and (F) command area. The five ROIs are assigned to Luciola cruciata fireflies individually in the image display space (A), and the fireflies are then traced within the ROI manually frame by frame. The time course of light intensity for each ROI is simultaneously displayed as a graph in the data analysis space (B). The flash patterns are described as four flashes of single peak within 8.0 sec except for the blue one (three peaks), and the flash duration and peak interval are 1.3 and 1.9 sec, respectively. The flashes are loosely synchronous. The red vertical line indicates the frame number of the image displayed.

Results and Discussion

Example 1

Luciola cruciata is a species that displays synchronous flashes. After sun set, the males begin to fly, flashing and synchronizing their flashes slowly (Ohba 1983, 1984, 2004). The video of the male's synchronous flashes in the field (Kyoto, Japan) was converted to AVI‐format file and used for analysis by TiLIA. Figure 2A shows an image with assigned ROIs for five males, and Fig. 2B, the graph of time course of light intensity for the five males is indicated by red, brown, green, purple, and blue. The flash patterns were described as four flashes of single peak within 8.0 sec except for the blue one (three peaks). The average flash duration and peak interval were 1.3 and 1.9 sec, respectively. Their flashes were loosely synchronous, as they tended to show flash peaks around the same moment. The flight path was described using an orbit line and the light intensity was indicated with a circle size optimally allocated at seven steps on the orbit line (Fig. 3). With our previous system (Ohba 1984; Makino et al. 1994), it was very difficult to trace multiple individuals with photo‐sensor data because it required rewinding the tape and adjusting the starting point for each individual (i.e., ROI). Image analysis with TiLIA alleviates these issues.

Figure 3.

Figure 3

Open in a new tab

Flight path of male Luciola cruciata. Light intensity is indicated by a circle size optimally allocated at seven steps on the orbit line.

Example 2

Flash communication and mating behavior of L. lateralis have been intensively studied (Ohba 1983, 2004). This communication system can be described in five distinct phases: (1) A flying male emits single‐pulsed flashes and seeks out the female flash pattern that is unique to this species; (2) a female recognizes the male flashes and her interflash interval reduces gradually and the female's unique flash plays an important role in attracting the male; (3) the male approaches the female on the ground and changes his flash pattern to twinkling flashes that, upon visual inspection, appear to be a single flash accompanied with minor blinking flashes; (4) both sexes continue emitting their distinctive flashes, and their interflash intervals are keep on further reducing and the timing of the male–female response flashes does not appear to be critical; and (5) thereafter, they copulate.

A pair of L. lateralis was placed in a plastic container, and the flash communication between male and female was recorded by a video camera (Handycam EVCX10, Sony, Tokyo, Japan) equipped with an image intensifier (Star Light Scope, Hamamatsu Photonics, Shizuoka, Japan). The video of the courtship before copulation (phase 4) was converted to an AVI‐formatted file and used for analysis by TiLIA. Figure 4 shows an image with assigned ROIs 1 (red circle) and 2 (blue circle) for male and female fireflies, respectively (left panel), and the graph of the time course of the light intensity of the two ROIs for 3.3 sec (right panel). The male flash pattern (red line) was described as a bimodal double‐pulsed flash (indicated by arrows in the figure). The flash duration, the interflash interval, and the bimodal double‐pulsed interval were 566, 624, and 140 msec, respectively, in average of the five peaks in the graph. The image analysis revealed the twinkling flash recognized by naked eye as a bimodal double‐pulsed flash with an interval of 140 msec. On the other hand, the unique female flash pattern (blue line) was described as a very rapid double flash, triple to quadruple flash. Thus, male‐ and female‐specific flash patterns and the exchange of their flashes during courtship were represented in the graph. The AVI file of male and female activity together with the time‐course analysis is provided in Supporting Information, Appendix S1.

Figure 4.

Figure 4

Open in a new tab

Analysis of flash communication of Luciola lateralis by TiLIA. Male and female fireflies are represented by ROIs 1 (red circle) and 2 (blue circle), respectively (left panel), and the time course of light intensities for the two ROIs is displayed as a graph within 3.3 sec (right panel). The male flash pattern (red line) is described as a bimodal double‐pulsed flash (indicated by arrows), and the flash duration, interflash interval, and the bimodal double‐pulsed interval are 566, 624, and 140 msec, respectively. The female flash pattern comprises very rapid double flash, and triple to quadruple flashes. The timer is stamped, sec and msec. The red vertical line indicates the frame number of the image displayed. The integrated video file is provided in Supporting Information, Appendix S1.

Further Suggested Uses

TiLIA generates output in the form of time‐series text data of light intensity and coordinates of ROIs. These data can be used for further analyses with other software packages. For example, a power spectrum analysis can be applied to flash pattern data to assess flash–pulse component relationships with behavior or among species (Ohba et al. 1995; Ohba and Shimoyama 2009). A correlation function analysis can be used to assess the degree of synchrony of flashes among individuals. A fractal dimension analysis can be applied to the coordinate data to assess flight pattern characteristics in relation to behavior or environment (Shibue et al. 1995).

Software Accessibility

The TiLIA software and user's manual can be found at Google Drive (https://drive.google.com/open?id=0B2o7FRVs2VohMmx2QzBVX3ZDeDA).

Conflict of Interest

None declared.

Supporting information

Appendix S1. The integrated video results from TiLIA that traced male and female L. lateralis courtship behavior, with the time‐course analysis of flash patterns.

Click here for additional data file. (18.9MB, avi)

Acknowledgments

We are grateful to our colleagues at Olympus Corporation who provided support in testing TiLIA during its development. This work was a collaborative project between Queen Sirikit Botanic Garden (Chiang Mai, Thailand) and Olympus Corporation (Tokyo, Japan) registered under the National Research Council of Thailand (Project ID: 2008/089).

References

  1. Ballantyne, L. , Fu X., Lambkin C., Jeng M. L., Faust L., Wijekoon W. M. C. D., et al. 2013. Studies on south‐east Asian fireflies: Abscondita, a new genus with details of life history, flashing patterns and behaviour of Abs. chinensis (L.) and Abs. terminalis (Olivier) (Coleoptera: Lampyridae: Luciolinae). Zootaxa 3721:1–48. [DOI] [PubMed] [Google Scholar]
  2. Barber, H. S. , and McDermott F. A.. 1951. North American fireflies of the genus Photuris . Smithson. Misc. Coll. 117:1–58. [Google Scholar]
  3. Buck, J. , and Buck E.. 1968. Mechanism of rhythmic synchronous flashing of fireflies. Science 199:1319–1327. [DOI] [PubMed] [Google Scholar]
  4. Copeland, J. , and Moiseff A.. 1995. The occurrence of synchrony in the North American firefly Photinus carolinus (Coleoptera: Lampyridae). J. Insect Behav. 8:381–394. [Google Scholar]
  5. Copeland, J. , and Moiseff A.. 1997. The effect of flash duration and flash shape on entrainment in Pteroptyx malaccae, a synchronic Southeast Asian firefly. J. Insect Physiol. 43:965–971. [DOI] [PubMed] [Google Scholar]
  6. Emery, C. 1887. La luce negli amori delle lucciole. Bull. Soc. Entomol. Ital. 18:406–411. [Google Scholar]
  7. Fu, X. , and Ballantyne L.. 2008. Taxonomy and behaviour of lucioline fireflies (Coleoptera: Lampyridae: Luciolinae) with redefinition and new species of Pygoluciola Wittmer from mainland China and review of Luciola Laporte. Zootaxa 1733:1–44. [Google Scholar]
  8. Fu, X. , Ohba N., Venel F., and Lei C.. 2006. Life cycle and behaviour of the aquatic firefly, Luciola leii (Coleoptera: Lampyridae) from mainland China. Can. Entomol. 138:860–870. [Google Scholar]
  9. Kawano, K. 2009. Analyses on flash communication of two fireflies, Luciola cruciate and Curtos costipennis, in field habitats. Bull. Firefly Mus. Toyota Town 2:19–27. [Google Scholar]
  10. Lloyd, J. E. 1966. Studies on the flash communication system in Photinus fireflies. Misc. Pub. Mus. Zool. Univ. Mich. 130:1–95. [Google Scholar]
  11. Lloyd, J. E. 1969. Flashes, behavior and additional species of Nearctic Photinus fireflies (Coleoptera: Lampyridae). Coleopt. Bull. 23:29–40. [Google Scholar]
  12. Lloyd, J. E. 1973. Fireflies of Melanesia: bioluminescence, mating behavior, and synchronous flashing (Coleoptera: Lampyridae). Environ. Entomol. 2:991–1008. [Google Scholar]
  13. Lloyd, J. E. 1984. Evolution of a firefly flash code. Florida Entomol. 67:228–239. [Google Scholar]
  14. Makino, T. , Suzuki H., and Ohba N.. 1994. Computer analysis system for firefly flash patterns. Sci. Rep. Yokosuka City Mus. 42:27–56. [Google Scholar]
  15. Mast, S. O. 1912. Behavior of fire‐flies (Photinus pyralis)? With special reference to the problem of orientation. J. Anim. Behav. 2:256–272. [Google Scholar]
  16. McDermott, F. A. 1910. A note on the light‐emission of some American Lampyridae. Can. Entomol. 42:357–363. [Google Scholar]
  17. McDermott, F. A. 1914. The ecologic relations of the photogenic function among insects. Z. Wiss. Insek. 10:303–307. [Google Scholar]
  18. McDermott, F. A. 1917. Observation on the light‐emission of American Lampyridae: the photogenic function as a mating adaptation; 5th paper. Can. Entomol. 49:53–61. [Google Scholar]
  19. Moiseff, A. , and Copeland J.. 2000. A new type of synchronized flashing in a North American firefly. J. Insect Behav. 13:597–612. [Google Scholar]
  20. Ohba, N. . 1979. Flash patterns and behavior of some Japanese fireflies (Coleoptera: Lampyridae). Sci. Rep. Yokosuka City Mus. 26:21–30, plates 6–7. [Google Scholar]
  21. Ohba, N. 1980. Mating behavior of a Japanese Hotaria firefly (Coleoptera: lampyridae). Sci. Rep. Yokosuka City Mus. 27:13–18. [Google Scholar]
  22. Ohba, N. . 1983. Studies on the communication system of Japanese fireflies. Sci. Rep. Yokosuka City Mus. 30:1–62, plates 1–6. [Google Scholar]
  23. Ohba, N. . 1984. Synchronous flashing in the Japanese firefly, Luciola cruciata (Coleoptera: Lampyridae). Sci. Rep. Yokosuka City Mus. 32:23–32, plate 8. [Google Scholar]
  24. Ohba, N. 2000. Flash patterns and communication of the two ecotypes in Hotaria parvula (Coleoptera: Lampyridae). Sci. Rep. Yokosuka City Mus. 47:1–22. [Google Scholar]
  25. Ohba, N. 2001. Geographical variation, morphology, and flash pattern of the firefly Luciola cruciata (Coleoptera: Lampyridae). Sci. Rep. Yokosuka City Mus. 48:45–89. [Google Scholar]
  26. Ohba, N. 2004. Flash communication systems of Japanese fireflies. Integr. Comp. Biol. 44:225–233. [DOI] [PubMed] [Google Scholar]
  27. Ohba, N. , and Shimoyama A.. 2009. The synchronous flashing signal of Pteroptyx effulgens in Papua New Guinea used by P. tarsalis to form aggregations Pp. 229–242 in Meyer‐Rochow V. B., ed. Bioluminescence in focus – a collection of illuminating essays. Research Signpost, Kerala, India. [Google Scholar]
  28. Ohba, N. , and Wong C. H.. 2004. External morphology and ecological study of the firefly, Pteroptyx tener at Kampong Kuantan, Selangor, Malaysia. Sci. Rep. Yokosuka City Mus. 51:1–33. [Google Scholar]
  29. Ohba, N. , Suzuki H., and Yamasaki T.. 1995. Luminescent pattern of Japanese fireflies based on computer analysis. Sci. Rep. Yokosuka City Mus. 43:17–24. [Google Scholar]
  30. Ohba, N. , Kim E. S., and Kim J. G.. 2001. Flash patterns and morphology of the firefly Luciola lateralis in Japan and Korea. Sci. Rep. Yokosuka City Mus. 48:91–116. [Google Scholar]
  31. Osten‐Sacken, B. 1861. Entomologische Notizen. Entomologische Zeitung 22:51–55. [Google Scholar]
  32. Papi, F. . 1969. Light emission, sex attraction and male flash dialogues in a firefly, Luciola lusitanica (Charp.). Monitore Zool. Ital. (N. S.) 3:135–184. [Google Scholar]
  33. Seliger, H. H. , Buck J. B., Fastie W. G., and McElroy W. D.. 1964. Flash patterns in Jamaican fireflies. Biol. Bull. 127:159–172. [Google Scholar]
  34. Shibue, K. , Ohba N., and Fujii E.. 1995. Fractal analysis to assess how the adult male of the firefly, Luciola lateralis, draw flying locus. Sci. Rep. Yokosuka City Mus. 43:25–31. [Google Scholar]
  35. Yiu, V. 2013. A study of Rhagophthalmidae and Lampyridae in Hong Kong with descriptions of new species (Coleoptera): part 1. Lampyrid 3:100–126. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix S1. The integrated video results from TiLIA that traced male and female L. lateralis courtship behavior, with the time‐course analysis of flash patterns.

Click here for additional data file. (18.9MB, avi)

Articles from Ecology and Evolution are provided here courtesy of Wiley

RESOURCES