Abstract
For the Seventh Manufacturing Problem Contest, participants were challenged to fabricate an optical filter with transmittance specified for s-polarization at two incident angles 10° and 50° from 400 nm to 1100 nm. The problem required that contestants be equally versed in the design, deposition, and measurement of optical filters in order to achieve good results. Eight teams from five different countries participated in the contest using various deposition techniques. The fabricated filters had a total thickness between 8.2 xm and 17.6 μm and a total number of layers from 74 to 255 layers that were deposited onto one or both sides of the substrate. The performances of the filters were measured by two independent laboratories. The evaluation results were presented at the Topical Meeting on Optical Interference Coating conference held in Santa Ana Pueblo, New Mexico in June 2019.
1. Introduction
At OSA’s triennial Topical Meeting on Optical Interference Coatings (OIC), it is traditional to present the results of three events organized for this conference: design, measurement and manufacturing contests. Far from being solely competitive, these contests are instructive to the whole thin-film community, as they feature state-of-the-art technologies and methods while sometimes revealing shortcomings.
This article presents the results of the latest Manufacturing Problem Contest, which challenges participating teams to design and fabricate an optical coating with specific spectral characteristics. This exercise requires the participants to wisely combine their design and measurement skills with a fair dose of ‘craftsmanship’ in the precise control of complex multilayer optical coating deposition.
The 2019 Manufacturing Problem Contest is the seventh; results of the first six are found in Refs. [1–6]. This time, participants were required to design and fabricate a filter with specific transmittance T spectra at two different oblique angles of incidence (AOI) for polarized light. The substrates were identical N-BK7 blanks sent to all participants with a maximum of three per team. After design, deposition, and measurements, the teams submitted their samples to the organizers along with certain specifications, some of which were optional. The filters were then evaluated by two independent laboratories: Optical Data Associates (ODA) and the National Institute of Standards and Technology (NIST). After an analysis of these measurements, the results and the rankings were presented at the OIC 2019 conference.
In Sections 2 and 3, the problem is described and discussed. Sections 4 and 5 provide participant information and the independent evaluation, respectively. The results appear in Section 6, followed by conclusions in Section 7.
2. Formulation of the problem
Since the first Manufacturing Problem Contest in 2001, the difficult task for the contest organizers has been to challenge the state-of-art in optical coating capability. In addition, the project must not have any practical use or commercial value so that contestants would not have to reveal any intellectual property. In the past, filter profiles involving relevant geographical and other features have been involved [1–6].
For the 2019 contest, we chose an image of a yucca plant (Genus Yucca), one of the state symbols of New Mexico, the host state for this meeting. The plant’s profile defined the target spectra (see Fig. 1). The problem involved T spectra specified for wavelengths between 400 nm and 1100 nm, at 10° and 50° AOI and s-polarized light. One incentive was the continuing and unappreciated challenge of making accurate T measurements at oblique AOI. A list of numerical values for the specification appears as a supplement to this article [7].
Figure 1 –

The Yucca Problem, with s-polarization T targets at 10° and 50° AOI. For better visibility, the plots show a reduced number of target points.
For the filter design and evaluation, the performance of the filter was assessed using the following merit function (MF):
| (1) |
where Ts10,i, Ts10,iD are the measured and target T values at 10° AOI, and Ts50,i, Ts50,iD are the measured and target T values at 50° AOI, both for s-polarized light, at the specified wavelength λi; N10 and N50 are the total number of wavelengths defining Ts10,iD and Ts50,iD targets; ΔTsl0,i and ΔTs50,i are the T tolerances, set to 0.01 for all wavelengths.
No specific coating materials were required. However, toxic and radioactive materials, such as ZnSe and ThF4, were excluded; delicate, soft materials that would be degraded by handling during evaluation were discouraged. .
The coatings were to be deposited on N-BK7 substrates (50 mm diameter × 1 mm thick) provided by the contest organizers. To minimize the influence of any form of anisotropy or thickness non-uniformity on the evaluation, participants were asked to mark their preferred axis of rotation for measurements at oblique incidence with two black dots drew near the opposite edges on the coating side.
Along with the samples, the participants had to provide their refractive index profiles and their T data. It was suggested that they add some background on their design and fabrication processes.
Samples were evaluated as described in Section 5. Apart from T measurements, the samples were not subjected to any surface analyses, such as Auger spectroscopy or scanning/transmission electron microscopy (SEM/TEM). All samples were returned to the participants after the presentation of the results.
3. Problem Discussion
3.1. Design
To confirm that the Yucca problem could be solved using commonly available coating materials and deposition techniques, we explored designs in which the low index material was SiO2 and the high index material was selected from TiO2, Al2O3, Nb2O3, or Ta2O5. Most optical constants were from Palik [8]. Our investigation revealed that many relatively good design solutions (MF < 5) employed various materials, the thicker, more complex designs generally led to lower MF values.
We also examined the effect of coating one or both surfaces of the substrate. Figure 2 shows two examples of designs based on SiO2 and TiO2 layers:
50-layer, 3.8 μm thick coating on the front-side, and a
30-layer, 2.7 μm thick coating on the front and a 24-layer, 1.7 μm thick coating on the back.
Figure 2 –

Two examples of possible designs for the Manufacturing Problem, involving only the front side (Column A) or both the front and back sides (Column B) of a substrate. (Row 1) Refractive index profiles, (Row 2) Calculated transmittance spectra, compared to the target curves (darker dot-dash line at the bottom side of the green silhouettes).
Figures 2(b) and (d) compare the calculated T spectra for these designs with the target spectra (bottom darker part of the green leaf silhouettes). Interestingly, we see that although both designs have roughly the same number of layers and total film thicknesses, the design involving both surfaces of the substrate led to much smoother T spectrum and result in a lower MF value.
To investigate the cause of the smoother T spectra for double-sided designs, we simulated the separate T spectra of the front and back coatings from the design in Fig. 2c. In Fig. 3, we observe that at some wavelengths, T maxima from one side coincide with T minima from the other. This complementarity explains the relative smoothness of the T spectra when compared to a single side design with similar number of layers and total thicknesses. However, for much thicker double-sided solutions, one could expect that the alignment of much finer peaks and valleys required to lower the MF would be more sensitive to fabrication errors.
Figure 3 –

Comparison of the transmittance spectra from the front- and back-coatings of the two-side design presented in Fig. 2(c) and (d), simulated individually for (a) 10° AOI and (b) 50° AOI. One can observe the correspondence of position of peaks and valleys, particularly at shorter wavelengths.
3.2. Measurement
As mentioned, one of the anticipated difficulties with this problem was the accurate evaluation of filters at oblique angles with polarized light. Commercially available spectrophotometers allow for these measurements. However, this task still demands attention to details such as the exact positioning of the sample, the effect of the beam deviation as it refracts through the sample at non-normal angles, the control of the polarization, and potential anisotropic and non-uniformity effects from the coatings.
We start with the beam deviation by the sample, particularly for measurements involving both visible and near infrared (NIR) light. The common size difference between visible detectors, typically large area photomultipliers (PMs), and smaller area semiconductor-based detectors for the NIR can render the measurement sensitive to a slight misalignment or deviation of the light beam. This may lead to a discontinuity at the wavelength where the detector is switched [9]. Therefore, the introduction of a sample into the sample or reference beam can affect the measurement. For example, Fig. 4 shows the variation of T at 800 nm of an N-BK7 substrate when varying the polarization angle of the sample and reference beams at 0°, 10°, and 50° AOI with the Lambda 1050 instrument described in Table 2. Such a T variation is expected at oblique angles, particularly at 50° which is near the N-BK7 Brewster angle (56.5°) at this wavelength. At normal incidence, it result from a slight beam deviation induced by the polarizer rotation assuming no anisotropy in the N-BK7 substrate. Another example is shown in Fig. 5, where we plot the difference of two T spectra with s-polarization and the same AOI values, but with +10° and −10° azimuthal orientations. Again, these differences can be attributed to misalignment at the detectors due to beam deviation.
Figure 4.

Measured variation of the transmittance of a N-BK7 substrate when changing the polarization angle of the incoming light (at a wavelength of 800 nm), for (a) normal incidence 0°, (b) 10° AOI and (c) 50° AOI.
Table 2.
Summary of the Measurement Equipment
| ODA | NIST | |
|---|---|---|
| Instrument | Cary 5000 | Perkin Elmer Lambda 1050 |
| Beams | Double-grating and double beam | Double-beam |
| Wavelength range | 400 nm to 1100 nm, 1.0 nm steps | 400 nm to 1100 nm, 1.0 nm steps |
| Light-source | Tungsten-halogen / deuterium | Tungsten-halogen / deuterium |
| Detectors | Photomultiplier / PbS | Photomultiplier / InGaAs |
| Transmittance accuracy | ±0.4 % in visible and NIR | ±0.4 % in visible and NIR |
Figure 5.

Effect of the sample orientation in the plan of incidence, with light incident at the same AOI but from two different directions [(a), (b)]; the resulting differences in transmittance are shown in (c) for AOI ±10° and ±50°.
Issues of beam deviation were expected and led the organizers to select 1 mm thick substrates compared to 4 mm thick for previous contests. However, thinner substrates are more likely to be deformed by stress in thicker coating solutions, which could also affect the beam [10, 11]. Other than beam deviations, potential thin film structural anisotropy [12, 13] or thickness non-uniformity could also affect the measurements at oblique angles.
In addition, inaccurately setting the AOI could lead to error. In some accessories, the AOI is set manually with an imprecise goniometer; other instruments are equipped with automated goniometers that can lose their calibration.
4. Participation
The Yucca Problem was posted on the OSA website in September 2018. Nine teams requested substrates to participate; eight submitted nine coated samples on time, with one team generating two entries. The teams alphabetically listed in Table 1 were from five different countries and three continents and represent governmental laboratories and private companies.
Table 1.
List of participating teams and their affiliations
| Team | Organization |
|---|---|
| Zach Gerig | FiveNine Optics, Boulder CO, USA |
| Julien Lumeau, Thomas Begou, Antonin Moreau, Frédéric Lemarquis, Cihan Koc and Fabien Lemarchand | Institut Fresnel, Marseille, France. |
| Vladimir Ponomarev and Vitaly Dogel | OptiSpark, Moscow, Russian Federation. |
| Marc Lappschies and Jan Brossmann | Optics Balzers, Jena, Germany |
| Masahiro Akiba | TOPCON Corporation, Tokyo, Japan |
| Karen Hendrix | Viavi Solutions, Santa Rosa CA, USA |
| Tim Gustafson | Viavi Solutions, Santa Rosa CA, USA |
| Lucas Alves | Viavi Solutions, Santa Rosa CA, USA |
We applied the same anonymity rule as in previous contests. All participants and their affiliations were disclosed at the OIC 2019 conference, however, their identities were not linked to any particular sample, except for the winning team whose identity was announced at the conference. This anonymity rule stimulates participation. In the next sections, the samples are named only by number.
As required by the organizers, the participating teams provided the refractive index profile and measured T for each of their submitted samples. Some requested additional information about the deposition and thickness monitoring techniques were provided by some participants; these can be related to film performance.
5. Sample Evaluation
On delivery, the samples were randomly marked with a serial number from S01 to S09, and all references to the submitting team were removed. The samples were repackaged in identical boxes and sent for evaluation to the two independent laboratories, ODA and NIST. The s-polarization transmittance at the specified AOI and wavelengths were measured with double-beam spectrophotometers with specifications listed in Table 2. The spectra from both labs were collected, averaged, compared to the targets by MFs calculated from Eq. (1). These MF averages were then used to rate and rank the samples.
6. Results and Discussion
The results obtained for the nine submitted samples appear in Table 3, with the following information included:
Number of layers and total thicknesses on the front and back sides of the substrate,
Calculated MFs from the designs,
Measured MF values evaluated by the participants, and from the measurements done at ODA and NIST, and
Fabrication methods.
Table 3.
Summary of the submitted filter designs,MF values and final ranking
| Information from participants | Information from evaluation labs | Additional information from participants | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Nb. layers (front+back coatings) | Total thickness (μm) | MF Design | MF Meas | MF ODA | MF NIST | MF Ave. | Rank | ||
| S01 | 224 | 17.6 | 2.030 | 3.671 | 3.574+/−0.011 | 3.830+/−0.011 | 3.020+/−0.008 | 8 | Magnetron sputtering |
| S02 | 125 (68+57) | 11.8 (6.5+5.3) | 0.59 | 0.88 | 0.891+/−0.011 | 1.436+/−0.011 | 1.164+/−0.008 | 1 | (Winning Sample!) Ion-beam sputtering; deposition time monitoring |
| S03 | 178 | 12.9 | 2.14 | 2.53 | 2.530+/−0.011 | 2.640+/−0.011 | 2.585+/−0.008 | 5 | Magnetron sputtering |
| S04 | 255 | 16.1 | 1.106 | 2.411 | 2.539+/−0.011 | 2.451+/−0.011 | 2.495+/−0.008 | 4 | Magnetron sputtering |
| S05 | 164 (86+78) | 12.97 (6.57+6.40) | 0.445 | 1.608 | 1.728+/−0.011 | 1.644+/−0.011 | 1.686+/−0.008 | 2 | Plasma-assisted magnetron sputtering (Bühler Helios800), optical broadband monitoring |
| S06 | 126 (64+62) | 13.92 (7.32+6.60) | 0.740 | 7.130 | 6.478+/−0.011 | 6.141+/−0.011 | 6.309+/−0.008 | 9 | Plasma-assisted evaporation (Bühler SYRUSpro); Buhler 0MS5000 monitoring |
| S07 | 138 (70+68) | 12.53 (6.47+6.06) | 0.439 | 2.353 | 2.272+/−0.011 | 2.090+/−0.011 | 2.181+/−0.008 | 3 | |
| S08 | 74 (42+32) | 8.24 (3.73+4.51) | 0.46 | 2.646 | 2.890+/−0.011 | 3.001+/−0.011 | 2.945+/−0.008 | 7 | Plasma-assisted evaporation (Bühler SYRUSpro710); Bühler 0MS5000 monitoring |
| S09 | 138 (74+64) | 14.03 (7.36+6.67) | 0.46 | 2.61 | 2.489+/−0.011 | 2.852+/−0.011 | 2.670+/−0.008 | 6 | Plasma-assisted magnetron sputtering (Bühler HELIOS); monitoring Bühler 0MS5100 |
Error estimates were added to certain MFs, based on known measurement uncertainties provided by the evaluation labs (Table 2) [14].
Many participants employed sputtering, which is not surprising given the known film stability and quality of sputtered films. Two teams used plasma-assisted evaporation. Most participants utilized optical monitioring systems for film thickness control; at least one team employed direct [stop-watch] deposition-time monitoring.
The results, with the ODA/NIST spectra compared to the target curves and the participants’ measurements, are shown in Figs. 6 and 7 and Table 3. On these figures, left-hand graphs represent each sample by its corresponding refractive index profile; right-hand graphs compare its s-polarization T spectra for both 10° and 50° AOI, measured by the participant and both evaluation labs, with the target curves. The result for the average MF from ODA/NIST is also displayed for each sample, along with its total thickness and number of layers.
Figure 6.

Evaluation results for the submitted samples S01 to S05. For each sample: left graph is the refractive index profile provided by the participant, and right graph shows the spectra measured by the participant and the two evaluation labs, ODA and NIST, compared with the target spectra (represented as a dot-dash green line along the lower sides of lighter green leaf silhouettes). The merit function MFavg is an average of MFs calculated from ODA and NIST spectra (details are in the text).
Figure 7.

Evaluation results for the submitted samples S06 to S09. (Continuation of Figure 6)
Even though the sample identification numbers were not linked to particular participants, they should be able to recognize their samples based on the information that they provided. We prefer that they maintain anonymity to facilitate future contest participation. The only exception is the winner, Zach Gerig, from FiveNine Optics in Boulder CO, USA. His sample, S02, had the lowest evaluated average MF value, 1.164, as shown in Table 3. Surprisingly, a stop-watch was used to control layer thicknesses in this effort!
Comparing different solutions, we gain insight into the importance of manufacturability. Table 3 indicates that total thicknesses ranged from 8.24 μm (S08) to 17.6 μm (S01), and layer totals from 74 (S08) to 224 (S01).
Looking at the spectra in Figs. 6 and 7, we can easily observe that the problem was not trivial and that no team succeeded in reproducing the targets with high fidelity. From a design perspective, it is instructive to note that no samples incorporated more than two materials. For problems involving only normal AOIs, it has been proved that the optimal design solution never involves more than two materials [15]; however, for oblique or multiple angle problems specifying one or both polarizations, intermediate index materials can simplify the design [16, 17].
Figure 8 compares the different designs and their expected and measured performances (read the figure caption for a detailed description). Figures 9 and 10 provide complementary information. From these figures, as well as Table 3, several observations can be made:
Only three samples (S01, S03 and S04) were single-sided; although they were among the thickest, none of them were in the top three performers.
The three samples with the lowest MF values were double-sided.
All of the double-sided samples had relatively similar total thicknesses and layer numbers on each side.
Two samples, S02 and S03, had smaller differences between design and measured MFvalues, indicating a good knowledge and control of the fabrication process, as well as a design less sensitive to small thickness errors.
Some samples with complex designs and low design MFs resulted in relatively high measured MFs, suggesting unforeseen sensitivity issues.
Figure 8.

Comparison of the results for all the samples: The area of the pie symbols is proportional to the total thickness of the represented coating, with the red slice representing the back-side coating; their ordinates are the averaged MFs, and the triangle symbols represent the corresponding calculated design MFs (as provided by the participants).
Figure 9.

Comparison of the numbers of layers (red, left axis) and the total thicknesses (blue, right axis) for all the samples, ordered according to their measured MF (and rank).
Figure 10.

Comparison of the calculated MFs and the average measured MFs, for all the samples (ordered according to their rank).
As mentioned, most teams employed sputtering. Samples S06 and S08, the only two samples made using evaporation had largest differences between design and measured MF. This could be an indication of more difficult deposition rate control (although S08 MF matches well with sputtering samples). Without further details, we cannot elaborate further on the relative merits of these techniques.
Generally, the T spectra generated by the participants and by ODA/NIST matched well, with slightly higher discrepancies for those done at 50° AOI. This indicates that the previously considered difficulties related to T measurements at oblique angles did not significantly affect the data presented. This result may also be a strong vindication of our choice to supply thin substrates. Measurement accuracy is a key contributor to the manufacture of high performance coatings.
7. Conclusions
We conclude that the 2019 Manufacturing Problem Contest was a success, with diverse participation (nine teams from France, Germany, Japan, Russia and USA), a well-attended presentation at the conference, and an eclectic mix of sample designs and fabrication techniques.
Given the number of samples, it is difficult to draw general conclusions that could provide precepts for making filters of this type. However, we can still make some observations based on the results:
The smoother results favor double-sided solutions with similar optical thickness coatings on both sides;
Although measurements at oblique AOI still demand attention and care, the data provided by the participating teams were similar to those from our evaluation labs, showing that these issues are under control.
The next contest announcement should be posted in September 2021. Team interested in participating to a future edition of the contest can contact the corresponding author (DP).
Acknowledgments
We are grateful to all the participants and their respective institutions for their considerable efforts involving the design, fabrication, and analysis of their samples. We also thank Edmund Optics for its continuing support through the donation of substrates to these contests.
Footnotes
Publisher's Disclaimer: Disclaimer Certain commercial equipment, instruments, or materials are reported in this paper to specify the experimental procedure adequately. Such identification is not intended as a recommendation or endorsement by the NIST, nor is it intended to imply that the materials or equipment identified are necessarily the best available for the purpose.
References
- 1.Dobrowolski JA, Browning S, Jacobson M and Nadal M, “Topical Meeting on Optical Interference Coatings (OIC’ 2001): Manufacturing Problem,“ Appl. Opt 41, 3039–3052 (2002). [DOI] [PubMed] [Google Scholar]
- 2.Dobrowolski JA, Browning S, Jacobson M and Nadal M, “2004 Optical Society of America’s Topical Meeting on Optical Interference Coatings: Manufacturing Problem,“ Appl. Opt 45, 1303–1311 (2006). [DOI] [PubMed] [Google Scholar]
- 3.Dobrowolski JA, Browning S, Jacobson M and Nadal M, “2007 Topical Meeting on Optical Interference Coatings: Manufacturing Problem,“ Appl. Opt 47, C231–C245 (2008). [DOI] [PubMed] [Google Scholar]
- 4.Dobrowolski JA, Li Li, Jacobson M and Allen DW, “2010 Topical Meeting on Optical Interference Coatings: Manufacturing Problem,“ Appl. Opt 50, C408–C419 (2011). [DOI] [PubMed] [Google Scholar]
- 5.Li Li, Dobrowolski JA, Jacobson M and Cooksey C, “Broadband transmission filters from the 2013 Optical Interference Coatings manufacturing problem contest,“ Appl. Opt 53, A248–A258 (2014). [DOI] [PubMed] [Google Scholar]
- 6.Poitras D, Li Li, Jacobson M and Cooksey C, “Manufacturing Problem Contest [invited],“ Appl. Opt 56, C1–C10 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Poitras D, Li Li, Jacobson M and Cooksey C, “Supplement information for the 2019 OIC Manufacturing Problem Contest,” Figshare (2019). https://figshare.com/articles/OIC2019_Manufacturing_Problem_Contest_Target_data_and_Results/9915800 [Google Scholar]
- 8.Palik ED, Handbook of Optical Constants of Solids (Academic, 1991), Vol. 1 and 2. [Google Scholar]
- 9.Zwinkels JC, Noël M, and Dodd CX, “Procedures and Standards for Accurate Spectrophotometric Measurements of Specular Reflectance.” Appl. Opt 33, 7933–7944 (1994). 10.1364/AO.33.007933. [DOI] [PubMed] [Google Scholar]
- 10.Abadias G, Chason E, Keckes J, Sebastiani M, Thompson GB, Barthel E, Doll GL, Murray CE, Stoessel CH, and Martinu L, “Review Article: Stress in Thin Films and Coatings: Current Status, Challenges, and Prospects.” J. of Vac. Sci. & Technol. A 36, 020801(2018). 10.1116/1.5011790. [DOI] [Google Scholar]
- 11.Doerner MF and Nix WD, “Stresses and Deformation Processes in Thin Films on Substrates.” Critical Reviews in Solid State and Materials Sciences 14, 225–268 (1988). 10.1080/10408438808243734. [DOI] [Google Scholar]
- 12.Horowitz F, “Structure-induced optical anisotropy in thin films,” Ph.D. Thesis U. Arizona: (1983). [Google Scholar]
- 13.Wang H, “Assessment of Optical Constants of Multilayer Thin Films with Columnar-Structure-Induced Anisotropy.” J. of Phys. D: Appl. Phys 28, 571–575 (1995). 10.1088/0022-3727/28/3/019. [DOI] [Google Scholar]
- 14.Error propagation was calculated using the package Uncertainties: a Python package for calculations with uncertainties, Lebigot Eric O., http://pythonhosted.org/uncertainties/.uncertainties
- 15.Tikhonravov AV, “Some theoretical aspects of thin film optics and their applications,” Appl. Opt 32, 5417–5426 (1993). [DOI] [PubMed] [Google Scholar]
- 16.Verly PG, Tikhonravov AV, and Trubetskov MK. “Efficient Refinement Algorithm for the Synthesis of Inhomogeneous Optical Coatings.” Appl. Opt 36, 1487–1495 (1997). 10.1364/AO.36.001487. [DOI] [PubMed] [Google Scholar]
- 17.Larouche S and Martinu L. “Step Method: A New Synthesis Method for the Design of Optical Filters with Intermediate Refractive Indices.” Appl. Opt 47, 4321–4330 (2008). 10.1364/AO.47.004321. [DOI] [PubMed] [Google Scholar]
