Testing the Digital Thread in Support of Model-Based Manufacturing and Inspection

Abstract

A number of manufacturing companies have reported anecdotal evidence describing the benefits of Model-Based Enterprise (MBE). Based on this evidence, major players in industry have embraced a vision to deploy MBE. In our view, the best chance of realizing this vision is the creation of a single “digital thread.” Under MBE, there exists a Model-Based Definition (MBD), created by the Engineering function, that downstream functions reuse to complete Model-Based Manufacturing and Model-Based Inspection activities. The ensemble of data that enables the combination of model-based definition, manufacturing, and inspection defines this digital thread. Such a digital thread would enable real-time design and analysis, collaborative process-flow development, automated artifact creation, and full-process traceability in a seamless real-time collaborative development among project participants. This paper documents the strengths and weaknesses in the current, industry strategies for implementing MBE. It also identifies gaps in the transition and/or exchange of data between various manufacturing processes. Lastly, this paper presents measured results from a study of model-based processes compared to drawing-based processes and provides evidence to support the anecdotal evidence and vision made by industry.

1 Introduction

We live in an age of digitization. Information technology advances such as big data, service-oriented architectures, and networking have triggered a digital revolution [1]. Until recently, most engineering and manufacturing activities relied on hardcopy and/or digital documents (including 2D drawings) to convey engineering data and to drive manufacturing processes. Those leading the efforts to bring the digital revolution to the manufacturing of complex products coined the term “digital thread” to convey the data flows between engineering, manufacturing, business processes, and across supply chains [2]. With the advent of new manufacturing-data standards [3] and more powerful engineering software, it is now possible to perform all engineering functions using a model-based definition (MBD). A MBD is a 3D digital-product model that defines the requirements and specifications of the product. A model-based enterprise (MBE) approach uses these models, rather than documents, as the data source for all engineering activities throughout the product lifecycle. The core MBE tenets are models are used to drive all aspects of the product lifecycle and data is created once and reused by all downstream data consumers.

One manufacturer after another is using information technology to become more productive, improve quality, and drive down business costs. However, interoperability remains a barrier to achieving these benefits. Industry hopes to capitalize the use of vendor-neutral data-exchange formats to counteract the staggering costs of data interoperability [4]. One such format is STEP (STandard for the Exchange of Product model data) ISO 10303-242:2014 [3] titled “Managed Model Based 3D Engineering” or known commonly as AP242. Barnard Feeney et al. [2] says, “The intent of STEP AP 242 is to support a manufacturing enterprise with a range of standardized information models that flow through a long and wide ‘digital thread’ that makes the manufacturing systems in the enterprise smart.” Digital data plays a central role in achieving the intent of AP242.

Digital data enables information exchange between software applications and automated information processing. Today, digital data is becoming the link that binds the manufacturing processes that design, produce, and maintain all modern industrial equipment, automobiles, airplanes, and power systems. But even the most technologically advanced manufacturing process is of limited use unless 1) it can interpret and act upon the digital data inputs created by other lifecycle processes, and 2) it can share its digital data outputs with those same processes.

Overcoming these challenges requires a MBD that contains not only the nominal geometry, but also additional information needed for manufacturing and inspection. This additional information is known as Product and Manufacturing Information (PMI). PMI includes geometric dimensions and tolerances (GD&T), material specifications, component lists, process specifications, and inspection requirements. PMI has the potential to make many lifecycle processes run faster, with fewer errors, and at lower cost.

In this paper, we focus on two of those processes: manufacturing and quality. Reaching the aforementioned potential requires improving their integration with the product design process. This means these processes must share the same semantics for the words and symbols used to communicate PMI in MBDs. Two different representations have been developed to facilitate this sharing. “Representation PMI,” the first way, is machine-readable PMI [2]. Having this semantics-related capability allows software developers to automate various design, manufacturing, and inspection functions. The second way is where only the presentation of symbols is defined to be human interpretable [2]. This approach is useful for visualizing design intent using a partly automated process.

Unambiguous “representation” and “presentation” of PMI are critical for digital-product definition, the major output of design, as noted. The semantics of PMI representation data are defined in various US and international standards, but these standards are complex [5]. Nevertheless, ensuring the same PMI data is interpreted and presented consistently by different engineering and manufacturing applications is still a challenge. Incorrect presentation and misinterpretation of PMI are additional challenges that can result in significant delays and costly errors.

Computer interpretability and data associativity are two essential MBD characteristics that enable significant advantages over 2D drawings or document-based approaches [6]. Computer interpretability enables manufacturing and inspection applications to read MBDs and understand the semantics of the data. This would result in fewer errors and a drastic reduction in processing time. Data associativity between PMI – such as, tolerances, material specifications, surface finish, hardness, and specific features in the MBD geometry, is critical for the correct MBD interpretation by downstream software applications. For example, components in assembly MBDs must be associated with, and oriented toward, the correct mating components.

Together, computer interpretability and data associativity enable the MBD to be the critical foundation that enables MBE to work. This, as noted, will improve the chance of industry's vision to become reality, at least for design, production, and inspection.

We conducted a comparison study to test our hypothesis that model-based processes outperform drawing-based processes. We selected three prismatic mechanical piece parts for our test cases that could be machined with common subtractive processes. We developed both a model-based definition and a drawing-based definition for each test case. We then manufactured and inspected the test cases by following the best-available design practices and standards. We were able to quantify both model-based processes and drawing-based processes by tracking metrics related to each task,

In this paper, we describe the design of that study in detail – including the standards and metrics definitions we used. After presenting the results of the study, we provide an in-depth discussion comparing model-based processes to drawing-based processes. We provide several recommendations for improving model-based processes through enhancements of both the standards and the processes. Lastly, we provide suggestions for further research that could enable wider adoption of the model-based enterprise concept.

2 Study Methodology

Because AP242 was not published at the time of this study and many of the manufacturing and metrology solution providers did not support AP242, this study makes use of two underlying assumptions. The research team assumed the capabilities of the manufacturing and/or quality system consuming AP242 [3] were equivalent to the same systems consuming native CAD data. We made this assumption because the AP242 standard-development community and ISO Technical Committee 184 / Sub-Committee 4 have the goal of enabling AP242 to support capabilities equivalent to the native CAD systems. In addition, ISO 10303-203:2011 [7], known commonly as AP203, was provided as a backup to the native CAD data. However, AP242 replaced AP203 and the geometry and presentation PMI elements of AP203 were added to AP242 – ensuring backwards capability. Thus, we assumed the end user should see little to no difference in data processing between the use of AP242, AP203, or native CAD data in the process.

The second assumption was based on the probability that people would work more efficiently when they knew they were being monitored since this could introduce bias into the results of the study. The assumption has equal probability of being relevant in both the model-based and drawing-based tests.

The research methodology for testing our hypothesis follows the process outlined in Figure 1. The following subsections will present and discuss the tasks as they relate to data collection, data reviews of designs, process plans, metrics and gaps analysis.

Fig. 1. Process for the pilot project.

2.1 Industry Input for Test Case and Metric Definitions

The research team consisted of two industry members, two consortium members, two solution providers, one academic member, and two government members. Each team member spoke with representatives from their respective organizations. The rest of the research team used their input to identify test cases and metrics for the pilot project. The metrics are associated with important manufacturing elements such as time, cost, and quality. The paper describes those metrics in detail in Section 2.3.

2.2 Test Cases

The team reviewed multiple product designs suggested by the industry members. The team's selection criteria for choosing designs to use for the test cases were 1) they must be realizable by a large part of the industry, 2) they must not be overly simple, and 3) they must not be too costly to fabricate. Three designs were chosen and any intellectual property was removed from each design. For each test case, the designs were modeled in three major CAD systems using common industry recommended practices. Three different CAD systems were used, deliberately, to mimic how supply chain partners receive models from their customers.

2.2.1 Model Organization Schema

MBDs must be structured so as to ensure the proper communication and interpretation of design intent. Various standards of ASME Y14 Committee [8] and ISO Technical Committee 10 [9] define the organization and presentation of information in drawings. That structure, which may be considered a drawing-organization schema, is a human-interpretable presentation of product-definition requirements. An organization method and a schema for models is equally important.

ASME Y14.41-2012 [10] defines the basic structure for a MBD. It does not, however, provide the user a full model-organization schema that is a computer-readable representation of product definition.

Appendix B of MIL-STD-31000 Revision A [11] includes a definition for a model-organization schema. The definition says the schema must provide, “recommendations and guidelines for the application and display management of the product definition data [11, p. 36].” The schema should enable the necessary product-definition exchanges to all downstream data consumers. But the management of the product-definition data is still part of presentation PMI only. Furthermore, the model-organization schema from MIL-STD-31000 does not provide guidance for the organization of representation data.

For example, in Figure 2 the PMI for a “hole” feature is depicted three ways. In Figure 2(A) presentation PMI for the hole is displayed in a drawing. In Figure 2(B) presentation PMI for the hole is displayed in a MBD. In Figure 2(C) the XML data associated with the representation PMI for the hole is defined in a MBD. All three methods in Figure 2 define the same feature: but methods (A) and (B) are intended for human consumption and method (C) is intended to for computer consumption. Although there are standards that specify how to define both, there are no standards that govern how CAD systems structure representation PMI.

Fig. 2. Examples of representation and presentation PMI in a drawing and model.

This situation, no standards, has enabled each CAD vendor to develop its own implementation strategy, which, of course, decreases interoperability. We must note that semantic-representation PMI approaches do exist [12, 13]. CAD technology and practice standards, however, do not use those approaches. According to [5], both must catch up before their full benefit can be realized.

The model-organization schema for the test case models were defined in accordance with ASME Y14.41-2012 [10] and use MIL-STD-31000 Revision A Appendix B [11] because those standards were the best available at the time. Since there was no representation guidance in the standards, we developed extensions for the purpose of studying different representation capabilities for possible recommendation to the standard-developing organizations. It is important to note that ASME launched a new subcommittee (Y14.41.1) in 2014 to develop a “3D model organization schema” derived from MIL-STD-31000 Revision A and the publication of the new ASME Y14.41.1 standard is expected in 2016.

2.2.2 Test Case 1

Test Case 1 is a hollowed-rectangular part requiring milling operations for manufacture. Figure 3 shows an image of the MBD for Test Case 1. The Test Case 1 MBD utilizes a full-dimension annotation method that combines dimensions, basic dimensions, and reference dimensions with bilateral and geometric tolerances.

Fig. 3. 3D model of Test Case 1.

ASME Y14.5-2009 [10] standardizes the definition and presentation of dimensioning and tolerancing. ASME Y14.5-2009 defines a dimension as “a numerical value(s) or mathematical expression in appropriate units of measure used to define the form, size, orientation or location, of a part or feature [10, p. 3].” A basic dimension is “a theoretically exact dimension [10, p. 3].” A reference dimension is “a dimension, usually without a tolerance, that is used for information purposes only [10, p. 3].” A bilateral tolerance is “a tolerance in which variation is permitted in both directions from the specified dimension [10, p. 7].” Whereas, a geometric tolerance is “the general term applied to the category of tolerances to control size, form, profile, orientation, location, and runout [10, p. 7].”

The Test Case 1 annotation-presentation style is equivalent to the presentation of a 2D drawing. All of the annotations presented can stand on their own without the need of a supplemental CAD model. The annotations in the MBD have representation data [see Figure 2(C)] associated with the presentation PMI [see Figure 2(B)] of the dimension and/or tolerance. In Figure 3, a feature-control frame annotation is selected to show how cross-highlighting presents the associativity between the tolerance and the part entities controlled by the tolerance.

2.2.3 Test Case 2

Test Case 2 is a cylindrical part requiring turning and milling operations for manufacture. Figure 4 shows an image of the MBD for Test Case 2. The Test Case 2 MBD utilizes a hybrid-annotation method that combines dimensions with bilateral and geometric tolerances. Test Case 2 reduces the annotation-presentation burden on the downstream-process user by not displaying basic and reference dimensions. In Figure 4, a datum annotation is selected to show how cross-highlighting presents the associativity between the datum and the part entity identified as the datum. Similar to Test Case 1, the annotations in the MBD for Test Case 2 have representation data [see Figure 2(C)] associated with the presentation PMI [see Figure 2(B)] of the dimension and/or tolerance.

Fig. 4.

3D model of Test Case 2

2.2.4 Test Case 3

Test Case 3 is a solid-block part requiring milling operations for manufacture. Figure 5 shows an image of the MBD for Test Case 3. The Test Case 3 MBD utilizes a reduced-annotation method that combines dimensions with only geometric tolerances. Test Case 3 reduces the annotation-presentation burden on the downstream-process user by not displaying basic and reference dimensions. In Figure 5, a dimension annotation with bilateral and geometric tolerances is selected to show how cross-highlighting presents the associativity between the dimension, tolerances, and the part entity controlled by the dimension and tolerances. Similar to Test Cases 1 and 2, the annotations in the 3D model for Test Case 3 have representation data [see Figure 2(C)] associated with the presentation PMI [see Figure 2(B)] of the dimension and/or tolerance.

Fig. 5. 3D model of Test Case 3.

2.2.5 Product Data Quality

Product data quality (PDQ) assessment and definition are not trivial issues. The first question one can ask is “What are the criteria that will tell us whether a given CAD model is of high quality or not?” International standards (e.g., SASIG, VDA, JAMA, MIL-STD-31000A) are beginning to describe PDQ recommendations for MBD entities. In addition, quality-criteria definition may also be based on process-driven criteria, which is a smaller subset of the international standards. Using process-driven criteria, a model is checked for adherence to quality requirements for Finite Element Method analysis, manufacturing, metrology, and/or long-term data archiving. Checking a model against a set of criteria will answer the question, “What is the MBD quality?” If a model passes all of the criteria checks, then the MBD is “certified” at a defined baselined-quality level. Thus, the risk of model interoperability issues is minimized.

MIL-STD-31000 Revision A Appendix C [11] provides a set of recommended numerical thresholds for geometry-validation criteria. The research team verified the test-case models to the automotive industry manufacturing criteria highlighted in Table 1 using a commercially-available verification and validation product.

Table 1. Automotive industry threshold values from MIL-STD-31000 Rev. A, Appx. C [11].

	Native Data			Derivative Data
Criteria/Use Case	Design	Analysis	Manufacturing	ISO 10303	Visualization	Translation
Curve criteria
Large curve or segment gap (G-CU-LG)	.01 mm max	.01 mm max	.01 mm max	.01 mm max	NR	.01 mm max
Non-tangent curves or segments (G-CU-NT)	2 max	O	O	3 max	O	O
Tiny Curve or segment (G-CU-TI)	.01 mm min.	.01 mm min.	.005 mm min.	.005 mm min.	NR	.005 mm min.
Self-Intersecting curve (G-CU-IS)	0.01 mm	0.01 mm	0.01 mm	.01 mm	NR	0.01 mm
Surface criteria
Non-Tangent surfaces or patches (G-SU-NT)	2 max	O	O	3 max	NR	3 max
Narrow surface or patch (G-SU-NA)	.01 mm min.	A	.005 mm min.	.005 mm min.	NR	.005 mm min.
Self-intersecting surface (G-SU-IS)	.01 mm	0.01 mm	.01 mm	.01 mm	NR	.01 mm
Edge loop criteria
Self-intersecting loop (G-LO-IS)	.01 mm	0.01 mm	0.01 mm	.01 mm	NR	.01 mm
Face criteria
Large edge face gap (G-FA-EG)	.01 mm max	.01 mm max	.01 mm max	.01 mm max	NR	.01 mm max
Narrow face (G-FA-NA)	.01 mm min.	.01 mm min.	.01 mm min.	.01 mm min.	NR	.01 mm min.
Embedded faces (G-FA-EM)	NA	A	.01 mm min.	.01 mm min.	NR	.01 mm min.
Inconsistent face on surface (G-FA-IT)	T	T	T	T	T	T
Shell criteria
Large face gap (G-SH-LG)	.01 mm max	.01 mm max	.01 mm max	.01 mm max	NR	.01 mm max
Over-used edge (G-SH-NM)	>2	A	>2	>2	>2	>2

O – Optional when required by design, A – value agreed by sender and receiver, U – user preference, NR – not required, T – logical value true

All of the test-case models passed the selected criteria with the exception of the Inconsistent face on surface (G-FA-IT) check. A CAD system may or may not assign a normal vector to all faces and surfaces in a model. At times, a face and its associated surfaces may have differing normal-vector directions, which could cause some inconsistencies in data exchange [11]. A description of G-FA-IT criteria from MIL-STD-31000 Revision A Appendix C [11, p. 112] states the failure of G-FA-IT criteria has no effect on the numerically-controlled manufacturing process. Therefore, the research team concluded the three test-case models pass the PDQ verification analysis.

2.3 Metrics and Analysis

The research team defined a set of metrics for analyzing the research question. The metrics are used to study the model-based process against the drawing-based process. The metrics analyze elements such as time, cost, and quality. The study measures time elements using observation, interviews, reports, and system logging. United States Bureau of Labor Statistics [14] provide cost estimates. Quality measures come from the review of a task's adherence to its requirements. The following subsections describe each metric in detail.

2.3.1 Cycle Time

The cycle-time metric measured the difference in time to complete processes when using 2D-annotated drawings versus MBDs. The research team reviewed detailed cycle times of Manufacturing functions and Quality functions. The research team reviewed engineering-design cycle time in a limited scope. Time measures are full-time-equivalent (FTE) hours and cost measures are in U.S. Dollars, both of which are key attributes for the cycle-time metric. The research measures the cycle-time metric using the time to

• Generate Design

• Revise Design

• Exchange Design

• Understand/Interpret Design

• Manufacture to Design requirements

• Inspect to Design requirements

The cycle-time metric includes the time to collect, interpret, and consume 2D drawings versus MBDs. The research team examined the processes where manual and automated activities are occurring and measured the delta in time between them. The key attributes are

• Distribution of data processed in typical year measured as a percentage

• Time measured in FTE hours

• Delta time measured as a percentage

• Cost measured in U.S. Dollars

The research measured the following

• Time for model-based manufacturing and model-based inspection workflows

• Supplier Package: Drawing only

• Supplier Package: MBD only

• Delta time of drawing-based workflow versus model-based workflow

• Manufacturing and inspection plan creation

• Identifying deatures and characteristics

• Matching manufacturing and/or measurement tasks to dimensional-metrology-equipment resources

For the Manufacture and Inspection phases of the study, the research team developed common lists of planning and operation tasks, referred to as the workflows. The research team quantified cycle time for each workflow. The manufacture workflow included managing the product definition, determining the production scope and requirements, decomposing manufacturing features, and planning and managing the production process. The inspection workflow included managing the product definition, determining inspection scope and requirements, decomposing selected tolerances and features, planning and managing inspection resources, planning and managing inspection processes, collecting inspection data, and tracking the product through inspection.

2.3.2 Creation Time of Annotations

The creation-time-of-annotations metric measured the difference in time to annotate 2D drawings versus MBDs. The metric is related to the generate-design attribute of the cycle-time metric. The creation-time-of-annotations metric examined modeling techniques (i.e. design intent vs. topology). The key attributes are time measured in FTE hours and cost measured in U.S. Dollars. The research team measured the representation (semantic) annotation feature as a count. In addition, the research team measured the time to apply annotations (designer) and verify annotations (checker). The measures of the research were the cost associated with each time parameter and the number of representation annotations versus polyline annotations to fully define the design.

2.4 Supplier Package

The product-definition data for the suppliers is divided into two packages: drawing-only and model-based definition. The drawing-only package contains only a 2D-PDF rendering of the CAD drawing. The drawing was created in accordance with the applicable ASME Y14 standards. The drawing-only package is the common and contractual product-definition data exchanged in industry today. Industrial exchanges may include an AP203 model along with the 2D drawing: but our industry partners said that the majority of their suppliers use the included AP203 model as reference only. These suppliers create their own 3D models from the included drawing because the 2D drawing is the contractual documentation to which the suppliers must deliver product. For the test cases we selected, the suppliers were asked to use the provided product-definition data in their already defined process.

The MBD package contains a 3D-PRC PDF with an AP203 file and the native CAD model attached to the PDF. The AP203 file was meant to represent the basic AP242 capabilities. The MBD package is consistent with the type of data industry will exchange in the MBE. The MBD was generated in accordance with ASME Y14.41-2012 [10] and MIL-STD-31000 Revision A Appendix B [11]. The GD&T information presented in the MBD as 3D annotations and/or PMI was generated in accordance with the applicable ASME Y14 standards. The 3D-PRC PDF and attachments were combined to form a technical-data package in accordance with MIL-STD-31000 Revision A.

3 Results

The research team tracked the time to annotate the design definitions used in the model-based and drawing-based processes. The team did not track the time to create model geometry because current industry practice uses the model geometry in both processes. Thus, the team expects the creation time for model geometry to be equal regardless of process. Table 2 presents the observed times to annotate the MBDs and the drawings.

Table 2. Observed time to annotate the design definition.

Test Case	Number of Characteristics	Drawing Time [hrs]	MBD Time [hrs]	Delta [hrs]
1	84	3.1	8.7	5.6
2	57	2.7	2.1	-0.6
3	53	2.2	2.0	-0.2
Total	194	7.9	12.8	4.9

The research team used 3D-CAD models in both the model-based and drawing-based processes. The model used in the model-based process came from the engineering function, which was the released MBD. In the drawing-based process, the model was reproduced by the manufacturing function using the 2D drawing as the source of the product definition. Figure 6 compares the drawing-based product model to the MBD for Test Case 2. The research team observed a change in volume, surface area, and center of mass with the comparison of the two models for Test Case 2. Volume went to 103.7 cubic centimeters (drawing-based) from 102.8 cubic centimeters (model-based) and surface area went to 505.1 square centimeters (drawing-based) to 509.1 square centimeters (model-based). Returning to Figure 6, the model-geometry variation ranged from -0.2489 centimeters to 1.064 centimeters. The research team observed similar results for Test Case 1 and Test Case 3.

Fig. 6. Comparison of the Test Case 2 models used in the model-based and drawing-based processes.

Similar to the design cycle-time, the research team tracked the cycle-time for the manufacture and inspection phases of work. As just mentioned, the model-based processes reused the MBD sent by the engineering function and all of the drawing-based processes reused the model regenerated by the drawing-based manufacture function. Table 3 presents the observed cycle-time for the manufacture and inspection phases.

Table 3. Observed cycle-time for manufacture and inspection phases.

Function	Drawing Time [hrs]	MBD Time [hrs]	Delta [hrs]
Test Case 1
Manufacture	73.9	3.7	-70.2
Inspection §	6.1	5.7	-0.4
Total	80.0	9.4	-70.6
Test Case 2
Manufacture	51.5	9.1	-42.4
Inspection §	6.0	2.8	-3.2
Total	57.5	11.9	-45.6
Test Case 3
Manufacture	32.0	8.8	-23.3
Inspection §	3.5	2.7	-0.8
Total	35.5	11.4	-24.1

^§It is important to note here, the delta-time for the inspection function is much lower than the research team expected. The cause of this is because the drawing-based inspection function reused the manufacture-built 3D model. Overall, the drawing-based inspection is more model-centric than a pure drawing-based inspection. A more detailed discussion of the cause and effects of reusing the manufacture function's model is presented in Section 4.

The delivered parts from the drawing-based process and model-based process are visually equivalent in all cases but one. There was a major discrepancy in Test Case 1 between the parts delivered by the drawing-based and model-based processes. In the parts delivered from the drawing-based process, a through-hole existed that should not have been a through-hole. Figure 7 compares a delivered Test Case 1 part from the drawing-based process to the model-based process. The model generated by the drawing-based manufacture function included the through hole.

Fig. 7. Comparison of a delivered Test Case 1 part showing the addition of an unintended through-hole in the drawing-based part.

4 Discussion and Recommendations

4.1 Analysis of Cycle Time

Figure 8 presents the total drawing-based and model-based cycle times in a bar chart aggregated from Table 2 and Table 3. On average, the total time to complete one design annotation, manufacture, and inspection cycle was 60.3 hours with a standard deviation of 18.5 hours for the drawing-based cycle compared to 15.2 hours with a standard deviation of 2.1 hours for the model-based cycle. The difference equates to a 74.8% average reduction with a standard deviation of 6.2%-points when the design to manufacturing and inspection process is converted from drawing-based definitions to model-based definitions.

Fig. 8. Comparison of drawing-based and model-based processes.

A closer look at the data from the study shows design-cycle time is negligible after the designer attains proper proficiency levels. The design-annotation cycle-time results from Table 2 show the time differential between the drawing-based process and the model-based process is less than one hour for Test Case 2 and Test Case 3. Test Case 1 shows a larger increase in time to annotate the MBD compared to annotating the drawing because the designer struggled with how to properly use the CAD system's 3D annotation tools to ensure the MBDs could be reusable in the downstream processes. A cause of the extra time was the additional work required to ensure the PMI associativity was applied correctly. In addition, the designer for Test Case 1 was well trained for using the CAD system's 2D annotation tools. Therefore, it is important CAD users receive training on the proper and recommended practices for using a system's 3D annotation tools.

In addition, the study confirmed a lack of documented recommended practices for defining PMI with both representation and presentation data. As mentioned earlier, the current available MBD standards define presentation PMI, but lack guidance for representation PMI. The research team developed extended representation methods [see Figure 2(C)] in addition to the current MBD standards. However, the developed solutions were one-off solutions for the particular CAM and CMM packages the model-based supplier had available. To enable a MBE fully, vendor-neutral practice standards are needed to provide recommendations to industry and solution providers for defining and reading representation PMI.

The drawing-based supplier used models in the manufacture and inspection process. However, the drawing-based supplier used the supplied 2D drawing to rebuild the models used in its process. As expected, the drawing-based process generated several product-definition-interpretation questions. The drawing-based supplier asked 12 questions related to interpreting the product definition from the drawing. All of the product-definition interpretation questions forced the drawing-based supplier to stop working on the supplier model being rebuilt from the drawing. The average cycle-time for answering a question was 2.8 calendar days. That equates to a total of 34 calendar days of work stoppage due to product-definition interpretation using the 2D drawings. In contrast, the model-based supplier asked no questions during its manufacture and inspection work.

Each time the drawing-based supplier asked a question that caused a stoppage, the supplier took the research team's part out of the work queue and went to working other jobs. When the drawing-based supplier received a satisfactory answer to its question, the supplier put the research team's part back in the work queue. The study results show the manufacture and inspection functions took 57.7 hours and 10.9 hours for the drawing-based process and the model-based process, respectively. Based on an eight-hour work day, this equates to on average 7.2 days versus 1.4 days to complete the manufacture and inspection functions for drawing-based process and model-based process, respectively.

The difference of 5.8 days between the drawing-based process and model-based process on the surface is deceiving. The model-based supplier delivered parts in approximately five weeks, but the drawing-based supplier took approximately eight months to deliver parts. From discussions with industry members, anecdotally “the difference in delivering parts model-based versus drawing-based is three weeks versus three months.” Thus, the study supports the anecdotal statement from industry.

More analysis of the results show the majority of the cycle-time difference between the drawing-based process and the model-based process is due to the manufacturing function. This is because the drawing-based supplier reproduced the models from the drawings provided by the research team. The model regeneration time accounts for 52.5, 18.5, and 10.7 hours for Test Case 1, Test Case 2, and Test Case 3, respectively. The slope of the fit curves shown in Figure 8 from Test Case 1 to Test Case 3 is a negative value for both the drawing-based process and model-based process. Recall Test Case 1 used the most burdensome annotation method and Test Case 3 used the least burdensome annotation method. The decreasing cycle-time from Test Case 1 to Test Case 3 appears to correlate to the annotation method. The research team cannot say definitively if the annotation method is the causation of the decreased cycle-time between test cases, but the result suggests further study is worthwhile to support a determination.

Another interesting result worth further study is the slope of the fit curve for the drawing-based process is much steeper than the slope of the fit curve for model-based process. A future research question should be, “Does the decreased slope in the fit curve for the model-based process show that the annotation method has less effect on the overall cycle-time?” The study of this future research question can support the testing of another anecdotal industry statement, which is “over-tolerancing costs you and me money.” The study results suggest the model-based process may diminish the effects of annotations methods once thought to be cost-prohibitive in a drawing-based process: but, more research is needed in this area.

The last analysis in the area of cycle-time is the inspection function. The study results show only a slight reduction in cycle-time between the drawing-based process and model-based process. It is important to note that the inspection function in the drawing-based process reused the manufacturing function's model, which decreased significantly the inspection cycle-time for the drawing-based process. This skews the cycle-time for inspection in favor of the the drawing-based process. If the drawing-based parts were inspected using only the 2D drawing, the cycle-time for the drawing-based inspection would be much longer than observed during the study. However, the reuse of the manufacturing function's model to reduce inspection cycle-time comes with risk to the product data quality of the models, which is not factored into the drawing-based process.

4.2 Product Data Quality Effects

Product-data quality is related directly to the final product quality because the product data is used directly in the manufacture and inspection functions. Model-based and drawing-based product definitions are communication mechanisms. Reproducing data exactly or equivalently from one point to another point [15] is a challenge with all communications. Therefore, communicated data must be verified and validated for accuracy and precision before the data can be processed.

In a model-based process, the research team recommends verifying the MBD against product-data-quality criteria such as the criteria in Appendix C of MIL-STD-31000 Revision A [11]. This ensures the MBD is of sufficient quality to support downstream processes. When the MBD is translated to a derivative format, the translated model should be validated against the native model to ensure the translated model is also of sufficient quality. Using a recommended, computer-based, verification and validation process ensures traceability of the MBD throughout the product lifecycle.

In a drawing-based process, verification and validation are difficult. The verification and validation of drawing-based processes are typically completed manually based on the person's domain knowledge and experience. Rezakhanlou and Villani [16] describe Shannon's [15] theory as “the entropy measures the optimal rate of compression that one can apply to a signal without losing any of the information which it carries.” The drawing is a compressed form for communicating the product definition.

The largest challenge in a drawing-based process is the verification and validation of the translations from the 2D presentation of information to the interpretation of the information in the human's brain. The same challenge exists in a model-based process, but information in a model-based process is presented in 3D and computer-based systems support the human's processing of the data. However, the communication mechanism of a drawing-based process is similar to the children's game of telephone.

In the children's game, a child whispers a message to another child. The message is passed through a full line of children until the message reaches the last child in line. The message is then announced aloud. The final message rarely matches the original message. Errors accumulate in the message as it is passed through the line. The same happens to the drawing-based product definition as it is communicated throughout the product lifecycle. However, the model is less prone to the problem because a computer is making sense of the model, not the human.

The model used in the study's drawing-based inspection came from the manufacture function's model created using the 2D drawing. Reusing the drawing-based manufacture model in inspection creates a risk of introducing errors into the quality of parts. The comparison of the drawing-based regenerated models against the model-based test-case models shows variation in each drawing-based model. The variations are model defects, which lowers the product-data quality. We could consider the model variations as synonymous with the accumulated errors in the children's game.

Luckily, the delivered parts from the drawing-based and model-based processes conformed mostly to the intended product definition. But in the case of Test Case 1, the defect highlighted in Figure 7 is a direct result of errors introduced through the process of creating a manufacturing model from a 2D drawing. The product geometry presented in the drawing was created from the native CAD model. The drawing was annotated manually by an experienced designer. The drawing was then checked by an expert. However, the depth callout for the hole feature was omitted resulting in a through hole. Figure 9 shows the dimensional callout where the depth callout should be presented.

Fig. 9. Excerpt of drawing for Test Case 1 highlighting the annotation group missing the hole depth callout.

The same depth callout was also mistakenly omitted from the MBD, however, the hole feature was modeled accurately. In the model-based process, the delivered parts were manufactured in conformance with the communicated product definition. because the hole feature was represented in its basic definition and a product-definition note mandated by default “all basic dimension surfaces shall be simultaneously located within a profile tolerance zone of .060 relative to datums A, B, and C.” The basic-model geometry coupled with the default profile tolerance provided coverage for the omitted presentation PMI depth callout. Thus Test Case 1 model-based parts were delivered as conforming parts.

The drawing-based process didn't have a backup definition. Therefore, the human in the drawing-based manufacture function interpreted the drawing as presented and read the product definition for the hole as a feature that passed completely through the product. The hole feature was then modeled as such in the model created by the manufacture function. The parts in the drawing-based process were manufactured from the recreated model with the defect injected. The drawing-based inspection process also missed the defect and signed off the parts for Test Case 1 as conforming. The Test Case 1 drawing-based parts were not accepted as conforming parts and the only remedy was to scrap the parts and produce a new lot.

The reproduction of Test Case 1 parts required rework based on the recreated model from the drawing. The rework time is not accounted for in the results of this study. The defect in Test Case 1 highlights the risks of reusing manufacturing-supplied models in the inspection process. Instead, the drawing-based inspection process should create its own models or inspect parts directly to the drawing-based product definition.

4.3 Identified Process Gaps

The study supports the hypothesis that model-based processes have the potential to provide significant benefit to the communication of product definitions by enabling model-based manufacturing and inspection. While a model-based process can complete a design, manufacture, and inspect cycle, gaps remain in the overall model-based process.

The first gap, the lack of standards for defining representation PMI, was discussed earlier. That gap affects all aspects of the product lifecycle that must interact with the product definition. No other gaps affected the model-based manufacture function. Five additional gaps discovered during the study affect the model-based inspection function. Those gaps relate to identification of key and critical characteristics, CAD model precision, certification of model quality, persistent identification of product definition elements, and reporting inspection results.

4.3.1 Key and Critical Characteristics

SAE International's AS9102B [17] defines the documentation requirements for conducting and reporting a First Article Inspection. AS9102B requires an organization to determine the key characteristic and critical-item requirements of the entities identified in a product definition [17]. AS9102B defines key characteristics as “an attribute or feature whose variation has a significant effect on product fit, form, function, performance, service life, or producibility, that requires specific action for the purpose of controlling variation [17].” AS9102B defines critical items as “those items that have significant effect on the product realization and use of the product, and hence require specific actions to assure they are adequately managed [17].” Key characteristics are also considered critical items.

ASME Y14.5 [18] and Y14.41 [10] does not address key characteristics or critical items – including the definition of symbols for each. Industry defined several different methods for identifying the items, but the methods differ from company to company. This creates a significant challenge for the inspection function in the product lifecycle. The research team recommends that a standard, perhaps Y14.5, should define a symbol for identifying key characteristics and critical items. However, the definition of a symbol only addresses the presentation-PMI concerns. Model-based inspection also requires a representation-PMI method for defining key characteristics and critical items. This would enable model-based inspection systems to identify quickly what entities in a model-based definition need to be managed carefully and ensure the human is kept aware of the entities during the model-based inspection process.

4.3.2 CAD Model Precision

In CAD systems, model precision is distinct from display precision. Commercial CAD systems are commonly accurate to 10⁻¹⁰ for computing purposes. Computational accuracy is dependent upon model precision. The dimension displayed as 1.25 or 1.250 has no effect on the computational accuracy of the model. The research team observed that model-based inspection systems do not know the model precision as a ratio of characteristic tolerances for the CAD models being released by the engineering function.

For example, the default model precision may be set by the designer to 2.5 μm. Therefore, a cylinder with an applied cylindricity geometric tolerance of 25 μm added to the model by the design will have a model precision of 2.5 μm. The ratio of cylinder precision to defined cylindricity tolerance is 1:10 or 10%. Understanding the ratio of model precision to applied tolerance is important because the metrology function must be able to determine the ratio's effect on the metrology uncertainty budget. A method is needed for defining, certifying, and communicating the model precision of the MBD. The method would need to support the preservation and traceability in the metrology uncertainty budgeting.

4.3.3 Certification of Model Quality

The metrology uncertainty budgeting is also affected by the quality of the product data model. A small discrepancy in the association of a feature with a PMI entity causes significant challenges for the model-based inspection function. For example, suppose a cylindrical surface and a circular edge are associated with a dimension and tolerance, which are defined with representation and presentation PMI. This causes an ambiguity in the product definition because different inspection rules apply to the measurement of a cylindrical surface and a circular edge. The inspector must resolve all ambiguity in the MBD before an inspection plan can be generated.

Unfortunately, there is no certification or program for recognizing the proficiency of CAD systems. Commercial tools for checking the application of datums, dimensions, and tolerance are available to industry. But the tools are not integrated fully with the CAD systems and they only check the presentation of the PMI against a standard (e.g., ASME Y14.5 [18]). ASME manages a personnel certification program “as a means to recognize proficiency in the [human] understanding and application of the geometric dimensioning and tolerance principles expressed in Y14.5 [19].” CAD systems need a similar program.

The tools must also check the representation PMI against a standard – although this may be difficult to do because standards like ASME Y14.5 [18] define more presentation requirements than representation requirement. The research team recommends a new tool to provide user-feedback and guidance to the designer applying PMI to the model. Such a tool 1) would enable a feedback loop to the designer when the designer is adding PMI to the model and 2) should notify the designer of any discrepancy at the point the PMI is applied to the model. The tool would ensure the model meets a certain level of adherence to the product-definition standards. The MBDs could also go through a verification-and-validation process such as discussed in Section 4.2 to further check model conformance. The last step in the process should be the engineering function “certifying” to all parties the quality of the model and how the model could be used.

4.3.4 Persistent Identification of Product Definition Elements

Persistent identification of product definition elements is critical for traceability. The CAD system and derivative data formats must support persistent entity identification to be able to track the relationships of entities. The identification must persist through design-versioning and the product-lifecycle workflows.

It is also critical for the model-based inspection function to include a model tree that identifies explicit metrology features. Typical MBDs contain boundary-representation (b-rep) entities (e.g., faces, edges, vertices). Supplemental entities (e.g., surfaces, curves, points) are also included and some design practices use those entities to further define product characteristics. In some cases, the b-rep entities may be grouped together to identify features (e.g., planes, cylinders). Characteristics entities are also included to identify datums, dimensions, and tolerances. Those characteristics may be included in the model as PMI. All of the entities together may be considered a “Bill of Characteristics.” The Bill of Characteristics can be considered the list of entities that must be accounted for and reported on in an inspection, especially in a First Article Inspection. The Bill of Characteristics in a model-based definition may go as far as describing the relationship between each entity in the accounting.

Commercial CAD systems do not generate an explicit Bill of Characteristics. We suggest tracking the relationship of entities in a model-based definition with a parent-child method to dynamically generate the Bill of Characteristics. The MBD would be considered the root node and all the entities in the model would be a child of the model-based definition. Characteristic entities would be parent nodes to feature entities, which would be parent nodes of b-rep and supplemental entities.

4.3.5 Reporting Inspection Results

A MBD provides the downstream product-lifecycle functions with a rich graphical and interactive 3D model with associated features and characteristics. However, the First Article Inspection report and other reporting mechanisms are typically sparse and disconnected static forms with optional 2D-drawing views. Significant usability and information is lost during model-based inspections when decomposing the MBD data into a text-based form – even when 2D-static graphics are added.

AS9102B [17] recognizes the use of digital-product definitions. A MBD is a type of digital-product definition. Section 4.7.1(c) of AS9102B [17] states, “Forms other than those depicted in Appendix B may be used; however, they shall contain all ‘Required’ information and have the same field reference numbers.” The research team recommends investigating alternative AS9102B forms to better convey MBD data combined with model-based inspection results.

5 Conclusions

The research team set out to test the potential benefits of the industrial digital-thread concept. The research team measured the benefits provided by the digital thread using a comparison of model-based processes against drawing-based processes. Three test cases were developed to support the study. On average, the study results showed that the model-based processes provided a 74.8% reduction in cycle-time compared to the drawing-based processes. Using a set of metrics, the research team came to several conclusions. The main conclusion was both the model-based processes and drawing-based processes have their own challenges related to completing the design-manufacture-inspect phases of the product lifecycle. We also made several observations that highlight opportunities for improving the current state of the model-based process.

First, the team concluded the difference in design time between a model-based process and drawing-based process is potentially negligible once the design function achieves an adequate level of proficiency in using the design tools. The research team also concluded there is a lack of representation PMI standards and the standards must be developed in order to enable effective verification and validation in the MBE.

Moving the focus from design to manufacturing, the team concluded the time differential between model-based processes and drawing-based processes is potentially significant. This means the effects of the time differential becomes more evident when considered in the number of working days to complete the processes instead of simply total hours worked. Additionally, the team concluded: 1) providing a MBD as the legal documentation for delivering product may shorten the cycle-time considerably and 2) increasing the product data quality may increase the overall product quality, by reducing the risk of error from data re-entry.

Lastly, the research team concluded gaps remain in the overall concept of MBE, but the concept is on a potential path to provide significant benefits to industry. As a result, we believe the current state of model-based processes has advanced to a sufficient point for starting wide-spread deployment of model-based practices to the industrial complex. Computer-aided manufacturing systems have been reading models consistently for some time and computer-aided inspection systems are starting to catch up. The next step for computer-aided systems in manufacturing and inspection is the automation of various planning activities (e.g., process, schedule, job routing) using the MBD.

In closing, the research team identified several research questions for future study. Future research should determine if the decreased slope in the fit curve for the model-based process shows that the annotation method affects the overall cycle-time less. Moreover, does model-based process diminish the effects of the annotation methods once thought to be cost-prohibitive in a drawing-based process? The research teams recommends the need for a tool to provide user-feedback and guidance to the designer when applying PMI to the model and further research is needed in that area to determine the best method for providing that feedback. Another questions is how should a designer identify and represent key and critical characteristics in model-based definitions? Also, how should the designer represent a bill of characteristics in a model-based definition? A research question looking at the transition from design to manufacturing is what are the effects of model-precision on the error budgets for manufacturing and inspection? In addition, investigating alternative AS9102B forms that better convey model-based definition data combined with model-based inspection results requires further research. The research team believes answering these additional questions will provide considerable support of the model-based enterprise concept and enable mass adoption by industry.