Table 1:

The analysis trustworthiness in product data using digital manufacturing certificates, proposed trust structure, and hierarchy

Main Attribute	Sub-category	Addressed	Description of the method’s congruence
Security	Accountability	•	The use of X.509-PKI with embedded metadata ensures a person and/or system can be called upon to account for actions performed on data.
	Audit-ability & Traceability	•	The digital signatures generate a reliable and secure audit trail.
	Confidentiality		This sub-category is outside the scope of the data and is the responsibility of the data-managing systems.
	Integrity	•	The digital signatures would become invalid if the data changes or becomes corrupt. The X.509-PKI digital certificate have a high-level of integrity if generated in conformance to the standard. Therefore, both accidental and malicious alterations in the data are discoverable, which ensures the integrity of the data.
	Safety		This sub-category is outside the scope of the data and is the responsibility of the data-managing systems.
	Non-Repudiation		This sub-category is outside the scope of the data and is the responsibility of the data-managing systems.
Compatibility	Openness	•	The method takes advantage of the X.509-PKI standard, which is widely adopted in information technology and systems. Therefore, the method is open and transparent in how it works.
	Re-usability	•	The method is interoperable between information technologies and systems, while also allowing the metadata to be extensible to support multiple use cases and/or domains.
Configuration-related Quality	Change Cycle & Stability	•	The method meets an acceptable level of stability because it utilizes the widely-adopted X.509-PKI standard.
	Completeness		Future work is required to develop a complete standardized metadata schema to assist in defining semantic representation of the metadata such that it can be computer-processable.
Compliance		•	A governance policy that addresses most needs of regulated and non-regulated industries is proposed for using the method. The method also conforms to industry-led consensus-based standards.
Privacy		○	The method and governance policy partially address the ability to control the usage of private information. The method, being built upon X.509-PKI, supports controlling public and private keys. However, the method and governance policy must be used in concert with a properly configured data-management system to fully control the usage of private information.
Cost			Cost is considered out of scope for the work described in this paper. However, the use of standards should help in minimizing cost.
Data-Related Quality	Data Integrity	•	Human errors, malicious attacks, intentional data modifications, transmission errors, system/software bugs or viruses, and hard-ware malfunctions are discoverable because any change in the data would invalidate the digital certificates / signatures embedded in the data.
	Data Reliability	•	The provenance of the data is traceable. Further, usage (authorization) of the data is controlled by embedded metadata. Therefore, the correctness of the data used by the user is controllable.
	Data Timeliness		Timeliness is considered out of scope for the work described in this paper. Timeliness depends on the systems managing, transmitting, and monitoring the data.
	Data Validity	○	The method partially addresses data validity by enabling the capture of verification and validation results in the metadata. However, a trusted verification and/or validation system must be used to perform the analysis.
Dependability	Accuracy	•	The method is highly accurate if the certificates are created and managed in accordance with the X.509-PKI standard.
	Availability	•	The use of X.509-PKI are intended to operate in asynchronous environments. Therefore, the ability to verify and validate the certificates should be highly available.
	Failure Tolerance	•	X.509-PKI is a well-established and stable technology. There are mechanisms in the standard for handling failures
	Flexibility & Robustness	○	The method and governance policy partially address this subcategory. A standardized metadata schema is required to ensure the method is robust enough to handle changes in context. Further study into all industry sectors the method and governance policy may apply to is required. The method and governance policy are minimally viable for usage in highly regulated industries, such as aerospace, automotive, and medical devices.
	Reliability	○	Uncertainty exists as to how reliable the method can be without a standardized metadata schema that is semantically representable. However, the X.509-PKI technology has been proven to be sufficiently reliable.
	Scalability	○	The method and governance policy is intended to be extensible to enable scalability. However, further work is need in establishing a metadata schema that covers the minimum information required for all desired domains.
	Maintainability	•	Using well-established consensus-based standards ensures the highest level of maintainability as system undergo evolution.
Performance	Transaction Time	○	The method was designed to be as efficient as possible. However, the transaction time will also depend on several variables outside the control of the method.
	Throughput	○	The method was designed to be as efficient as possible. How-ever, the throughput will also depend on several variables outside the control of the method.
	Response Time	○	The method was designed to be as efficient as possible. However, the response time will also depend on several variables outside the control of the method.
Usability	Satisfaction	•	The method takes advantage of standards that are well-establish and widely adopted.
	Learn-ability		This sub-category is outside the scope of the data and is the responsibility of the data-managing systems.
	Effectiveness	○	The method and governance policy were studied in the context of a limited set of use cases. Therefore, further work is required to determine all domains where the method and governance policy would enable users to achieve the specified goals of this work.
	Efficiency of Use	○	The method was designed to be as efficient as possible. However, the efficiency of use will also depend on several variables outside the control of the method.
Correctness		○	The method conforms mostly to the utilized standards and specifications. The method is in full compliance with the X.509-PKI standard. However, extensions and/or modifications to several data-format standards are required to fully take advantage of the method and governance policy. [7] provide several modifications required.
Complexity		•	The complexity sub-category addresses inverse relationships such that more service fragmentation is typically considered less trustworthy than monolithic services. The method and governance policy presented in this paper were designed to take those inverse relationships into account. Using the X.509-PKI standard assists in managing the fragmentation by enabling the traceability of the data. Therefore, as data moves between systems and services, the digital certificates can be verified and validated to support trusting the data.

^•= fully addressed.

^○= partially addressed.