Papers Presented at the SSCET 2022 Systems Engineering Track E

The following papers were peer reviewed and accepted for presentation at the 2022 SSCET based on the contributions to systems engineering theory and applications. A copy of the presented paper has been archived by the SSCET. However, the authors retain copyright ownership of their article. Please contact the authors to obtain a copy of the respective paper of interest.

Systems Approach for Characterizing and Understanding Large-scale Additive Manufacturing Process

James T. Stinson, Elton L. Freeman, Guillermo A. Riveros, Owen J. Eslinger

Author Contact: James T. Stinson, USACE ERDC Information Technology Lab, Vicksburg, MS: James.T.Stinson@usace.army.mil

Abstract—Large Scale Additive Manufacturing for polymer applications is a relatively new research area. Manufacturing objects with at least one dimension greater than 10m can take many days to complete. Because of the long run time, a large-scale print provides a greater opportunity for product defects to occur. These defects are very expensive and result in wasting human, material, and print process resources. Understanding and capturing the material characteristics, machine execution, product design, geometry, tool paths, temperature effects, and cooling curves are pivotal variables that help determine quality and product success. This work focuses on a systems engineering approach to help quantify the metrics associated with the additive manufacturing process. Prototyping, sensing/monitoring, data warehousing, information analysis, high performance computational modeling, print verification, and the structural/functional testing of large prints will help gain insight and understanding into physical and logistical limitations. This work explores how the information collected can be used to develop a “smart” enabled printer that makes adjustments in real-time using in situ monitoring, High Performance Computing (HPC) on-the-edge analysis, and feedback loops to mitigate the probability of failure.

Feasibility of AI-Driven Autonomous Systems for Target Detection in Operational Environment in Army Missions

Daniel Pham, Kristin Weger, Thomas Davis, Vineetha Menon, Bryan Mesmer, Nathan Tenhundfeld, Sampson Gholston

Author Contact: Daniel Pham, Department of Computer Science, The University of Alabama in Huntsville, Huntsville, AL: pdp0008@uah.edu

Abstract—Latest technological advancements in autonomous systems make them an excellent candidate for remote monitoring and detection of targets-of-interest in the operational environment, while ensuring the safety of our soldiers/human operators in intense battlefield scenarios. This research presents a feasibility study of an Artificial Intelligence (AI) operated autonomous drone system that is employed in a combat search and rescue (CSAR) scenario. In these types of scenarios, hostages may be spread out over a larger geographical area. Reliance on ground vision alone may not only be obscured and insufficient but also can expose soldiers or human rescuers to danger. Since time is of great essence, such an intense situation calls for swift decision making with the lives of hostages at stake, making it a classic scenario to evaluate the effectiveness of an AI-driven drone autonomous system for CSAR missions. Here the focus is to gather information about the operational environment (detect hostages, and other targets-of-interest) in a short amount of time that spans a varying geographic terrain. This allows for targeted rescue operations with AI-informed decision-making outcomes that ensures that the rescuers at ground level can be directed along the most optimal path, and casualties will be either minimized or entirely averted. In this research, we study the feasibility of our drone-based AI target detection techniques to evaluate the classification accuracy of multi-view targets-of- interest in the CSAR environment. The overarching goal of this study is to verify whether or not unmanned rescue vehicles are more efficient in swift decision making, navigating a complex environment, and identification of targets.

A Study of Drone-based Explainable AI for Enhanced Human-AI Trust and Informed Decision Making in Human-AI Interactive Virtual Environments

Joseph Schwalb; Kristin Weger; Vineetha Menon; Bryan Mesmer; Nathan Tenhundfeld; Sampson Gholston.

Author Contact: Joseph Schwalb, Department of Computer Science The University of Alabama Huntsville, United States: jds0099@uah.edu

Abstract—Modern artificial intelligence (AI)-based assistive technologies are poised to augment user capabilities and to ultimately engage with society more effectively. Of the variety of assistive technology available, we are particularly interested in assistive automation (AA) such as autonomous systems, which are driven by explainable AI (XAI) technology. Object or target detection is one of the core components of AI-based autonomous systems that enables real-time decision making in tasks such as identification of targets-of-interest including pedestrians, traffic signs, vehicles, monitoring the environment, etc. Often, the rationale behind the AI-based decision-making process is hidden or unavailable to the user or human operator, thus causing a lack of trust in deploying autonomous systems for Army missions. This research presents a comprehensive study of scope of XAI techniques for drone-based autonomous systems particular to a combat search and rescue (CSAR) scenario designed using the Unity environment. In this paper, we propose XAI techniques to model the human (player) – AI (drone) agent exchanges through a CSAR scenario-based game where each interaction is observed. The performance evaluation of the human-AI interactions is studied in both with and without AI-assistive decision-making situations to understand the role of XAI in Army mission success. We introduce several evaluation criteria to break down these interactions into key metrics such as time to complete mission, time to rescue hostages, time to spot hostages, and the frequency of switches between the drone and player perspectives to allow for a deeper understanding of the enhanced informed decision- making capabilities that an AI Agent brings to a Human Agent.

Integration of Weather and Traffic Data Analytics for Installation Decision Dashboards

John P. Richards, PhD, PE; Randy Buchanan, PhD; Christina Rinaudo; George E. Gallarno; Shelia K. Barnett; Erin Williams, Natalie Myers.

Author Contact: John P. Richards, PhD, PE Institute for Systems Engineering Research, US Army Engineer Research and Development Center, Devens, MA: john.p.richards@erdc.dren.mil

Abstract— The Smart Base Artificial Intelligence (AI) for Traffic and Weather project aims to support modernization of installation inclement weather-related decision-making processes by applying complex computational analytics and high performance computing assets. Current inclement-weather decision processes are based solely on weather data and require extensive human interactions and ad-hoc community coordination. This research seeks to integrate weather and traffic data with real-time analytics in order to develop a decision dashboard that can more effectively communicate the impact that weather may have on transportation safety. The goal is to create a more data-driven decision on installation early closures, delayed reporting, or reporting for mission essential personnel only. This approach identifies, captures, processes, analyzes and leverages various data streams to inform decision-making process in a methodology not currently implemented by military installations. Project deliverables will enable informed decisions for the management of weather-related operations on installations, reducing risk to the installation population and increase decision- making efficiency.

GOMS analysis of virtual reality menu designs

Emily S. Wall, Ph.D. P.E.; Reuben Burch, Ph.D.; Michael Hamilton, Ph.D.; Daniel Carruth, Ph.D.; Brian Smith, Ph.D.; Ginnie Hsu.

Author Contact: Emily S. Wall, Ph.D. P.E. Center for Advanced Vehicular Systems Extension, Mississippi State University Starkville, MS: ewall@cavse.msstate.edu

Abstract— The use of menus within virtual reality (VR) environments has often been overshadowed by the novelty of the VR environment itself and has created a gap in the literature for acceptable design options while in VR. When technology is difficult to use, users tend to dislike it and effort that is supposed to be put into the goal of the technology instead is used in overcoming steep learning curves or high mental workloads caused by the product. Overall usability of systems is a vital part of the overall design and management of a product throughout its lifecycle and the use analysis tools such as a GOMS (Goals, Operators, Methods, Selection Rules) analysis allows researchers to predict probable human performance outcomes and thus better manage a systems future use. This study focuses on giving users two different VR menu workflows to determine if a top-down (Method-TD), or a bottom-up (Method-BU) workflow approach influences overall user performance in terms of five variables: time to complete, accuracy, usability, intuitiveness, and user preference when performing tasks in VR. The methods will also be compared using a GOMS analysis to determine any differences between the predicted completion times and the recorded prediction times. The results of this study show that there is no evidence of a significant difference in task performance, accuracy, usability, intuitiveness, or user preference between Method-TD (top-down) and Method-BU (bottom-up) menu organization in a VR environment. This matches the prediction of the initial GOMS which inferred that organization does not have an effect on task performance. When focusing on the subjective variables of usability, intuitiveness, and user preference, the results also confirmed that there were no differences found between Method- TD and Method-BU. This study does show significant differences between the GOMS model predicted results and the reported results for the five variables. The results of this study allow the researcher to make better design decisions for a VR menu system to improve overall usability which should have positive system lifecycle effects for the project.

Augmenting Military Wargaming with Deep Reinforcement Learning

William Leonard; Christina Rinaudo; Phillip Bond; Jaylen Hopson; Theresa Coumbe; Christopher Morey; Robert Hilborn; Christian Darken; Jonathan Alt.

Author Contact: William Leonard, U.S. Army Engineer Research & Development Center Vicksburg, MS, USA: William.B.Leonard@erdc.dren.m il

Abstract—Advancements in deep reinforcement learning (RL), a sub-field of machine learning, provide the opportunity to develop intelligent systems capable of credibly competing with human experts in military wargame settings. This research includes training of software agents using methods from RL in a proof-of-concept framework with naval military scenarios. Experimentation allows for the evaluation of trained-agent performance using multiple algorithms and parameter configurations in each scenario, in order to determine the most effective agent for use within the wargaming framework. This paper provides a summary of proof-of-concept efforts to develop and analyze intelligent systems for use in course-of-action wargaming to support mission planning, and it discusses challenges and opportunities presented by applying deep RL to this problem space.

Forecasting Melt Pool Temperature Distribution in an Additive Manufacturing Process via Tensor Rank Decomposition and Ensemble Learning

Jonathan Storey; Linkan Bian.

Author Contact: Jonathan Storey, Institute for Systems Engineering Research, Mississippi State University, Vicksburg, Mississippi: jstorey@iser.msstate.edu

Abstract—In an additive manufacturing (AM) process, deficiencies in the printing process can lead to costly mistakes for the manufacturer if left unaddressed. Some of the more significant deficiencies that can occur, affecting the microstructural properties of a part, are porosity and lack of fusion within a part. Previous research has indicated defects in a part can occur from fluctuations in the melt pool temperature. The size of the melt pool and the scan pattern are key factors associated with part defects. Thus, it is critical to be able to adjust an AM process when certain criteria are met. To know when the criteria are met, we must anticipate when a process is tending towards an undesirable outcome in order that we might avoid damage to a part thereby saving time and reducing unnecessary expenses. This requires accurate forecasts of the process. In-situ pyrometer data of a direct laser deposition process for a thin-wall construction has been collected at Mississippi State University which captures the temperature distribution of the melt pool. The dimensionality of the data and the presence of noise pose unique challenges in developing accurate forecasting models. In this paper, we propose a four dimensional tensor representation of pyrometer sensor data that enables identification of layer-wise trends. Using a rank decomposition of the tensor enables construction of an ensemble of models, which provides an effective means of reducing both the dimensionality and the noise within the data, enabling accurate forecasts.

A Manufacturability assessment framework using a rule-based expert system

Emily Wall, Ph.D. P.E.; Larry Dalton, P.E.; Gehendra Sharma; Alexander Sommers; Tonya McCall; Shahram Rahimi, Ph.D.; Althea Henslee

Author Contact: Emily Wall, Ph.D. P.E. Mississippi State University Starkville, MS: ewall@cavse.msstate.edu

Abstract – The ease with which a conceptualized product design may be manufactured is an important consideration during early lifecycle decision analysis, especially when there are potentially many viable alternative designs under consideration. Valid assessments of the “manufacturability” of these designs typically requires the practiced knowledge of qualified subject matter experts (SMEs) in various technical realms within the broader domains of supply chain and operations. However, for a variety of reasons, the required SMEs may not be available to support completion of the needed evaluations. Accordingly, there is a need for enhanced methods for accomplishing such assessments in an efficient manner. In many similar instances (ex: medical, legal, etc.), the development and implementation of computerized rule-based expert systems have been successful and beneficial. Such precedents validate that expert systems, which effectively capture the pertinent knowledge of SMEs, can be enabled via the programming of structured, rule-based queries. These adroit inquiries can then be answered by the available, albeit less experienced, technical personnel to provide for a relevant assessment. This research outlines the initial development efforts for a computerized early lifecycle expert system for the manufacturability assessment of alternative conceptual product designs. The evolving methodology involves the establishment of primary and secondary criteria, an analogous questionnaire pertaining to the criteria, and swing weighting of the criteria based on case-specific decision analysis objectives. The results culminate in a ranking of calculated scores for the various possible design alternatives.

Smart Base Installations: Bayesian Network for Decision Analysis to Support the Decision-Making Process During Severe Weather Events

Rolando S. Orellana Rivera; Gregory S. Parnell; Edward A. Pohl; Eric A. Specking

Author Contact: Rolando S. Orellana Rivera, Department of Industrial Engineering, University of Arkansas, Fayetteville, AR: rsorella@uark.edu

Abstract—Developing a warning decision-making process for severe weather events involving uncertainty has a critical mission and safety impact. Adverse weather can lead to the loss of human lives, accidents, or impact mission success on military bases. This project supports the development of the United States Army’s smart base initiative at Fort Carson, Colorado Springs. We identified significant weather attributes and data sources and built a Bayesian Network as a quantitative decision model to assess alternatives and support the decision process. Our prototype decision model built in Python enables us to utilize existing tools for analysis and visualizations. The expected outcome is to offer insights to the decision-makers through a weather assessment probability distribution of potential warnings.

Roadway crash trend analysis with Innovative Trend Analysis and Mann-Kendall Test

Ahmed Hossain; Ashifur Rahman; Xiaoduan Sun

Author Contact: Ahmed Hossain, Department of Civil Engineering, University of Louisiana at Lafayette, Lafayette, LA: ahmed.hossain1@louisiana.edu

Abstract— Following the national goal of crash reduction, the state of Louisiana set an ambitious goal of ‘Destination Zero Deaths’ (DZD), aiming to halve the number of fatalities and severe injuries by 2030. The goal of this study is to detect and quantify fatal and severe injury (FSI) trends in several DZD focus areas, including young drivers, alcohol, intersection, single vehicle, inattentive and distracted driving, and pedestrian. The database contains monthly FSI observations from two different periods: pre-DZD (1999-2008) and post-DZD (2009-2018). Methods employed in this investigation include Innovative Trend Analysis (ITA) and Mann-Kendall (MK) statistical significance test. The analysis reveals several significant results: (a) trend of FSI involving young drivers followed a decreasing pattern for both pre-DZD and post-DZD periods, (b) alcohol involved FSI trend followed two opposite types of a pattern (decreasing trend for the pre-DZD period while increasing trend for the post-DZD), (c) decreasing trend was observed for intersection related FSI during the pre-DZD period, (d) single vehicle involved FSI trend followed an increasing pattern for both pre-DZD and post-DZD periods, (e) pedestrian-involved FSI trend followed increasing pattern during the post-DZD period. The findings of the analysis can assist safety professionals in identifying key safety trends and prioritizing associated safety projects. Additionally, the concept of using ITA and MK tests to identify trends can be implemented as a decision- support tool within the context of systems engineering.