Actions for Discrete structural optimization as a sequential decision process solved using deep reinforcement learning

Email RIS file
Bookmark
Report an Issue

Discrete structural optimization as a sequential decision process solved using deep reinforcement learning

Author
Ororbia, Maximilian Edward
Published
[University Park, Pennsylvania] : Pennsylvania State University, 2023.
Physical Description
1 electronic document
Additional Creators
Simpson, Timothy W.
Access Online

Availability

I Want It
I Want It

Finding items...

Graduate Program
Restrictions on Access
Restricted (PSU Only).
Summary
Topology optimization is a mathematical framework that in general seeks to determine the optimal layout of material in a design domain. If the design variables are continuous, then often sensitivities can be derived and used with gradient based optimization techniques. However, for certain applications the design variables are inherently discrete, for example the design of steel structures that are constructed of standardized steel sections. This work focuses on the latter, whereby the optimization of structures with discrete elements and discrete design variables is formulated as a sequential decision process solved using reinforcement learning (RL) and deep reinforcement learning (DRL), which has been shown to efficiently provide adept solutions to a variety of high-dimensional planning and learning problems. Hence, this work mathematically models the discrete optimization of planar structures, including trusses and frames, as a Markov decision process (MDP). By modeling discrete structural optimization as an MDP, the set of all feasible design solutions can be precisely represented and the MDP naturally, but not exclusively, accommodates discrete actions. Within this framing, the MDP states correspond to specific structural designs represented as finite graph configurations and the actions correspond to specific topological and parametric grammars that are applied to alter the structure, transitioning the design to a new state and graph configuration. Key to modeling discrete optimization as an MDP is the relation of the rewards to the change in the design's performance as the agent explores alternate design configurations. Through this relation, the agent learns an optimal policy, that is, a sequence of necessary alteration actions, that maximizes its cumulative reward and simultaneously synthesizes a high-performing design solution, if not the global optimal, with respect to the specified design problem's objective and specified constraints. The discrete MDP model solved using RL and DRL is applied to the discrete optimization of planar truss and frame structures. To demonstrate the merit of the idea and because elements of the MDP tuple are unknown to the agent a priori, a tabular implementation of reinforcement learning (RL), specifically a Q-learning algorithm, was employed to solve the MDP due to its strong convergence properties. However, for problems with high dimensional state and action spaces, that is, with a large set of feasible design solutions, the number of state visits required to converge to the optimal policy becomes intractable, a problem commonly referred to as the curse of dimensionality. Tabular RL algorithms can become inefficient for solving design problems with relatively large state and action spaces due to their memory limitation and need for an excessive number of experiences to learn the optimal policy. Hence by extending the general discrete structural optimization MDP to be solved using DRL, a deep neural network architecture is specifically developed to approximate the state-action value function, such that the network has far fewer parameters than the cardinality of the state space of feasible design solutions. This enables the MDP framework to adeptly solve discrete topology optimization design problems with large state and action spaces. A benefit of the suggested method, in comparison to other discrete optimization approaches, is that the MDP framework is grounded in mathematics and is not dependent on the specifics of the structural model. The framework is evaluated in the context of the discrete structural optimization of planar trusses and frames with discrete elements and multiple discrete cross-sectional areas, and its utility is investigated through several numerical examples, each with different state space cardinalities. The objective of the design task is to determine both the layout of structural elements and the assignment of cross-sectional areas that minimize the displacement at a specified node for a given external force(s) determined using either linear or nonlinear finite element analysis, subject to stability and volume constraints. In this work, both the agent's learned optimal policy and the resulting synthesized design solution are validated against the policy determined by using a state-action value iteration dynamic programming algorithm, chosen for its strong convergence guarantees, and the global optimal design configuration identified from an exhaustive evaluation of all feasible design solutions, respectively. Also through qualitative and quantitative comparison with other considered alternative methods, the MDP framework is observed to adeptly learn optimal policies that synthesize optimal design solutions with lower computational effort.
Other Subject(s)
Genre(s)
Dissertation Note
Ph.D. Pennsylvania State University 2023.
Technical Details
The full text of the dissertation is available as an Adobe Acrobat .pdf file ; Adobe Acrobat Reader required to view the file.
View MARC record | catkey: 42248274