Conference Agenda

Recently, challenges such as climate change and social equity have become important considerations for decision making, including for chemical process systems [1]. Traditional decision-making models that prioritize financial objectives alone are insufficient for addressing these complex issues. Instead, finding a solution which balances these tradeoffs requires solving a multi-objective optimization problem (MOP). Results of MOPs are Pareto frontiers, which depict a manifold of solutions representing the best one objective can do without making another one worse. However, challenges arise in dealing with problems that consider four or more objectives (many-objective problems, or MaOPs), where visualization of objective trade-offs becomes less intuitive and rigorously generating a complete set of solution points becomes computationally prohibitive.

In our previous work, we have developed an algorithm capable of systematically reducing objective dimensionality for (mixed integer) linear MaOPs [3]. In this work, we will extend the algorithm to reduce the dimensionality of nonlinear MaOPs. An outer approximation-like method is used to systematically replace nonlinear objectives and constraints with a set of linear approximations that, when the nonlinear problem is convex, provides a relaxation of the original problem [4]. Additional linear outer approximation constraints likely to be active in determining the Pareto frontier are generated by taking random steps within the cone defined by all objective gradient vectors. We demonstrate that identifying correlation strengths based on the linearly relaxed constraint space using our previously developed method can be sufficient for developing correlation strength weights for objective grouping.

The nonlinear objective reduction algorithm is validated through its application to various systems. First, an illustrative example with elliptical constraints and nonlinear objectives is presented. Next, the algorithm is applied to the well-known DTLZ5 benchmark problems, for which the correlations of objectives are known a priori. By tuning the step size for generating outer approximation constraints, the ability of the nonlinear objective reduction algorithm to successfully divide the objectives into the appropriate groupings is demonstrated for problems with up to 12 objectives. Finally, we demonstrate the algorithm's utility in a practical case study involving the optimal design of a hydrogen production system, underscoring its versatility and effectiveness in solving complex and practical many-objective optimization problems.

[1] Bolis, I., Morioka, S.N. and Sznelwar, L.I., 2017. Are we making decisions in a sustainable way? A comprehensive literature review about rationalities for sustainable development. Journal of cleaner production, 145, pp.310-322.

[3] Russell, J.M., Allman, A., 2023. Sustainable decision making for chemical process systems via dimensionality reduction of many objective problems. AIChE Journal, 69(2), e17692

[4] Viswanathan, J. and Grossmann, I.E., 1990. A combined penalty function and outer-approximation method for MINLP optimization. Computers & Chemical Engineering, 14(7), pp.769-782.

[5] Deb, K., Thiele, L., Laumanns, M. and Zitzler, E., 2005. Scalable test problems for evolutionary multiobjective optimization. In Evolutionary multiobjective optimization: theoretical advances and applications (pp. 105-145). London: Springer London.

[6] Zitzler, E., Deb, K. and Thiele, L., 2000. Comparison of multiobjective evolutionary algorithms: Empirical results. Evolutionary computation, 8(2), pp.173-195.

Rubber mixing is a crucial process in the rubber industry, where raw rubber is combined with various additives to produce a composite with the properties expected for tire performance. Conducted in internal mixers, this batch process poses a significant challenge in tracking the composite quality. Measuring the quality requires costly experimental analysis that can take several hours, while a batch is completed in 30 minutes. Developing physics-based models to predict mixing quality is challenging due to several factors: (i) the complex, non-linear, and heterogeneous processes involved, including mechanical mixing, chemical reactions, and thermal effects; (ii) the distinct chemical and physical properties of additives; and (iii) the difficulty of accounting for the machinery’s degradation that evolves over time.

Machine learning (ML) techniques have recently emerged as soft sensors to estimate composite quality by mapping easily measurable online variables to quality indicators. Common approaches such as just-in-time learning, ensemble methods, and window-based methods have shown promise, but they are limited by their lack of explainability. These methods offer little insight into the underlying physical processes or the influence of conditions on the final quality — an understanding that is crucial for control and optimization. They consider basic process variables (e.g. power and temperature) as the sole input for the ML model, neglecting critical factors like material characteristics and weather conditions, which can be vital, especially when ultra-quality grades with minimal variation must be achieved. Additionally, these approaches lack robust Uncertainty Quantification (UQ) frameworks, affecting their reliability.

This work addresses these challenges by proposing an explainable and robust ML-based method with UQ to analyze the hidden relationships between final batch quality and influencing factors, including process variables, material properties, and weather conditions. Our study focuses on one of Michelin’s rubber mixing lines and utilizes a comprehensive dataset covering 35125 production batches. The dataset has a dimensionality of 329 variables, which includes both direct measurements and features extracted using expert knowledge.

Our methodology centers on the XGBoost model which is selected as the most accurate after extensive comparisons. The approach consists of three phases: Feature Selection, Explainability, and UQ. First, we apply recursive feature elimination to reduce dimensionality, retaining the most informative variables, which are then ranked using SHapley Additive exPlanations (SHAP) to further shrink the dimensionality. Subsequently, the explainability phase integrates SHAP values into other techniques like Partial Dependence Plots to ensure consistency across different quality levels and randomized data subsets. Finally, UQ is introduced through conformal predictions and ensemble methods to generate reliable confidence intervals for predictions, providing accurate model predictions and robust uncertainty estimates.

The results of our approach show promising improvements in both accuracy and interpretability. Feature selection eliminated 82% of redundant features, boosting prediction performance by 17% compared to the baseline model. Additionally, our analysis revealed the importance of previously overlooked factors in the literature, such as initial material properties and weather conditions, in predicting composite quality. The UQ module achieved 90% coverage, offering strong mathematical guarantees, and improving the model's robustness and reliability in real-world industrial applications.

Process optimization under uncertainty is inherent to many PSE applications such as optimal experiment design, RTO, etc. Extremum seeking, modifier adaptation, policy search, or Bayesian optimization (BO) are typical approaches used in this context to drive the process to the real optimum by acquiring experimental information. But actual experiments involve a cost (economic, resources, time) and a limit budget usually exists. However, none of the above techniques handle the accumulated exploration cost explicitly as part of a tradeoff in the optimization objective.

The problem of finding the best tradeoff on cumulative process performance and experimental cost over a finite budget is a Markov Decision Process (MDP) whose states are uncertain process beliefs. The general way to approach these problems is evaluating belief trees of candidate actions and plausible observations via dynamic programming, as Monte Carlo Tree Search algorithms do. But their computational cost is prohibitive.

If the belief is modeled by a Gaussian process (GP), the nonmyopic BO acquisition functions developed by the machine learning community are a more tractable and smart way to approach the above MDP problem. The key idea of nonmyopic BO is to look ahead several steps and compute the best decisions based on an estimation on the cumulative expected value or improvement over the budget horizon [1]. Anyway, solving one-shot multi-step trees is also challenging, and rollout algorithms with default myopic BO acquisition functions are normally used to complete the value function estimation after the first lookahead steps, which can lead to conservative results.

Recently we proposed a variant tailored for process experimental optimization [2] in which a few well-known standard BO acquisition functions are dynamically selected in each node of the tree to find the best expected value over the decision horizon. Moreover, we employ Gauss-Hermite Quadrature in each observation node as an efficient way to approximate the value function. Although the approach is more tractable than other nonmyopic BO, its expected optimallity is a bit lower due to just relying on standard BO acquisition functions.

To remove such conservativeness and getting an optimallity comparable with nonmyopic BO approaches in the literature, but at lower computational cost, here we propose modelling the value function of the first-stage decision also with a GP whose data will correspond to evaluations of our decision tree in [2] for subsequent stages, instead of typical rollouts. In this way, the first-stage decision will be efficiently optimized via the dynamically learned value-function GP and Bayesian Adaptive Direct Search [3].

[1] Jiang, S., Jiang, D., Balandat, M., Karrer, B., Gardner, J., & Garnett, R. 2020. Efficient nonmyopic bayesian optimization via one-shot multi-step trees. Advances in Neural Information Processing Systems, 33, 18039-18049.

[2] Pitarch, J.L., Armesto, L., Sala, A. 2024. POMDP non-myopic Bayesian optimization for processes with operation constraints and a finite budget. Revista Iberoamericana de Automática e Informática Industrial 21, 328-338.

[3] Acerbi, L., & Ma, W. J. 2017. Practical Bayesian optimization for model fitting with Bayesian adaptive direct search. Advances in neural information processing systems, 30.

Bayesian classification (BC) is a powerful supervised machine learning technique for modelling the relationship between a set of continuous variables (causal variables or inputs) and a set of discrete variables (response variables or outputs) that are represented as classes. BC has proven successful in several computational intelligence applications¹ (e.g. clinical diagnosis and feasibility analysis). It adopts a probabilistic approach to learning and inference, where the relationship between inputs and outputs is expressed as probability distributions via Gaussian process² (GP) models. Upon gathering data, posterior GP models are used to predict the class probabilities and to decide the most probable class corresponding to an input.

One way to efficiently implement BC using scarce data is to implement the method in closed loop using active learning³ (AL) methods. The existing AL methods are either based on the values of relative class probabilities or based on the prediction uncertainty of the GP model. While the former methods exclude the uncertainty associated with the inference problem, the latter use uncertainty in the latent function values (the GP model predictions). This makes the AL methods converge slowly towards the true decision boundary separating the classes⁴. Here, we propose an AL method based on the uncertainty propagated from the space of latent function values to the space of relative class probabilities. We compare our method with existing AL methods in a simulated case study motivated by the identification of the feasible (fouling-free) operating region in a flow reactor for drug particles synthesis. Inputs in the case study are antisolvent flowrate, antisolvent-to-solvent flowrate ratio, and additive concentration, while outputs consist of class labels 0 and 1 for infeasible and feasible experiments, respectively.

The true model assumption in the simulated case study helped obtain the true input-space decision boundary between feasible and infeasible regions, compared against the predicted boundaries to evaluate the classification performance. The results indicate that the probability-based AL method predicts a decision boundary closest to the assumed true boundary, with violations due to misclassification. The other two approaches, the function uncertainty-based approach particularly, provide a more conservative decision boundary compared to the true one. The propagated uncertainty-based approach provides a boundary that is conservative and close to the assumed true decision boundary. This study helps to design widely applicable adaptive BC methods with improved accuracy and reliability, ideal for autonomous systems applications.

References

1 G. Cosma, D. Brown, M. Archer, M. Khan and A. Graham Pockley, Expert Syst Appl, 2017, 70.

2 C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning, 2018.

3 D. D. Lewis and J. Catlett, in Proceedings of the 11th International Conference on Machine Learning, ICML 1994, 1994.

4 D. Khatamsaz, B. Vela, P. Singh, D. D. Johnson, D. Allaire and R. Arróyave, NPJ Comput Mater, 2023.

The global food system contributes to and at the same time is impacted by climate change, making crucial to ensure climate resilience. Quantitative insights into how climate change affects food production are essential for this effort, and impact modeling serves as a key tool for gaining such insights. Typically, this involves integrating climate projections with impact models. In this study, we develop a machine learning-based impact model to assess the effects of climate change on the food system, with a focus on the dairy sector.

Dairy was chosen for this case study due to the limited research addressing how climate change affects the sector, as most studies focus on its contribution to greenhouse gas emissions. Using a comprehensive dataset of raw bovine milk from multiple farms across three countries (Malta, Spain, Belgium), spanning several years, we built a machine learning model to evaluate the future impacts of climate change. While milk yield is commonly studied, this research places special emphasis on milk fat, another milk attribute likely to be affected by changing climatic conditions.

A key pillar in this work is the uncertainty analysis. The impact model was built using Gaussian process regression, which offers a straightforward quantification of the model uncertainty. We also account for uncertainties in climate models and variability between farms, i.e., inter-farm variability. Therefore our approach yields a robust framework for uncertainty quantification. The results indicate that inter-farm variability contributes the most to the overall uncertainty, suggesting that on-farm measures may be the most effective for climate-proofing the dairy sector.

References

Katsini L. Bhonsale S., Akkermans S., Roufou S. Griffin S., Valdramidis V., Misiou O., Koutsoumanis K., Muñoz López C.A., Polanska M., Van Impe J.F.M., 2022. Quantitative methods to predict the effect of climate change on microbial food safety: A needs analysis. Trends Food Sci 126

Lehner, F., Deser, C., Maher, N., Marotzke, J., Fischer, E. M., Brunner, L., Knutti, R., and Hawkins, E., 2020. Partitioning climate projection uncertainty with multiple large ensembles and CMIP5/6. Earth Syst Dynam, 11, 491–508