ADMMBO

ADMMBO combines the Alternating Direction Method of Multipliers (ADMM) with Bayesian Optimization (BO) to optimize unknown objective functions subject to unknown constraints.

Key Features:

• Augmented Objective Function: Augments the original objective with auxiliary variables corresponding to each constraint to transform the constrained problem into an augmented form aligning new variables with the primary variable.
• Iterative Solution via BO and ADMM: Solves multiple unconstrained subproblems iteratively using Bayesian Optimization while ADMM coordinates subproblem solutions to enforce constraints and promote convergence.
• Handling Infeasible Initial Points: Enables optimization to commence from infeasible starting points without requiring an initial feasible solution.
• Insensitivity to Initialization: Reduces sensitivity to the choice of initialization parameters, decreasing dependence on prior parameter settings.
• Efficient Management of Decoupled Problems: Treats decoupled constraints independently to streamline the optimization process and improve computational efficiency.
• Concrete Stopping Criterion: Employs a defined stopping criterion to terminate iterations once a satisfactory solution is reached.

Scientific Applications:

• Machine Learning: Hyperparameter tuning in models with complex constraint structures.
• Engineering Design: Optimization of design parameters subject to safety and performance constraints.
• Biological Systems: Parameter estimation in biological models with inherent regulatory constraints.

Methodology:

Reformulates the constrained problem via function augmentation with auxiliary variables, then applies Bayesian Optimization iteratively to the resulting unconstrained subproblems while using ADMM to coordinate subproblem solutions and enforce constraint satisfaction.