Course Syllabus
The textbook can be used for a variety of course formats. The template below is offered for a typical audience for entry graduate/advanced senior engineering students, similar to the course Design Optimization (ME 555) that has been taught at the University of Michigan in Ann Arbor, first introduced in 1980.
Objective
The course aims at providing the student with a first exposure to a rational integration of traditional design methodologies with concepts and techniques of modern optimization theory and practice. “Design” is defined in a broad context and students from diverse disciplines, including outside engineering, can benefit from the course. The student learns to create appropriate mathematical optimization models and to use analytical and computational techniques to solve them. Availability of software analysis models, such as finite element analysis, makes possible optimization of increasingly complex designs.
Content
The exposition is designed for students without prior knowledge of optimization who wish to apply optimization in practical design situations or who may pursue advanced research work. The treatment of topics is mathematically rigorous but with an emphasis on practical use.
Topics relate primarily to nonlinear models with a finite number of variables (nonlinear programming) including the following: mathematical modeling for design optimization, boundedness and monotonicity analysis of models, differential theory and optimality conditions, numerical methods of nonlinear programming, hierarchical system design, partitioning and coordination, and industrial case studies. Knowledge of basic differential calculus and linear algebra is assumed.
Coursework
There are three parts in the course work: (i) A set of homework assignments and three in-class exams; these are intended as aids to understanding the theoretical content of the course; (ii) an individual project where a design problem chosen by each student is formulated, analyzed and solved, as a independent subsystem of the larger system; (iii) a team project where 2-4 students combine their individual subsystem design problems into a system design synthesis. The teams for Part (iii) are formed early on so that the individual design projects can be selected to fit into the final system design problem. There is no final exam other than project presenations. A typical grade distribution can be
| Homework |
20%
|
| Exams each |
10%
|
| Individual subsystem |
30%
|
| System synthesis |
20%
|
The project grading can be based on the model’s originality and on the successful application of the course theory for design optimization model analysis, numerical solution, and interpretation of results.
An alternative is to omit Part III above in order to simplify project work or to de-emphasize team work. For large classes a single project can be assigned to a group, while for smaller classes the projects can be conducted individually.
Computer Requirements
Several state-of-the-art optimization software packages should be available for student use in assignments and design project work, including commercial and in-house products. Students are expected to program the analysis models needed to compute objective and constraint functions and to interface them with the available optimization packages. However, students are not typically expected to code their own optimization algorithms, except as practice in homework. Modeling in matlab is quite common and interfacing matlab/simulink models with optimization packages is quite easy.
Mathematics Background
The course uses mathematics from several classical areas typically taught at the first or second year in typical engineering or scheince curricula. This mathematical knowledge is required at a working level: the student must have a good understanding of the concepts and techniques rather than superficial familiarity. Mathematical understanding is needed because design optimization is treated as a mathematical problem that can be solved by numerical algorithms. The problem itself (nonlinear optimization) is a very difficult one to solve and the available techniques (commercial codes or otherwise) can fail to give an answer or, worse, give an answer that is wrong. A deeper knowledge of the mathematical reasoning behind the algorithms is important for two reasons: one, so that the design engineer can formulate a problem is a way that some of the potential difficulties are alleviated; two, so that when numerical results are generated, proper interpretation can be given to them and steps taken to achieve solution robustness.
Although some of math concepts can be reviewed in the course, lack of familiarity will increase the student’s workload, at least the first few weeks of the course. However, this knowledge is what will distinguish an informed user from a casual one (who may get wrong answers and not even know it).
The main mathematical concepts needed are as follows.
| Vector Calculus and Linear Algebra | |
| Vector addition, multiplication, dot products, euclidean norms Matrix addition, multiplication, product, inversion, partition Linear combinations, linear dependence/independence Basis of a vector space, subspace, null space Eigenvalues, eigenvectors Solution of linear systems of equations |
|
| Sets and Functions | |
| Open, closed sets, neighborhoods, sequences, limits of sequences Continuity and differentiability of functions Partial derivatives, gradient vector Taylor’s theorem |
|
A brief summary of the mathematical concepts needed for the course can be obtained below.
Syllabus Overview
For most programs of study, the book contains much more material than would be necessary for three lecture hours a week for one semester.
Any course that requires an optimal design project should include Chapters 1, 2, and 9. Placing emphasis on problem formulation should include Chapter 3. A strong theme on gradient-based solution methods would include material from Chapters 4, 5, and 6. A selection from the nongradient approaches in Chapter 7 would round out the basic ideas for all algorithms in use today. Chapter 8 on systems optimization would be the basis for a course that emphasizes what has become known as multidisciplinary design optimization strategies.
Linear programming for problems with purely linear functions is included in Chapter 5 on boundary optima, as a special case of boundary-tracking, active set strategy algorithms, thus avoiding the overhead of the specialized terminology traditionally associated with
the subject. Problems with discrete variables first encountered in Chapter 3 can be
addressed with the methods described in Chapter 7.
An approximate allocation of topics throughout a very ambitious semester is as follows.
| Weeks 1 – 2 | Design Optimization Process Overview |
| Optimization formulations and engineering design. Basic concepts of models. Optimization and Quality-Function-Deployment. Preference structures and Pareto optimality. Boundedness of models. Parametric solutions. Optimality conditions. Numerical methods. Computer software introduction. Simple case studies. [Chapter 1] | |
| Weeks 3 – 4 | Modeling and Monotonicity Analysis |
| Building optimization models. Simulation-based models, surrogate models. Constraint activity. Monotonicity Principles. Well-bounded models. Monotonicity table. Model transformations. Activity maps. [Chapters 2, 3] | |
| Weeks 5 – 7 | Differential Theory: Interior and Boundary Optima |
| Local Approximations of functions. Necessary and sufficient conditions. Convexity. Gradient and Newton’s methods. Line search. Trust regions. [Chapter 4] Feasible directions. Regular points. Necessary and sufficient conditions for equality constrained problems. Lagrange multipliers. Generalized reduced gradient method. Karush-Kuhn-Tucker conditions. Sensitivity analysis. Linear programming as a special case of nonlinear programming [Chapter 5] |
|
| Weeks 8 – 9 | Gradient-based Numerical Methods: Basic Strategies |
| Local and Global Convergence. Termination criteria. Exact and inexact line search. Quasi-Newton Methods. Penalty methods. Sequential Quadratic Programming. Trust region algorithms. [Chapter 6] | |
| Weeks 10-11 | Nongradient Numerical Methods |
| Coordinate Search. Generalized Pattern Search. Nelder–Mead Algorithm. Simulated Annealing. Genetic Algorithm. Multiobjective Genetic Algorithm. Particle Swarm Optimization. Black-Box Methods. Dividing Rectangles (DIRECT). Efficient Global Optimization. [Chapter 7} |
|
| Week 12 | Implementation issues |
| Derivative approximation. Automatic differentiation. Scaling. Active set strategies. Degeneracy. Model transformations. [Chapter 9] | |
| Weeks 12-13 | System Design Optimization |
| System interactions. Distributed optimization. Partitioning and coordination strategies .Hierarchical and nonhierarchical methods. MDF and IDF. Target cascading. Modular and shared component architectures. Design optimization and product development. [Chapter 8] | |
| Weeks 14-15 | Design Optimization Practice |
| Presentation of several case studies and student projects. |