Testimonials
The Third Edition
In this third edition, the chapters on model analysis, particularly with respect to boundedness and monotonicity, have been consolidated into a new Chapter 3. The discussion on metamodels using neural nets and kriging has been updated. A completely new chapter on nongradient methods has been added, recognizing that these methods are now mature and part of our toolkit. A new chapter on systems design optimization has also been added to address the reality that most design problems today must be viewed as system problems. The final chapter on optimization practice has been expanded to include a short discussion on global optimization and when it may be worthwhile investing in this most elusive optimization goal.
Many colleagues and students have reviewed or studied parts of the manuscript and offered valuable comments. We are particularly grateful to all of the students at Michigan and other institutions who found various errors in the first two editions and also pointed to desired improvements in the manuscript. For this third edition, we especially acknowledge Alparslan Emrah Bayrak, Alex Burnap, Namwoo Kang, and Max Yi Ren, who provided extensive help in editing the new parts of the book and using them for teaching the design optimization course at Michigan. Comments and feedback by James Allison, Harrison Kim, Michael Kokkolaras, Jeremy Michalek, and Steven Hoffenson were most valuable in improving clarity and catching errors. The material on neural nets and kriging was based on guest lectures prepared for the Michigan course by Sigurd Nelson and updated by Max Yi Ren and Alex Burnap. The material on trust regions was also a contribution by Sigurd Nelson based on his dissertation. Chapter 3 was carefully edited by Alparslan Emrah Bayrak. The new Chapter 7 uses materials from the masters theses of Ryan Fellini and John Whitefoot, further expanded by Alex Burnap and Max Yi Ren during their teaching of the Michigan course. The newChapter 8 usesmaterials from the dissertations of JamesAllison, Namwoo Kang, Ramprasad Krishnamachari, Harrison Kim, Diane Peters, and Terry Wagner. Allison’s design examples in that chapter were originally developed for his dissertation. Special thanks go to Michael Kokkolaras for sustained advice on how to improve the textbook from his own experiences teaching the design optimization course both at Michigan and at McGill.
Academic and Industrial Users
Principles of Optimal Design has been used for courses offered at several universities and industrial firms including:
Carnegie Mellon University, Pittsburgh, Pennsylvania, USADelft Technical University, Delft, Netherlands
Eindhoven Institute of Technology, Eindhoven, Netherlands
Ford Motor Company, Dearborn, Michigan, USA
General Motors Corporation, Detroit, Michigan, USA
Kuwait University, Safat, Kuwait
University of Houston, Houston, Texas, USA
University of Illinois-Urbana Champaign, Urbana, Illinois, USA
Jaypee Institute of Information Technology, Uttar Pradesh, India
University of Kyoto, Kyoto, Japan
Massachusetts Institute of Technology, Cambridge, Massachusetts National University of Singapore, Singapore
NWFP Engineering and Technology University, Peshawar, Pakistan
Oregon State University, Corvallis, Oregon, USA
Politecnico di Milano, Milano, Italy
University of Patras, Patras, Greece
Simon Fraser University, Burnaby, British Columbia, Canada
Stanford University, Stanford, California, USA
St. Cloud State University, St. Cloud, Minnesota, USAUniversity of Michigan, Ann Arbor, Michigan, USA
University of Texas-Austin, Austin, Texas, USA
University of Texas-Pan American, Edinburg, Texas, USA
Waseda University, Tokyo, Japan
Reviews of Previous Editions
Below are some reviews published since the first edition appeared in 1988.
” The writer of a textbook has a conflict to resolve. On the one hand it is desirable to include in the textbook material that would be of interest to the widest audience, whereas on the other hand it is desirable to insertmaterial close to the author’s heart. The first edition of Papalambros and Wilde’s Principles of Optimal Design leaned in the direction of emphasis on material close to the heart of the authors, in particular, monotonicity analysis. The second edition improves the balance, and the result is a textbook that should be useful to a much wider audience. The new Chapter 2, Model Construction, includes popular topics, and the old specialized Chapter 6, Global Bound Construction, which also included geometric programming, is gone. There are still some popular topics that are left out or shortchanged, but this is difficult to avoid without greatly expanding the thickness and cost of the textbook. The second edition is available in paperback, and at the list price of $44.95 it is a bargain. On the day I checked on the Web, I could get it from Barnes and Noble with a coupon for $42.90 including FedEx shipping.
The second edition retains and strengthens some of the excellent features of the first. These include a wealth of good examples, solved in detail and well illustrated, as well as very nice illustrations of various concepts of optimization. The Notation section at the beginning of the book is useful, and every chapter ends with a Notes section that provides information on additional sources of material. There is also a large number of homework problems in each chapter. The book is well written and easy to follow and provides an excellent basis for a first graduate course on design optimization in Mechanical Engineering, Aerospace Engineering, or Mechanics departments.
The division of material surprised me at first. In particular, some search methods for unconstrained and constrained problems are discussed in Chapters 4 and 5, respectively. Others are given in Chapter 7, entitled Local Computation. On second thought, I could see the benefit of this arrangement, as it allows the teacher to more easily skip algorithmic descriptions. Indeed, I find that, as the years pass, I tend to teach fewer and fewer algorithms in my optimization classes because of the increasing availability of good software.
A brief description of the contents of the book follows: Chapter 1,Optimization Models, discusses the important topic of the formulation of the optimization problem. As the authors say in the preface, “a good model can make optimization almost trivial, whereas a bad one can make correct optimization difficult or impossible.” Compared to the first edition, this chapter now has material on multicriterion optimization as well as expanded coverage of hierarchical models.
Chapter 2, Model Construction, is a new chapter that includesmaterial on fitting models from data, originally in Chapter 1, but now with kriging methods and neural networks added. The chapter also includes a good treatment of natural and practical constraints. Chapter 3, Model Boundedness, begins with a very thorough discussion of the effects of constraints on optima and continues with monotonicity analysis.
Chapter 4, Interior Optima, provides mostly theoretical aspects of unconstrained optimization, including optimality conditions and the concept of convexity. Numericalmethods in this chapter are limited to steepest descent and Newton’s method, as well as a general discussion of trust region algorithms. More methods are discussed in Chapter 7. Chapter 5, Boundary Optima, includes optimality conditions but also the gradient projection and generalized reduced gradient methods, as well as linear programming. I would have liked to see more in this chapter on sensitivity of optimal solutions to problem parameters. Chapter 6, Parametric and Discrete Optima, is mostly about the use of monotonicity theory for model reduction, like Chapter 5 from the first edition. There is much more on discrete variables but not with standard methods, like branch and bound for linear problems, or stochastic methods, such as genetic algorithms.
Chapter 7, Local Computation, includes search algorithms for constrained and unconstrained optimization. Compared to the first edition, it includes trust regions and convex approximations. The absence of the conjugate gradient method is conspicuous (even though quasi-Newton is there). I would have also liked to see methods that do not require derivatives, such as pattern search methods. Chapter 8, Principles and Practice, describes the calculation of derivatives, scaling, problem formulations, checklists, and a road map of where to find things in the book, as well as lists of software and Internet sites.”
– Raphael T. Haftka, University of Florida, AIAA Journal Vol. 39, No. 7, July 2001
” The book written by professors Panos Y. Papalambros and Douglas J. Wilde on the principles of optimal design is a very useful course both to students and practicing engineers. The continuing push for reducing design costs and cycle time using computer-based models makes the use of optimization tools inevitable.
The book is organized into eight chapters. Chapter 1 presents an introduction in the basic concepts as system, mathematical model, optimization design etc. There are defined the well posed problems for that the feasibility and boundedness are discussed. The global and local optima are introduced and the chapter is concluded with a discussion of the dependency between modeling and computation.
In Chapter 2, after a brief review of curve fitting, regression analysis, neural networks and kriging, some very useful and interesting modeling-examples are given, in order to illustrate the concepts introduced before.
Chapter 3 contains an in-depth treatment of the methods concerning with model reduction and verification process. The bounds and the impact of constraints upon them are the target of a careful identifying process. A significant reduction in model size both increases the designer’s understanding of the problem and eases the subsequent computational effort.
In the Chapters 4 and 5 the classical theory of differential optimization is developed, starting with the derivation optimality conditions and than showing how iterative algorithms can be naturally constructed from them. The basic ideas for optimality of constrained problems are explored. Starting from the optimality conditions, some basic search procedures that can be used to reach optima through numerical iteration are derived.
Chapter 6 analyzes how optimal designs are affected by changes in the design environment defined by problem parameters. Special and original emphasis fell on monotonicity analysis application to construct parametric routines that generate optimal designs, with a minimum of iterative searching.
The local search algorithms are discussed in Chapter 7. The influences and limitations that can affect the numerical optimization are emphasized and, finally, a small number of considered preferable methods are described. Remarkable is the use of active set strategies in local computation, in direct analogy with the case decomposition and global computation.
Chapter 8, focused on optimization practice, summarizes the tactics in conducting practical design studies. Containing also a very useful checklist, this final chapter shows the experience of the authors and makes the book more attractive for design project work.
A special attention is given to the clarity in exposition. In order to give a rigorous proof of principles, the new concepts definitions are followed by immediate applications to simple examples. Each chapter ends with proposed exercises and the figures are both clear and concise.
This book is undoubtedly a valuable reference for students, academics, researchers, and industrial engineers interested in optimization design. It can be characterized, as a comprehensive and self-contained exposition, which would arm the reader with all the relevant tools, required for analyzing, modeling and computing in the optimization practice.”
– Dorin LUCACHE, IASI POLYTECHNIC MAGAZINE, Vol. 13, No. 3-4
“ This is a text book for a one-semester course in optimal design. It is suitable for seniors or first year graduate students. It can also be used by researchers concerned with design, operations research and many other areas in which computers are heavily used for designs and planning. Classical optimization theory and numerical algorithms are integrated with the newer ideas of monotonicity analysis and model boundedness. While the book has a heavy engineering flavor this should not discourage others from its use.”
– Mathematics & Computers in Simulation, Vol. 31, 1989
“ This book grew out of courses given at the University of Michigan and Stanford University to an audience coming from engineering sciences. Its flavour is thus in this spirit, and it may – in considerable part also by its many examples which are worked out – be of interest to mathematicians which have to give lectures to such a community or to work on such topics.”
– H.Muthsam, Padiatrie und Padologie (Wien) 5th of Feb. 1989
“ This is an excellent book for anyone interested in modeling, model building, optimization of models, and the interaction between optimization and the modeling process. It combines classical optimization theory with new ideas of monotonicity and model boundedness that provide valuable information for determining the most efficient and correct formulation of a model. This book is definitely aimed at the engineering design students, and it will be a valuable addition to the libraries of operation researchers, economists, numerical analysts and some computer scientist. However, it ma appear little uninspiring for physicists and mathematicians because of its strong engineering flavor, and honest simplicity throughout. There are eight chapters in the book, each of which has a concise introduction to set the stage for the concepts to be discussed, and a summary which re-enforces and ties all the ideas presented. The first seven chapters discuss mathematical modeling, different types of models, model boundedness, interior optima, boundary optima, Karush-Kuhn-Tucker conditions, global bound construction and some discussion on non-linear problems. The Chapter 8 reviews modeling techniques and approaches presented in the previous seven chapters. The optimization checklist included in this chapter should also provide good help to student readers.
Overall this appears to be a good, comprehensive textbook for senior level undergraduate engineering students. The equations and illustrations are also adequate. However, some advanced researchers in the physical science may find this book uninteresting and uninspiring mainly because of its content and the way it is presented.”
– Wooil M. Moon, Physics in Canada, Vol. 46, Jul. 1990
“ This book is concerned with the analysis and solution of models that represent an engineering device. Its philosophy is that an engineering model, before plunging into the uncertain area of numerical computation to solve a non-linear program, increases the likelihood of finding an optimal solution to the design problem. The proposed analyses aim to identify the relevant variables and critical constraints, to check that the model is feasible and bounded, and discuss approaches to simplifying the model. These methods are discussed in Chapters 1 to 6, and in Chapter 7 a small number of non-linear programming algorithms are described. Chapter 8 presents a checklist of issues that should be addressed by anyone, design engineer or not, concerned with building and solving a mathematical model.
This textbook is suitable for a final year undergraduate course; it assumes knowledge of linear algebra and differential calculus. There are exercises at the end of each chapter.”
– S. Powell, International Statistical Reviews
“ The present (interesting and important) book deals with optimization as well as with the modeling process. As sources of the book the authors mention WILDE’s monograph “Globally Optimal Design” (1978) and graduate engineering design course at the University of Michigan and the Stanford University. Comparing with the first source the authors emphasized more precise, explicit, and broader treatment and many examples for the support of understanding. Each chapter is finished by a section “Summary. Notes. Exercises”. Unfortunately, no solutions are offered concerning these exercises. It is the declared intention of the authors to integrate the (classical) optimization theory and numerical methods with the newer concepts of monotonicity analysis and model boundedness in order to establish a procedure of design optimization where global analysis and local iterative methods are really integrated.
Chapter 1 “Optimization models” includes fundamental notions and stresses mathematical modeling, the optimal design concept, different kinds of optima, question of modeling data, and the connection between solution and computation.
Chapter 2 “Model boundedness”, where bounds, extrema, and optima, the constrained optimum, underconstrained models, the way to well bounded models are included. Beyond monotonicity, inequality and equality constraints and the model preparation procedure will be discussed.
Chapter 3 “Interior optima” is devoted to fundamental notions of optimization such as the existence of optima, local approximation, optimality, convexity, local exploration (descent), searching along a line, and stabilization.
Chapter 4 “Boundary optima” deals with feasible directions, tangency, equality constraints, the Hessian, feasible iterations, inequality constraints, Karush-Kuhn-Tucker conditions, linear programming, and sensitivity.
Chapter 5 “Model reduction”: parametric solution, the monotonicity table, hidden monotonicity, the activity map, overconstrained models, finding the optimal case, starting case selections, discrete variables.
Chapter 6 “Global bound constructions” treats geometric programming, where the geometric inequality, unconstrained and constrained geometric inequality, unconstrained and constrained geometric programming problems (including duality) are considered.
Chapter 7 “Local computation” contains local and global convergence, single variable minimization, quasi-Newton methods, finite differences and scaling, active set strategies, penalties and barriers, sequential quadratic programming.
Chapter 8 “Principles and practices”, at the end of the book, tries to summarize and to organize theories and techniques of the previous chapters with the aim to get a problem-solving strategy as a guidance in practical design optimizations. Thus modeling consideration prior to computation and for local computation and moreover, an optimization checklist and finally a review with respect to concepts, rules and principles are given.
The book is supplemented by an extensive reference list and an author list.
To summarize: we recommend highly this book not only for engineering students but for operations analysts, economists, and all those who are interested on (applied) optimization. “
– K.H. Elster and R. Tichatsohk, Optimization, Vol. 20, 1989
” The availability of cheap, powerful computers, together with the promise of artificial intelligence has drawn the attention of engineering designers and others to the subject of modeling – the mathematical description of physical or economical description of a physical or economic system. A natural purpose of modeling is optimization – finding the best value for some goal associated with the model.
This is therefore a text book about modeling for design optimization and the mutual interaction between these two processes. It presents a condensed version of classical optimization theory and numerical algorithms, which it integrates with the newer ideas of monotonicity analysis and model boundedness.”
– Engineering Designer,1989
” This book is primarily intended for engineering and applied science graduate students, and possibly upper level seniors. It is furthermore intended for use in a course which covers the theoretical principles and applications of engineering design optimization. Sufficient worked example problems are included. It would also serve very nicely for use by practicing engineering with an interest in optimal design. While the example problems are predominantly oriented towards mechanical engineering design applications this text book could also be adapted for use in other engineering disciplines including civil, industrial, chemical, and manufacturing engineering.
The book begins with a treatise on models for engineering design optimization and stresses the importance of properly posed, hierarchically structured models. The authors go on to discuss the unified interplay of modeling and computation in optimization, and continue to carry this concept throughout the book.
The concepts related to the mathematics necessary to achieve model boundedness are covered next, and the groundwork for the important concept of monotonicity analysis is laid down. Constraint activity, the concept of cases as subsets of an overall optimization problem, and constraint criticality and constraint relaxation conditions are discussed. The author’s first and second monotonicity principles are also included. These principles are of significant importance in recognizing a (1) well constrained objective function and (2) characteristics associated with variables in the constraints (nonobjective variables) possessing relevance (ie, occurring in an active constraint). It is important to note that the authors speak from first hand experience when discussing this topic since it represents a currently active research direction for them. At the conclusion of the second chapter they succinctly describe in summary form the manner in which the concepts discussed can and should be systematically applied to real problem if problem size reduction is to be successfully achieved. A more detained discussion of model reduction follows in chapter 5.
Optimum solutions lying within (rather than on) the boundaries of the design space occupy the next topic of discussion in the book. Necessary and sufficient conditions are covered through the use of vector calculus. The concepts of convexity and convex functions are described, followed by a discussion of first-order gradient following techniques and second-order algorithms (Newton’s methods) which employ the Hessian matrix. This chapter, in effect, concentrates on the solution of unconstrained optimization problems.
Optima lying on (rather than within) the boundary of the feasible regions, and algorithms capable of locating them comprise the next topic of discussion. The authors address the questions “What happens if the optimization problem includes constraints and the objective function has a minimum on a boundary?” and “What are the appropriate optimality conditions for constrained problem that can be operationally useful without explicitly eliminating constraints?” Definition of, and explanations for, the concepts of feasible directions, feasible perturbations, and the feasible domain are given and graphically depicted. The classical mathematical formulation of the constrained optimization problems is developed.
Included among the topics covered are Lagrange multipliers, the constrained Hessian and its relation to curvature at the boundary of the feasible region, and techniques such as GRG and gradient projection which guarantee feasible iterations while simultaneously decreasing the objective function. The Krush-Kuhn-Tucker (KKT) conditions satisfying the necessary optimality conditions for a problem containing equality and inequality constraints are described and geometrically interpreted. Finally, concepts behind linear programming (LP) problems are discussed along with algorithms for solving problems of this type, and the chapter concludes with mention of sensitivity analysis.
Acquiring information germane to model reduction or -as the authors call it- case decomposition, is discussed in chapter 5. This is important for efficiency regions since it is applied prior to the implementation of any numerical iterative calculations. The authors discuss numerous approaches for “steering through a small fraction of well bounded cases, activating or relaxing constraints until further local improvement to the optimum solution is no longer possible.” Included among these approaches are the use of the monotonicity table (MT); a systematic approach for keeping track of the reasoning processes involved in detecting and eliminating underconstrained cases based on monotonicity analysis; and the activity map for detecting and screening out overconstrained cases. Activity mapping when combined with the use of monotonicity tables are useful for organizing well-constrained cases and displaying their iterelations. The authors go on to discuss the Maximal activity principle for studying the consistency of constraints, and a heuristic referred to as the Coincidence rule which provides guidelines for selecting good starting cases for the location of the optimum in a tractable number of iterations.
In Local computation (chapter 7) the authors provide an appreciation of what is involved in numerical optimization and they describe a few methods that are considered to work reasonably well within the limits of current understanding of these methods. The concepts of local and global convergence are defined, where the former corresponds to efficiency of an algorithm and the latter corresponds to the robustness of an algorithm. Quasi-Newton methods, methods which rely on Hessian matrix update techniques, and the classic DFP and BFGS algorithms are included. The use of finite difference calculus and variable scaling is also discussed as are Lagrange multiplier estimates and penalty function methods. The chapter concludes with a summary of sequential quadratic programming. The final chapter covers what the author term “principles and practice.” Two optimization checklists are provided which offer guidance for the systematic solution of optimization problems. That is quite useful.
This reviewer strongly recommends that engineering faculty adopt textbook for a course they may teach on design optimization. The authors, being leaders in a number of important and timely topics in the area of design optimization research (particularly monotonicity analysis), provide a modern presentation of the mathematics and computational aspect of design optimization in a manner which flows smoothly and is enjoyable to read. Readers particularly interested in the subject of design optimization will find the book to be extremely interesting and difficult to put aside as the authors build the concepts of optimization in a gradual and logical manner.
It is exciting to find a “fresh” approach to design optimization such as is discussed in this book. It is hoped that through this book the authors can inspire new researchers to seek new challenges, and to explore new areas that have the potential to lead to new techniques and discoveries in the field of design optimization. Current concern for worldwide competitiveness in manufactured goods should, in part, provide the necessary incentive for the pursuit of new developments in the field of design optimization, while books like this one can provide a firm background in the principles of optimization which are necessary to explore new areas holding significant promise such as knowledge-based programming, machine learning, and expert system.
Principle of Optimal Design: Modeling and Computation has been well designed and thought out, and the authors are to be congratulated on their fine contribution to the field of engineeing design optimization. One expects that many students will benefits from the knowledge they will acquire from this book. One last comment concerning optimization software is in order. While the reviewer understands and sympathizes with the authors’ intent to emphasize underlying principles rather than numerical details, he hopes that as this textbook begins to be used in engineeing optimization courses throughout the world that interested faculty would pool the software subroutines developed in their courses into a library (these could be sent to the authors or the publisher) which could serve as a basis for an educational optimization software library to be included with the textbook or made available by the publisher. Oftentimes, the amount of time required to develop such subroutines from scratch exceeds the time available in a one semester course. This would also add more of the computer aided “flavor” to the book.”
– David A. Hoeltzel, Applied Mechanics Reviews, Vol. 42, No. 6, 1989
“ Explains the concept of optimal design and demonstrates the relationship between the mathematical model that describes a design and the solution methods that optimize it. This second edition takes into account developments in computer power and optimization, and includes a discussion of trust region and convex approximation algorithms. A new chapter focuses on how to construct optimal design models. The final chapter on optimization practice has been expanded to include computation of derivatives, interpretation of algorithmic results, and selection of algorithms and software.”
– Book News, Inc. (from Amazon.com)
“ Textbook puts the concept of optimal design on a rigorous foundation and demonstrates the intimate relationship between the mathematical model that describes a design and the solution methods that optimize it. This edition has been thoroughly updated to reflect new developments. Softcover; hardcover also available. DLC: Mathematical optimization.”
– Book Info (from Amazon.com)
“ Since the first edition was published, computers have become ever more powerful, design engineers are tackling more complex systems, and the term “optimization” is now routinely used to denote a design process with increased speed and quality. This second edition takes account of these developments and brings the original text thoroughly up to date. The book now discusses trust region and convex approximation algorithms. A new chapter focuses on how to construct optimal design models. Three new case studies illustrate the creation of optimization models. The final chapter on optimization practice has been expanded to include computation of derivatives, interpretation of algorithmic results, and selection of algorithms and software.”
– Book Description (from Amazon.com)