6. A guided tour


6.1. Introduction

One of the big advantages of MATLAB is that it makes it very easy to do experiments and try out things without doing a lot of programming and read big manuals. The MOSEK optimization toolbox has been designed with this in mind. Hence, it should be very easy to solve optimization problems using MOSEK.

Moreover, a guide tour to the optimization toolbox has been designed which introduces the toolbox using examples. The intention is that after studying these examples, then the reader should be able to solve his or her own optimization problems without much further effort. Nevertheless, for the user who is interested in exploiting the toolbox to the limits, then a detailed discussion and command reference is provided in the subsequent chapters.

6.2. The tour starts

The MOSEK optimization toolbox consists of two layers of functions. The procedures in the top layer are application specific functions which has an easy to use interface. Currently, there are four procedures in the top layer and they are:

msklpopt

Performs linear optimization.

mskqpopt

Performs quadratic optimization.

mskenopt

Performs entropy optimization.

mskgpopt

Performs geometric optimization (posynomial case).

mskscopt

Performs separable convex optimization.

The bottom layer of MOSEK optimization toolbox consists of one procedure named mosekopt. This procedure provide a very flexible and powerful interface to the MOSEK optimization package. However, the price for this flexibility is a more complicated calling procedure.

For compatibility with the MATLAB optimization toolbox then MOSEK also provides an implementation of linprog, quadprog and so forth. For details about these functions we refer the reader to Chapter 7.

In the subsequent sections the use of MOSEK optimization toolbox is demonstrated using examples. Most of these examples are available in the directory

mosek\5\toolbox\examp\

6.3. The MOSEK terminolgy

First some MOSEK terminology is introduced which will make the subsequent sections easy to understand.

The MOSEK optimization toolbox can solve different classes of optimization problems such as linear, quadratic, conic, and mixed integer optimization problems. Each of these problems are solved by one of the optimizers in MOSEK. Indeed MOSEK includes the following optimizers:

Depending on the optimizer different solution types may be produced. For example the interior-point optimizers produces a general interior-point solution whereas the simplex optimizer produces a basic solution.

6.4. Linear optimization

The first example is the linear optimization problem

\begin{math}\nonumber{}\begin{array}{lccccl}\nonumber{}\mbox{minimize} &  &  & x_{1}+2x_{2} &  & \\\nonumber{}\mbox{subject to} & 4 & \leq{} & x_{1}+x_{3} & \leq{} & 6,\\\nonumber{} & 1 & \leq{} & x_{1}+x_{2}, &  & \\\nonumber{} &  &  & 0\leq{}x_{1},x_{2},x_{3.} &  &\end{array}\end{math} (6.4.1)

6.4.1. Using msklpopt

A linear optimization problem such as (6.4.1) can be solved using the msklpopt function which is designed for solution of the problem

\begin{math}\nonumber{}\begin{array}{llcccc}\nonumber{}\mbox{minimize} &  &  & c^{T}x &  & \\\nonumber{}\mbox{subject to} & l^{c} & \leq{} & Ax & \leq{} & u^{c},\\\nonumber{} & l^{x} & \leq{} & x & \leq{} & u^{x}.\end{array}\end{math} (6.4.2)

[[MathCmd 5]] and [[MathCmd 6]] are called constraint bounds whereas [[MathCmd 7]] and [[MathCmd 8]] are variable bounds.

The first step of solving the example (6.4.1) is to setup the data for problem (6.4.2) i.e. the c, A, etc. Afterwards the problem is solved using an appropriate call to msklpopt.

% lo1.m 

c     = [1 2 0]';
a     = [[1 0 1];[1 1 0]];
blc   = [4 1]';
buc   = [6 inf]';
blx   = sparse(3,1);
bux   = [];
[res] = msklpopt(c,a,blc,buc,blx,bux);
sol   = res.sol;

% Ineterior-point solution.

sol.itr.xx'      % x solution.
sol.itr.sux'     % Dual variables corresponding to buc.
sol.itr.slx'     % Dual variables corresponding to blx.

% Basic solution

sol.bas.xx'      % x solution in basic solution.

Note that

  • Infinite bounds are specified using -inf and inf. Moreover, the bux = [] means that all upper bounds [[MathCmd 8]] are plus infinite.
  • The call [res] = msklpopt(c,a,blc,buc) implies that the lower and upper bounds on x is minus and plus infinity respectively.
  • The lines after the msklpopt can of course be omitted, but the purpose of those lines is to view different parts of the solutions. The field res.sol contains one or more solution. In this case both the interior-point solution sol.itr and the basic solution sol.bas is defined.

6.4.2. Using mosekopt

The function msklpopt is in fact just a wrapper around the real optimization routine mosekopt. Therefore, an alternative to use the msklpopt is to call mosekopt directly if desired. In general the syntax for a mosekopt call is

[rcode,res] = mosekopt(cmd,prob,param) 

The arguments prob and param are optional. The purpose of the arguments are as follows:

cmd

Is a string telling what mosekopt should do. For example 'minimize info' tells mosekopt that the objective should be minimized and information about the optimization should be returned.

prob

A MATLAB structure specifying the problem that should be optimized.

param

A MATLAB structure specifying parameters controlling the behaviour of the MOSEK optimizer. However, in general it should not be necessary to change the parameters.

The following MATLAB commands demonstrate how to setup the prob structure for the example (6.4.1) and solve the problem using mosekopt:

% lo2.m

clear prob;

% Specifies c vector.
prob.c = [ 1 2 0]';

% Specify a in sparse format.
subi   = [1 2 2 1];
subj   = [1 1 2 3];
valij  = [1.0 1.0 1.0 1.0];

prob.a = sparse(subi,subj,valij);

% Specify lower bounds on the constraints.
prob.blc  = [4.0 1.0]';

% Specify  upper bounds on the constraints.
prob.buc  = [6.0 inf]';

% Specify lower bounds on the variables.
prob.blx  = sparse(3,1);

% Specify upper bounds on the variables.
prob.bux = [];   % There are no bounds.

% Perform the optimization.
[r,res] = mosekopt('minimize',prob); 

% Show the optimal x solution.
res.sol.bas.xx

Observe that

  • A MATLAB structure named prob containing all the relevant problem data is defined.
  • All fields of this structure are optional except prob.a which is required to be a sparse matrix.
  • Different parts of the solution can be viewed by inspecting the solution field res.sol.

6.5. Convex quadratic optimization

A frequently occurring problem type is the quadratic optimization problem which consists of minimizing a quadratic objective function subject to linear constraints. One example of such a problem is:

\begin{math}\nonumber{}\begin{array}{lcccl}\nonumber{}\mbox{minimize} &  &  & x_{1}^{2}+0.1x_{2}^{2}+x_{3}^{2}-x_{1}x_{3}-x_{2} & \\\nonumber{}\mbox{subject to} & 1 & \leq{} & x_{1}+x_{2}+x_{3} & \\\nonumber{} &  &  & x\geq{}0. &\end{array}\end{math} (6.5.1)

In general a quadratic optimization problem has the form

\begin{math}\nonumber{}\begin{array}{lccccl}\nonumber{}\mbox{minimize} &  &  & \frac{1}{2}x^{T}Qx+c^{T}x &  & \\\nonumber{}\mbox{subject to} & l^{c} & \leq{} & Ax, & \leq{} & u^{c},\\\nonumber{} & l^{x} & \leq{} & x & \leq{} & u^{x},\end{array}\end{math} (6.5.2)

which for the example (6.5.1) implies

\begin{math}\nonumber{}Q=\left[\begin{array}{ccc}\nonumber{}2 & 0 & -1\\\nonumber{}0 & 0.2 & 0\\\nonumber{}-1 & 0 & 2\end{array}\right],\quad{}c=\left[\begin{array}{c}\nonumber{}0\\\nonumber{}-1\\\nonumber{}0\end{array}\right],\quad{}A=\left[\begin{array}{ccc}\nonumber{}1 & 1 & 1\end{array}\right],\end{math} (6.5.3)

and

\begin{displaymath}\nonumber{}l^{c}=1,\quad{}u^{c}=\infty ,\quad{}l^{x}=\left[\begin{array}{c}\nonumber{}0\\\nonumber{}0\\\nonumber{}0\end{array}\right]\mbox{ and }u^{x}=\left[\begin{array}{c}\nonumber{}\infty \\\nonumber{}\infty\end{array}\right]\end{displaymath}

Note the explicit [[MathCmd 14]] in the objective function of (6.5.1) which implies diagonal elements must be doubled in Q i.e. [[MathCmd 15]], whereas the coefficient in (6.5.2) is 1 in front of [[MathCmd 16]].

6.5.1. Two important assumptions

MOSEK assumes that the Q matrix is symmetric i.e.

\begin{displaymath}\nonumber{}Q=Q^{T}\end{displaymath}

and Q is positive semi-definite . A matrix is positive semi-definite if the smallest eigenvalue of the matrix is nonnegative. An alternative statement of positive semi-definite requirement is

\begin{displaymath}\nonumber{}x^{T}Qx\geq{}0,~\forall x.\end{displaymath}

If Q is not positive semi-definite, then MOSEK will not produce reliable results or work at all.

One way of checking whether Q is positive semi-definite is to check whether all the eigenvalues of Q are nonnegative. The MATLAB command eig computes all eigenvalues of a matrix.

6.5.2. Using mskqpopt

The subsequent MATLAB statements solve the problem (6.5.1) using the MOSEK function mskqpopt

% qo1.m

% Setup Q.
q     = [[2 0 -1];[0 0.2 0];[-1 0 2]];

% Setup the linear part of the problem.
c     = [0 -1 0]';
a     = ones(1,3);
blc   = [1.0];
buc   = [inf];
blx   = sparse(3,1);
bux   = [];

% Optimize the problem.
[res] = mskqpopt(q,c,a,blc,buc,blx,bux);

% Show the primal solution.
res.sol.itr.xx

It should be clear that the format for calling mskqpopt is very similar to calling msklpopt except that the Q matrix is included as the first argument of the call. Similarly, the solution can be inspected by viewing the field res.sol.

6.5.3. Using mosekopt

The following sequence of MATLAB commands solve the quadratic optimization example by calling mosekopt directly.

% qo2.m

clear prob;

% c vector.
prob.c = [0 -1 0]';

% Defining the data.

% First the lower triangular part of q in the objective 
% is specified in a sparse format. The format is:
%
%   Q(prob.qosubi(t),prob.qosubj(t)) = prob.qoval(t), t=1,...,4

prob.qosubi = [ 1  3 2   3]';
prob.qosubj = [ 1  1 2   3]';
prob.qoval  = [ 2 -1 0.2 2]';

% a, the constraint matrix
subi  = ones(3,1);
subj  = 1:3;
valij = ones(3,1);

prob.a = sparse(subi,subj,valij);

% Lower bounds on constraints
prob.blc  = [1.0]';

% Upper bounds on constraints
prob.buc  = [inf]';

% Lower bounds on variables
prob.blx  = sparse(3,1);

% Upper bounds on variables.
prob.bux = [];   % There are no bounds.

[r,res] = mosekopt('minimize',prob);

% Display return code
fprintf('Return code: %d\n',r);

% Display primal solution for the constraints
res.sol.itr.xc'

% Display primal solution for the variables
res.sol.itr.xx'

This sequence of commands looks much like the one that was used to solve the linear optimization example using mosekoptexcept for the definition of the Q matrix in prob. mosekopt requires that Q is specified in a sparse format. Indeed the vectors qosubi, qosubj, and qoval are used to specify the coefficients of Q in the objective using the principle

\begin{displaymath}\nonumber{}Q_{{\mbox{{qosubi(t)}},\mbox{{qosubj(t)}}}}=\mbox{{qoval(t)}},\quad{}\mbox{for}~t=1,\ldots ,\mbox{length}(\mbox{{qosubi}}).\end{displaymath}

An important observation is that due to Q is symmetric, then only the lower triangular part of Q should be specified.

6.6. Conic optimization

One way of generalizing a linear optimization problem is to include a constraint of the form

\begin{displaymath}\nonumber{}x\in{}\mathcal{C}\end{displaymath}

in the problem definition where [[MathCmd 21]] is required to be a convex cone. The resulting class of problems is known as conic optimization.

MOSEK can solve a subset of all conic problems and subsequently it is demonstrated how to solve this subset using the toolbox function mosekopt.

6.6.1. The conic optimization problem

To be specific a conic optimization problem has the following form

\begin{math}\nonumber{}\begin{array}{lcccccl}\nonumber{}\mbox{minimize} &  &  & c^{T}x+c^{f} &  &  & \\\nonumber{}\mbox{subject to} & l^{c} & \leq{} & Ax & \leq{} & u^{c}, & \\\nonumber{} & l^{x} & \leq{} & x & \leq{} & u^{x}, & \\\nonumber{} &  &  & x\in{}\mathcal{C}, &  &  &\end{array}\end{math} (6.6.1)

where [[MathCmd 21]] must satisfy the following requirements. Let

\begin{displaymath}\nonumber{}x^{t}\in{}{R}^{{n^{t}}},~t=1,\ldots ,k\end{displaymath}

be vectors comprised of parts of the decision variables x such that each decision variable is a member of exactly one vector [[MathCmd 25]]. For example it could be the case that

\begin{displaymath}\nonumber{}x^{1}=\left[\begin{array}{c}\nonumber{}x_{1}\\\nonumber{}x_{4}\\\nonumber{}x_{7}\end{array}\right]\mbox{ and }x^{2}=\left[\begin{array}{c}\nonumber{}x_{6}\\\nonumber{}x_{5}\\\nonumber{}x_{{3}}\\\nonumber{}x_{2}\end{array}\right].\end{displaymath}

Next define

\begin{displaymath}\nonumber{}\mathcal{C}:=\left\lbrace{}x\in{}{R}^{n}:~x^{t}\in{}\mathcal{C}_{t},~t=1,2,\ldots ,k\right\rbrace{}\end{displaymath}

where [[MathCmd 28]] must have one of the following forms.

  • [[MathCmd 29]] set:

    \begin{displaymath}\nonumber{}\mathcal{C}_{t}=\lbrace{}x\in{}{R}^{{n^{t}}}\rbrace{}.\end{displaymath}
  • Quadratic cone:

    \begin{displaymath}\nonumber{}\mathcal{C}_{t}=\left\lbrace{}x\in{}{R}^{{n^{t}}}:x_{1}\geq{}\sqrt{\sum \limits _{{j=2}}^{{n^{t}}}x_{j}^{2}}\right\rbrace{}.\end{displaymath}
  • Rotated quadratic cone:

    \begin{displaymath}\nonumber{}\mathcal{C}_{t}=\left\lbrace{}x\in{}{R}^{{n^{t}}}:2x_{1}x_{2}\geq{}\sum \limits _{{j=3}}^{{n^{t}}}x_{j}^{2},~x_{1},x_{2}\geq{}0\right\rbrace{}.\end{displaymath}

The [[MathCmd 29]] set is never specified explicitly, because if a variable is not a member of any other cone, then it is member of this cone.

Although the cones MOSEK can handle give rise to a limited class of conic problems, then it includes linear, quadratic, quadratically constrained optimization, and other classes of nonlinear convex optimization problems. See Section 9.4 for a discussion.

6.6.2. Solving an example

The problem

\begin{math}\nonumber{}\begin{array}{lccccc}\nonumber{}\mbox{minimize} & x_{5}+x_{6} &  & \\\nonumber{}\mbox{subject to} & x_{1}+x_{2}+x_{3}+x_{4} & = & 1,\\\nonumber{} & x_{1},x_{2},x_{3},x_{4} & \geq{} & 0,\\\nonumber{} & x_{5}\geq{}\sqrt{x_{1}^{2} + x_{3}^{2}}, &  & \\\nonumber{} & x_{6}\geq{}\sqrt{x_{2}^{2} + x_{4}^{2}} &  &\end{array}\end{math} (6.6.2)

is an example of a conic quadratic optimization problem. The problem involves some linear constraints and two quadratic cones. The linear constraints are specified just as if the problem where a linear problem whereas the cones are specified using a MATLAB cell array named cones. cones must contain one cell per cone, where a cell must contain the two fields type and sub. type is used to specify the the type of the cone and sub is used to specify the member variables of the cone.

The following MATLAB code demonstrates how to solve the example (6.6.2) using MOSEK.

% cqo1.m

clear prob;

% First the non conic part of the problem is specified.

prob.c   = [0 0 0 0 1 1];
prob.a   = sparse([1 1 1 1 0 0]);
prob.blc = 1;
prob.buc = 1;
prob.blx = [0 0 0 0 -inf -inf];
prob.bux = inf*ones(6,1);

% Next the cones are specified.

% First an empty cell array named
% cones is defined. It should contain
% one cell per cone.

prob.cones   = cell(2,1);

% The first cone is specified.

prob.cones{1}.type = 'MSK_CT_QUAD';
prob.cones{1}.sub  = [5 3 1];

% The subfield type specifies the cone type
% i.e. whether it is quadratic cone
% or rotated quadratic cone. The keys
% for the two cone types are MSK_CT_QUAD
% MSK_CT_RQUAD respectively.
%
% The subfield sub specifies the members
% of the cone. I.e. the above definition
% implies x(5) >= sqrt(x(3)^2+x(1)^2)

% The second cone is specified.

prob.cones{2}.type = 'MSK_CT_QUAD';
prob.cones{2}.sub  = [6 2 4];

% Finally, the problem is optimized.

[r,res]=mosekopt('minimize',prob);

% The primal solution is displayed.

res.sol.itr.xx'

A couple of important comments are:

  • No variable must be member of more than one cone. This is not serious restriction. See the subsequent section.
  • The R set is not specified explicitly.

6.6.3. Quadratic and conic optimization

The example

\begin{math}\nonumber{}\begin{array}{llcl}\nonumber{}\mbox{minimize} & x_{1}+x_{2}+x_{3} &  & \\\nonumber{}\mbox{subject to} & x_{1}^{2}+x_{2}^{2}+x_{3}^{2} & \leq{} & 1,\\\nonumber{} & x_{1}+0.5x_{2}^{2}+x_{3} & \leq{} & 0.5\end{array}\end{math} (6.6.3)

is not a conic quadratic optimization problem but can easily be reformulated as such.

Indeed the first constraint is equivalent to

\begin{math}\nonumber{}\begin{array}{rcl}\nonumber{}x_{4} & \geq{} & \sqrt{x_{1}^{2} + x_{2}^{2} + x_{3}^{2}},\\\nonumber{}x_{4} & = & 1\end{array}\end{math} (6.6.4)

where [[MathCmd 37]] is a new variable. This is quadratic cone and linear constraint. The second constraint in (6.6.3) is equivalent to

\begin{displaymath}\nonumber{}\begin{array}{rcl}\nonumber{}x_{1}+x_{3}+x_{5} & = & 0.5,\\\nonumber{}x_{2}-x_{7} & = & 0,\\\nonumber{}x_{5} & \geq{} & 0,\\\nonumber{}x_{6} & = & 1,\\\nonumber{}x_{7}^{2} & \leq{} & 2x_{5}x_{6},\end{array}\end{displaymath}

because this implies

\begin{displaymath}\nonumber{}x_{5}\geq{}0.5x_{7}^{2}=0.5x_{2}^{2.}\end{displaymath}

and

\begin{displaymath}\nonumber{}x_{1}+0.5x_{2}^{2}+x_{3}\leq{}x_{1}+x_{3}+x_{5}=0.5.\end{displaymath}

Observe that no variable can occur in more than one cone and therefore the additional constraint

\begin{displaymath}\nonumber{}x_{2}=x_{7}\end{displaymath}

is introduced and [[MathCmd 42]] has been included in second conic constraint instead of [[MathCmd 43]]. Using this “trick” then it is always possible to obtain a formulation where no variable occurs in more than one cone.

Therefore, the example (6.6.3) is equivalent to the conic quadratic optimization problem

\begin{math}\nonumber{}\begin{array}{lccl}\nonumber{}\mbox{minimize} & x_{1}+x_{2}+x_{3} &  & \\\nonumber{}\mbox{subject to} & x_{1}+x_{3}+x_{5} & = & 0.5,\\\nonumber{} & x_{2}-x_{7} & = & 0,\\\nonumber{} & x_{4} & = & 1,\\\nonumber{} & x_{5} & \geq{} & 0,\\\nonumber{} & x_{6} & = & 1,\\\nonumber{} & x_{4}\geq{}\sqrt{x_{1}^{2} + x_{2}^{2} + x_{3}^{2}}, &  & \\\nonumber{} & 2x_{5}x_{6}\geq{}x_{7}^{2.} &  &\end{array}\end{math} (6.6.5)

This problem can be solved using MOSEK as follows:

% cqo2.m

% Setup of the non conic part of the problem.

prob          = [];
prob.c        = [1 1 1 0 0 0 0]';
prob.a        = sparse([[1 0 1 0 1 0 0];...
                        [0 1 0 0 0 0 -1]]);
prob.blc      = [0.5 0];
prob.buc      = [0.5 0];
prob.blx      = [-inf -inf -inf 1 -inf 1 -inf];
prob.bux      = [inf   inf  inf 1  inf 1  inf];

% Setup of cone information.

prob.cones         = cell(2,1);
prob.cones{1}.type = 'MSK_CT_QUAD';
prob.cones{1}.sub  = [4 1 2 3];
prob.cones{2}.type = 'MSK_CT_RQUAD';
prob.cones{2}.sub  = [5 6 7];

[r,res]       = mosekopt('minimize',prob);

% Display the solution.
res.sol.itr.xx'

6.6.4. Conic duality and the dual solution

The dual problem corresponding to the conic optimization problem (6.6.1) is given by

\begin{math}\nonumber{}\begin{array}{lcccccl}\nonumber{}\mbox{maximize} & (l^{c})^{T}s_{l}^{c}-(u^{c})^{T}s_{u}^{c} &  & \\\nonumber{} & +(l^{x})^{T}s_{l}^{x}-(u^{x})^{T}s_{u}^{x}+c^{f} &  & \\\nonumber{}\mbox{subject to} & -y+s_{l}^{c}-s_{u}^{c} & = & 0,\\\nonumber{} & A^{T}y+s_{l}^{x}-s_{u}^{x}+s_{n}^{x} & = & c,\\\nonumber{} & s_{l}^{c},s_{u}^{c},s_{l}^{x},s_{u}^{x} & \geq{} & 0,\\\nonumber{} & s_{n}^{x}\in{}\mathcal{C}^{*} &  &\end{array}\end{math} (6.6.6)

where the dual cone [[MathCmd 46]] is defined as follows. Let [[MathCmd 47]] be partitioned similar to x i.e. if [[MathCmd 48]] is member of [[MathCmd 25]], then [[MathCmd 50]] is a member of [[MathCmd 51]] as well. Now the dual cone is defined by

\begin{displaymath}\nonumber{}\mathcal{C}^{*}:=\left\lbrace{}s_{n}^{x}\in{}{R}^{{n^{t}}}:~(s_{n}^{x})^{t}\in{}\mathcal{C}^{*}_{t},~t=1,\ldots ,k\right\rbrace{}\end{displaymath}

where the type of [[MathCmd 53]] is dependent on the type of [[MathCmd 28]]. For the cone types MOSEK can handle the relation between the primal and dual cones are given as follows:

  • [[MathCmd 29]] set:

    \begin{displaymath}\nonumber{}\mathcal{C}_{t}=\left\lbrace{}x\in{}{R}^{{n^{t}}}\right\rbrace{}\quad{}\Leftrightarrow \quad{}\mathcal{C}^{*}_{t}:=\left\lbrace{}s\in{}{R}^{{n^{t}}}:~s=0\right\rbrace{}.\end{displaymath}
  • Quadratic cone:

    \begin{displaymath}\nonumber{}\mathcal{C}_{t}:=\left\lbrace{}x\in{}{R}^{{n^{t}}}:x_{1}\geq{}\sqrt{\sum \limits _{{j=2}}^{{n^{t}}}x_{j}^{2}}\right\rbrace{}\quad{}\Leftrightarrow \quad{}\mathcal{C}^{*}_{t}=\mathcal{C}_{t}.\end{displaymath}
  • Rotated quadratic cone:

    \begin{displaymath}\nonumber{}\mathcal{C}_{t}:=\left\lbrace{}x\in{}{R}^{{n^{t}}}:2x_{1}x_{2}\geq{}\sqrt{\sum \limits _{{j=3}}^{{n^{t}}}x_{j}^{2}},~x_{1},x_{2}\geq{}0\right\rbrace{}.\quad{}\Leftrightarrow \quad{}\mathcal{C}^{*}_{t}=\mathcal{C}_{t}.\end{displaymath}

For a more detailed discussion about conic duality see Section 9.4.

6.6.4.1. How to obtain the dual solution

When solving a conic optimization problem using MOSEK then the dual solution is of course available. The following MATLAB code fragment shows where the dual solution is stored.

% cqo3.m

[r,res]=mosekopt('minimize',prob);

% Solution record.
res.sol

% Dual variables for lower
% bounds on constraints.
res.sol.itr.slc'

% Dual variables for upper
% bounds on constraints.
res.sol.itr.suc'

% Dual variables for lower
% bounds on variable.
res.sol.itr.slx'

% Dual variables for upper
% bounds on variables.
res.sol.itr.sux'

% Dual variables with respect
% to the conic constraints.
res.sol.itr.snx'

6.6.5. Setting accuracy parameters for the conic optimizer

Three parameters controls the accuracy of the solution obtained by conic interior-point optimizer. The following example demonstrates which parameters should be reduced to obtain a more accurate solution if required.

% How to change the parameters that controls
% the accuracy of a solution computed by the conic
% optimizer.

param = [];

% Primal feasibility tolerance for the primal solution
param.MSK_DPAR_INTPNT_CO_TOL_PFEAS = 1.0e-8;

% Dual feasibility tolerance for the dual solution
param.MSK_DPAR_INTPNT_CO_TOL_DFEAS = 1.0e-8;

% Relative primal-dual gap tolerance.
param.MSK_DPAR_INTPNT_CO_TOL_REL_GAP = 1.0e-8;

[r,res]=mosekopt('minimize',prob,param);

6.7. Quadratically constrained optimization

In the previous section a quadratically constrained optimization problem was solved using the conic optimizer. It is also possible to solve such a problem directly. One example of such an optimization problem is

\begin{math}\nonumber{}\begin{array}{llcl}\nonumber{}\mbox{minimize} & x_{1}+x_{2}+x_{3} &  & \\\nonumber{}\mbox{subject to} & x_{1}^{2}+x_{2}^{2}+x_{3}^{2} & \leq{} & 1,\\\nonumber{} & x_{1}+0.1x_{2}^{2}+x_{3} & \leq{} & 0.5.\end{array}\end{math} (6.7.1)

Note there are quadratic terms in both constraints. This problem can be solved using mosekopt as follows:

% qco1.m

clear prob;

% Specifying problem data.

% First c.
prob.c      = ones(3,1);

% Next quadratic terms in the constraints.
prob.qcsubk = [1   1   1   2  ]';
prob.qcsubi = [1   2   3   2  ]';
prob.qcsubj = [1   2   3   2  ]';
prob.qcval  = [2.0 2.0 2.0 0.2]';

% a
prob.a      = [sparse(1,3);sparse([1 0 1])];

prob.buc    = [1 0.5]';

[r,res]     = mosekopt('minimize',prob);

% Viewing the solution.
fprintf('\nx:');
fprintf(' %-.4e',res.sol.itr.xx');
fprintf('\n||x||: %-.4e',norm(res.sol.itr.xx));

Note the quadratic terms in the constraints are specified using the fields prob.qcsubk, prob.qcsubi, prob.qcsubj, and prob.qcval as follows

\begin{displaymath}\nonumber{}Q_{{\mbox{{qcsubi(t)}},\mbox{{qcsubj(t)}}}}^{{\mbox{{qcsubk(t)}}}}=\mbox{{qcval(t)}},\quad{}\mbox{for}\quad{}t=1,\ldots ,\mbox{length}(\mbox{{qcsubk}})\end{displaymath}

where [[MathCmd 61]] is the quadratic term in the kth constraint. Also observe that only the lower triangular part of the [[MathCmd 62]]s should be specified.

6.8. Linear least squares and related norm minimization problems

A frequently occurring problem in statistics and in many other areas of science is the problem

\begin{math}\nonumber{}\mbox{minimize}\quad{}\left\|Fx-b\right\|\end{math} (6.8.1)

where F and b is a matrix and vector of appropriate dimensions. x is the vector decision variables.

Typically the norm used is the 1, the 2, or the infinity norm.

6.8.1. The case of the 2 norm

However, initially let us focus on the 2 norm. In this case (6.8.1) is identical to the quadratic optimization problem

\begin{math}\nonumber{}\mbox{minimize}\quad{}1/2x^{T}F^{T}Fx+1/2b^{T}b-b^{T}Fx\end{math} (6.8.2)

in the sense that the set of optimal solutions for the two problems coincide. This fact follows from

\begin{displaymath}\nonumber{}\begin{array}{rcl}\nonumber{}\left\|Fx-b\right\|^{2} & = & (Fx-b)^{T}(Fx-b)\\\nonumber{} &  & x^{T}F^{T}Fx+b^{T}b+2b^{T}Fx.\end{array}\end{displaymath}

Subsequently, it is demonstrated how the quadratic optimization problem (6.8.2) is solved using mosekopt. In the example the problem data is first read from file. Next data for the problem (6.8.2) is constructed and finally the problem is solved.

% nrm1.m

% Read data form the file afiro.mps.
[r,res] = mosekopt('read(afiro.mps)');

% Getting data for the problem
%              minimize ||f x - b||_2
f = res.prob.a';
b = res.prob.c;

% The problem is solved by solving
%             minimize 0.5 xf'fx+0.5*b'*b-(f'*b)'*x

% Clearing prob
clear prob;

% Compute the fixed term in the objective.
% It is not really needed.
prob.cfix = 0.5*b'*b

% Forming c
prob.c = -f'*b;

% Forming q. Note only the lower triangular
% part of f'*f is used.
[prob.qosubi,prob.qosubj,prob.qoval] = find(sparse(tril(f'*f)))

% Obtain the matrix dimensions.
[m,n]   = size(f);

% a is specified
prob.a  = sparse(0,n);

[r,res] = mosekopt('minimize',prob);

% The optimality conditions are f'*(f x - b) = 0.
% Checking if they are satisfied:

fprintf('\nnorm(f^T(fx-b)): %e',norm(f'*(f*res.sol.itr.xx-b)));

Quite frequently the x variables must be within some bounds or satisfy some additional linear constraints. These requirements can easily be incorporated into the problem (6.8.2). For example the constraint [[MathCmd 66]] can be modeled as follows

% nrm2.m. Continuation of nrm1.m.

% Now assume the same objective should be
% minimized subject to -1 <= x <= 1

prob.blx = -ones(n,1);
prob.bux = ones(n,1);

[r,res] = mosekopt('minimize',prob);

% Checking if the solution is feasible
norm(res.sol.itr.xx,inf)

6.8.2. The case of the infinity norm

In some applications of the norm minimization problem (6.8.1) it is better to use the infinity norm than the 2 norm. However, the problem (6.8.1) stated as an infinity norm problem is equivalent to the linear optimization problem

\begin{math}\nonumber{}\begin{array}{lccl}\nonumber{}\mbox{minimize} & \tau  &  & \\\nonumber{}\mbox{subject to} & Fx+\tau e-b & \geq{} & 0,\\\nonumber{} & Fx-\tau e-b & \leq{} & 0,\end{array}\end{math} (6.8.3)

where e is the vector of all ones of appropriate dimension. This implies

\begin{displaymath}\nonumber{}\begin{array}{rcl}\nonumber{}\tau e & \geq{} & Fx-b\\\nonumber{}\tau e & \geq{} & -(Fx-b)\end{array}\end{displaymath}

and hence at optimum

\begin{displaymath}\nonumber{}\tau ^{*}=\left\|Fx^{*}-b\right\|_{\infty }\end{displaymath}

holds.

The problem (6.8.3) is straightforward to solve.

% nrm3.m. Continuation of nrm1.m.

% Let x(n+1) play the role as tau, then the problem is
% solved as follows.

clear prob;

prob.c   = sparse(n+1,1,1.0,n+1,1);
prob.a   = [[f,ones(m,1)];[f,-ones(m,1)]];
prob.blc = [b            ; -inf*ones(m,1)];
prob.buc = [inf*ones(m,1); b             ];

[r,res]  = mosekopt('minimize',prob);

% The optimal objective value is given by
norm(f*res.sol.itr.xx(1:n)-b,inf) 

6.8.3. The case of the one norm

Finally, in the case of the one norm then by definition we have that

\begin{displaymath}\nonumber{}\left\|Fx-b\right\|_{1}=\sum _{{i=1}}^{m}|f_{{i:}}x-b_{i}|\end{displaymath}

Therefore, the norm minimization problem can be formulated as follows

\begin{math}\nonumber{}\begin{array}{lccll}\nonumber{}\mbox{minimize} & \sum \limits _{{i=1}}^{m}t_{i} &  &  & \\\nonumber{}\mbox{subject to} & |f_{{i:}}x-b_{i}| & = & t_{i}, & i=1,\ldots ,m,\end{array}\end{math} (6.8.4)

which in turn is equivalent to

\begin{math}\nonumber{}\begin{array}{lccll}\nonumber{}\mbox{minimize} & \sum \limits _{{i=1}}^{m}t_{i} &  &  & \\\nonumber{}\mbox{subject to} & f_{{i:}}x-b_{i} & \leq{} & t_{i}, & i=1,\ldots ,m,\\\nonumber{} & -(f_{{i:}}x-b_{i}) & \leq{} & t_{i}, & i=1,\ldots ,m.\end{array}\end{math} (6.8.5)

The reader should verify that this is really the case.

In matrix notation this problem can be expressed as follows

\begin{math}\nonumber{}\begin{array}{lccl}\nonumber{}\mbox{minimize} & e^{T}t &  & \\\nonumber{}\mbox{subject to} & Fx-te & \leq{} & b,\\\nonumber{} & Fx+te & \geq{} & b,\end{array}\end{math} (6.8.6)

where [[MathCmd 74]]. Next this problem is solved.

% nrm4.m. Continuation of nrm1.m.

% Let x(n:(m+n)) play the role as t. Now
% the problem can be solved as follows

clear prob;

prob.c   = [sparse(n,1)   ; ones(m,1)];
prob.a   = [[f,-speye(m)] ; [f,speye(m)]];
prob.blc = [-inf*ones(m,1); b];
prob.buc = [b             ; inf*ones(m,1)];

[r,res]  = mosekopt('minimize',prob);

% The optimal objective value is given by:
norm(f*res.sol.itr.xx(1:n)-b,1) 

6.8.3.1. A better formulation

It is possible to improve upon the formulation of the problem (6.8.5). Indeed problem (6.8.5) is equivalent to

\begin{math}\nonumber{}\begin{array}{lccll}\nonumber{}\mbox{minimize} & \sum \limits _{{i=1}}^{m}t_{i} &  &  & \\\nonumber{}\mbox{subject to} & f_{{i:}}x-b_{i}-t_{i}+v_{i} & = & 0, & i=1,\ldots ,m,\\\nonumber{} & -(f_{{i:}}x-b_{i})-t_{i} & \leq{} & 0, & i=1,\ldots ,m,\\\nonumber{} & v_{i}\geq{}0, &  &  & i=1,\ldots ,m.\end{array}\end{math} (6.8.7)

After eliminating the t variables then this problem is equivalent to

\begin{math}\nonumber{}\begin{array}{lccll}\nonumber{}\mbox{minimize} & \sum \limits _{{i=1}}^{m}(f_{{i:}}x-b_{i}+v_{i}) &  &  & \\\nonumber{}\mbox{subject to} & -2(f_{{i:}}x-b_{i})-v_{i} & \leq{} & 0, & i=1,\ldots ,m,\\\nonumber{} & v_{i}\geq{}0, &  &  & i=1,\ldots ,m.\end{array}\end{math} (6.8.8)

Note this problem only have the half number of general constraints of problem (6.8.5). I.e. we have replaced constraints of the general form

\begin{displaymath}\nonumber{}f_{{i:}}x\leq{}b_{i}\end{displaymath}

with simpler constraints

\begin{displaymath}\nonumber{}v_{i}\geq{}0\end{displaymath}

which MOSEK treats in a special and highly efficient way. Also note MOSEK only stores the non-zeros in the coefficient matrix of the constraints. This implies that the problem (6.8.8) is likely to require much less space than the problem (6.8.7).

It is left as an exercise for the reader to implement this formulation in MATLAB.

6.9. More about solving linear least squares problems

Linear least squares problem with and without linear side constraints appear very frequently in practice and it is therefore important to know how such problems are solved efficiently using MOSEK.

Now assume that the problem of interest is the linear least squares problem

\begin{math}\nonumber{}\begin{array}{lccccc}\nonumber{}\mbox{minimize} & \frac{1}{2}\left\|Fx-f\right\|_{2}^{2} &  & \\\nonumber{}\mbox{subject to} & Ax & = & b,\\\nonumber{} & l^{x}\leq{}x\leq{}u^{x}, &  &\end{array}\end{math} (6.9.1)

where F and A are matrices and the remaining quantities are vectors. x is the vector of decision variables. The problem (6.9.1) as stated is a convex quadratic optimization problem and can be solved as such.

However, if F has much fewer rows than columns then it will usually be more efficient to solve the equivalent problem

\begin{math}\nonumber{}\begin{array}{lccccc}\nonumber{}\mbox{minimize} & \frac{1}{2}\left\|z\right\|_{2}^{2} &  & \\\nonumber{}\mbox{subject to} & Ax & = & b,\\\nonumber{} & Fx-z & = & f,\\\nonumber{} & l^{x}\leq{}x\leq{}u^{x}. &  &\end{array}\end{math} (6.9.2)

Note a number of new constraints and variables have been introduced which of course seem to be disadvantageous but on the other hand the Hessian of the objective in problem (6.9.2) is much sparser than in problem (6.9.1). This frequently turns out to be more important for the computational efficiency and therefore the latter formulation is usually the better one.

In the case F has many more rows than columns, then formulation (6.9.2) is not attractive but the corresponding dual problem is. Using the duality theory outlined in Section 9.5.1 we obtain the dual problem

\begin{math}\nonumber{}\begin{array}{lccccc}\nonumber{}\mbox{maximize} & b^{T}y+f^{T}\bar{y} &  & \\\nonumber{} & +(l^{x})^{T}s_{l}^{x}+(u^{x})^{T}s_{u}^{x} &  & \\\nonumber{} & -\frac{1}{2}\left\|z\right\|_{2}^{2} &  & \\\nonumber{}\mbox{subject to} & A^{T}y+F^{T}\bar{y}+s_{l}^{x}-s_{u}^{x} & = & 0,\\\nonumber{} & z-\bar{y} & = & 0,\\\nonumber{} & s_{l}^{x},s_{u}^{x}\geq{}0 &  &\end{array}\end{math} (6.9.3)

which can be simplified to

\begin{math}\nonumber{}\begin{array}{lccccc}\nonumber{}\mbox{maximize} & b^{T}y+f^{T}z &  & \\\nonumber{} & +(l^{x})^{T}s_{l}^{x}+(u^{x})^{T}s_{u}^{x} &  & \\\nonumber{} & -\frac{1}{2}\left\|z\right\|_{2}^{2} &  & \\\nonumber{}\mbox{subject to} & A^{T}y+F^{T}z+s_{l}^{x}-s_{u}^{x} & = & 0,\\\nonumber{} & s_{l}^{x},s_{u}^{x}\geq{}0 &  &\end{array}\end{math} (6.9.4)

after eliminating the [[MathCmd 83]] variables. Here we use the convention that

\begin{displaymath}\nonumber{}l_{j}^{x}=-\infty ~\Rightarrow (s_{l}^{x})_{j}=0\quad{}\mbox{and}\quad{}u_{j}^{x}=\infty \Rightarrow (s_{u}^{x})_{j}=0.\end{displaymath}

In practice such fixed variables in [[MathCmd 85]] and [[MathCmd 86]] should be removed from the problem.

Given our assumptions then the dual problem (6.9.4) will have much fewer constraints than the primal problem (6.9.2). This is important because MOSEK tends to be more efficient the fewer constraints there are in the problem. One obvious question is what if the dual problem (6.9.4) is solved instead of the primal problem (6.9.2), then how is the optimal x solution obtained. It turns that the dual variables corresponding to the constraint

\begin{displaymath}\nonumber{}A^{T}y+F^{T}z+s_{l}^{x}-s_{u}^{x}=0\end{displaymath}

is the optimal x solution. Therefore, due to MOSEK always reports this information as the

res.sol.itr.y

vector, then the optimal x solution can easily be obtained.

In the subsequent code fragment it is investigated whether it is attractive to solve the dual problem rather than the primal for a concrete numerical example. This example has no linear equalities and F is a 2000 by 400 matrix.

% nrm5.m

% First read data from a file.
[rcode,res] = mosekopt('read(lsqpd.mps) echo(0)');

% The problem data.
F           = res.prob.a;
f           = res.prob.blc;
blx         = res.prob.blx;
bux         = [];

% In this case there are no linear constraints
% First we solve the primal problem:
%
% minimize 0.5|| z ||^2
% subject  F x - z = f
%          l <= x <= u

% Note m»n
[m,n]       = size(F);

prob        = [];

prob.qosubi = n+(1:m);
prob.qosubj = n+(1:m);
prob.qoval  = ones(m,1);
prob.a      = [F,-speye(m,m)];
prob.blc    = f;
prob.buc    = f;
prob.blx    = [blx;-inf*ones(m,1)];
prob.bux    = bux;


fprintf('m=%d  n=%d\n',m,n);

fprintf('First try\n');

tic
[rcode,res] = mosekopt('minimize echo(0)',prob);

%Display the solution time
fprintf('Time           : %-.2f\n',toc);

try 
  % x solution
  x = res.sol.itr.xx;

  % objective value
  fprintf('Objective value: %-6e\n',norm(F*x(1:n)-f)^2);

  % Check feasibility
  fprintf('Feasibility    : %-6e\n',min(x(1:n)-blx(1:n)));
catch
  fprintf('MSKERROR: Could not get solution')
end

% Clearing prob again.
prob=[];

%
% Next we  solve the dual problem

% Index of lower bounds that are finite
lfin        = find(blx>-inf);

% Index of upper bounds that are finite
ufin        = find(bux<inf);

prob.qosubi = 1:m;
prob.qosubj = 1:m;
prob.qoval  = -ones(m,1);
prob.c      = [f;blx(lfin);-bux(ufin)];
prob.a      = [F',...
               sparse(lfin,(1:length(lfin))',...
                      ones(length(lfin),1),...
                      n,length(lfin)),...
               sparse(ufin,(1:length(ufin))',...
                      -ones(length(ufin),1),...
                      n,length(ufin))];
prob.blc    = sparse(n,1);
prob.buc    = sparse(n,1);
prob.blx    = [-inf*ones(m,1);...
               sparse(length(lfin)+length(ufin),1)];
prob.bux    = [];

fprintf('\n\nSecond try\n');
tic
[rcode,res] = mosekopt('maximize echo(0)',prob);

%Display the solution time
fprintf('Time           : %-.2f\n',toc);

try
  % x solution
  x = res.sol.itr.y;

  % objective value
  fprintf('Objective value: %-6e\n',...
          norm(F*x(1:n)-f)^2);

  % Check feasibility
  fprintf('Feasibility    : %-6e\n',...
          min(x(1:n)-blx(1:n)));
catch
  fprintf('MSKERROR: Could not get solution')
end

Here is the output produced:

m=2000  n=400
First try
Time           : 2.07
Objective value: 2.257945e+001
Feasibility    : 1.466434e-009


Second try
Time           : 0.47
Objective value: 2.257945e+001
Feasibility    : 2.379134e-009 

It can be observed that both formulations produced a strictly feasible solution having the same objective value. Moreover, using the dual formulation leads to a reduction in the solution time by about a factor 5. So in this case we can conclude that the dual formulation is far superior to the primal formulation of the problem.

6.9.1. Using conic optimization linear least squares problems

Linear least squares problems can also be solved using conic optimization because the linear least squares problem

\begin{math}\nonumber{}\begin{array}{lccccc}\nonumber{}\mbox{minimize} & \left\|Fx-f\right\|_{2} &  & \\\nonumber{}\mbox{subject to} & Ax & = & b,\\\nonumber{} & l^{x}\leq{}x\leq{}u^{x}, &  &\end{array}\end{math} (6.9.5)

is equivalent to

\begin{math}\nonumber{}\begin{array}{lccccc}\nonumber{}\mbox{minimize} &  &  & t &  & \\\nonumber{}\mbox{subject to} &  &  & Ax & = & b,\\\nonumber{} &  &  & Fx-z & = & f,\\\nonumber{} & l^{x} & \leq{} & x & \leq{} & u^{x},\\\nonumber{} &  &  & \left\|z\right\|_{2}\leq{}t. &  &\end{array}\end{math} (6.9.6)

This problem is a conic quadratic optimization problem having one quadratic cone and the corresponding dual problem is

\begin{math}\nonumber{}\begin{array}{lccl}\nonumber{}\mbox{maximize} & b^{T}y+f^{T}\bar{y}+(l^{x})^{T}s_{l}^{x}-(u^{x})^{T}s_{u}^{x} &  & \\\nonumber{}\mbox{subject to} & A^{T}y+F^{T}\bar{y}+s_{l}^{x}-s_{u}^{x} & = & 0,\\\nonumber{} & -\bar{y}+s_{z} & = & 0,\\\nonumber{} & s_{t} & = & 1,\\\nonumber{} & \left\|s_{z}\right\|\leq{}s_{t}. &  & \\\nonumber{} & s_{l}^{x},s_{u}^{x}\geq{}0 &  &\end{array}\end{math} (6.9.7)

which can be reduced to

\begin{math}\nonumber{}\begin{array}{lccl}\nonumber{}\mbox{maximize} & b^{T}y+f^{T}s_{z}+(l^{x})^{T}s_{l}^{x}-(u^{x})^{T}s_{u}^{x} &  & \\\nonumber{}\mbox{subject to} & A^{T}y-F^{T}\bar{s}_{z}+s_{l}^{x}-s_{u}^{x} & = & 0,\\\nonumber{} & s_{t} & = & 1,\\\nonumber{} & \left\|s_{z}\right\|\leq{}s_{t}, &  & \\\nonumber{} & s_{l}^{x},s_{u}^{x}\geq{}0. &  &\end{array}\end{math} (6.9.8)

Quite frequently the dual problem has much fewer constraints than the primal problem. In such cases it will be more efficient to solve the dual problem and obtain the primal solution x from the dual solution of the dual.

6.10. Entropy optimization

6.10.1. Using mskenopt

An entropy optimization problem has the following form

\begin{math}\nonumber{}\begin{array}{lccccl}\nonumber{}\mbox{minimize} &  &  & \sum \limits _{{j=1}}^{n}d_{j}x_{j}\ln (x_{j})+c^{T}x &  & \\\nonumber{}\mbox{subject to} & l^{c} & \leq{} & Ax & \leq{} & u^{c},\\\nonumber{} &  &  & 0\leq{}x, &  &\end{array}\end{math} (6.10.1)

where all the components of d must be nonnegative i.e. [[MathCmd 93]]. One example of an entropy optimization problem is

\begin{math}\nonumber{}\begin{array}{lccccl}\nonumber{}\mbox{minimize} &  &  & x_{1}\ln (x_{1})-x_{1}+x_{2}\ln (x_{2}) &  & \\\nonumber{}\mbox{subject to} & 1 & \leq{} & x_{1}+x_{2} & \leq{} & 1,\\\nonumber{} &  &  & 0\leq{}x_{1},x_{2} &  &\end{array}\end{math} (6.10.2)

This problem can be solved using the mskenopt command as follows

d     = [1 1]';
c     = [-1 0]';
a     = [1 1];
blc   = 1;
buc   = 1;
[res] = mskenopt(d,c,a,blc,buc);
res.sol.itr.xx;

6.11. Geometric optimization

A so-called geometric optimization problem can be stated as follows

\begin{math}\nonumber{}\begin{array}{lccll}\nonumber{}\mbox{minimize} & \sum \limits _{{k\in{}J_{0}}}c_{k}\prod \limits _{{j=1}}^{n}t_{j}^{{a_{{kj}}}} &  &  & \\\nonumber{}\mbox{subject to} & \sum \limits _{{k\in{}J_{i}}}c_{k}\prod \limits _{{j=1}}^{n}t_{j}^{{a_{{kj}}}} & \leq{} & 1, & i=1,\ldots ,m,\\\nonumber{} & t>0, &  &  &\end{array}\end{math} (6.11.1)

where it is assumed that

\begin{displaymath}\nonumber{}\cup _{{k=0}}^{m}J_{k}=\lbrace{}1,\ldots ,T\rbrace{}\end{displaymath}

and if [[MathCmd 97]], then

\begin{displaymath}\nonumber{}J_{i}\cap J_{j}=\emptyset .\end{displaymath}

Hence, A is an [[MathCmd 99]] matrix and c is a vector of length t. In general the problem (6.11.1) is very hard to solve, but the posynomial case where

\begin{displaymath}\nonumber{}c>0\end{displaymath}

is relatively easy. The problem (6.11.1) is in general not a convex optimization problem, but using the variable transformation

\begin{math}\nonumber{}t_{j}=e^{{x_{j}}}\end{math} (6.11.2)

we obtain the problem

\begin{math}\nonumber{}\begin{array}{lccll}\nonumber{}\mbox{minimize} & \sum \limits _{{k\in{}J_{0}}}c_{k}e^{{a_{{k:}}x}} &  &  & \\\nonumber{}\mbox{subject to} & \sum \limits _{{k\in{}J_{i}}}c_{k}e^{{a_{{k:}}x}} & \leq{} & 1, & i=1,\ldots ,m,\end{array}\end{math} (6.11.3)

Now using that the [[MathCmd 103]] function is an increasing function we obtain an equivalent problem

\begin{math}\nonumber{}\begin{array}{lccll}\nonumber{}\mbox{minimize} & \log (\sum \limits _{{k\in{}J_{0}}}c_{k}e^{{a_{{k:}}x}}) &  &  & \\\nonumber{}\mbox{subject to} & \log (\sum \limits _{{k\in{}J_{i}}}c_{k}e^{{a_{{k:}}x}}) & \leq{} & \log (1), & i=1,\ldots ,m,\end{array}\end{math} (6.11.4)

which is a convex optimization problem. Hence, the problem (6.11.4) can be solved by MOSEK.

For further details about geometric optimization we refer the reader to [18, pp. 531-538].

6.11.1. Using mskgpopt

MOSEK cannot solve a geometric optimization problem directly. However, after it has been transformed to the form (6.11.4), then it can be solved using the MOSEK optimization toolbox function mskgpopt. Note that the solution to the transformed problem can easily be converted into a solution to the original geometric optimization problem using relation (6.11.2).

Subsequently, we will use the example

\begin{math}\nonumber{}\begin{array}{lccccl}\nonumber{}\mbox{minimize} & 40t_{1}^{{-1}}t_{2}^{{-1/2}}t_{3}^{{-1}}+20t_{1}t_{3}+40t_{1}t_{2}t_{3} &  & \\\nonumber{}\mbox{subject to} & \frac{1}{3}t_{1}^{{-2}}t_{2}^{{-2}}+\frac{4}{3}t_{2}^{{1/2}}t_{3}^{{-1}} & \leq{} & 1,\\\nonumber{} & 0<t_{1},t_{2},t_{3} &  &\end{array}\end{math} (6.11.5)

to demonstrate how a geometric optimization problem is solved using mskgpopt. Note that both the objective and the constraint functions consists of a sum of similar type of terms. These terms and where they belong can be specified completely using the matrix

\begin{displaymath}\nonumber{}A=\left[\begin{array}{ccc}\nonumber{}-1 & -0.5 & -1\\\nonumber{}1 & 0 & 1\\\nonumber{}1 & 1 & 1\\\nonumber{}-2 & -2 & 0\\\nonumber{}0 & 0.5 & -1\end{array}\right],\end{displaymath}

and the vectors

\begin{displaymath}\nonumber{}c=\left[\begin{array}{c}\nonumber{}40\\\nonumber{}20\\\nonumber{}40\\\nonumber{}\frac{1}{3}\end{array}\right]~\mbox{and}~map=\left[\begin{array}{c}\nonumber{}0\\\nonumber{}0\\\nonumber{}0\\\nonumber{}1\\\nonumber{}1\end{array}\right].\end{displaymath}

The interpretation is that each row of A and c describe one term e.g. the first row of A and the first element of c describe the first term in the objective function. The vector map tells whether a term belongs to the objective or a constraint. If [[MathCmd 108]] is equal to zero, then the kth term belongs to the objective function. Otherwise it belongs to the [[MathCmd 108]]th constraint.

The following MATLAB code demonstrate how the example is solved using mskgpopt.

% go1.m

c     = [40 20 40 1/3 4/3]';
a     = sparse([[-1  -0.5  -1];[1 0 1];...
                [1 1 1];[-2 -2 0];[0 0.5 -1]]);
map   = [0 0 0 1 1]';
[res] = mskgpopt(c,a,map);

fprintf('\nPrimal optimal solution to original gp:');
fprintf(' %e',exp(res.sol.itr.xx));
fprintf('\n\n');

% Compute the optimal objective value and
% the constraint activities.
v = c.*exp(a*res.sol.itr.xx);

% Add appropriate terms together.
f = sparse(map+1,1:5,ones(size(map)))*v;

% First objective value. Then constraint values.
fprintf('Objective value: %e\n',log(f(1)));
fprintf('Constraint values:');
fprintf(' %e',log(f(2:end)));
fprintf('\n\n');

% Dual multipliers (should be negative)
fprintf('Dual variables (should be negative):');
fprintf(' %e',res.sol.itr.y);
fprintf('\n\n');

The code also computes the objective value and the constraint values at the optimal solution. Moreover, the optimal dual Lagrange multipliers for the constraints are shown and the gradient of the Lagrange function at the optimal point is computed.

6.11.2. Comments

6.11.2.1. Solving large scale problems

If you want to solve a large problem i.e. a problem where A has large dimensions, then A has to be sparse. Otherwise you will run out of space. Recall a sparse matrix is a matrix that contains few non zero elements. If A is a sparse matrix, then you should construct it using the MATLAB function sparse as follows

A = sparse(subi,subj,valij);

where

\begin{displaymath}\nonumber{}a_{{subi[k],subj[k]}}=valij[k].\end{displaymath}

Please try

help sparse

inside MATLAB for more details about the sparse function.

6.11.2.2. Preprocessing tip

Before solving a geometric optimization problem then it is worthwhile to check if a column of the A matrix inputted to mskgpopt contains only positive elements. It is easy to verify that if this is the case then the corresponding variable [[MathCmd 111]] can take the value zero in the optimal solution. This may cause MOSEK problems and it is better to remove such variables from the problem.

6.12. Separable convex optimization

In this section we will discuss solution of nonlinear separable convex optimization problems. A general separable nonlinear optimization problem can be specified as follows:

\begin{math}\nonumber{}\begin{array}{lccccc}\nonumber{}\mbox{minimize} &  &  & f(x)+c^{T}x &  & \\\nonumber{}\mbox{subject to } &  &  & g(x)+Ax-x^{c} & = & 0,\\\nonumber{} & l^{c} & \leq{} & x^{c} & \leq{} & u^{c},\\\nonumber{} & l^{x} & \leq{} & x & \leq{} & u^{x},\end{array}\end{math} (6.12.1)

where

This implies that the ith constraint essentially has the form

\begin{displaymath}\nonumber{}l_{i}^{c}\leq{}g_{i}(x)+\sum \limits _{{j=1}}^{n}a_{{ij}}x_{j}\leq{}u_{i}^{c}\end{displaymath}

when the [[MathCmd 124]] variable has been eliminated.

The problem (6.12.1) must satisfy the three important requirements:

  1. Separability: This requirement implies that all nonlinear functions can be written on the form

    \begin{displaymath}\nonumber{}f(x)=\sum \limits _{{j=1}}^{n}f^{j}(x_{j})\end{displaymath}

    and

    \begin{displaymath}\nonumber{}g_{i}(x)=\sum \limits _{{j=1}}^{n}g_{i}^{j}(x_{j}).\end{displaymath}

    Hence, the nonlinear functions can be written as a sum of functions which only depends on one variable.

  2. Differentiability: All functions be twice differentiable for all [[MathCmd 48]] satisfying

    \begin{displaymath}\nonumber{}l_{j}^{x}<x<u_{j}^{x}\end{displaymath}

    if [[MathCmd 48]] occur in at least one nonlinear function. Hence, if [[MathCmd 130]] appears in the problem, then the lower bound on [[MathCmd 43]] should be 0.

  3. Convexity: The problem should be a convex optimization problem. See Section 9.5 for a discussion of this requirement.

6.12.1. Using mskscopt

Subsequently, we will use the following example

\begin{math}\nonumber{}\begin{array}{lccl}\nonumber{}\mbox{minimize} & x_{1}-\ln (x_{1}+2x_{2}) &  & \\\nonumber{}\mbox{subject to} & x_{1}^{2}+x_{2}^{2} & \leq{} & 1\end{array}\end{math} (6.12.2)

to demonstrate the solution of a convex separable optimization problem using the MOSEK optimization toolbox function mskscopt.

First observe the problem (6.12.2) is not a separable optimization problem due to the logarithmic term in objective is not a function of a single variable. However, by introducing one additional constraint and variable then the problem can be made separable as follows

\begin{math}\nonumber{}\begin{array}{lccl}\nonumber{}\mbox{minimize} & x_{1}-\ln (x_{3}) &  & \\\nonumber{}\mbox{subject to} & x_{1}^{2}+x_{2}^{2} & \leq{} & 1,\\\nonumber{} & x_{1}+2x_{2}-x_{3} & = & 0,\\\nonumber{} & x_{3}\geq{}0. &  &\end{array}\end{math} (6.12.3)

This problem is obviously separable and equivalent to the previous problem. Moreover, note all nonlinear functions are well defined for values of x satisfying the variable bounds strictly i.e.

\begin{displaymath}\nonumber{}x_{3}>0.\end{displaymath}

This makes it (almost) sure that function evaluations errors will not occur during the optimization process because MOSEK will only evaluate [[MathCmd 135]] for [[MathCmd 136]].

When using the mskscopt function to solve problem (6.12.3), then the linear part of the problem such as a c and A are specified as usual using MATLAB vectors and matrices. However, the nonlinear functions must be specified using five arrays which in the case of problem (6.12.3) can have the form

opr  = ['log'; 'pow'; 'pow'];
opri = [0;     1;     1    ];
oprj = [3;     1;     2    ];
oprf = [-1;    1;     1    ];
oprg = [0;     2;     2;   ];

Hence, opr(k,:) specify the type of a nonlinear function, opri(k) specify in which constraint the nonlinear function should be added to (zero means objective), and oprj(k) means the nonlinear function should be taken of variable [[MathCmd 48]]. Finally, oprf(k) and oprg(k) are parameters used by the mskscopt function according to the table:

opr(k,:) opri(k) oprj(k) oprf(k) oprg(k) function
ent i j f (not used) [[MathCmd 138]]
exp i j f g [[MathCmd 139]]
log i j f (not used) [[MathCmd 140]]
pow i j f g [[MathCmd 141]]

The i value indicates which constraint the nonlinear function belongs too. However, if i is identical to zero, then the function belongs to the objective. Using this notation a separable convex optimization problem can be solved with the function:

mskscopt(opr,
         opri,
         oprj,
         oprf,
         oprg,
         c,
         a,
         blc,
         buc,
         blx,
         bux)

All the elements for solving a nonlinear convex separable optimization problem has now been discussed and therefore we will conclude this section by showing the MATLAB code that will solve the example problem (6.12.3).

% sco1.m

% First the linear part of the problem
% is specified.

c           = [1;0;0];
a           = sparse([[0 0 0];[1 2 -1]]);
blc         = [-inf; 0];
buc         = [1;0];
blx         = [-inf;-inf;0];

% Then the nonlinear part.

opr         = ['log'; 'pow'; 'pow'];
opri        = [0;     1;     1    ];
oprj        = [3;     1;     2    ];
oprf        = [-1;    1;     1    ];
oprg        = [0;     2;     2    ];

% Finally, the optimizer is called. Note that bux is an optional
% parameter which should be added if the variables has an upper
% bound. 

[res]       = mskscopt(opr,opri,oprj,oprf,oprg,c,a,blc,buc,blx); 
                                                                 

% Then the solution is printed.
res.sol.itr.xx

6.13. Mixed integer optimization

Up until now it has been assumed that the variables in an optimization problem are continuous. Hence, it has been assumed that any value between the bounds of a variable is feasible. In many case this is not a valid assumption because some variables are integer constrained. For example a variable may denote number of persons assigned to a given job and it may not be possible to assign a fractional person.

MOSEK is capable of solving linear and quadratic optimization problems where one or more of the variables are integer constrained using a mixed integer optimizer

6.13.1. Solving an example

Using the example

\begin{math}\nonumber{}\begin{array}{lccl}\nonumber{}\mbox{minimize} & -2x_{1}-3x_{2} &  & \\\nonumber{}\mbox{subject to } & 195x_{1}+273x_{2} & \leq{} & 1365,\\\nonumber{} & 4x_{1}+40x_{2} & \leq{} & 140,\\\nonumber{} & x_{1}\leq{}4, &  & \\\nonumber{} & x_{1},x_{2}\geq{}0, &  & \mbox{ and integer}\end{array}\end{math} (6.13.1)

we will demonstrate how to solve an integer optimization problem using MOSEK.

% milo1.m

% First specify the linear problem data as if
% the problem is a linear optimization
% problem.

clear prob       
prob.c        = [-2 -3];
prob.a        = sparse([[195 273];[4 40]]);
prob.blc      = -[inf inf];
prob.buc      = [1365 140];
prob.blx      = [0 0];
prob.bux      = [4 inf];

% Specifies indexes of variables that are integer
% constrained.

prob.ints.sub = [1 2];

% Optimize the problem.
[r,res] = mosekopt('minimize',prob);

try 
  % Display the optimal solution.
  res.sol.int
  res.sol.int.xx'
catch
  fprintf('MSKERROR: Could not get solution')
end

Observe that compared to a linear optimization problem with no integer constrained variables then:

  • The field prob.ints.sub is used to specify the indexes of the variables that are integer constrained.
  • The optimal integer solution is returned in the MATLAB structure res.sol.int.

6.13.2. Speeding up the solution of a mixed integer problem

In general a mixed integer optimization problem can be very difficult to solve. Therefore, in some case it may be necessary to improve upon the problem formulation and “assist” the mixed integer optimizer.

How to obtain a good problem formulation is beyond the scope of this section and the reader is referred to [20]. However, two methods for assisting the mixed integer optimizer are discussed subsequently.

6.13.2.1. Specifying an initial feasible solution

In many cases a good feasible integer solution may be known to the optimization problem. If that is the case, then it is worthwhile to inform the mixed integer optimizer about it. The reason is that this reduces the space in which the optimizer has to look for an optimal solution.

Consider the problem:

\begin{math}\nonumber{}\begin{array}{lccl}\nonumber{}\mbox{maximize} & 7x_{0}+10x_{1}+x_{2}+5x_{3} &  & \\\nonumber{}\mbox{subject to} & x_{0}+x_{1}+x_{2}+x_{3} & \leq{} & 2.5\\\nonumber{} & x_{3}\geq{}0 &  & \\\nonumber{} & x_{0},x_{1},x_{2}\geq{}0 &  & \mbox{and integer},\end{array}\end{math} (6.13.2)

where only some of the variables are integer and the remaining are continuous. A feasible solution to this problem is:

\begin{math}\nonumber{}x_{0}=0,x_{1}=2,x_{2}=0,x_{3}=0.5\end{math} (6.13.3)

The following example demonstrate how to input this initial solution to MOSEK.

% milo2.m

clear prob
clear param
[r,res]         = mosekopt('symbcon');
sc              = res.symbcon;


prob.c          = [7 10 1 5];
prob.a          = sparse([1 1 1 1 ]);
prob.blc        = -[inf];
prob.buc        = [2.5];
prob.blx        = [0 0 0 0];
prob.bux        = [inf inf inf inf];
prob.ints.sub   = [1 2 3];

prob.sol.int.xx = [0 2 0 0.5]';

%Optionally also set status keys 
%prob.sol.int.skx = [sc.MSK_SK_SUPBAS;sc.MSK_SK_SUPBAS;...
%                    sc.MSK_SK_SUPBAS;sc.MSK_SK_BAS] 
%prob.sol.int.skc = [sc.MSK_SK_UPR]

[r,res] = mosekopt('maximize',prob);

try
  % Display the optimal solution.
  res.sol.int.xx'
catch
  fprintf('MSKERROR: Could not get solution')
end

It is also possible to specify only the values of the integer variables and then let MOSEK computes values for the remaining continuous variables such that a feasible solution is obtained. If the parameter MSK_IPAR_MIO_CONSTRUCT_SOL is set to MSK_ON then MOSEK try to compute a feasible solution from the specified values of the integer variables. MOSEK generates the feasible solution by temporarily fixing all integer variables to the specified values and then optimizing the resulting continuous linear optimization problem. Hence, using this feature it is only necessary to specify the values of prob.sol.int.xx corresponding to the integer constrained variables.

Suppose it is known that [[MathCmd 145]] are candidates for good integer values to our problem, then the following example demonstrates how to optimize the problem () using a feasible starting solution generated from the integer values as [[MathCmd 145]].

% milo3.m

[r,res]       = mosekopt('symbcon');
sc            = res.symbcon;

clear prob

prob.c        = [7 10 1 5];
prob.a        = sparse([1 1 1 1 ]);
prob.blc      = -[inf];
prob.buc      = [2.5];
prob.blx      = [0 0 0 0];
prob.bux      = [inf inf inf inf];
prob.ints.sub = [1 2 3];

% Values for the integer variables are specified.
prob.sol.int.xx  = [0 2 0 0]';

% Tell Mosek to construct a feasible solution from a given integer
% values. 
param.MSK_IPAR_MIO_CONSTRUCT_SOL = sc.MSK_ON;

[r,res] = mosekopt('maximize',prob,param);

try
  % Display the optimal solution.
  res.sol.int.xx'
catch
  fprintf('MSKERROR: Could not get solution')
end

6.13.2.2. Using branching priorities

The mixed integer optimized in MOSEK employs the so-called branch-and-bound algorithm to search for the optimal solution. See [20, pp. 91-112] for details about the branch-and-bound algorithm. The branch-and-bound algorithm can benefit from knowing about priorities of the integer variables.

For instance in an optimization model some integer variables may denote which factories to build and other variables which products to make in the factories. It seems natural to decide upon which factories to build first and then decide upon which products to make in which factories. Hence, some integer variables are more important than others.

In MOSEK it is possible to assign priorities to all the integer variables. The higher priority that is assigned to a variable the more important the variable is considered by the branch-and-bound algorithm. Priorities are specified using the field prob.ints.pri as follows:

prob.ints.sub = [4 1 2 3];  % Integer variables.
prob.ints.pri = [5 10 2 4]; % Priorities.

This implies that variable 4 has been assigned priority 5 and so forth.

An example of the usage of priorities can be seen in [20, pp. 232-235].

6.14. Sensitivity analysis

Given an optimization problem it is often useful to obtain information about how the optimal objective value change when a problem parameter is perturbed. For instance the objective function may reflect the price of a raw material such as oil which may not be known with certainty. Therefore, it might be interesting to know how the optimal objective value changes as the oil price change.

Analyzing how the optimal objective value changes when the problem data is changed is called sensitivity analysis.

Consider the problem:

minimize

\begin{math}\nonumber{}\begin{array}{ccccccccccccccl}\nonumber{}1x_{{11}} & + & 2x_{{12}} & + & 5x_{{23}} & + & 2x_{{24}} & + & 1x_{{31}} & + & 2x_{{33}} & + & 1x_{{34}}\end{array}\end{math} (6.14.1)

subject to

\begin{math}\nonumber{}\begin{array}{ccccccccccccccl}\nonumber{}x_{{11}} & + & x_{{12}} &  &  &  &  &  &  &  &  &  &  & \leq{} & 400,\\\nonumber{} &  &  &  & x_{{23}} & + & x_{{24}} &  &  &  &  &  &  & \leq{} & 1200,\\\nonumber{} &  &  &  &  &  &  &  & x_{{31}} & + & x_{{33}} & + & x_{{34}} & \leq{} & 1000,\\\nonumber{}x_{{11}} &  &  &  &  &  &  & + & x_{{31}} &  &  &  &  & = & 800,\\\nonumber{} &  & x_{{12}} &  &  &  &  &  &  &  &  &  &  & = & 100,\\\nonumber{} &  &  &  & x_{{23}} & + &  &  &  &  & x_{{33}} &  &  & = & 500,\\\nonumber{} &  &  &  &  &  & x_{{24}} & + &  &  &  &  & x_{{34}} & = & 500,\\\nonumber{}x_{{11}}, &  & x_{{12}}, &  & x_{{23}}, &  & x_{{24}}, &  & x_{{31}}, &  & x_{{33}}, &  & x_{{34}} & \geq{} & 0.\end{array}\end{math} (6.14.2)

The example below demonstrate how sensitivity analysis can answer questions of the type: “What happens to the optimal solution if we decrease the upper bound on the first constraint with 1”. For more information on sensitivity analysis see Chapter 13.

% sensitivity2.m

%setup problem data
clear prob
prob.a = sparse([1,    1,    0,    0,    0,    0,    0;
                 0,    0,    1,    1,    0,    0,    0;
                 0,    0,    0,    0,    1,    1,    1;
                 1,    0,    0,    0,    1,    0,    0;
                 0,    1,    0,    0,    0,    0,    0;
                 0,    0,    1,    0,    0,    1,    0;
                 0,    0,    0,    1,    0,    0,    1]);

prob.c =  [1,2,5,2,1,2,1];
prob.blc = [-Inf,-Inf,-Inf,800,100,500, 500];
prob.buc =[400,1200,1000,800,100,500,500];
prob.bux(1:7) = Inf; 
prob.blx(1:7) = 0;

% analyse upper bound on constraint 1 
prob.prisen.cons.subu = [1];  

[r,res] = mosekopt('minimize echo(0)',prob); 
fprintf ('Optimal objective value: %e\n',prob.c * res.sol.bas.xx  );
fprintf('Sensitivity results for constraint 1:');
res.prisen.cons
%If we change the upper bound on constraint 1 with a
%value v in [res.prisen.cons.lr_bu(1),res.prisen.cons.rr_bu(1)] 
%then the optimal objective changes with - v * ls_bu(0) 
% e,g changing prob.buc(1) with -1
prob.buc(1) =  prob.buc(1) - 1;
new_sol_predicted = prob.c * res.sol.bas.xx  + 1 * res.prisen.cons.ls_bu(1);
fprintf ('New optimal objective after changing bound predicted to:%e\n', ...
         new_sol_predicted);
[r,res] = mosekopt('minimize echo(0)',prob); 
fprintf ('New optimal objective value: %e\n',prob.c * res.sol.bas.xx  );

The output from running the example is given below:

Optimal objective value: 3.000000e+03
Sensitivity results for constraint 1:
ans =

    lr_bl: []
    rr_bl: []
    ls_bl: []
    rs_bl: []
    lr_bu: -300
    rr_bu: 0
    ls_bu: 3
    rs_bu: 3

New optimal objective after changing bound predicted to:3.003000e+03
New optimal objective value: 3.003000e+03

6.15. The solutions

Whenever an optimization problem is solved using MOSEK, then one or more optimal solutions are reported depending on which optimizer is used. These solutions are available in the structure

res.sol

which have one or more of the subfields

res.sol.itr  % Interior solution.
res.sol.bas  % Basis solution
res.sol.int  % Integer solution

The interior (point) solution is an arbitrary optimal solution which is computed using the interior-point optimizer. The basis solution is only available for linear problems and is produced by the simplex optimizer or the basis identification process which is an add-on to the interior-point optimizer. Finally, the integer solution is only available for problems having integer constrained variables and is computed using integer optimizer.

Each of three solutions may contain one or more of the following subfields:

.prosta

Problem status. See Section D.34.

.solsta

Solution status. See Section D.44.

.skc

Constraint status keys. See Section 6.1 below.

.skx

Variable status keys. See Section 6.1 below.

.xc

Constraint activities.

.xx

Variable activities.

.y

Identical to -.slc+.suc.

.slc

Dual variables corresponding to lower constraint bounds.

.suc

Dual variables corresponding to upper constraint bounds.

.slx

Dual variables corresponding to lower variable bounds.

.sux

Dual variables corresponding to upper variable bounds.

.snx

Dual variables corresponding to the conic constraints.

6.15.1. The constraint and variable status keys

In a solution both constraints and variables are assigned a status key which indicates whether the constraint or variable is at its lower limit, its upper limit, is super basic and so forth in the optimal solution. For interior-point solutions these status keys are only indicators which the optimizer produces.

In Table 6.1 the possible values for the status keys are shown accompanied with an interpretation of the key.

Symbolic Numeric String Interpretation
constant constant code
MSK_SK_UNK 0 UN Unknown status
MSK_SK_BAS 1 BS Is basic
MSK_SK_SUPBAS 2 SB Is superbasic
MSK_SK_LOW 3 LL Is at the lower limit (bound)
MSK_SK_UPR 4 UL Is at the upper limit (bound)
MSK_SK_FIX 5 EQ Lower limit is identical to upper limit
MSK_SK_INF 6 ** Is infeasible i.e. the lower limit is
  greater than the upper limit.
Table 6.1: Constraint and variable status keys.

By default the constraint and variable status keys are reported using string codes but it is easy to have MOSEK report the numeric codes instead. Indeed in the example

  % Status keys in string format
[rcode,res]=mosekopt('minimize statuskeys(0)',prob);
res.sol.skc(1)
res.sol.prosta 

the status keys are represented using string codes whereas in the example

% Status keys in string format
[rcode,res]=mosekopt('minimize statuskeys(1)',prob);
res.sol.skc(1)
res.sol.prosta 

the status keys are represented using numeric codes.

6.16. Viewing the task information

In MOSEK the optimization problem and the related instructions with respect to the optimization process is called an optimization task or for short a task. Whenever MOSEK performs operations on a task then it stores information in task information database. Examples of information that is stored is the number of interior-point iterations performed to solve the problem and time taken to do the optimization.

All the items stored in the task information database are listed in Sections D.15 and D.11. It is of course possible to see the whole or part of information task database within MATLAB.

% Solve a problem and obtain
% the task information database.
[r,res]=mosekopt('minimize info',prob);

% View one item
res.info.MSK_IINF_INTPNT_ITER

% View the whole database
res.info

6.17. Inspecting and setting parameters

A large number of parameters controls the behavior of MOSEK. For example there is a parameter controlling which optimizer is used, one that limits the maximum number of iterations allowed, and several parameters specifying the termination tolerance. All these parameters are stored in a database internally in MOSEK. The complete parameter database can be obtained and viewed using the commands:

[r,res]=mosekopt('param');
res.param 

We will not describe the purpose of each parameter here but instead refer the reader to Appendix C where all the parameters are presented in details.

In general it should not be necessary to change any of the parameters but if it is required, then it is easy to do so. In the subsequent example code it is demonstrated how to modify a few parameters and afterwards performing the optimization using these parameters.

% Obtain all symbolic constants
% defined by MOSEK.

[r,res]  = mosekopt('symbcon');
sc       = res.symbcon;

param    = [];

% Basis identification is unnecessary.
param.MSK_IPAR_INTPNT_BASIS   = sc.MSK_OFF;

% Alternatively you can use
%
%  param.MSK_IPAR_INTPNT_BASIS   = 'MSK_OFF';
%

% Use another termination tolerance.
param.MSK_DPAR_INTPNT_TOLRGAP = 1.0e-9;

% Perform optimization using the
% modified parameters.

[r,res] =  mosekopt('minimize',prob,param);

6.18. Advanced start (warmstart)

In practice it frequently occurs that when an optimization problem has been solved, then the same problem slightly modified should be reoptimized. Moreover, if the modification is only small, then it can be expected that the optimal solution to the original problem is a good approximation to the modified problem. Therefore, it should be efficient to start the optimization of the modified problem from the previous optimal solution.

Currently, the interior-point optimizer in MOSEK cannot take advantage of a previous optimal solution. Indeed this is important topic for research. However, the simplex optimizer can exploit any basic solution.

6.18.1. Some examples using warmstart

Using the example

\begin{math}\nonumber{}\begin{array}{lccccl}\nonumber{}\mbox{minimize} &  &  & x_{1}+2x_{2} &  & \\\nonumber{}\mbox{subject to} & 4 & \leq{} & x_{1}+x_{3} & \leq{} & 6,\\\nonumber{} & 1 & \leq{} & x_{1}+x_{2}, &  & \\\nonumber{} &  &  & 0\leq{}x_{1},x_{2},x_{3.} &  &\end{array}\end{math} (6.18.1)

the warmstart facility using the simplex optimizer will be demonstrated. A quick inspection of the problem indicates that [[MathCmd 150]] and [[MathCmd 151]] is an optimal solution. Hence, it seems to be a good idea to let the initial basis consists of [[MathCmd 152]] and [[MathCmd 153]] and the all the other variables be at their lower bound. This idea is used in the example code:

% advs1.m

clear prob param bas

% Specify an initial basic solution.
bas.skc      = ['LL';'LL'];
bas.skx      = ['BS';'LL';'BS'];
bas.xc       = [4 1]';
bas.xx       = [1 3 0]';

prob.sol.bas = bas;

% Specify the problem data.
prob.c       = [ 1 2 0]';
subi         = [1 2 2 1];
subj         = [1 1 2 3];
valij        = [1.0 1.0 1.0 1.0];
prob.a       = sparse(subi,subj,valij);
prob.blc     = [4.0 1.0]';
prob.buc     = [6.0 inf]';
prob.blx     = sparse(3,1);
prob.bux     = [];

% Use the primal simplex optimizer.
param.MSK_IPAR_OPTIMIZER = 'MSK_OPTIMIZER_PRIMAL_SIMPLEX';
[r,res] = mosekopt('minimize',prob,param)

Some comments are:

  • In the example the dual solution is defined. This is acceptable because the primal simplex optimizer is used for the re optimization and it does not exploit a dual solution. In the future MOSEK will also contain a dual simplex optimizer and if that optimizer is used, then it will be important that a “good” dual solution is specified.
  • The status keys bas.skc and bas.skx must only contain the entries BS, EQ, LL, UL, and SB. Moreover, for example EQ must only be specified for a fixed constraint or variable. LL and UL can only be used for a variable that has a finite lower and upper bound respectively.
  • The number of constraints and variables defined to be basic must correspond exactly to the number of constraints i.e. the row dimension of A.

6.18.2. Adding a new variable

Next assume that the problem

\begin{math}\nonumber{}\begin{array}{lccccl}\nonumber{}\mbox{minimize} &  &  & x_{1}+2x_{2}-x_{4} &  & \\\nonumber{}\mbox{subject to} & 4 & \leq{} & x_{1}+x_{3}+x_{4} & \leq{} & 6,\\\nonumber{} & 1 & \leq{} & x_{1}+x_{2}, &  & \\\nonumber{} &  &  & 0\leq{}x_{1},x_{2},x_{3},x_{4.} &  &\end{array}\end{math} (6.18.2)

should be solved which is identical to the problem (6.18.1) except a new variable [[MathCmd 37]] has been added. In continuation to the previous example this problem can be solved as follows (using warmstart):

% advs2.m. Continuation of advs1.m.

prob.c       = [prob.c;-1.0];
prob.a       = [prob.a,sparse([1.0 0.0]')];
prob.blx     = sparse(4,1);

% Reuse old optimal basis solution.
bas          = res.sol.bas;

% Add to status key.
bas.skx      = [res.sol.bas.skx;'LL'];

% New variable is at the lower limit
bas.xx       = [res.sol.bas.xx;0.0];
bas.slx      = [res.sol.bas.slx;0.0];
bas.sux      = [res.sol.bas.sux;0.0];

prob.sol.bas = bas;

[rcode,res]  = mosekopt('minimize',prob,param);

% New primal optimal solution
res.sol.bas.xx'

6.18.3. Fixing a variable

In for example branch and bound methods for integer programming problems it is necessary to reoptimize the problem after a variable has been fixed to a value. This can easily be achieved as follows:

% advs3.m. Continuation of advs2.m.

prob.blx(4)  = 1;
prob.bux     = [inf inf inf 1]';

% Reuse the basis.
prob.sol.bas = res.sol.bas;

[rcode,res]  = mosekopt('minimize',prob,param);

% Display the optimal solution.
res.sol.bas.xx'

The variable [[MathCmd 37]] is simply fixed at the value 1 and the problem is re optimized. Note the basis from the previous optimization can immediately be reused.

6.18.4. Adding a new constraint

Now assume that the constraint

\begin{math}\nonumber{}x_{1}+x_{2}\geq{}2\end{math} (6.18.3)

should be added to the problem and the problem should be reoptimized. The following example demonstrates how to do this.

% advs4.m. A continuation of advs3.m. 

% Modify the problem.
prob.a       = [prob.a;sparse([1.0 1.0 0.0 0.0])];
prob.blc     = [prob.blc;2.0];
prob.buc     = [prob.buc;inf];

% Obtain the previous optimal basis.
bas          = res.sol.bas;

% Setting of the solution to modified problem.
bas.skc      = [bas.skc;'BS'];
bas.xc       = [bas.xc;bas.xx(1)+bas.xx(2)];
bas.y        = [bas.y;0.0];
bas.slc      = [bas.slc;0.0];
bas.suc      = [bas.suc;0.0];

% Reuse the basis.
prob.sol.bas = bas;

% Reoptimize.
[rcode,res]  = mosekopt('minimize',prob,param);

res.sol.bas.xx' 

Note the slack variable corresponding to the new constraint are declared basic. This implies that the new basis is nonsingular and can be reused.

6.18.5. Using numeric values to represent status key codes

In the previous examples the constraint and variable status keys are represented using string codes. Although the status keys are easy to read then they are sometimes difficult to work with in a program. Therefore, the status keys can also be represented using numeric values as demonstrated in the example:

% sk1.m

% Obtain all symbolic constants
% defined in MOSEK.

clear prob bas;

[r,res]  = mosekopt('symbcon');
sc       = res.symbcon;

% Specify an initial basic solution.
% Note symbolic constants are used.
% I.e. sc.MSK_SK_LOW instead of 4.
bas.skc      = [sc.MSK_SK_LOW;sc.MSK_SK_LOW];
bas.skx      = [sc.MSK_SK_BAS;sc.MSK_SK_LOW;sc.MSK_SK_BAS];
bas.xc       = [4 1]';
bas.xx       = [1 3 0]';
prob.sol.bas = bas;

% Specify the problem data.
prob.c   = [ 1 2 0]';
subi     = [1 2 2 1];
subj     = [1 1 2 3];
valij    = [1.0 1.0 1.0 1.0];
prob.a   = sparse(subi,subj,valij);
prob.blc = [4.0 1.0]';
prob.buc = [6.0 inf]';
prob.blx = sparse(3,1);
prob.bux = [];

% Use the primal simplex optimizer.
clear param;
param.MSK_IPAR_OPTIMIZER = sc.MSK_OPTIMIZER_PRIMAL_SIMPLEX;

[r,res] = mosekopt('minimize statuskeys(1)',prob,param)

% Status keys will be numeric now i.e.

res.sol.bas.skc'

% is vector of numeric values.

Note using the command

[r,res]  = mosekopt('symbcon');
sc       = res.symbcon;

then all the symbolic constants defined within MOSEK are obtained and those constants are then used in the lines

bas.skc  = [sc.MSK_SK_LOW;sc.MSK_SK_LOW];
bas.skx  = [sc.MSK_SK_BAS;sc.MSK_SK_LOW;sc.MSK_SK_BAS]; 

These two lines are in fact equivalent to

bas.skc  = [1;1];
bas.skx  = [3;1;3];

However, it is not recommended to specify the constraint and variable status keys this way because it is less readable and portable. Indeed if for example MOSEK later change the definition that 1 is equivalent `LL', then all programs using numerical keys are incorrect whereas using the symbolic constants the programs remain correct.

6.19. Using names

In MOSEK it possible to give the objective, each constraint, each variable, and each cone a name. In general there is not much use for such names except in connection with reading and writing MPS files. See Section 6.20 for details.

All the names are specified in the prob.names structure.

% The problem is given a name.
prob.names.name   = 'CQO example';

% Objective name.
prob.names.obj    = 'cost';

% The two constraints are given a name.
prob.names.con{1} = 'constraint_1';
prob.names.con{2} = 'constraint_2';

% The six variables are given a name.
prob.names.var    = cell(6,1);
for j=1:6
  prob.names.var{j} = sprintf('x%d',j);
end

% Finally the two cones are given a name.
prob.names.cone{1} = 'cone_a';
prob.names.cone{2} = 'cone_b';

6.19.1. Blanks in names

Although legal then is strongly advised not to use blanks in names except for the problem name. For instance 'x 1' should be avoided if possible.

6.20. MPS files

An industry standard format for storing linear optimization problems in a ASCII file is the so-called MPS format. For readers not familiar with the MPS format then a specification of the MPS format supported by MOSEK can be seen in Appendix A.

The advantage of the MPS format is that problems stored in this format can be read by any commercial optimization software, so it facilitates easy communication of optimization problems.

6.20.1. Reading a MPS file

It is possible to use mosekopt to read a MPS file containing the problem data. In that case mosekopt reads data from a MPS file and returns both the problem data and the optimal solution if required. Assume afiro.mps is the MPS file that mosekopt should read the problem data from, then this task is performed using the command

[r,res] = mosekopt('read(afiro.mps'));

In this case res.prob will contain several fields which contains the problem data as read from the MPS file. For example the MATLAB command

res.prob.c'

will display c on the screen.

The names used in the MPS file is also available in the prob.names structure.

% All names.
prob.names

% Constraint names.
prob.names.con

It is of course also possible to read problems having quadratic terms in the objective function or the constraints. The following set of MATLAB commands demonstrates how to read such a problem and viewing the data.

% mpsrd.m

% Read data from the file wp12-20.mps.

[r,res] = mosekopt('read(wp12-20.mps)');

% Looking at the problem data
prob = res.prob;
clear res;

% Form the quadratic term in the objective.
q = sparse(prob.qosubi,prob.qosubj,prob.qoval);

% Get a graphical picture.
spy(q) % Notice only the lower triangular part is defined.

6.20.2. Writing a MPS files

It is also possible to write an MPS file using MOSEK. Indeed assume that a problem defined by the MATLAB structure prob should be written to a MPS file. This happens in the example:

% Write the data defined by prob to an MPS file
% named datafile.mps
mosekopt('write(datafile.mps)',prob);

If the field prob.names is defined, then MOSEK will use those names when writing the MPS file. Otherwise MOSEK will use generic names.

6.21. User call-back functions

A call-back function is a user defined MATLAB function, to be called by MOSEK on a given event. The optimization toolbox supports two types of call-back functions which are presented below.

6.21.1. Controlling log printing via call-back

When using mosekopt it is possible to control the amount of information that mosekopt prints to the screen. For instance the command

[r,res] = mosekopt('minimize echo(0)',prob)

forces mosekopt to print no log information because echo(0) has been added to the command string. A high number in the commandecho(n) i.e. for instance echo(3) forces MOSEK prints more log information to the screen.

It is possible to redirect the MOSEK log printing almost anywhere using a user defined log call-back function. It works as follows. First create an m-file with a function looking like

function myprint(handle,str)
% handle: Is user defined data structure
% str   : Is a log string.
%

fprintf(handle,'%s',str); 

It is not important what the function is called and it is not important what the function does. However, it is important it accepts two input arguments. The first argument is handle which is user defined MATLAB structure and the second argument is str which is a line of MOSEK log. (Note myprint prints the log line to the screen and to a file.)

The following code fragment shows how to inform MOSEK about the function myprint.

%
% In this example the MOSEK log info
% should be printed to the screen and to a file named
% mosek.log.
%

fid                = fopen('mosek.log','wt');
callback.log       = 'myprint';
callback.loghandle = fid;

%
% The argument handle in myprint() will be identical to
% callback.loghandle when called.
%

mosekopt('minimize',prob,[],callback);

6.21.2. The iteration call-back function

It is possible to specify an iteration callback function to be called frequently during the optimization. A typically use for this call-back function is to displays information about the optimization process or to terminate it.

The iteration call-back function takes the following form:

function [r] =  myiter(handle,where,info)
% handle: Is a user defined data structure
% where : Is an integer indicating from where in the optimization
%         process the callback was invoked .
% info  : A MATLAB structure containing information about the state of the
%         optimization.

r = 0;   % r should always be assigned a value.

if handle.symbcon.MSK_CALLBACK_BEGIN_INTPNT==where
    fprintf('Interior point optimizer started\n');
end

if handle.symbcon. MSK_CALLBACK_INTPNT==where
    % print primal objective
    fprintf('Interior-point primal obj.: %e\n',info.MSK_DINF_INTPNT_PRIMAL_OBJ);

    % terminate when cputime > handle.maxtime
    if info.MSK_DINF_INTPNT_CPUTIME > handle.maxtime
      r = 1;
    else
      r = 0;
    end
end     

if handle.symbcon. MSK_CALLBACK_END_INTPNT==where
    fprintf('Interior point optimizer terminated\n');
end

The function takes three arguments. The first argument handle is a user defined MATLAB structure, the second argument where indicates from where in the optimization process the callback was invoked and the third argument info is a structure containing information about the process. For details about info see Section 7.1.7. If the functions return argument which is nonzero, then optimization process is terminated immediately.

In order to inform MOSEK about the iteration call-back function the fields iter and iterhandle are initialized as shown in the following example.

[r,res]             = mosekopt('symbcon');
data.maxtime        = 100.0;
data.symbcon        = res.symbcon;

callback.iter       = 'myiter';
callback.iterhandle = data;

mosekopt('minimize',prob,[],callback);
Mon Sep 14 15:56:07 2009