Getting started
Assuming you have the SHOT executable (SHOT.exe on Windows) installed on your system, you can call SHOT with the following syntax:
./SHOT problemfile.[nl,osil,xml,gms]
If you need instructions the flag --help shows SHOT's syntax.
The valid options for SHOT are listed here.
Settings for SHOT can be specified either through an options file (either as a text file or as an OSoL-file in XML syntax).
To generate an example options file pass the flag --opt which will create the file options.opt in the current directory. To use an option file pass the parameter --opt options.opt.
To create an empty OSoL-file add the flag --osol, which creates the file options.xml. To use this file pass the parameter --osol options.xml.
It is also possible to pass options through the command line using the following syntax
./SHOT problemfile Termination.TimeLimit=100.0 Termination.ObjectiveGap.Absolute=0.1
Note that the options are case sensitive and that there are no spacing between option names, =, and value.
There are also a number of short hand options available.
Flag | Description |
| Assumes that the problem is convex. |
| Activates the debug mode (see below). Note that if no directory is specified, a named option from the list above cannot be used directly after |
| Sets the MIP solver to either Cbc, CPLEX or Gurobi. |
| Sets the NLP solver to use. If GAMS is specified, the default NLP solver in GAMS will be used; it is possible to change what GAMS NLP solver to use with the setting |
| Activates the single- or multi-tree strategy. |
| Sets the maximum number of threads to use in the MIP solver. |
| Sets the absolute objective gap tolerance, for example, e.g, |
| Sets the relative objective gap tolerance. |
| Sets the time-limit in seconds. |
After a successful termination, SHOT will create an result file <problemname>.osrl in the current directory. This file is in the Optimization Services OSrL syntax, which is an XML-based format. It is also possible to specify where the result file should be written by adding the parameter --osrl path/to/resultfile.osrl as an argument when calling SHOT.
It is also possible to create a GAMS trace file in a similar way by adding the --trc <filename.trc> flag. If no filename is specified, a filename consisting of the problem file and the suffix .trc is created in the current directory.
To create a file in AMPL sol-format add the parameter --sol <filename.sol>.
The level of output that SHOT writes to the console can be controlled with the setting Output.Console.LogLevel , where values between 0 and 6 are allowed. Assuming that the default value 2 is used, first some basic information about the solver, including the current version is provided:
╶ Supporting Hyperplane Optimization Toolkit (SHOT) ──────────────────────────────────────────────────────────────────╴Andreas Lundell and Jan Kronqvist, Åbo Akademi University, Finland.See documentation for full list of contributors and utilized software libraries.Version: 1.0.alpha 1. Git hash: 6e43619-dirty. Released Apr 11 2020.For more information visit https://shotsolver.dev
Then information about the modeling system or file format used is given as well as information about possible bound tightening performed on the variable bounds.
╶ Modeling system ────────────────────────────────────────────────────────────────────────────────────────────────────╴Modeling system: GAMSProblem read from file: /home/andreas/Research/minlplib/21122018/minlplib/gms/tls4.gmsPerforming bound tightening on original problem.- Bounds for 18 variables tightened in 0.06 s.Performing bound tightening on reformulated problem.- Bounds for 34 variables tightened in 0.08 s.
Then information about the provided problem instance is shown. SHOT automatically reformulates the problem into a form that is more suitable for the underlying algorithm, and statistics about the reformulated problem is given as well.
╶ Problem instance ───────────────────────────────────────────────────────────────────────────────────────────────────╴Original ReformulatedProblem classification: MINLP, convex MINLP, convexObjective function direction: minimize minimizeObjective function type: linear linearNumber of constraints: 64 80- linear: 60 64- convex nonlinear: 4 16Number of variables: 105 121- real: 16 32- binary: 85 85- integer: 4 4- nonlinear: 20 20Number of transformations performed: 16- signomial terms partitioning: 16
After this, the (nondefault) settings used are shown; here the MIP solver utilized (CPLEX) considered all variable bounds over +/- 1e20 to be infinite, so this setting will be used by SHOT as well.
Below the used settings, the names and versions of the utilized subsolvers are shown.
╶ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────╴No options file specified.Nondefault options used:- Model.Variables.Continuous.MaximumUpperBound = 1e+20- Model.Variables.Continuous.MinimumLowerBound = -1e+20Dual strategy: Single-tree- cut algorithm: ESH- solver: CPLEX 12.10Primal NLP solver: Ipopt 3.12 (with default linear solver)
Now, since the ESH algorithm was used for generating linearizations of nonlinear expressions in SHOT, an interior point is required, which will be found by utilizing a minimax solver implemented by solving linear programming (LP) problems.
╶ Interior point search ──────────────────────────────────────────────────────────────────────────────────────────────╴Strategy selected: cutting plane minimaxIteration │ Time │ Cuts │ Objective value │ Objective diff.#: type │ tot. │ + | tot. │ problem | line srch │ abs. | rel.╶─────────────────┴────────┴─────────────┴─────────────────────────┴──────────────────╴1: LP 0.38 -1e+12 | -1e+12 inf. | inf.2: LP 0.38 4 | 4 -43.5 | 68.8668 1.1e+02 | 2.6e+003: LP 0.40 4 | 8 -37.462 | 59.892 9.7e+01 | 2.6e+004: LP 0.40 4 | 12 -32.6284 | 46 7.9e+01 | 2.4e+005: LP 0.40 4 | 16 -24.1783 | 25.641 5.0e+01 | 2.1e+006: LP 0.40 4 | 20 -12.5657 | 16.3322 2.9e+01 | 2.3e+007: LP 0.40 4 | 24 -11.5673 | 8.10095 2.0e+01 | 1.7e+008: LP 0.41 4 | 28 -9.36657 | 6.87839 1.6e+01 | 1.7e+009: LP 0.41 4 | 32 -8.18222 | 5.50341 1.4e+01 | 1.7e+0010: LP 0.41 4 | 36 -8.10926 | 3.05517 1.1e+01 | 1.4e+0011: LP 0.42 4 | 40 -8.0093 | 1.69059 9.7e+00 | 1.2e+0012: LP 0.42 4 | 44 -7.96557 | 0.652575 8.6e+00 | 1.1e+0013: LP 0.42 4 | 48 -7.96557 | -1.97056 6.0e+00 | 7.5e-01Valid interior point with constraint deviation -1.971 found.
The
Iterationcolumn shows the current iteration number, as well as the problem type solved (LP=linear programming).The
Timecolumn shows the wall clock time in seconds.In the
Cutscolumn, the number of linear constraints added in each iteration, as well as the total number of added constraints, are shown.In the
Objective valuecolumn, there are two values, the first corresponding to the solution of the LP problem and the other of the minimal value on the line between the previous solution point and the current.The last column gives the absolute and relative difference between the two objective values in the previous column.
So, in the example above, 13 LP problems were solved and a total of 48 linear constraints were generated to find an integer-relaxed interior point to the reformulated MINLP problem. Now we can continue to solve the actual problem.
Iteration │ Time │ Dual cuts │ Objective value │ Objective gap │ Current solution#: type │ tot. │ + | tot. │ dual | primal │ abs. | rel. │ obj.fn. | max.err.╶─────────────────┴────────┴─────────────┴─────────────────────────┴───────────────────┴───────────────────────────╴1: LP-O 0.43 0 | inf. inf. | 1.0e+00 0 | 65: 8.76e+016: LP-O 0.48 8 | 40 1.04491 | inf. inf. | 1.0e+00 1.04491 | 71: 3.04e+007: LP-O 0.48 7 | 47 1.55977 | inf. inf. | 1.0e+00 1.55977 | 73: 1.50e+008: LP-O 0.49 8 | 55 1.64102 | inf. inf. | 1.0e+00 1.64102 | 68: 1.49e+009: LP-O 0.49 2 | 57 1.64102 | inf. inf. | 1.0e+00 1.64102 | 68: 4.06e-0110: CB (th: 0) 0.60 3.67836 | inf. inf. | 1.0e+00 7.8 | 72: 6.17e+0012: CB (th: 0) 0.67 2 | 71 3.99122 | inf. inf. | 1.0e+00 8.8 | 78: 1.60e+0018: CB (th: 0) 0.69 4 | 110 3.99122 | 10.8 6.8e+00 | 6.3e-01 10.8 | 75: 1.97e-0719: CB (th: 0) 0.69 3 | 113 3.99122 | 10.8 6.8e+00 | 6.3e-01 10.8 | 75: 1.97e-0720: CB (th: 0) 0.72 4.01615 | 10.8 6.8e+00 | 6.3e-01 9 | 65: 7.82e-012: NLPSOLPT-O 0.75 4.01615 | 9 5.0e+00 | 5.5e-01 9 | 63: 1.37e-1026: CB (th: 0) 0.77 2 | 143 4.02143 | 9 5.0e+00 | 5.5e-01 8.8 | 66: 8.83e-0127: CB (th: 0) 0.80 4 | 147 4.31494 | 9 4.7e+00 | 5.2e-01 7.9 | 65: 1.02e+0131: CB (th: 0) 0.85 2 | 153 4.44449 | 9 4.6e+00 | 5.1e-01 8.5 | 70: 1.96e+003: NLPSOLPT-O 0.88 4.44449 | 8.5 4.1e+00 | 4.8e-01 8.5 | 60: -1.16e-1039: CB (th: 0) 1.19 5.28294 | 8.5 3.2e+00 | 3.8e-01 7.7 | 67: 7.65e+0040: CB (th: 0) 1.25 1 | 187 5.3468 | 8.5 3.2e+00 | 3.7e-01 8.4 | 66: 3.25e+0046: CB (th: 0) 1.41 1 | 206 5.49899 | 8.4 2.9e+00 | 3.5e-01 7.7 | 72: 2.93e+0047: CB (th: 0) 2.50 2 | 208 6.60651 | 8.4 1.8e+00 | 2.1e-01 8.3 | 67: 4.75e+0048: CB (th: 0) 2.68 2 | 210 6.71446 | 8.4 1.7e+00 | 2.0e-01 8.3 | 72: 8.20e-0153: CB (th: 0) 2.68 3 | 222 6.71446 | 8.3 1.6e+00 | 1.9e-01 8.3 | 70: 2.27e-0653: MIP-O 3.16 8.3 | 8.3 1.8e-15 | 2.1e-16 8.3 | 70: 2.27e-06
The first thing to notice is that not all iterations are written to console, but this condensed output can be deactivated with the switch Output.Console.Iteration.Detail=0.
The columns are similar to those of the minimax solver mentioned above.
The
Iterationcolumn shows the iteration number and type. The type identifier consists of a part showing the problem type, and a part showing the solution status of the iteration.The following types are available for the linearized problem solved in SHOT's dual strategy:
LP: linear programming problem.
QP: quadratic programming problem.
QCQP: quadratically constrained quadratic programming problem.
MILP: mixed-integer linear programming problem.
MIQP: mixed-integer quadratic program.
MIQCQP: mixed-integer quadratically constrained quadratic program.
CB: New incumbent solution candidate found in callback by MIP solver. For some solvers, also the thread number of the callback is shown.
The following types are available for the primal heuristic in SHOT:
NLPSOLPT: An NLP problem has been solved with fixed integer values originating from a MIP solution that is classified as optimal (or found in a callback).
NLPNEWDB: An NLP problem has been solved with fixed integer values originating from a MIP solution that has improved the lower bound of a minimization problem (or upper bound of a maximization problem).
The following statuses are appended after the dash:
O: Solved to optimality.
F: Solved to feasibility.
I: Infeasible problem detected.
U: Unbounded problem detected.
TL: Time limit reached.
IL: Iteration limit reached.
SL: Solution limit reached.
The
Timecolumn shows the wall clock time in seconds.In the
Cutscolumn, the number of linear constraints added in each iteration, as well as the total number of added constraints, are shown.In the
Objective value - dualcolumn, the current lower bound (if minimizing) on the objective value is shown. Unless an asterisk*is added to the value, this is a global lower bound. This value is provided by the underlying MIP solver.In the
Objective value - primalcolumn, the current upper bound (if minimizing) on the objective value is shown. This value is the current best know solution to the problem and is given by either the MIP solver's solution pool or a primal heuristic, such as solving integer-fixed NLP problems.The
objective gapcolumn shows the absolute and relative gap between the lower and upper bounds in the previous columns.The final column
Current solutionprovides some insight into the objective value and constraint fulfillment of the current iteration (either from the MIP or NLP solver).In the
max.errcolumn, the nonlinear constraint index with the maximal error in the solution point is shown in addition to the error.
Finally, after terminating the iterative solution procedure, SHOT shows a final report that explains the status of the solution, and why the solver terminated.
╶ Solution report ────────────────────────────────────────────────────────────────────────────────────────────────────╴Terminated since absolute gap met requirements.Globally optimal primal solution found.Objective bound (minimization) [dual, primal]: [8.3, 8.3]Objective gap absolute / relative: 1.77636e-15 / 2.14019e-16Fulfilled termination criteria:- absolute objective gap tolerance 1.77636e-15 <= 0.001- relative objective gap tolerance 2.14019e-16 <= 0.001Unfulfilled termination criteria:- iteration limit 53 <= 200000- solution time limit (s) 3.16338 <= 1.79769e+308Dual problems solved in main step: 10- LP problems 9- MILP problems, optimal 1Number of explored nodes: 4988Problems solved during interior point search:- LP problems: 13Fixed primal NLP problems solved: 4
In the case above, the single-tree strategy was used, and the linearizations were added during the solution of only one MILP problem, which was solved to optimality.
After this some insight into the number of primal solutions, i.e., solutions to the original MINLP problem, found, and which method was used to find them:
Number of primal solutions found: 10- root search: 5- NLP problem with fixed integers: 2- MILP solution pool: 1- lazy constraint callback: 2
A report over how much time the individual components took is also provided:
Total solution time: 3.16344- problem reformulation: 0.0875337- bound tightening: 0.138686- feasibility based (original problem): 0.0603996- feasibility based (reformulated problem): 0.0776433- interior point search: 0.110094- solving relaxed problems: 0.0685161- dual strategy: 2.76156- solving MIP problems: 2.67002- root search for constraint cuts: 0.0667068- primal strategy: 0.19353- solving NLP problems: 0.155881- performing root searches: 0.00566103
Finally, SHOT shows where the log file and solution file(s) have been written:
╶─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╴Results written to: /home/andreas/Research/SHOT/COIN-OR/shot/build/clang/tls4.osrlLog written to: /home/andreas/Research/SHOT/COIN-OR/shot/build/clang/SHOT.log
By default SHOT creates a log file SHOT.log in the current directory containing the same type of output as is written to the console. To specify the location of this file use the parameter --log <filename.log>. It is possible to control the verbosity of the log file with the parameter Output.File.LogLevel setting.
SHOT does not normally create any auxiliary files during run-time so it might be difficult to debug what is happening, e.g. in each dual iteration. There is however a setting Output.Debug flag that, if activated, creates a folder SHOT_debug_***** in the system temp folder. The destination folder can be specified with the parameter Output.Debug.Path. or with--debug=<directory>, where directory is where the debug files should be saved.
For example, there is the file originalproblem.txt and reformulatedproblem.txt which shows how SHOT interprets the given problem and how it is reformulated by SHOT. Also, the intermediate linearized problems solved by the MIP solver in each iteration is provided as LP*.lp.
To use SHOT in Python, you need to have a SHOT executable installed somewhere on your system.
The location can either be somewhere in your path (so that you can call SHOT from a terminal in any directory), or you will need to specify where the SHOT executable is.
Assuming SHOT is in your path, you can solve problems with the following command:
pyomo problemfile.py --solver SHOT
Otherwise you will need to specify where the SHOT executable is with the command
pyomo problemfile.py --solver "path/to/SHOT[.exe]"
The solution will be written to a file results.yml in the current directory. You can also write the solution to the screen by adding the flag --show-result.
You can specify any of SHOT's options (as described here) by adding the --solver-optionsflag. For example: --solver-options="Termination.TimeLimit=100.0" will set a time limit of 100 seconds.
If you want to specify multiple settings, separate them with a space, e.g. --solver-options="Termination.TimeLimit=100.0 Termination.ObjectiveGap.Relative=0.01".
Note that some given parameters may not always be considered; the used settings are indicated at the top of the log file SHOT.log.
If you have a GAMS version with built in support of SHOT, you can simply specify <problem-class> = SHOT (e.g. MINLP=SHOT), either in the model itself or in the command line.
To use your own version of SHOT with GAMS, you can modify the gamsconfig.yaml file. See the GAMS documentation for instructions.
In short, you need to create a file called gamsconfig.yaml in either the GAMS system directory or one of the other directories specified here.
To make available a solver named SHOT2 in GAMS, which uses the SHOT library from D:\Users\janeDoe\Downloads\gamslink\GamsShot-20.dll, add the following contents to the yaml file:
solverConfig:- SHOT2:minVersion: 33fileType: 1001dictType: 5licCodes: 000102030405scriptName: gmsgennt.cmdexecutableName: gmsgennx.exelibrary:libName: D:\Users\janeDoe\Downloads\gamslink\GamsShot-20.dllauditCode: shtsolverInterfaceType: 1threadSafeIndic: TruemodelTypes:- MINLP- MIQCP
To make available a solver named SHOT2 in GAMS, which uses the SHOT library from /path/to/libSHOTSolver.so , add the following contents to the yaml file:
solverConfig:- SHOT2:minVersion: 33fileType: 1001dictType: 5licCodes: 000102030405scriptName: gmsgenus.runexecutableName: gmsgenux.outlibrary:libName: /path/to/libSHOTSolver.soauditCode: shtsolverInterfaceType: 1threadSafeIndic: TruemodelTypes:- MINLP- MIQCP
The settings are explained in detail here.
It is possible to get SHOT working together with the Julia-based optimization modeling system JuMP using its AmplNLWriter.jl subproject. For more information see the Github repository.
In short, assuming you have everything set up and a version of SHOT installed and working on your system you can use the following command.
julia> using JuMP, AmplNLWriterjulia> m = Model(() -> AmplNLWriter.Optimizer("SHOT"))
This assumes that the SHOT executable is found in your path, otherwise you will need to specify the full path to where the executable is, i.e.
julia> using JuMP, AmplNLWriterjulia> m = Model(() -> AmplNLWriter.Optimizer("/path/to/SHOT"))
If you have GAMS installed you can also use SHOT from JuMP and Pyomo by using the GAMS interface described in this blog post.
SHOT should be compatible with any modeling system using ASL (models in nl and solution in sol formats). In practice this works so that the following syntax is used:
./SHOT path/to/problemfile.nl --AMPL
Then SHOT solves the problem given in the nl-file and writes a solution file problemfile.sol. This file is written in the same directory as the nl-file.
