Getting started

Running SHOT from the command line

Assuming you have the SHOT executable (SHOT.exe on Windows) installed on your system, you can call SHOT with the following syntax:

./SHOT problemfile.[nl,osil,xml,gms]

If you need instructions the flag --help shows SHOT's syntax.

Options

The valid options for SHOT are listed here.

Pass options using a file

Settings for SHOT can be specified either through an options file (either as a text file or as an OSoL-file in XML syntax).

To generate an example options file pass the flag --opt which will create the file options.opt in the current directory. To use an option file pass the parameter --opt options.opt.

To create an empty OSoL-file add the flag --osol, which creates the file options.xml. To use this file pass the parameter --osol options.xml.

Pass options using the command line

It is also possible to pass options through the command line using the following syntax

./SHOT problemfile Termination.TimeLimit=100.0 Termination.ObjectiveGap.Absolute=0.1

Note that the options are case sensitive and that there are no spacing between option names, =, and value.

There are also a number of short hand options available.

Flag

Description

--convex

Assumes that the problem is convex.

--debug or --debug=<directory>

Activates the debug mode (see below). Note that if no directory is specified, a named option from the list above cannot be used directly after --debugas it will be regarded as the debug directory name. If no directory is specified a directory will be created in the system temp folder.

--mip=<cbc|cplex|gurobi>

Sets the MIP solver to either Cbc, CPLEX or Gurobi.

--nlp=<ipopt|gams>

Sets the NLP solver to use. If GAMS is specified, the default NLP solver in GAMS will be used; it is possible to change what GAMS NLP solver to use with the setting Subsolver.GAMS.NLP.Solver.

--tree=<single|multi>

Activates the single- or multi-tree strategy.

--threads=<int>

Sets the maximum number of threads to use in the MIP solver.

--absgap=<float

Sets the absolute objective gap tolerance, for example, e.g, 1e-3.

--relgap=<float>

Sets the relative objective gap tolerance.

--timelimit=<float>

Sets the time-limit in seconds.

Results

OSrL (Optimization Services result Language) file

After a successful termination, SHOT will create an result file <problemname>.osrl in the current directory. This file is in the Optimization Services OSrL syntax, which is an XML-based format. It is also possible to specify where the result file should be written by adding the parameter --osrl path/to/resultfile.osrl as an argument when calling SHOT.

GAMS trace file

It is also possible to create a GAMS trace file in a similar way by adding the --trc <filename.trc> flag. If no filename is specified, a filename consisting of the problem file and the suffix .trc is created in the current directory.

AMPL solution file

To create a file in AMPL sol-format add the parameter --sol <filename.sol>.

Console output

The level of output that SHOT writes to the console can be controlled with the setting Output.Console.LogLevel , where values between 0 and 6 are allowed. Assuming that the default value 2 is used, first some basic information about the solver, including the current version is provided:

╶ Supporting Hyperplane Optimization Toolkit (SHOT) ──────────────────────────────────────────────────────────────────╴

 Andreas Lundell and Jan Kronqvist, Åbo Akademi University, Finland.
 See documentation for full list of contributors and utilized software libraries.

 Version: 1.0.alpha 1. Git hash: 6e43619-dirty. Released Apr 11 2020. 

 For more information visit https://shotsolver.dev

Then information about the modeling system or file format used is given as well as information about possible bound tightening performed on the variable bounds.

╶ Modeling system ────────────────────────────────────────────────────────────────────────────────────────────────────╴

 Modeling system:            GAMS
 Problem read from file:     /home/andreas/Research/minlplib/21122018/minlplib/gms/tls4.gms

 Performing bound tightening on original problem.
  - Bounds for 18 variables tightened in 0.06 s.
 Performing bound tightening on reformulated problem.
  - Bounds for 34 variables tightened in 0.08 s.

Then information about the provided problem instance is shown. SHOT automatically reformulates the problem into a form that is more suitable for the underlying algorithm, and statistics about the reformulated problem is given as well.

╶ Problem instance ───────────────────────────────────────────────────────────────────────────────────────────────────╴

                                    Original             Reformulated

 Problem classification:            MINLP, convex        MINLP, convex

 Objective function direction:      minimize             minimize
 Objective function type:           linear               linear

 Number of constraints:             64                   80
  - linear:                         60                   64
  - convex nonlinear:               4                    16

 Number of variables:               105                  121
  - real:                           16                   32
  - binary:                         85                   85
  - integer:                        4                    4
  - nonlinear:                      20                   20

 Number of transformations performed:                    16
 - signomial terms partitioning:                         16

After this, the (nondefault) settings used are shown; here the MIP solver utilized (CPLEX) considered all variable bounds over +/- 1e20 to be infinite, so this setting will be used by SHOT as well.

Below the used settings, the names and versions of the utilized subsolvers are shown.

╶ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────╴

 No options file specified.

 Nondefault options used:

  - Model.Variables.Continuous.MaximumUpperBound = 1e+20
  - Model.Variables.Continuous.MinimumLowerBound = -1e+20

 Dual strategy:              Single-tree
  - cut algorithm:           ESH
  - solver:                  CPLEX 12.10

 Primal NLP solver:          Ipopt 3.12 (with default linear solver)

Now, since the ESH algorithm was used for generating linearizations of nonlinear expressions in SHOT, an interior point is required, which will be found by utilizing a minimax solver implemented by solving linear programming (LP) problems. 

╶ Interior point search ──────────────────────────────────────────────────────────────────────────────────────────────╴

 Strategy selected:          cutting plane minimax
                                                                                     
    Iteration     │  Time  │    Cuts     │     Objective value     │  Objective diff.   
     #: type      │  tot.  │   + | tot.  │    problem | line srch  │    abs. | rel.    
╶─────────────────┴────────┴─────────────┴─────────────────────────┴──────────────────╴

     1: LP           0.38                      -1e+12 | -1e+12          inf. | inf.    
     2: LP           0.38      4 | 4            -43.5 | 68.8668      1.1e+02 | 2.6e+00 
     3: LP           0.40      4 | 8          -37.462 | 59.892       9.7e+01 | 2.6e+00 
     4: LP           0.40      4 | 12        -32.6284 | 46           7.9e+01 | 2.4e+00 
     5: LP           0.40      4 | 16        -24.1783 | 25.641       5.0e+01 | 2.1e+00 
     6: LP           0.40      4 | 20        -12.5657 | 16.3322      2.9e+01 | 2.3e+00 
     7: LP           0.40      4 | 24        -11.5673 | 8.10095      2.0e+01 | 1.7e+00 
     8: LP           0.41      4 | 28        -9.36657 | 6.87839      1.6e+01 | 1.7e+00 
     9: LP           0.41      4 | 32        -8.18222 | 5.50341      1.4e+01 | 1.7e+00 
    10: LP           0.41      4 | 36        -8.10926 | 3.05517      1.1e+01 | 1.4e+00 
    11: LP           0.42      4 | 40         -8.0093 | 1.69059      9.7e+00 | 1.2e+00 
    12: LP           0.42      4 | 44        -7.96557 | 0.652575     8.6e+00 | 1.1e+00 
    13: LP           0.42      4 | 48        -7.96557 | -1.97056     6.0e+00 | 7.5e-01 

 Valid interior point with constraint deviation -1.971 found.

  • The Iteration column shows the current iteration number, as well as the problem type solved (LP=linear programming).

  • The Time column shows the wall clock time in seconds.

  • In the Cuts column, the number of linear constraints added in each iteration, as well as the total number of added constraints, are shown.

  • In the Objective value column, there are two values, the first corresponding to the solution of the LP problem and the other of the minimal value on the line between the previous solution point and the current.

  • The last column gives the absolute and relative difference between the two objective values in the previous column.

So, in the example above, 13 LP problems were solved and a total of 48 linear constraints were generated to find an integer-relaxed interior point to the reformulated MINLP problem. Now we can continue to solve the actual problem. 

    Iteration     │  Time  │  Dual cuts  │     Objective value     │   Objective gap   │     Current solution
     #: type      │  tot.  │   + | tot.  │       dual | primal     │    abs. | rel.    │    obj.fn. | max.err.
╶─────────────────┴────────┴─────────────┴─────────────────────────┴───────────────────┴───────────────────────────╴

     1: LP-O         0.43                           0 | inf.            inf. | 1.0e+00            0 | 65: 8.76e+01     
     6: LP-O         0.48      8 | 40         1.04491 | inf.            inf. | 1.0e+00      1.04491 | 71: 3.04e+00     
     7: LP-O         0.48      7 | 47         1.55977 | inf.            inf. | 1.0e+00      1.55977 | 73: 1.50e+00     
     8: LP-O         0.49      8 | 55         1.64102 | inf.            inf. | 1.0e+00      1.64102 | 68: 1.49e+00     
     9: LP-O         0.49      2 | 57         1.64102 | inf.            inf. | 1.0e+00      1.64102 | 68: 4.06e-01     
    10: CB (th: 0)   0.60                     3.67836 | inf.            inf. | 1.0e+00          7.8 | 72: 6.17e+00     
    12: CB (th: 0)   0.67      2 | 71         3.99122 | inf.            inf. | 1.0e+00          8.8 | 78: 1.60e+00     
    18: CB (th: 0)   0.69      4 | 110        3.99122 | 10.8         6.8e+00 | 6.3e-01         10.8 | 75: 1.97e-07     
    19: CB (th: 0)   0.69      3 | 113        3.99122 | 10.8         6.8e+00 | 6.3e-01         10.8 | 75: 1.97e-07     
    20: CB (th: 0)   0.72                     4.01615 | 10.8         6.8e+00 | 6.3e-01            9 | 65: 7.82e-01     
     2: NLPSOLPT-O   0.75                     4.01615 | 9            5.0e+00 | 5.5e-01            9 | 63: 1.37e-10     
    26: CB (th: 0)   0.77      2 | 143        4.02143 | 9            5.0e+00 | 5.5e-01          8.8 | 66: 8.83e-01     
    27: CB (th: 0)   0.80      4 | 147        4.31494 | 9            4.7e+00 | 5.2e-01          7.9 | 65: 1.02e+01     
    31: CB (th: 0)   0.85      2 | 153        4.44449 | 9            4.6e+00 | 5.1e-01          8.5 | 70: 1.96e+00     
     3: NLPSOLPT-O   0.88                     4.44449 | 8.5          4.1e+00 | 4.8e-01          8.5 | 60: -1.16e-10    
    39: CB (th: 0)   1.19                     5.28294 | 8.5          3.2e+00 | 3.8e-01          7.7 | 67: 7.65e+00     
    40: CB (th: 0)   1.25      1 | 187         5.3468 | 8.5          3.2e+00 | 3.7e-01          8.4 | 66: 3.25e+00     
    46: CB (th: 0)   1.41      1 | 206        5.49899 | 8.4          2.9e+00 | 3.5e-01          7.7 | 72: 2.93e+00     
    47: CB (th: 0)   2.50      2 | 208        6.60651 | 8.4          1.8e+00 | 2.1e-01          8.3 | 67: 4.75e+00     
    48: CB (th: 0)   2.68      2 | 210        6.71446 | 8.4          1.7e+00 | 2.0e-01          8.3 | 72: 8.20e-01     
    53: CB (th: 0)   2.68      3 | 222        6.71446 | 8.3          1.6e+00 | 1.9e-01          8.3 | 70: 2.27e-06     
    53: MIP-O        3.16                         8.3 | 8.3          1.8e-15 | 2.1e-16          8.3 | 70: 2.27e-06

The first thing to notice is that not all iterations are written to console, but this condensed output can be deactivated with the switch Output.Console.Iteration.Detail=0.

The columns are similar to those of the minimax solver mentioned above.

  • The Iteration column shows the iteration number and type. The type identifier consists of a part showing the problem type, and a part showing the solution status of the iteration.

    • The following types are available for the linearized problem solved in SHOT's dual strategy:

      • LP: linear programming problem.

      • QP: quadratic programming problem.

      • QCQP: quadratically constrained quadratic programming problem.

      • MILP: mixed-integer linear programming problem.

      • MIQP: mixed-integer quadratic program.

      • MIQCQP: mixed-integer quadratically constrained quadratic program.

      • CB: New incumbent solution candidate found in callback by MIP solver. For some solvers, also the thread number of the callback is shown.

    • The following types are available for the primal heuristic in SHOT:

      • NLPSOLPT: An NLP problem has been solved with fixed integer values originating from a MIP solution that is classified as optimal (or found in a callback).

      • NLPNEWDB: An NLP problem has been solved with fixed integer values originating from a MIP solution that has improved the lower bound of a minimization problem (or upper bound of a maximization problem).

    • The following statuses are appended after the dash:

      • O: Solved to optimality.

      • F: Solved to feasibility.

      • I: Infeasible problem detected.

      • U: Unbounded problem detected.

      • TL: Time limit reached.

      • IL: Iteration limit reached.

      • SL: Solution limit reached.

  • The Time column shows the wall clock time in seconds.

  • In the Cuts column, the number of linear constraints added in each iteration, as well as the total number of added constraints, are shown.

  • In the Objective value - dual column, the current lower bound (if minimizing) on the objective value is shown. Unless an asterisk * is added to the value, this is a global lower bound. This value is provided by the underlying MIP solver.

  • In the Objective value - primal column, the current upper bound (if minimizing) on the objective value is shown. This value is the current best know solution to the problem and is given by either the MIP solver's solution pool or a primal heuristic, such as solving integer-fixed NLP problems.

  • The objective gap column shows the absolute and relative gap between the lower and upper bounds in the previous columns.

  • The final column Current solution provides some insight into the objective value and constraint fulfillment of the current iteration (either from the MIP or NLP solver).

    • In the max.err column, the nonlinear constraint index with the maximal error in the solution point is shown in addition to the error.

Finally, after terminating the iterative solution procedure, SHOT shows a final report that explains the status of the solution, and why the solver terminated.

╶ Solution report ────────────────────────────────────────────────────────────────────────────────────────────────────╴

 Terminated since absolute gap met requirements.

 Globally optimal primal solution found.

 Objective bound (minimization) [dual, primal]:  [8.3, 8.3]
 Objective gap absolute / relative:              1.77636e-15 / 2.14019e-16

 Fulfilled termination criteria: 
  - absolute objective gap tolerance             1.77636e-15 <= 0.001
  - relative objective gap tolerance             2.14019e-16 <= 0.001

 Unfulfilled termination criteria: 
  - iteration limit                              53 <= 200000
  - solution time limit (s)                      3.16338 <= 1.79769e+308
  
 Dual problems solved in main step:              10
  - LP problems                                  9
  - MILP problems, optimal                       1

 Number of explored nodes:                       4988

 Problems solved during interior point search: 
 - LP problems:                                  13

 Fixed primal NLP problems solved:               4

In the case above, the single-tree strategy was used, and the linearizations were added during the solution of only one MILP problem, which was solved to optimality.

After this some insight into the number of primal solutions, i.e., solutions to the original MINLP problem, found, and which method was used to find them:

 Number of primal solutions found:               10
 - root search:                                  5
 - NLP problem with fixed integers:              2
 - MILP solution pool:                           1
 - lazy constraint callback:                     2

A report over how much time the individual components took is also provided:

 Total solution time:                            3.16344
  - problem reformulation:                       0.0875337
  - bound tightening:                            0.138686
    - feasibility based (original problem):      0.0603996
    - feasibility based (reformulated problem):  0.0776433
  - interior point search:                       0.110094
    - solving relaxed problems:                  0.0685161
  - dual strategy:                               2.76156
    - solving MIP problems:                      2.67002
    - root search for constraint cuts:           0.0667068
  - primal strategy:                             0.19353
    - solving NLP problems:                      0.155881
    - performing root searches:                  0.00566103

Finally, SHOT shows where the log file and solution file(s) have been written:

╶─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╴

 Results written to: /home/andreas/Research/SHOT/COIN-OR/shot/build/clang/tls4.osrl
 Log written to:     /home/andreas/Research/SHOT/COIN-OR/shot/build/clang/SHOT.log

File output

By default SHOT creates a log file SHOT.log in the current directory containing the same type of output as is written to the console. To specify the location of this file use the parameter --log <filename.log>. It is possible to control the verbosity of the log file with the parameter Output.File.LogLevel setting.

Debugging information and intermediate problems

SHOT does not normally create any auxiliary files during run-time so it might be difficult to debug what is happening, e.g. in each dual iteration. There is however a setting Output.Debug flag that, if activated, creates a folder SHOT_debug_***** in the system temp folder. The destination folder can be specified with the parameter Output.Debug.Path. or with--debug=<directory>, where directory is where the debug files should be saved.

For example, there is the file originalproblem.txt and reformulatedproblem.txt which shows how SHOT interprets the given problem and how it is reformulated by SHOT. Also, the intermediate linearized problems solved by the MIP solver in each iteration is provided as LP*.lp.

Using SHOT in Pyomo

To use SHOT in Python, you need to have a SHOT executable installed somewhere on your system.

The location can either be somewhere in your path (so that you can call SHOT from a terminal in any directory), or you will need to specify where the SHOT executable is.

Assuming SHOT is in your path, you can solve problems with the following command:

pyomo problemfile.py --solver SHOT

Otherwise you will need to specify where the SHOT executable is with the command

pyomo problemfile.py --solver "path/to/SHOT[.exe]"

The solution will be written to a file results.yml in the current directory. You can also write the solution to the screen by adding the flag --show-result.

Specifying solver options

You can specify any of SHOT's options (as described here) by adding the --solver-optionsflag. For example: --solver-options="Termination.TimeLimit=100.0" will set a time limit of 100 seconds.

If you want to specify multiple settings, separate them with a space, e.g. --solver-options="Termination.TimeLimit=100.0 Termination.ObjectiveGap.Relative=0.01".

Note that some given parameters may not always be considered; the used settings are indicated at the top of the log file SHOT.log.

Using SHOT in GAMS

If you have a GAMS version with built in support of SHOT, you can simply specify <problem-class> = SHOT (e.g. MINLP=SHOT), either in the model itself or in the command line.

Use your own SHOT version

To use your own version of SHOT with GAMS, you can modify the gamsconfig.yaml file. See the GAMS documentation for instructions.

In short, you need to create a file called gamsconfig.yaml in either the GAMS system directory or one of the other directories specified here.

On Windows

To make available a solver named SHOT2 in GAMS, which uses the SHOT library from D:\Users\janeDoe\Downloads\gamslink\GamsShot-20.dll, add the following contents to the yaml file:

solverConfig:
- SHOT2:
  minVersion: 33
  fileType: 1001
  dictType: 5
  licCodes: 000102030405
  scriptName: gmsgennt.cmd
  executableName: gmsgennx.exe
  library:
    libName: D:\Users\janeDoe\Downloads\gamslink\GamsShot-20.dll
    auditCode: sht
    solverInterfaceType: 1
    threadSafeIndic: True
  modelTypes:
    - MINLP
    - MIQCP

On Linux/MAC

To make available a solver named SHOT2 in GAMS, which uses the SHOT library from /path/to/libSHOTSolver.so , add the following contents to the yaml file:

solverConfig:
- SHOT2:
   minVersion: 33
   fileType: 1001
   dictType: 5
   licCodes: 000102030405
   scriptName: gmsgenus.run
   executableName: gmsgenux.out
   library:
     libName: /path/to/libSHOTSolver.so
     auditCode: sht
     solverInterfaceType: 1
     threadSafeIndic: True
   modelTypes:
     - MINLP
     - MIQCP

The settings are explained in detail here.

Using SHOT in JuMP

It is possible to get SHOT working together with the Julia-based optimization modeling system JuMP using its AmplNLWriter.jl subproject. For more information see the Github repository.

In short, assuming you have everything set up and a version of SHOT installed and working on your system you can use the following command. 

julia> using JuMP, AmplNLWriter
julia> m = Model(() -> AmplNLWriter.Optimizer("SHOT"))

This assumes that the SHOT executable is found in your path, otherwise you will need to specify the full path to where the executable is, i.e. 

julia> using JuMP, AmplNLWriter
julia> m = Model(() -> AmplNLWriter.Optimizer("/path/to/SHOT"))

Call SHOT from JuMP using GAMS

If you have GAMS installed you can also use SHOT from JuMP and Pyomo by using the GAMS interface described in this blog post.

Using SHOT through ASL (AMPL Solver Library)

SHOT should be compatible with any modeling system using ASL (models in nl and solution in sol formats). In practice this works so that the following syntax is used:

./SHOT path/to/problemfile.nl --AMPL

Then SHOT solves the problem given in the nl-file and writes a solution file problemfile.sol. This file is written in the same directory as the nl-file.

PreviousBenchmarksNextSolver options

Last updated