Summary of all benchmarks in the MGS competition

7.3	call script runs 7.3 seconds on the benchmark instance
1.1	call script runs 1.1 seconds on the benchmark instance (first place)
1.2	call script runs 1.2 seconds on the benchmark instance (second place)
2.3	call script runs 2.3 seconds on the benchmark instance (third place)
R	call script runs out of time
S	call script runs out of memory
I	computed answer set is incorrect
NF	answer set was not found (only for incomplete solvers)
CR	call script/solver can't read the input format e.g. extended rules

BoundedSpanningTree

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	R	R	R	R	R	R	R	R
nomore-localprop	R	R	R	R	R	R	R	R
nomore-default	R	R	R	R	R	R	R	R
pbmodels-wsatcc-MGS	R	R	R	R	R	R	R	R
smodels_rsn	R	R	R	R	R	103.0	246.3	R
smodels_rs	R	R	R	R	399.8	235.3	261.3	519.1
smodels	R	R	R	R	309.9	235.3	260.0	R
script.assat.normal	CR	CR	CR	CR	CR	CR	CR	CR
dlv-contest-special	0.3	0.3	0.3	0.3	0.2	0.3	0.3	0.2
dlv-contest	0.3	0.3	0.3	0.3	0.2	0.3	0.2	0.2
clasp_cmp_score	36.3	43.8	38.5	37.6	22.1	24.3	21.5	21.7
pbmodels-minisat+-MGS	44.0	53.5	38.4	39.0	21.8	21.6	21.7	22.1
clasp_cmp_score2	46.0	47.5	38.2	38.6	22.6	22.0	19.2	19.1
clasp_score_def	87.2	82.4	80.8	84.0	48.0	40.4	54.0	52.0
pbmodels-pueble-MGS	92.5	68.1	55.5	70.4	35.8	28.4	26.7	29.1
default	100.4	167.3	80.8	95.3	22.3	13.8	11.8	17.0

HamiltonianCycle

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-localprop	R	R	19.5	R	R	R	R	R
pbmodels-wsatcc-MGS	S	S	S	S	S	S	R	R
pbmodels-pueble-MGS	R	R	R	R	R	R	R	R
pbmodels-minisat+-MGS	S	S	S	S	S	S	R	R
smodels_rsn	R	R	R	R	S	S	R	R
smodels_rs	R	R	R	R	S	S	R	R
smodels	R	R	R	R	S	S	R	R
default (tight encoding)	R	R	R	R	S	S	R	R
default (non-tight encoding)	5.0	7.6	4.3	6.2	25.3	14.5	56.6	453.9
script.assat.normal	CR	S	S	S	S	S	S	S
clasp_cmp_score	1.1	1.0	1.0	4.6	9.0	1.7	24.4	50.7
clasp_score_def	1.3	1.1	1.1	1.3	2.2	1.8	3.1	5.2
clasp_cmp_score2	1.4	0.9	0.9	3.0	9.2	1.4	2.4	50.6
scriptAtomreasonLp	6.9	10.9	6.7	3.6	100.6	58.5	222.8	259.9
dlv-contest-special	21.6	19.8	19.8	21.2	48.2	54.8	133.2	149.7
dlv-contest	23.5	19.8	20.7	21.2	48.8	55.1	134.0	149.9
nomore-D	140.2	R	108.8	171.2	R	R	R	R
nomore-default	254.9	183.5	69.2	R	R	364.9	R	R

WeightedSpanningTree

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	R	473.9	R	R	R	R	R	R
pbmodels-wsatcc-MGS	R	R	R	R	R	R	R	R
smodels_rsn	R	R	R	R	R	579.1	R	R
smodels_rs	R	R	324.0	407.3	213.3	126.5	539.3	R
smodels	R	126.7	R	160.3	213.9	126.8	304.1	434.6
script.assat.normal	CR	CR	CR	CR	CR	CR	CR	CR
dlv-contest	0.1	0.1	0.1	0.1	0.1	0.1	0.1	0.1
dlv-contest-special	0.1	0.1	0.1	0.1	0.1	0.1	0.1	0.1
default	5.2	8.5	14.7	6.6	5.4	7.8	9.5	9.8
clasp_cmp_score2	9.8	11.6	11.7	10.5	10.2	10.8	12.1	14.1
clasp_cmp_score	10.4	10.7	11.4	11.2	11.9	10.8	12.4	12.4
pbmodels-minisat+-MGS	11.5	11.1	14.3	11.4	11.6	11.1	15.6	14.5
clasp_score_def	13.8	12.9	14.7	12.2	13.5	12.1	14.3	13.5
pbmodels-pueble-MGS	14.7	15.7	17.3	15.8	16.0	23.6	36.7	17.0
nomore-default	444.8	396.0	511.1	453.8	480.1	R	R	521.1
nomore-localprop	445.0	395.5	511.7	452.6	479.6	R	R	521.0

Sokoban

	Instances
Call Script	1	2	3	4	5	6	7	8
dlv-contest-special	109.4	68.4	76.8	14.5	35.5	103.4	12.7	13.7
dlv-contest	109.5	68.1	77.7	14.7	35.6	103.7	12.7	13.6

Fastfood

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	S	S	S	S	S	S	S	S
nomore-localprop	S	S	S	S	S	S	S	S
nomore-default	S	S	S	S	S	S	S	S
pbmodels-wsatcc-MGS	R	R	R	R	R	R	R	R
pbmodels-pueble-MGS	R	R	R	R	R	R	R	R
pbmodels-minisat+-MGS	R	R	R	R	R	R	R	R
clasp_cmp_score2	S	S	S	S	S	S	S	S
clasp_cmp_score	S	S	S	S	S	S	S	S
clasp_score_def	S	S	S	S	S	S	S	S
smodels_rsn	S	S	S	350.7	S	S	S	S
smodels_rs	S	S	S	351.4	S	S	S	S
smodels	S	S	S	352.1	S	S	S	S
default	S	S	S	S	S	S	S	S
script.assat.normal	CR	CR	CR	CR	CR	CR	CR	CR
dlv-contest-special	155.4	91.6	5.3	263.6	5.6	5.4	9.0	9.0
dlv-contest	157.4	92.5	5.3	265.5	5.5	5.4	9.0	9.0

BlockedN-QueensCompetition

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	R	R	R	R	R	R	R	R
nomore-localprop	R	R	R	R	R	R	R	R
nomore-default	R	R	R	R	R	R	R	R
clasp_cmp_score	R	R	R	398.4	S	426.2	202.1	469.5
smodels_rsn	R	R	R	R	396.6	322.7	R	R
dlv-contest	R	R	R	171.7	R	540.5	R	R
dlv-contest-special	R	R	R	171.8	R	536.6	R	R
pbmodels-wsatcc-MGS	29.5	NF	NF	37.8	27.3	42.8	NF	29.4
pbmodels-pueble-MGS	36.5	R	R	R	R	R	345.2	R
clasp_score_def	111.5	542.6	469.9	25.9	192.4	47.2	63.2	166.7
smodels	112.6	105.8	56.0	87.2	24.6	8.6	12.3	143.0
pbmodels-minisat+-MGS	227.4	225.0	214.2	6.5	122.9	11.6	48.8	75.4
smodels_rs	252.0	103.1	56.2	45.8	12.0	11.2	12.5	136.2
default	442.7	R	R	220.6	221.1	80.6	66.4	75.2
clasp_cmp_score2	525.8	R	R	348.2	R	276.7	194.7	534.7

WeightBoundedDominatingSetSuite

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	R	R	R	R	R	R	R	R
nomore-localprop	R	R	21.9	R	R	R	R	R
nomore-default	R	R	22.0	R	R	R	R	R
smodels	R	R	R	R	R	R	R	R
dlv-contest	R	R	R	R	R	R	R	R
dlv-contest-special	R	R	R	R	R	R	R	R
script.assat.normal	CR	CR	CR	CR	CR	CR	CR	CR
clasp_score_def	2.1	1.6	7.8	54.7	R	R	R	R
clasp_cmp_score	4.7	4.7	0.8	225.7	162.9	R	210.0	122.8
pbmodels-minisat+-MGS	5.2	15.2	563.4	R	8.8	R	R	R
clasp_cmp_score2	13.5	2.8	5.0	484.7	304.7	292.6	185.8	R
smodels_rs	20.5	R	R	R	R	R	R	R
default	34.9	34.7	18.2	R	R	R	R	R
pbmodels-wsatcc-MGS	87.6	40.5	111.6	250.0	R	421.6	196.2	469.6
pbmodels-pueble-MGS	151.6	24.0	10.2	6.0	R	25.7	R	R
smodels_rsn	399.6	R	345.0	R	R	R	R	R

TravelingSalespersonSuite

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	S	S	S	S	S	S	S	S
nomore-localprop	S	S	S	S	S	S	S	S
nomore-default	S	S	S	S	S	S	S	S
pbmodels-wsatcc-MGS	R	R	R	R	R	R	R	R
pbmodels-pueble-MGS	S	S	S	S	S	S	S	S
pbmodels-minisat+-MGS	S	S	S	S	S	S	S	S
clasp_cmp_score2	S	S	S	S	S	S	S	S
clasp_cmp_score	S	S	S	S	S	S	S	S
clasp_score_def	S	S	S	S	S	S	S	S
smodels_rsn	R	R	R	R	R	R	R	R
smodels_rs	R	R	R	R	R	R	R	R
smodels	R	R	R	R	R	R	R	R
default	S	S	S	S	S	S	S	S
script.assat.normal	CR	CR	CR	CR	CR	CR	CR	CR
dlv-contest-special	0.7	0.8	0.8	1.0	0.9	1.0	11.1	0.9
dlv-contest	0.8	0.8	0.8	1.0	1.0	1.1	11.2	0.9

WeightedLatinSquareCompetition

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	S	S	S	S	S	S	S	S
nomore-localprop	S	S	S	S	S	S	S	S
nomore-default	S	S	S	S	S	S	S	S
pbmodels-wsatcc-MGS	NF	3.3	NF	NF	NF	NF	2.9	1.4
clasp_cmp_score2	S	S	S	S	S	S	S	S
clasp_cmp_score	S	S	S	S	S	S	S	S
clasp_score_def	S	S	S	S	S	S	S	S
dlv-contest	R	286.2	R	R	R	R	R	109.2
dlv-contest-special	R	288.1	R	R	R	R	R	107.9
default	S	S	S	S	S	S	S	S
pbmodels-pueble-MGS	0.4	0.4	0.3	0.3	0.3	0.3	0.4	0.4
pbmodels-minisat+-MGS	0.5	1.1	0.6	1.0	0.9	0.7	1.2	0.6
smodels_rsn	28.1	0.6	198.6	89.3	219.0	258.6	0.9	0.1
smodels_rs	363.7	0.3	R	R	393.3	R	5.1	1.5
smodels	477.7	346.2	R	R	454.8	R	2.2	166.3

SocialGolfer

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	R	77.8	193.9	R	48.5	127.2	116.4	R
nomore-localprop	R	60.8	R	R	R	R	R	R
nomore-default	R	60.7	R	R	R	R	R	R
pbmodels-wsatcc-MGS	NF	5.4	NF	NF	NF	NF	NF	NF
pbmodels-pueble-MGS	R	4.6	8.4	2.8	3.0	10.1	4.5	41.4
pbmodels-minisat+-MGS	R	2.6	6.1	0.8	2.0	8.5	3.0	208.6
clasp_cmp_score2	R	1.1	2.2	3.3	0.9	1.8	1.3	436.6
clasp_cmp_score	R	1.4	2.2	15.8	1.3	1.8	1.3	R
clasp_score_def	R	0.9	2.4	24.8	1.0	2.5	1.4	R
smodels_rsn	R	1.2	2.2	R	1.0	6.8	1.3	R
smodels_rs	R	1.7	3.4	54.4	2.6	9.2	2.5	R
smodels	R	1.1	24.5	59.2	1.7	1.7	1.5	R
default	R	0.9	17.1	10.7	0.9	2.0	1.3	0.2
dlv-contest-special	565.4	21.5	R	45.9	3.4	R	14.1	R
dlv-contest	579.1	21.5	R	46.5	3.4	R	14.2	R

CarSequencing

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	R	R	R	R	R	R	R	R
nomore-localprop	R	R	R	R	R	340.6	R	R
nomore-default	R	R	R	R	R	342.9	R	R
pbmodels-pueble-MGS	R	R	10.9	R	R	0.7	3.4	R
pbmodels-minisat+-MGS	R	R	R	R	R	4.7	179.1	R
clasp_cmp_score2	R	R	R	R	R	9.1	43.6	R
clasp_cmp_score	R	R	R	R	R	5.1	105.9	R
clasp_score_def	R	R	R	R	R	4.7	126.3	R
smodels_rsn	R	R	R	R	R	38.3	R	R
smodels_rs	R	R	R	R	R	40.5	R	R
smodels	R	R	R	R	R	R	R	R
dlv-contest	R	R	R	R	R	0.8	R	R
dlv-contest-special	R	R	R	R	R	0.8	R	R
default	R	R	R	R	S	1.2	R	R
pbmodels-wsatcc-MGS	328.8	484.9	R	3.2	563.5	1.0	2.1	9.9

15-PuzzleCompetition

	Instances
Call Script	1	2	3	4	5	6	7
nomore-D	R	R	R	R	R	R	R
nomore-localprop	R	R	R	R	R	R	R
nomore-default	R	R	R	R	R	R	R
dlv-contest	R	R	R	R	R	R	R
dlv-contest-special	R	R	R	R	R	R	R
smodels_rsn	R	R	R	R	R	R	R
smodels_rs	R	R	R	R	R	R	R
default	R	R	R	340.4	R	R	R
clasp_cmp_score	R	R	72.1	71.2	R	325.2	R
clasp_score_def	R	R	136.2	76.7	422.0	455.8	166.2
pbmodels-wsatcc-MGS	NF	NF	NF	NF	NF	NF	NF
pbmodels-minisat+-MGS	383.8	286.2	270.7	52.9	526.6	283.7	103.5
pbmodels-pueble-MGS	431.8	490.8	R	148.5	R	R	402.8
clasp_cmp_score2	579.1	R	R	71.1	R	477.3	414.0

SchurCompetition

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	R	R	R	R	R	R	R	R
nomore-localprop	R	R	R	R	R	R	R	R
nomore-default	R	R	R	R	R	R	R	R
pbmodels-wsatcc-MGS	R	R	R	R	R	R	R	R
pbmodels-pueble-MGS	R	R	R	R	R	52.0	45.1	15.9
pbmodels-minisat+-MGS	R	R	45.2	468.8	R	102.0	15.5	10.2
clasp_cmp_score	R	233.7	R	240.4	R	R	13.7	7.8
smodels_rsn	R	R	R	R	R	R	R	R
smodels_rs	R	R	R	R	R	R	R	R
smodels	R	R	R	R	R	R	R	R
dlv-contest	R	R	R	R	R	R	R	R
dlv-contest-special	R	R	R	R	R	R	281.4	204.5
default	R	137.7	R	3.3	R	8.3	13.2	6.7
clasp_score_def	20.5	R	R	437.7	R	439.6	20.2	8.1
clasp_cmp_score2	26.3	272.8	359.2	205.8	146.7	7.1	13.9	7.7

TowersOfHanoiCompetition

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	R	R	R	R	R	R	R	R
nomore-localprop	R	R	R	R	R	R	R	R
nomore-default	R	R	R	R	R	R	R	R
pbmodels-wsatcc-MGS	NF	NF	NF	NF	NF	NF	NF	NF
smodels_rsn	R	R	R	R	R	R	R	R
dlv-contest-special	1.7	1.8	2.0	3.6	2.8	2.9	3.2	4.6
dlv-contest	2.3	2.3	2.7	5.1	3.7	3.9	4.3	6.4
clasp_score_def	3.6	3.7	18.8	43.2	R	24.4	R	26.2
pbmodels-minisat+-MGS	7.6	7.6	29.3	64.8	72.9	198.0	10.2	163.8
clasp_cmp_score2	10.9	10.9	35.4	169.4	97.1	95.0	299.9	R
clasp_cmp_score	19.1	19.1	13.0	166.2	110.2	136.4	439.2	408.8
scriptAtomreasonLp	75.6	75.7	49.8	R	223.0	R	R	R
scriptEloopLp	75.7	75.7	49.9	R	223.0	R	R	R
default	75.8	75.7	49.9	R	223.6	R	R	R
pbmodels-pueble-MGS	85.2	85.3	75.8	R	442.9	255.2	R	584.3
smodels_rs	101.0	101.0	124.3	R	R	R	R	R
smodels	101.2	100.3	114.6	584.1	R	R	391.3	R

Reachability

	Instances
Call Script	1	2	3	4	5	6	7	8
nomore-D	R	R	R	R	R	R	R	R
nomore-localprop	R	R	R	R	R	R	R	R
nomore-default	R	R	R	R	R	R	R	R
pbmodels-wsatcc-MGS	S	S	S	S	S	S	S	S
pbmodels-pueble-MGS	S	S	S	S	S	S	S	S
pbmodels-minisat+-MGS	S	S	S	S	S	S	S	S
clasp_cmp_score2	S	S	S	S	S	S	S	S
clasp_cmp_score	S	S	S	S	S	S	S	S
clasp_score_def	S	S	S	S	S	S	S	S
smodels_rsn	S	S	S	S	S	S	S	S
smodels_rs	S	S	S	S	S	S	S	S
smodels	S	S	S	S	S	S	S	S
dlv-contest	R	R	R	R	R	R	R	R
scriptEloopLp	S	S	S	S	S	S	S	S
scriptAtomreasonLp	S	S	S	S	S	S	S	S
default	S	S	S	S	S	S	S	S
dencode_bc+gnt	S	S	S	S	S	S	S	S
dencode+gnt	S	S	S	S	S	S	S	S
gnt	S	S	S	S	S	S	S	S
dlv-contest-special	27.6	32.6	71.2	87.9	169.9	5.9	0.3	0.5