Summary of all solvers in the MGS competition

This section presents the results for the solvers/call scripts of the First Answer Set Programming System Competition. The placement of solvers was determined according to the number of instances they solved within the allocated time and space as the primary measure of performance. Running times were used as tie breakers.
Each single run was limited to 600 seconds execution time and 448 MB RAM memory usage.
We give below tables providing a complete placement of all call scripts participating in each competition class. We report for each call script the absolute and relative number of solved instances ("Solved" and "%"), its minimum, maximum, and average run times, where "avg" gives the average run time on all solved instances and "avg²" gives the average run time on all instances, where a timeout is taken as 600 seconds. The last column given the Euclidean distance between the vector of all run times of a call script and the virtually best solver taken to be the vector of all minimum run times.

Results for SAT and UNSAT instances

	Call Script	Solved	%	min	max	avg	avg²	EuclDist
1	dlv-contest-special	76/119	63.87	0.07	565.38	54.31	251.49	3542.79
2	dlv-contest	66/119	55.46	0.06	579.09	49.73	294.81	3948.3
3	pbmodels-minisat+-MGS	65/111	58.56	0.52	563.39	83.27	297.41	4142.88
4	clasp_cmp_score2	64/111	57.66	0.87	579.14	115.35	320.56	4285.59
5	clasp_score_def	60/111	54.05	0.91	542.64	80.09	318.97	4277.69
6	clasp_cmp_score	58/111	52.25	0.83	469.46	87.40	332.16	4280.92
7	pbmodels-pueble-MGS	54/111	48.65	0.34	584.31	80.94	347.49	4491.85
8	default	51/119	42.86	0.23	453.88	64.96	370.7	4543.12
9	smodels_rs	34/111	30.63	0.30	539.33	153.86	471.45	5349.11
10	smodels	34/104	32.69	1.14	584.06	173.60	460.6	4974.14
11	pbmodels-wsatcc-MGS	23/111	20.72	1.05	563.52	136.97	504.06	5409.17
12	smodels_rsn	22/111	19.82	0.12	579.10	163.14	513.42	5471.81
13	nomore-default	13/111	11.71	22.04	521.11	315.78	566.71	5714.67
14	scriptAtomreasonLp	12/24	50.00	3.59	259.88	91.18	345.59	2121.13
15	nomore-localprop	10/111	9.01	19.54	521.03	324.83	575.21	5787.71
16	nomore-D	9/111	8.11	48.50	473.89	161.99	564.49	5763.96
17	scriptEloopLp	4/16	25.00	49.92	223.03	106.09	476.52	2088.98
18	gnt	0/8	0.00				600	1696.31
19	dencode_bc+gnt	0/8	0.00				600	1696.31
20	dencode+gnt	0/8	0.00				600	1696.31
21	script.assat.normal	0/50	0.00				600	4101.66

Results for SAT instances only

	Call Script	Solved	%	min	max	avg	avg²	EuclDist
1	dlv-contest-special	61/94	64.89	0.07	536.65	37.43	234.93	3007.53
2	clasp_cmp_score2	59/91	64.84	0.87	579.14	114.00	284.9	3494.98
3	dlv-contest	55/94	58.51	0.06	540.51	34.66	269.22	3283.04
4	clasp_score_def	54/91	59.34	0.91	455.81	68.09	284.36	3506.49
5	clasp_cmp_score	54/91	59.34	0.83	469.46	89.45	297.03	3565.68
6	pbmodels-minisat+-MGS	53/91	58.24	0.56	563.39	88.41	302.04	3643.78
7	default	46/99	46.46	0.86	453.88	69.90	353.69	4237.81
8	pbmodels-pueble-MGS	43/91	47.25	0.35	584.31	90.88	359.43	4004.3
9	smodels_rs	27/98	27.55	0.30	539.33	144.32	474.46	4853.29
10	smodels	27/84	32.14	1.14	584.06	162.39	459.34	4377.48
11	pbmodels-wsatcc-MGS	23/91	25.27	1.05	563.52	136.97	482.97	4838.48
12	smodels_rsn	16/91	17.58	0.12	579.10	152.79	521.37	4976.94
13	nomore-default	13/91	14.29	22.04	521.11	315.78	559.4	5122.74
14	scriptAtomreasonLp	12/22	54.55	3.59	259.88	91.18	322.46	1944.06
15	nomore-localprop	10/91	10.99	19.54	521.03	324.83	569.76	5204.19
16	nomore-D	9/91	9.89	48.50	473.89	161.99	556.68	5110.12
17	scriptEloopLp	4/14	28.57	49.92	223.03	106.09	458.88	1911.24
18	gnt	0/6	0.00				600	1469.07
19	dencode_bc+gnt	0/6	0.00				600	1469.07
20	dencode+gnt	0/6	0.00				600	1469.07
21	script.assat.normal	0/47	0.00				600	4032.5

Results for UNSAT instances only

	Call Script	Solved	%	min	max	avg	avg²	EuclDist
1	dlv-contest-special	15/25	60.00	0.26	565.38	122.96	313.78	1872.46
2	pbmodels-minisat+-MGS	12/20	60.00	0.52	224.98	60.58	276.35	1595.24
3	pbmodels-pueble-MGS	11/20	55.00	0.34	345.25	42.11	293.16	1658.77
4	dlv-contest	11/25	44.00	9.02	579.09	125.11	391.05	2193.33
5	smodels_rs	7/20	35.00	12.52	393.29	190.65	456.73	2079.01
6	smodels	7/20	35.00	12.30	477.67	216.84	465.89	2108.16
7	clasp_score_def	6/20	30.00	8.11	542.64	188.14	476.44	2156.36
8	smodels_rsn	6/20	30.00	28.12	350.68	190.73	477.22	2062.06
9	default	5/20	25.00	0.23	66.35	19.45	454.86	2117.09
10	clasp_cmp_score2	5/20	25.00	3.26	436.61	131.23	482.81	2164.42
11	clasp_cmp_score	4/20	20.00	7.75	202.11	59.84	491.97	2201.53
12	gnt	0/2	0.00				600	723.56
13	dencode_bc+gnt	0/2	0.00				600	723.56
14	scriptEloopLp	0/2	0.00				600	723.56
15	dencode+gnt	0/2	0.00				600	723.56
16	scriptAtomreasonLp	0/2	0.00				600	723.56
17	script.assat.normal	0/3	0.00				600	930.76
18	pbmodels-wsatcc-MGS	0/20	0.00				600	2420.69
19	nomore-localprop	0/20	0.00				600	2420.69
20	nomore-default	0/20	0.00				600	2420.69
21	nomore-D	0/20	0.00				600	2420.69