Summary of all solvers in the SLparse competition

This section presents the results for the solvers/call scripts of the First Answer Set Programming System Competition. The placement of solvers was determined according to the number of instances they solved within the allocated time and space as the primary measure of performance. Running times were used as tie breakers.
Each single run was limited to 600 seconds execution time and 448 MB RAM memory usage.
We give below tables providing a complete placement of all call scripts participating in each competition class. We report for each call script the absolute and relative number of solved instances ("Solved" and "%"), its minimum, maximum, and average run times, where "avg" gives the average run time on all solved instances and "avg²" gives the average run time on all instances, where a timeout is taken as 600 seconds. The last column given the Euclidean distance between the vector of all run times of a call script and the virtually best solver taken to be the vector of all minimum run times.

Results for SAT and UNSAT instances

	Call Script	Solved	%	min	max	avg	avg²	EuclDist
1	clasp_cmp_slparse2	100/127	78.74	0.38	556.49	75.96	187.37	2791.89
2	clasp_cmp_slparse	94/127	74.02	0.41	502.53	61.37	201.33	2919.46
3	pbmodels-minisat+-SLparse	91/127	71.65	0.49	503.57	76.69	225.03	3241.06
4	clasp_slparse_def	89/127	70.08	0.37	546.50	55.62	218.5	3152.34
5	smodels_rs_slparse	87/127	68.50	0.23	576.28	95.92	254.69	3403.9
6	groundedDefault	81/127	63.78	0.25	407.49	46.20	246.79	3448.34
7	scriptAtomreasonGr	81/127	63.78	0.25	407.46	50.55	249.56	3465.67
8	scriptEloopGr	78/127	61.42	0.24	407.48	46.15	259.84	3598.7
9	smodels_slparse	75/127	59.06	0.26	518.08	102.76	306.35	3958.86
10	smodels_rsn_slparse	74/127	58.27	0.35	596.39	70.52	291.49	3815.31
11	pbmodels-pueblo-SLparse	69/127	54.33	0.25	593.05	87.25	321.42	4189.03
12	nomore-D-slparse	54/127	42.52	1.08	530.39	152.78	409.84	4765.06
13	nomore-localprop-slparse	50/127	39.37	1.08	517.63	120.80	411.34	4846.58
14	nomore-default-slparse	49/127	38.58	1.08	549.80	143.44	423.85	4920.23
15	gnt_slparse	35/127	27.56	2.10	482.13	81.40	457.08	5276.26
16	dencode+gnt_slparse	35/127	27.56	2.18	482.81	81.64	457.15	5276.38
17	dencode_bc+gnt_slparse	35/127	27.56	2.10	485.36	81.70	457.16	5276.53
18	script.assat.lparse-output	30/127	23.62	1.00	225.28	38.64	467.4	5379.18
19	pbmodels-wsatcc-SLparse	25/127	19.69	1.12	272.98	46.56	491.05	5585.82

Results for SAT instances only

	Call Script	Solved	%	min	max	avg	avg²	EuclDist
1	clasp_cmp_slparse2	79/96	82.29	0.38	556.49	91.19	181.29	2281.43
2	clasp_cmp_slparse	73/96	76.04	0.41	502.53	73.81	199.87	2436.42
3	pbmodels-minisat+-SLparse	68/96	70.83	0.64	503.57	95.19	242.43	2934.4
4	clasp_slparse_def	67/96	69.79	0.37	467.51	59.50	222.77	2721.21
5	groundedDefault	62/96	64.58	0.25	407.49	51.75	245.92	2931.25
6	scriptAtomreasonGr	62/96	64.58	0.25	407.46	57.89	249.89	2953.31
7	scriptEloopGr	61/96	63.54	0.24	407.48	50.33	250.73	2988.7
8	smodels_rs_slparse	61/96	63.54	0.23	478.30	86.28	273.58	3036.86
9	smodels_rsn_slparse	52/96	54.17	0.35	596.39	74.23	315.21	3483.84
10	smodels_slparse	50/96	52.08	0.26	518.08	103.33	341.32	3643.95
11	pbmodels-pueblo-SLparse	45/96	46.88	0.33	593.05	119.17	374.61	3976.95
12	nomore-D-slparse	37/96	38.54	1.08	530.39	146.69	425.28	4230.8
13	nomore-default-slparse	32/96	33.33	1.08	518.33	149.36	449.79	4417.04
14	nomore-localprop-slparse	31/96	32.29	1.08	517.63	123.43	446.11	4418.97
15	pbmodels-wsatcc-SLparse	25/96	26.04	1.12	272.98	46.56	455.87	4605.66
16	gnt_slparse	20/96	20.83	2.10	482.13	101.31	496.11	4798.9
17	dencode+gnt_slparse	20/96	20.83	2.18	482.81	101.60	496.17	4799.02
18	dencode_bc+gnt_slparse	20/96	20.83	2.10	485.36	101.66	496.18	4799.2
19	script.assat.lparse-output	15/96	15.62	1.00	210.78	33.03	511.41	4916.68

Results for UNSAT instances only

	Call Script	Solved	%	min	max	avg	avg²	EuclDist
1	smodels_rs_slparse	26/31	83.87	2.23	576.28	118.54	196.2	1537.54
2	smodels_slparse	25/31	80.65	2.16	473.90	101.61	198.07	1547.31
3	pbmodels-pueblo-SLparse	24/31	77.42	0.25	266.83	27.40	156.7	1315.99
4	pbmodels-minisat+-SLparse	23/31	74.19	0.49	209.18	21.97	171.14	1376.13
5	clasp_slparse_def	22/31	70.97	1.11	546.50	43.81	205.28	1591.32
6	smodels_rsn_slparse	22/31	70.97	2.25	321.58	61.75	218.02	1555.44
7	clasp_cmp_slparse	21/31	67.74	0.84	109.21	18.13	205.83	1608.46
8	clasp_cmp_slparse2	21/31	67.74	0.75	109.19	18.68	206.2	1609.25
9	scriptAtomreasonGr	19/31	61.29	0.31	213.91	26.61	248.57	1813.52
10	groundedDefault	19/31	61.29	0.30	229.54	28.10	249.48	1816.26
11	nomore-localprop-slparse	19/31	61.29	6.64	496.82	116.51	303.67	1990.49
12	scriptEloopGr	17/31	54.84	0.31	229.56	31.12	288.03	2004.56
13	nomore-default-slparse	17/31	54.84	6.67	549.80	132.30	343.52	2167.57
14	nomore-D-slparse	17/31	54.84	6.65	465.69	166.05	362.03	2192.31
15	script.assat.lparse-output	15/31	48.39	5.22	225.28	44.26	331.09	2182.17
16	gnt_slparse	15/31	48.39	4.39	288.00	54.86	336.22	2193.06
17	dencode+gnt_slparse	15/31	48.39	4.41	287.87	55.03	336.31	2193.06
18	dencode_bc+gnt_slparse	15/31	48.39	4.28	287.81	55.10	336.34	2193.05
19	pbmodels-wsatcc-SLparse	0/31	0.00				600	3160.58