Summary of all solvers in the SCore competition

This section presents the results for the solvers/call scripts of the First Answer Set Programming System Competition. The placement of solvers was determined according to the number of instances they solved within the allocated time and space as the primary measure of performance. Running times were used as tie breakers.
Each single run was limited to 600 seconds execution time and 448 MB RAM memory usage.
We give below tables providing a complete placement of all call scripts participating in each competition class. We report for each call script the absolute and relative number of solved instances ("Solved" and "%"), its minimum, maximum, and average run times, where "avg" gives the average run time on all solved instances and "avg²" gives the average run time on all instances, where a timeout is taken as 600 seconds. The last column given the Euclidean distance between the vector of all run times of a call script and the virtually best solver taken to be the vector of all minimum run times.

Results for SAT and UNSAT instances

	Call Script	Solved	%	min	max	avg	avg²	EuclDist
1	clasp_cmp_score_glp2	89/95	93.68	0.56	530.21	29.81	65.82	1080.46
2	clasp_cmp_score_glp	89/95	93.68	0.75	504.49	30.36	66.34	1099.14
3	clasp_score_glp_def	86/95	90.53	0.75	431.66	25.20	79.66	1386.63
4	smodels_rs_score	81/95	85.26	1.21	346.36	38.93	121.61	1872.81
5	defaultGlparse.sh	81/95	85.26	1.35	597.97	46.86	128.38	2089.18
6	scriptAtomreasonGlparse	80/95	84.21	1.30	576.80	42.40	130.44	2107.83
7	pbmodels-minisat+-SCore	80/95	84.21	0.72	436.11	57.18	142.89	2170.4
8	pbmodels-pueblo-SCore	78/95	82.11	0.34	452.84	41.00	141.03	2210.39
9	dencode+gnt_score	78/95	82.11	1.27	363.19	42.80	142.51	2162.64
10	smodels_score	77/95	81.05	1.28	352.41	40.40	146.43	2217.61
11	dencode_bc+gnt_score	77/95	81.05	1.27	360.70	42.52	148.15	2228.65
12	gnt_score	77/95	81.05	1.27	359.77	42.56	148.18	2228.83
13	scriptEloopGlparse	75/95	78.95	1.36	598.20	42.86	160.15	2493.41
14	smodels_rsn_score	75/95	78.95	1.21	486.23	63.00	176.05	2503.32
15	lp2sat+minisat	75/95	78.95	1.10	561.06	79.89	189.39	2621.13
16	wf+lp2sat+minisat	73/95	76.84	1.56	587.40	86.42	205.35	2792.51
17	dlv-contest-special	69/95	72.63	0.24	586.62	102.47	238.64	3090.71
18	dlv-contest	68/95	71.58	0.24	587.83	96.69	239.74	3110.36
19	lp2sat+siege	68/95	71.58	1.11	471.36	97.50	240.32	3052.8
20	nomore-localprop-SCore	64/95	67.37	2.45	550.43	103.23	265.34	3316.33
21	nomore-default-SCore	63/95	66.32	2.45	554.76	124.62	284.75	3415.78
22	nomore-D-SCore	62/95	65.26	2.77	559.88	161.15	313.59	3583.85
23	ASPeR-call-script	24/95	25.26	1.47	592.24	98.28	473.25	4906.79
24	ASPeRS30-call-script	21/95	22.11	1.51	561.20	88.99	487.04	4995.78
25	ASPeRS20-call-script	21/95	22.11	1.49	381.33	89.40	487.13	4980.24
26	pbmodels-wsatcc-SCore	6/95	6.32	25.57	529.80	208.15	575.25	5514.97

Results for SAT instances only

	Call Script	Solved	%	min	max	avg	avg²	EuclDist
1	clasp_score_glp_def	29/33	87.88	0.75	30.19	5.44	77.51	396.73
2	clasp_cmp_score_glp	29/33	87.88	0.96	46.67	6.31	78.27	398.12
3	clasp_cmp_score_glp2	29/33	87.88	0.56	70.94	8.05	79.8	403.03
4	scriptEloopGlparse	29/33	87.88	1.36	229.87	22.30	92.33	638.86
5	defaultGlparse.sh	29/33	87.88	1.37	230.35	22.31	92.33	638.8
6	scriptAtomreasonGlparse	29/33	87.88	2.99	287.06	28.25	97.55	675
7	smodels_rs_score	28/33	84.85	1.21	263.58	23.64	110.97	769.87
8	lp2sat+minisat	28/33	84.85	2.76	479.87	62.66	144.07	1004.04
9	wf+lp2sat+minisat	28/33	84.85	2.71	587.40	87.03	164.75	1160.26
10	pbmodels-minisat+-SCore	25/33	75.76	0.75	436.11	94.22	216.83	1536.98
11	dencode_bc+gnt_score	24/33	72.73	1.27	132.33	16.57	175.68	1404.43
12	gnt_score	24/33	72.73	1.27	132.43	16.57	175.69	1404.44
13	dencode+gnt_score	24/33	72.73	1.27	132.84	16.58	175.7	1404.48
14	smodels_score	24/33	72.73	1.28	154.65	18.99	177.45	1407.72
15	lp2sat+siege	23/33	69.70	3.16	307.13	87.23	242.61	1654.01
16	smodels_rsn_score	22/33	66.67	1.33	380.68	80.83	253.89	1754.12
17	pbmodels-pueblo-SCore	21/33	63.64	3.77	452.84	81.79	270.23	1868.56
18	nomore-D-SCore	20/33	60.61	4.14	559.88	130.88	315.69	2049.96
19	nomore-default-SCore	19/33	57.58	3.46	530.65	124.84	326.43	2137.61
20	dlv-contest-special	18/33	54.55	0.24	479.23	93.02	323.46	2136.12
21	dlv-contest	17/33	51.52	0.24	328.46	69.54	326.73	2164.75
22	nomore-localprop-SCore	16/33	48.48	3.46	542.37	96.65	355.95	2304.28
23	ASPeRS20-call-script	11/33	33.33	5.06	381.33	104.70	434.9	2615.36
24	ASPeRS30-call-script	11/33	33.33	5.06	561.20	138.65	446.22	2665.19
25	ASPeR-call-script	7/33	21.21	4.92	176.45	55.00	484.39	2835.31
26	pbmodels-wsatcc-SCore	6/33	18.18	25.57	529.80	208.15	528.75	3001.49

Results for UNSAT instances only

	Call Script	Solved	%	min	max	avg	avg²	EuclDist
1	clasp_cmp_score_glp2	60/62	96.77	0.84	530.21	40.33	58.38	1002.48
2	clasp_cmp_score_glp	60/62	96.77	0.75	504.49	41.99	59.99	1024.5
3	pbmodels-pueblo-SCore	57/62	91.94	0.34	391.18	25.97	72.26	1180.82
4	clasp_score_glp_def	57/62	91.94	1.02	431.66	35.26	80.8	1328.67
5	pbmodels-minisat+-SCore	55/62	88.71	0.72	348.60	40.34	103.53	1532.43
6	dencode+gnt_score	54/62	87.10	1.43	363.19	54.46	124.85	1644.52
7	smodels_rs_score	53/62	85.48	1.42	346.36	47.01	127.28	1707.25
8	smodels_score	53/62	85.48	1.40	352.41	50.10	129.92	1713.51
9	dencode_bc+gnt_score	53/62	85.48	1.44	360.70	54.28	133.49	1730.45
10	gnt_score	53/62	85.48	1.43	359.77	54.33	133.54	1730.67
11	smodels_rsn_score	53/62	85.48	1.21	486.23	55.60	134.63	1785.96
12	defaultGlparse.sh	52/62	83.87	1.35	597.97	60.55	147.56	1989.13
13	scriptAtomreasonGlparse	51/62	82.26	1.30	576.80	50.45	147.95	1996.83
14	dlv-contest	51/62	82.26	0.66	587.83	105.74	193.43	2233.43
15	dlv-contest-special	51/62	82.26	0.67	586.62	105.81	193.49	2233.72
16	nomore-localprop-SCore	48/62	77.42	2.45	550.43	105.43	217.1	2385.02
17	lp2sat+minisat	47/62	75.81	1.10	561.06	90.16	213.5	2421.21
18	scriptEloopGlparse	46/62	74.19	1.58	598.20	55.81	196.25	2410.17
19	wf+lp2sat+minisat	45/62	72.58	1.56	581.98	86.03	226.96	2540.06
20	lp2sat+siege	45/62	72.58	1.11	471.36	102.75	239.09	2565.91
21	nomore-default-SCore	44/62	70.97	2.45	554.76	124.53	262.57	2664.24
22	nomore-D-SCore	42/62	67.74	2.77	490.51	175.56	312.48	2939.66
23	ASPeR-call-script	17/62	27.42	1.47	592.24	116.10	467.32	4004.69
24	ASPeRS30-call-script	10/62	16.13	1.51	116.73	34.36	508.77	4225.47
25	ASPeRS20-call-script	10/62	16.13	1.49	240.37	72.56	514.93	4238.24
26	pbmodels-wsatcc-SCore	0/62	0.00				600	4626.66