Duckietown challenges

Duckietown challenges server

This is the Duckietown challenges server. For more information, look at duckietown.org.

Challenges

Mooc 2022 challenges
Validation challenges
Variations on AI-DO challenges 🏋
AI-DO (full state)

These are the challenges for the 2022 mooc.

Challenge LX - Braitenberg 1

#1 Bea Baselines `1664`	#2 Bea Baselines `1734`	#3 Bea Baselines `1809`

copy link	copy link	copy link

Rank (user)	User	Submission	complete	User label	Distance from starting point. ?↑
1	Bea Baselines 🐤	`1664`	1/5	baseline-duckietown	2.5
-	Bea Baselines 🐤	`1734`	1/5	minimal-agent	1.5
-	Bea Baselines 🐤	`1809`	1/5	template-ros	1.35
-	Bea Baselines 🐤	`1759`	1/5	template-pytorch	0.7
-	Bea Baselines 🐤	`1784`	1/5	template-random	0.6
2	Kathryn Hite	`1859`	1/5	exercises_braitenberg	0.05
-	Bea Baselines 🐤	`1639`	1/5	baseline-duckietown-ml	0.05

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge MOOC - Visual Lane Servoing

This is the challenge to test the Visual Lane Servoing of the MOOC’s robot vision exercise.

For more information about the “Self-Driving Cars with Duckietown” MOOC, visit duckietown.org.

#1 Kathryn Hite `1863`	#2 Bea Baselines `1637`	#3 Bea Baselines `1807`

copy link	copy link	copy link

Rank (user)	User	Submission	complete	User label	Major infractions ? ↓	Lateral deviation ? ↓	Traveled distance ?↑	Survival time ?↑
1	Kathryn Hite	`1863`	1/1	lx22-visservoing	0.	0.67	6.6	60.
2	Bea Baselines 🐤	`1637`	1/1	baseline-duckietown-ml	0.	0.9	0.6	60.
-	Bea Baselines 🐤	`1807`	1/1	template-ros	2.6	0.15	0.6	6.
-	Bea Baselines 🐤	`1732`	1/1	minimal-agent	2.8	0.18	1.2	10.
-	Bea Baselines 🐤	`1757`	1/1	template-pytorch	21.2	0.84	1.2	32.
-	Bea Baselines 🐤	`1782`	1/1	template-random	23.2	1.19	0.6	36.
-	Bea Baselines 🐤	`1662`	1/1	baseline-duckietown	25.2	4.65	4.8	60.

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge MOOC - State Estimation

This is the challenge is part of the the MOOC’s state estimation exercise.

For more information about the “Self-Driving Cars with Duckietown” MOOC, visit duckietown.org.

#1 Bea Baselines `1663`	#2 Bea Baselines `1758`	#3 Bea Baselines `1808`

copy link	copy link	copy link

Rank (user)	User	Submission	complete	User label	Traveled distance ?↑	Survival time ?↑	Lateral deviation ? ↓	Major infractions ? ↓
1	Bea Baselines 🐤	`1663`	1/1	baseline-duckietown	7.44	60.	4.21	6.
-	Bea Baselines 🐤	`1758`	1/1	template-pytorch	2.48	60.	1.09	47.8
-	Bea Baselines 🐤	`1808`	1/1	template-ros	1.24	12.	0.26	7.6
-	Bea Baselines 🐤	`1783`	1/1	template-random	0.62	60.	0.68	49.6
-	Bea Baselines 🐤	`1733`	1/1	minimal-agent	0.62	12.	0.26	6.8
-	Bea Baselines 🐤	`1638`	1/1	baseline-duckietown-ml	0.	60.	1.22	0.

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge MOOC - ModCon

This is the challenge to test the PID controller of the MOOC’s modcon exercise.

For more information about the “Self-Driving Cars with Duckietown” MOOC, visit duckietown.org.

#1 Kathryn Hite `1864`	#2 Bea Baselines `1660`	#3 Bea Baselines `1805`

copy link	copy link	copy link

Rank (user)	User	Submission	complete	User label	Major infractions ? ↓	Lateral deviation ? ↓	Traveled distance ?↑	Survival time ?↑
1	Kathryn Hite	`1864`	1/1	lx22-modcon	2.4	1.8	6.	46.
2	Bea Baselines 🐤	`1660`	1/1	baseline-duckietown	3.4	0.96	4.2	32.
-	Bea Baselines 🐤	`1805`	1/1	template-ros	5.4	0.65	2.4	14.
-	Bea Baselines 🐤	`1780`	1/1	template-random	6.	0.12	-0.	8.
-	Bea Baselines 🐤	`1730`	1/1	minimal-agent	15.4	0.	0.	16.
-	Bea Baselines 🐤	`1755`	1/1	template-pytorch	36.	0.73	0.6	44.
-	Bea Baselines 🐤	`1635`	1/1	baseline-duckietown-ml	60.	0.	0.	60.

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge MOOC - Object Detection

This is the challenge is part of the the MOOC’s object detection exercise.

For more information about the “Self-Driving Cars with Duckietown” MOOC, visit duckietown.org.

#1 Kathryn Hite `1862`	#2 Bea Baselines `1731`	#3 Bea Baselines `1756`

copy link	copy link	copy link

Rank (user)	User	Submission	complete	User label	Survival time ?↑	Traveled distance ?↑
1	Kathryn Hite	`1862`	1/1	objdet exercise	60.	4.4
2	Bea Baselines 🐤	`1731`	1/1	minimal-agent	60.	3.8
-	Bea Baselines 🐤	`1756`	1/1	template-pytorch	60.	1.8
-	Bea Baselines 🐤	`1636`	1/1	baseline-duckietown-ml	60.	0.1
-	Bea Baselines 🐤	`1781`	1/1	template-random	54.5	0.3
-	Bea Baselines 🐤	`1661`	1/1	baseline-duckietown	37.5	4.8
-	Bea Baselines 🐤	`1806`	1/1	template-ros	25.3	1.3

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge Planning dynamic (validation)

Planning intermediate (validation)

Rank (user)	User	Submission	complete	User label	mistake ratio ? ↓	Success ratio ?↑	Duration ? ↓	Plan complexity ? ↓	Min distance ?↑
-	Andrea Daniele	`1588`	1/4	planning-exercise	0.25	0.25	755.6	752.	0.
-	Andrea Censi	`1573`	1/4	planning-exercise	0.25	0.25	755.6	752.	0.

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge Planning constrained (validation)

Planning intermediate (validation)

Rank (user)	User	Submission	complete	User label	mistake ratio ? ↓	Success ratio ?↑	Duration ? ↓	Plan complexity ? ↓	Min distance ?↑
-	Andrea Daniele	`1589`	1/4	planning-exercise	0.25	0.25	755.6	752.	0.
-	Andrea Censi	`1574`	1/4	planning-exercise	0.25	0.25	755.6	752.	0.
1	Bea Baselines 🐤	`914`	1/4	baseline-solution	0.25	0.25	763.	780.2	0.06

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge Planning (empty env)

Planning test with empty environment.

Rank (user)	User	Submission	complete	User label	mistake ratio ? ↓	Success ratio ?↑	Duration ? ↓	Plan complexity ? ↓	Min distance ?↑
1	Bea Baselines 🐤	`915`	1/1	baseline-solution	0.93	1.	8.8	16.6	0.87
-	Andrea Daniele	`1590`	1/1	planning-exercise	1.	1.	22.	8.	0.27
-	Andrea Censi	`1575`	1/1	planning-exercise	1.	1.	22.	8.	0.27

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge Planning dynamic (validation)

Planning intermediate (validation)

Rank (user)	User	Submission	complete	User label	mistake ratio ? ↓	Success ratio ?↑	Duration ? ↓	Plan complexity ? ↓	Min distance ?↑
-	Andrea Daniele	`1585`	1/4	planning-exercise	0.25	0.25	755.6	752.	0.
-	Andrea Censi	`1570`	1/4	planning-exercise	0.25	0.25	755.6	752.	0.

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge Planning intermediate (validation)

Planning intermediate (validation)

Rank (user)	User	Submission	complete	User label	mistake ratio ? ↓	Success ratio ?↑	Duration ? ↓	Plan complexity ? ↓	Min distance ?↑
-	Andrea Daniele	`1586`	1/4	planning-exercise	1.	1.	22.	8.	0.01
-	Andrea Censi	`1571`	1/4	planning-exercise	1.	1.	22.	8.	0.01

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge Planning (empty env)

Planning test with empty environment.

Rank (user)	User	Submission	complete	User label	mistake ratio ? ↓	Success ratio ?↑	Duration ? ↓	Plan complexity ? ↓	Min distance ?↑
1	Bea Baselines 🐤	`912`	1/1	baseline-solution	0.	1.	11.2	16.	0.88
-	Andrea Daniele	`1587`	1/1	planning-exercise	0.97	1.	22.	8.	0.27
-	Andrea Censi	`1572`	1/1	planning-exercise	0.97	1.	22.	8.	0.27

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge Collision checking

Collision checking (validation)

Rank (user)	User	Submission	complete	User label	false negative rate ? ↓	false positive rate ? ↓
1	Bea Baselines 🐤	`922`	1/1	baseline-solution	0.	0.
-	Andrea Censi	`1582`	1/1	collision-check-exercise	0.39	0.47

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

In these challenges you must guide a Duckiebot through the streets of Duckietown with only a camera and encoders available.

There are 3 variations:

`LF`	`LFP`	`LFV_multi`
Follow the lane	Avoid the duckies pedestrians	Control multiple Duckiebots

To get started, read the docs.

Note: these are the validation 🏋 challenge - you can see the output but these are not the scores that count. Select the “AI-DO 5 Testing 🥇” tab to see the scores that count.

Challenge Sanity check (full)

Very short “challenge”. Sanity check.

Rank (user)	User	Submission	complete	User label	Survival time ?↑	Major infractions ? ↓	Traveled distance ?↑	Lateral deviation ? ↓
1	Bea Baselines 🐤	`1722`	1/1	minimal-agent-state	2.	0.	0.62	0.01
-	Bea Baselines 🐤	`1692`	1/1	baseline-simple-yield	2.	0.	0.62	0.01

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge Sanity check (full)

Very short “challenge”. Sanity check.

#1 Bea Baselines 1707

copy link

Rank (user)	User	Submission	complete	User label	Survival time ?↑	Major infractions ? ↓	Traveled distance ?↑	Lateral deviation ? ↓
1	Bea Baselines 🐤	`1707`	1/1	minimal-agent-full	2.	0.	0.62	0.01

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

Challenge Sanity check

Very short “challenge”. Sanity check.

#1 user5 `1869`	#1 Bea Baselines `1812`	#1 Bea Baselines `1787`

copy link	copy link	copy link

Rank (user)	User	Submission	complete	User label	Survival time ?↑	Major infractions ? ↓	Traveled distance ?↑	Lateral deviation ? ↓
1	user5	`1869`	1/1	template-random	2.	0.	0.62	0.01
2	Bea Baselines 🐤	`1812`	1/1	template-ros	2.	0.	0.62	0.01
-	Bea Baselines 🐤	`1787`	1/1	template-random	2.	0.	0.62	0.01
-	Bea Baselines 🐤	`1762`	1/1	template-pytorch	2.	0.	0.62	0.01
-	Bea Baselines 🐤	`1737`	1/1	minimal-agent	2.	0.	0.62	0.01
-	Bea Baselines 🐤	`1667`	1/1	baseline-duckietown	2.	0.	0.62	0.01
-	Bea Baselines 🐤	`1642`	1/1	baseline-duckietown-ml	2.	0.	0.62	0.01
-	Bea Baselines 🐤	`805`	1/1	challenge-aido_LF-baseline-duckietown:daffy-staging:06423b9f834b346c04889a28f919db3673a8d2c1	2.	0.	0.62	0.01
-	Bea Baselines 🐤	`804`	1/1	challenge-aido_LF-template-ros:daffy-staging:3d125842a05be049b6ee6fe0059908c18c57ebb2	2.	0.	0.62	0.01
-	Bea Baselines 🐤	`803`	1/1	challenge-aido_LF-template-ros:daffy-staging:3b1546b6959e8affea32711104eb83074ec984ea	2.	0.	0.62	0.01

See also the extended leaderboard.

This is a validation challenge: the output is visible to everybody. This challenge is CLOSED for submissions.

These are support challenges. There is no prize for them for AI-DO 5.

These challenges are the full-state information version as the other ones.

Statistics

11	users
72	challenges
1751	submissions
246012	evaluation jobs