Duckietown Challenges Home Challenges Submissions

Submission 542

Submission542
Competingyes
Challengeaido-LFVI_multi-sim-testing
UserBea Baselines 🐤
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 17058 sim-1of4: 17062 sim-2of4: 17060 sim-3of4: 17056
Next
User labelbaseline-dagger-pytorch
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

The highlights are available only to the owner and the admins.

The highlights are available only to the owner and the admins.

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
17062sim-1of4successyes0:25:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median6.469323635565996
deviation-center-line_median4.3143707239626


other stats
deviation-center-line_max7.310521768919497
deviation-center-line_mean4.814337196931958
deviation-center-line_min3.3180855708831354
deviation-heading_max28.327641092414815
deviation-heading_mean19.158021927773344
deviation-heading_median17.18986465543658
deviation-heading_min13.924717307805402
distance-from-start_max3.259907041099038
distance-from-start_mean1.9328320676150583
distance-from-start_median2.003233765591634
distance-from-start_min0.464953698177928
driven_any_max10.683513621314466
driven_any_mean6.390438341258884
driven_any_median7.196585458744389
driven_any_min0.4850688262322927
driven_lanedir_consec_max9.728056709458372
driven_lanedir_consec_mean5.781293297136001
driven_lanedir_consec_min0.4584692079536401
driven_lanedir_max10.089690948759772
driven_lanedir_mean6.049094264497043
driven_lanedir_median6.8241084506373815
driven_lanedir_min0.4584692079536401
in-drivable-lane_max0.29999999999998295
in-drivable-lane_mean0.07499999999999574
in-drivable-lane_min0.0
per-episodes
details{"4way-000-ego0": {"driven_any": 0.4850688262322927, "survival_time": 59.99999999999873, "driven_lanedir": 0.4584692079536401, "in-drivable-lane": 0.0, "deviation-heading": 19.74992981274655, "distance-from-start": 0.464953698177928, "deviation-center-line": 7.310521768919497, "driven_lanedir_consec": 0.4584692079536401}, "4way-000-ego1": {"driven_any": 3.793757079890853, "survival_time": 59.99999999999873, "driven_lanedir": 3.62270018335154, "in-drivable-lane": 0.0, "deviation-heading": 28.327641092414815, "distance-from-start": 0.973574810162383, "deviation-center-line": 5.139649988648461, "driven_lanedir_consec": 3.62270018335154}, "4way-000-ego2": {"driven_any": 10.683513621314466, "survival_time": 59.99999999999873, "driven_lanedir": 10.089690948759772, "in-drivable-lane": 0.29999999999998295, "deviation-heading": 13.924717307805402, "distance-from-start": 3.0328927210208847, "deviation-center-line": 3.3180855708831354, "driven_lanedir_consec": 9.315947087780453}, "4way-000-ego3": {"driven_any": 10.599413837597924, "survival_time": 59.99999999999873, "driven_lanedir": 10.025516717923225, "in-drivable-lane": 0.0, "deviation-heading": 14.629799498126603, "distance-from-start": 3.259907041099038, "deviation-center-line": 3.4890914592767404, "driven_lanedir_consec": 9.728056709458372}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
17060sim-2of4successyes0:01:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
17058sim-0of4successyes0:23:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median1.56589238522859
deviation-center-line_median3.802804521997847


other stats
deviation-center-line_max5.866814890013379
deviation-center-line_mean3.469955024185722
deviation-center-line_min0.40739616273381496
deviation-heading_max30.359343508168372
deviation-heading_mean18.20296961498894
deviation-heading_median20.078524639617353
deviation-heading_min2.295485672552684
distance-from-start_max3.2425757479733353
distance-from-start_mean1.7343315259887837
distance-from-start_median1.398540741269016
distance-from-start_min0.8976688734437688
driven_any_max10.521229500607122
driven_any_mean3.7779677848550577
driven_any_median1.665398985053879
driven_any_min1.2598436687053487
driven_lanedir_consec_max9.420735207041629
driven_lanedir_consec_mean3.439789095590744
driven_lanedir_consec_min1.206636404864171
driven_lanedir_max10.014922925203486
driven_lanedir_mean3.589984149205945
driven_lanedir_median1.5691886333780625
driven_lanedir_min1.206636404864171
in-drivable-lane_max49.99999999999872
in-drivable-lane_mean12.49999999999968
in-drivable-lane_min0.0
per-episodes
details{"udem-000-ego0": {"driven_any": 10.521229500607122, "survival_time": 59.99999999999873, "driven_lanedir": 10.014922925203486, "in-drivable-lane": 0.0, "deviation-heading": 13.950459094911098, "distance-from-start": 3.2425757479733353, "deviation-center-line": 3.089851631926232, "driven_lanedir_consec": 9.420735207041629}, "udem-000-ego1": {"driven_any": 1.2598436687053487, "survival_time": 59.99999999999873, "driven_lanedir": 1.206636404864171, "in-drivable-lane": 0.0, "deviation-heading": 30.359343508168372, "distance-from-start": 0.8976688734437688, "deviation-center-line": 5.866814890013379, "driven_lanedir_consec": 1.206636404864171}, "udem-000-ego2": {"driven_any": 1.6298682034883272, "survival_time": 59.99999999999873, "driven_lanedir": 1.507997066958198, "in-drivable-lane": 49.99999999999872, "deviation-heading": 2.295485672552684, "distance-from-start": 1.1672129852216788, "deviation-center-line": 0.40739616273381496, "driven_lanedir_consec": 1.5019980779466138}, "udem-000-ego3": {"driven_any": 1.700929766619431, "survival_time": 59.99999999999873, "driven_lanedir": 1.6303801997979266, "in-drivable-lane": 0.0, "deviation-heading": 26.20659018432361, "distance-from-start": 1.629868497316353, "deviation-center-line": 4.515757412069462, "driven_lanedir_consec": 1.6297866925105662}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
17056sim-3of4successyes0:01:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
15921sim-0of4successno0:16:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
15920sim-1of4successno0:14:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
15919sim-2of4successno0:01:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
15918sim-3of4successno0:01:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14848sim-0of4failedno0:01:03
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14847sim-2of4successno0:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14845sim-3of4successno0:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14843sim-1of4successno0:20:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13712sim-2of4successno0:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13710sim-3of4successno0:01:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13709sim-0of4errorno0:01:13
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13708sim-1of4errorno0:01:16
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12348sim-1of4successno0:19:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12347sim-0of4successno0:19:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12346sim-3of4successno0:01:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12345sim-2of4successno0:01:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7248sim-2of4successno0:01:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7247sim-1of4successno0:15:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7240sim-0of4successno0:15:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7239sim-3of4successno0:01:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6058sim-2of4failedno0:03:08
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6057sim-0of4failedno0:03:13
The container "solut [...]
The container "solution-ego1" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6056sim-1of4failedno0:03:19
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6053sim-3of4failedno0:03:40
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4453sim-3of4successno0:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4450sim-2of4successno0:04:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4447sim-1of4successno0:09:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4446sim-0of4successno0:14:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible