Duckietown Challenges Home Challenges Submissions

Evaluator 117

ID117
evaluatorArchimede-stage-05-6.2.50
owneruser1000001
machineArchimede-stage
processArchimede-stage-05-6.2.50
version6.2.50
first heard
last heard
statusinactive
# evaluating
# success3 13988
# timeout1 14007
# failed3 13992
# error
# aborted
# host-error
arm0
x86_641
Mac0
gpu available1
Number of processors40
Processor frequency (MHz)2.1 GHz
Free % of processors50%
RAM total (MB)376.6 GB
RAM free (MB)259.4 GB
Disk (MB)74209.1 GB
Disk available (MB)63739.1 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, ETH_small_intersect
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
140071194Bea Baselines 🐤template-pytorchlx22-state-estimationsimtimeoutyesArchimede-stage-05-6.2.50----No reset possible
140011199Bea Baselines 🐤template-roslx22-modconsimsuccessyesArchimede-stage-05-6.2.500:04:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
in-drivable-lane_median7.274999999999991
deviation-center-line_median0.05025046181523703
driven_lanedir_consec_median0.12494310983546296
survival_time_median7.799999999999998


other stats
deviation-center-line_max0.10050092363047404
deviation-center-line_mean0.05025046181523703
deviation-center-line_min0.0
deviation-heading_max0.7365119801843227
deviation-heading_mean0.36825599009216137
deviation-heading_median0.36825599009216137
deviation-heading_min0.0
distance-from-start_max2.4975922035162244
distance-from-start_mean1.8887023431482757
distance-from-start_median1.8887023431482757
distance-from-start_min1.2798124827803266
driven_any_max2.618971401449815
driven_any_mean1.9805182230111156
driven_any_median1.9805182230111156
driven_any_min1.3420650445724165
driven_lanedir_consec_max0.24988621967092595
driven_lanedir_consec_mean0.12494310983546296
driven_lanedir_consec_min0.0
driven_lanedir_max0.24988621967092595
driven_lanedir_mean0.12494310983546296
driven_lanedir_median0.12494310983546296
driven_lanedir_min0.0
in-drivable-lane_max8.999999999999993
in-drivable-lane_mean7.274999999999991
in-drivable-lane_min5.549999999999988
per-episodes
details{"LF-full-loop-000-ego0": {"driven_any": 2.618971401449815, "survival_time": 10.050000000000008, "driven_lanedir": 0.24988621967092595, "in-drivable-lane": 8.999999999999993, "deviation-heading": 0.7365119801843227, "distance-from-start": 2.4975922035162244, "deviation-center-line": 0.10050092363047404, "driven_lanedir_consec": 0.24988621967092595}, "LF-full-loop-001-ego0": {"driven_any": 1.3420650445724165, "survival_time": 5.549999999999988, "driven_lanedir": 0.0, "in-drivable-lane": 5.549999999999988, "deviation-heading": 0.0, "distance-from-start": 1.2798124827803266, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max10.050000000000008
survival_time_mean7.799999999999998
survival_time_min5.549999999999988
No reset possible
139971201Bea Baselines 🐤template-roslx22-visservoingsimsuccessyesArchimede-stage-05-6.2.500:03:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
in-drivable-lane_median3.17499999999999
deviation-center-line_median0.1007538168388081
driven_lanedir_consec_median0.4872921902768395
survival_time_median5.549999999999988


other stats
deviation-center-line_max0.12479161690404578
deviation-center-line_mean0.1007538168388081
deviation-center-line_min0.07671601677357044
deviation-heading_max1.2605363175599835
deviation-heading_mean0.8712578939381439
deviation-heading_median0.8712578939381439
deviation-heading_min0.48197947031630417
distance-from-start_max1.4284872860746782
distance-from-start_mean1.3160755884422677
distance-from-start_median1.3160755884422677
distance-from-start_min1.2036638908098565
driven_any_max1.4985839119668376
driven_any_mean1.367294398162346
driven_any_median1.367294398162346
driven_any_min1.2360048843578546
driven_lanedir_consec_max0.709394736501356
driven_lanedir_consec_mean0.4872921902768395
driven_lanedir_consec_min0.26518964405232304
driven_lanedir_max0.709394736501356
driven_lanedir_mean0.4872921902768395
driven_lanedir_median0.4872921902768395
driven_lanedir_min0.26518964405232304
in-drivable-lane_max3.9499999999999886
in-drivable-lane_mean3.17499999999999
in-drivable-lane_min2.3999999999999915
per-episodes
details{"LF-small-loop-000-ego0": {"driven_any": 1.2360048843578546, "survival_time": 5.1999999999999895, "driven_lanedir": 0.709394736501356, "in-drivable-lane": 2.3999999999999915, "deviation-heading": 0.48197947031630417, "distance-from-start": 1.2036638908098565, "deviation-center-line": 0.07671601677357044, "driven_lanedir_consec": 0.709394736501356}, "LF-small-loop-001-ego0": {"driven_any": 1.4985839119668376, "survival_time": 5.899999999999987, "driven_lanedir": 0.26518964405232304, "in-drivable-lane": 3.9499999999999886, "deviation-heading": 1.2605363175599835, "distance-from-start": 1.4284872860746782, "deviation-center-line": 0.12479161690404578, "driven_lanedir_consec": 0.26518964405232304}}
simulation-passed1
survival_time_max5.899999999999987
survival_time_mean5.549999999999988
survival_time_min5.1999999999999895
No reset possible
139961205Bea Baselines 🐤template-tensorflowlx22-visservoingsimfailedyesArchimede-stage-05-6.2.500:01:35
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 298, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 307, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Getting agent protocol
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
139941204Bea Baselines 🐤template-tensorflowlx22-objdetsimfailedyesArchimede-stage-05-6.2.500:01:20
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 298, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 307, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Getting agent protocol
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
139921206Bea Baselines 🐤template-tensorflowlx22-state-estimationsimfailedyesArchimede-stage-05-6.2.500:02:49
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 298, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 307, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Getting agent protocol
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13988785Andrea Danieletemplate-rosaido-LF-sim-validationsim-0of4successnoArchimede-stage-05-6.2.500:06:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.3757647780505371
survival_time_median7.499999999999981
deviation-center-line_median0.17973532373638276
in-drivable-lane_median5.299999999999984


other stats
deviation-center-line_max0.17973532373638276
deviation-center-line_mean0.17973532373638276
deviation-center-line_min0.17973532373638276
deviation-heading_max1.5754684707878093
deviation-heading_mean1.5754684707878093
deviation-heading_median1.5754684707878093
deviation-heading_min1.5754684707878093
distance-from-start_max1.8673834251416923
distance-from-start_mean1.8673834251416923
distance-from-start_median1.8673834251416923
distance-from-start_min1.8673834251416923
driven_any_max1.8833113778288255
driven_any_mean1.8833113778288255
driven_any_median1.8833113778288255
driven_any_min1.8833113778288255
driven_lanedir_consec_max0.3757647780505371
driven_lanedir_consec_mean0.3757647780505371
driven_lanedir_consec_min0.3757647780505371
driven_lanedir_max0.3757647780505371
driven_lanedir_mean0.3757647780505371
driven_lanedir_median0.3757647780505371
driven_lanedir_min0.3757647780505371
in-drivable-lane_max5.299999999999984
in-drivable-lane_mean5.299999999999984
in-drivable-lane_min5.299999999999984
per-episodes
details{"zigzag-000-ego0": {"driven_any": 1.8833113778288255, "survival_time": 7.499999999999981, "driven_lanedir": 0.3757647780505371, "in-drivable-lane": 5.299999999999984, "deviation-heading": 1.5754684707878093, "distance-from-start": 1.8673834251416923, "deviation-center-line": 0.17973532373638276, "driven_lanedir_consec": 0.3757647780505371}}
simulation-passed1
survival_time_max7.499999999999981
survival_time_mean7.499999999999981
survival_time_min7.499999999999981
No reset possible