Duckietown Challenges Home Challenges Submissions

Evaluator 52

ID52
evaluatorArchimede-stage-03
owneruser1000001
machineArchimede-stage
processArchimede-stage-03
version6.2.42
first heard
last heard
statusinactive
# evaluating
# success2 7921
# timeout1 7928
# failed1 7917
# error
# aborted
# host-error
arm0
x86_641
Mac0
gpu available1
Number of processors40
Processor frequency (MHz)2.1 GHz
Free % of processors49%
RAM total (MB)376.6 GB
RAM free (MB)241.6 GB
Disk (MB)74209.1 GB
Disk available (MB)63576.0 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, ETH_small_intersect
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
7928526Bea Baselines 🐤baseline-behavior-cloningaido-LF-sim-testingsim-2of4timeoutnoArchimede-stage-03----No reset possible
7925658Bea Baselines 🐤template-pytorchaido-LF-sim-testingsim-3of4successnoArchimede-stage-030:05:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.8724808544643783
survival_time_median23.300000000000196
deviation-center-line_median0.4305016267821178
in-drivable-lane_median18.650000000000176


other stats
deviation-center-line_max0.4305016267821178
deviation-center-line_mean0.4305016267821178
deviation-center-line_min0.4305016267821178
deviation-heading_max2.123438787321369
deviation-heading_mean2.123438787321369
deviation-heading_median2.123438787321369
deviation-heading_min2.123438787321369
distance-from-start_max0.7348309202471706
distance-from-start_mean0.7348309202471706
distance-from-start_median0.7348309202471706
distance-from-start_min0.7348309202471706
driven_any_max5.1907218116173555
driven_any_mean5.1907218116173555
driven_any_median5.1907218116173555
driven_any_min5.1907218116173555
driven_lanedir_consec_max0.8724808544643783
driven_lanedir_consec_mean0.8724808544643783
driven_lanedir_consec_min0.8724808544643783
driven_lanedir_max0.8724808544643783
driven_lanedir_mean0.8724808544643783
driven_lanedir_median0.8724808544643783
driven_lanedir_min0.8724808544643783
in-drivable-lane_max18.650000000000176
in-drivable-lane_mean18.650000000000176
in-drivable-lane_min18.650000000000176
per-episodes
details{"autolab-000-ego0": {"driven_any": 5.1907218116173555, "survival_time": 23.300000000000196, "driven_lanedir": 0.8724808544643783, "in-drivable-lane": 18.650000000000176, "deviation-heading": 2.123438787321369, "distance-from-start": 0.7348309202471706, "deviation-center-line": 0.4305016267821178, "driven_lanedir_consec": 0.8724808544643783}}
simulation-passed1
survival_time_max23.300000000000196
survival_time_mean23.300000000000196
survival_time_min23.300000000000196
No reset possible
7921715Bea Baselines 🐤template-randomaido-LF-sim-testingsim-2of4successnoArchimede-stage-030:03:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.026068993087872805
survival_time_median11.950000000000037
deviation-center-line_median0.3267562024225741
in-drivable-lane_median8.500000000000023


other stats
deviation-center-line_max0.3267562024225741
deviation-center-line_mean0.3267562024225741
deviation-center-line_min0.3267562024225741
deviation-heading_max2.5569354764043557
deviation-heading_mean2.5569354764043557
deviation-heading_median2.5569354764043557
deviation-heading_min2.5569354764043557
distance-from-start_max0.6291689351090229
distance-from-start_mean0.6291689351090229
distance-from-start_median0.6291689351090229
distance-from-start_min0.6291689351090229
driven_any_max2.61737903576574
driven_any_mean2.61737903576574
driven_any_median2.61737903576574
driven_any_min2.61737903576574
driven_lanedir_consec_max0.026068993087872805
driven_lanedir_consec_mean0.026068993087872805
driven_lanedir_consec_min0.026068993087872805
driven_lanedir_max0.029415931241439264
driven_lanedir_mean0.029415931241439264
driven_lanedir_median0.029415931241439264
driven_lanedir_min0.029415931241439264
in-drivable-lane_max8.500000000000023
in-drivable-lane_mean8.500000000000023
in-drivable-lane_min8.500000000000023
per-episodes
details{"loop-000-ego0": {"driven_any": 2.61737903576574, "survival_time": 11.950000000000037, "driven_lanedir": 0.029415931241439264, "in-drivable-lane": 8.500000000000023, "deviation-heading": 2.5569354764043557, "distance-from-start": 0.6291689351090229, "deviation-center-line": 0.3267562024225741, "driven_lanedir_consec": 0.026068993087872805}}
simulation-passed1
survival_time_max11.950000000000037
survival_time_mean11.950000000000037
survival_time_min11.950000000000037
No reset possible
7917672Bea Baselines 🐤template-tensorflowaido-LF-sim-testingsim-0of4failednoArchimede-stage-030:02:53
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 298, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 307, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Getting agent protocol
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible