Duckietown Challenges Home Challenges Submissions

Evaluator 67

ID67
evaluatordavinci-stage-02
owneruser1000001
machinedavinci-stage
processdavinci-stage-02
version6.2.43
first heard
last heard
statusinactive
# evaluating
# success6 8100
# timeout1 8226
# failed1 8093
# error
# aborted
# host-error1 8198
arm0
x86_641
Mac0
gpu available1
Number of processors64
Processor frequency (MHz)2.8 GHz
Free % of processors57%
RAM total (MB)251.6 GB
RAM free (MB)213.8 GB
Disk (MB)491.1 GB
Disk available (MB)283.9 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, ETH_small_intersect
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
8226875Bea Baselines 🐤template-pytorchaido-LF-sim-testingsim-2of4timeoutnodavinci-stage-02----No reset possible
8218875Bea Baselines 🐤template-pytorchaido-LF-sim-testingsim-3of4successnodavinci-stage-020:09:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6165195742437861
survival_time_median59.99999999999873
deviation-center-line_median2.15100308381559
in-drivable-lane_median38.94999999999909


other stats
deviation-center-line_max2.15100308381559
deviation-center-line_mean2.15100308381559
deviation-center-line_min2.15100308381559
deviation-heading_max17.85962790155537
deviation-heading_mean17.85962790155537
deviation-heading_median17.85962790155537
deviation-heading_min17.85962790155537
distance-from-start_max0.32775698442258777
distance-from-start_mean0.32775698442258777
distance-from-start_median0.32775698442258777
distance-from-start_min0.32775698442258777
driven_any_max5.145380951573823
driven_any_mean5.145380951573823
driven_any_median5.145380951573823
driven_any_min5.145380951573823
driven_lanedir_consec_max0.6165195742437861
driven_lanedir_consec_mean0.6165195742437861
driven_lanedir_consec_min0.6165195742437861
driven_lanedir_max1.0470675069455817
driven_lanedir_mean1.0470675069455817
driven_lanedir_median1.0470675069455817
driven_lanedir_min1.0470675069455817
in-drivable-lane_max38.94999999999909
in-drivable-lane_mean38.94999999999909
in-drivable-lane_min38.94999999999909
per-episodes
details{"zigzag-000-ego0": {"driven_any": 5.145380951573823, "survival_time": 59.99999999999873, "driven_lanedir": 1.0470675069455817, "in-drivable-lane": 38.94999999999909, "deviation-heading": 17.85962790155537, "distance-from-start": 0.32775698442258777, "deviation-center-line": 2.15100308381559, "driven_lanedir_consec": 0.6165195742437861}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
8213875Bea Baselines 🐤template-pytorchaido-LF-sim-testingsim-0of4successnodavinci-stage-020:01:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.29639679717001366
survival_time_median8.14999999999998
deviation-center-line_median0.19133956133056915
in-drivable-lane_median5.899999999999984


other stats
deviation-center-line_max0.19133956133056915
deviation-center-line_mean0.19133956133056915
deviation-center-line_min0.19133956133056915
deviation-heading_max1.8211069668974944
deviation-heading_mean1.8211069668974944
deviation-heading_median1.8211069668974944
deviation-heading_min1.8211069668974944
distance-from-start_max0.6451911104636501
distance-from-start_mean0.6451911104636501
distance-from-start_median0.6451911104636501
distance-from-start_min0.6451911104636501
driven_any_max1.879747568244325
driven_any_mean1.879747568244325
driven_any_median1.879747568244325
driven_any_min1.879747568244325
driven_lanedir_consec_max0.29639679717001366
driven_lanedir_consec_mean0.29639679717001366
driven_lanedir_consec_min0.29639679717001366
driven_lanedir_max0.29639679717001366
driven_lanedir_mean0.29639679717001366
driven_lanedir_median0.29639679717001366
driven_lanedir_min0.29639679717001366
in-drivable-lane_max5.899999999999984
in-drivable-lane_mean5.899999999999984
in-drivable-lane_min5.899999999999984
per-episodes
details{"autolab-000-ego0": {"driven_any": 1.879747568244325, "survival_time": 8.14999999999998, "driven_lanedir": 0.29639679717001366, "in-drivable-lane": 5.899999999999984, "deviation-heading": 1.8211069668974944, "distance-from-start": 0.6451911104636501, "deviation-center-line": 0.19133956133056915, "driven_lanedir_consec": 0.29639679717001366}}
simulation-passed1
survival_time_max8.14999999999998
survival_time_mean8.14999999999998
survival_time_min8.14999999999998
No reset possible
8203876Bea Baselines 🐤template-pytorchaido-LFVI_multi-sim-testingsim-3of4successnodavinci-stage-020:02:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
8198876Bea Baselines 🐤template-pytorchaido-LFVI_multi-sim-testingsim-1of4host-errornodavinci-stage-020:01:13
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/auth

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 36, in docker_login
    client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e) from e
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 39, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation) from e
docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/auth: Internal Server Error ("Get https://registry-stage2.duckietown.org/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 863, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1065, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 64, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 42, in docker_login
    raise DockerLoginError(f"Could not login to {registry!r} as {docker_username!r} {pwd}") from e
duckietown_build_utils.docker_logging_in.DockerLoginError: Could not login to 'registry-stage2.duckietown.org' as 'duckietowndaemon' 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
8146877Bea Baselines 🐤template-pytorchaido-LFV_multi-sim-testingsim-3of4successnodavinci-stage-020:25:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median42.499999999998906
driven_lanedir_consec_median1.149727583107605
deviation-center-line_median1.5288205034462348


other stats
deviation-center-line_max3.8601322789364434
deviation-center-line_mean1.8382631321931528
deviation-center-line_min0.4352792429436976
deviation-heading_max40.8856998046741
deviation-heading_mean17.964725178779137
deviation-heading_median12.959571995067034
deviation-heading_min5.054056920308383
distance-from-start_max0.8214792196813558
distance-from-start_mean0.3815964698570216
distance-from-start_median0.34364048218811655
distance-from-start_min0.017625695370497522
driven_any_max7.358980988129897
driven_any_mean3.6207374208149967
driven_any_median3.5531706643508514
driven_any_min0.017627366428387588
driven_lanedir_consec_max1.6854013240921597
driven_lanedir_consec_mean0.9996122729475508
driven_lanedir_consec_min0.013592601482833366
driven_lanedir_max1.6854013240921597
driven_lanedir_mean0.9996122729475508
driven_lanedir_median1.149727583107605
driven_lanedir_min0.013592601482833366
in-drivable-lane_max53.09999999999873
in-drivable-lane_mean34.52499999999914
in-drivable-lane_min0.0
per-episodes
details{"zigzag-000-ego0": {"driven_any": 5.775036417496241, "survival_time": 59.99999999999873, "driven_lanedir": 1.6849009661694954, "in-drivable-lane": 49.99999999999865, "deviation-heading": 5.520379365852265, "distance-from-start": 0.5332894188725494, "deviation-center-line": 0.9009286046809304, "driven_lanedir_consec": 1.6849009661694954}, "zigzag-000-ego1": {"driven_any": 1.3313049112054616, "survival_time": 59.99999999999873, "driven_lanedir": 0.6145542000457147, "in-drivable-lane": 53.09999999999873, "deviation-heading": 5.054056920308383, "distance-from-start": 0.15399154550368366, "deviation-center-line": 0.4352792429436976, "driven_lanedir_consec": 0.6145542000457147}, "zigzag-000-ego2": {"driven_any": 7.358980988129897, "survival_time": 59.99999999999873, "driven_lanedir": 1.6854013240921597, "in-drivable-lane": 34.99999999999917, "deviation-heading": 20.398764624281803, "distance-from-start": 0.8214792196813558, "deviation-center-line": 2.156712402211539, "driven_lanedir_consec": 1.6854013240921597}, "zigzag-000-ego3": {"driven_any": 0.017627366428387588, "survival_time": 59.99999999999873, "driven_lanedir": 0.013592601482833366, "in-drivable-lane": 0.0, "deviation-heading": 40.8856998046741, "distance-from-start": 0.017625695370497522, "deviation-center-line": 3.8601322789364434, "driven_lanedir_consec": 0.013592601482833366}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
8113878Bea Baselines 🐤template-pytorchaido-LFP-sim-testingsim-1of4successnodavinci-stage-020:11:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median30.599999999999355
driven_lanedir_consec_median0.8971847843718168
deviation-center-line_median1.7967176566615473


other stats
deviation-center-line_max1.7967176566615473
deviation-center-line_mean1.7967176566615473
deviation-center-line_min1.7967176566615473
deviation-heading_max22.942273763172334
deviation-heading_mean22.942273763172334
deviation-heading_median22.942273763172334
deviation-heading_min22.942273763172334
distance-from-start_max0.3101010804386363
distance-from-start_mean0.3101010804386363
distance-from-start_median0.3101010804386363
distance-from-start_min0.3101010804386363
driven_any_max3.091717091852439
driven_any_mean3.091717091852439
driven_any_median3.091717091852439
driven_any_min3.091717091852439
driven_lanedir_consec_max0.8971847843718168
driven_lanedir_consec_mean0.8971847843718168
driven_lanedir_consec_min0.8971847843718168
driven_lanedir_max0.8971847843718168
driven_lanedir_mean0.8971847843718168
driven_lanedir_median0.8971847843718168
driven_lanedir_min0.8971847843718168
in-drivable-lane_max30.599999999999355
in-drivable-lane_mean30.599999999999355
in-drivable-lane_min30.599999999999355
per-episodes
details{"loop-000-ego0": {"driven_any": 3.091717091852439, "survival_time": 59.99999999999873, "driven_lanedir": 0.8971847843718168, "in-drivable-lane": 30.599999999999355, "deviation-heading": 22.942273763172334, "distance-from-start": 0.3101010804386363, "deviation-center-line": 1.7967176566615473, "driven_lanedir_consec": 0.8971847843718168}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
8100882Bea Baselines 🐤template-randomaido-LFV-sim-validationsim-3of4successnodavinci-stage-020:06:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median18.400000000000126
in-drivable-lane_median9.6500000000001
driven_lanedir_consec_median0.2737818968396458
deviation-center-line_median0.799573856657706


other stats
deviation-center-line_max0.799573856657706
deviation-center-line_mean0.799573856657706
deviation-center-line_min0.799573856657706
deviation-heading_max6.366511388189685
deviation-heading_mean6.366511388189685
deviation-heading_median6.366511388189685
deviation-heading_min6.366511388189685
distance-from-start_max0.6736010379604951
distance-from-start_mean0.6736010379604951
distance-from-start_median0.6736010379604951
distance-from-start_min0.6736010379604951
driven_any_max4.170232497072811
driven_any_mean4.170232497072811
driven_any_median4.170232497072811
driven_any_min4.170232497072811
driven_lanedir_consec_max0.2737818968396458
driven_lanedir_consec_mean0.2737818968396458
driven_lanedir_consec_min0.2737818968396458
driven_lanedir_max0.2565970421753513
driven_lanedir_mean0.2565970421753513
driven_lanedir_median0.2565970421753513
driven_lanedir_min0.2565970421753513
in-drivable-lane_max9.6500000000001
in-drivable-lane_mean9.6500000000001
in-drivable-lane_min9.6500000000001
per-episodes
details{"zigzag-000-ego0": {"driven_any": 4.170232497072811, "survival_time": 18.400000000000126, "driven_lanedir": 0.2565970421753513, "in-drivable-lane": 9.6500000000001, "deviation-heading": 6.366511388189685, "distance-from-start": 0.6736010379604951, "deviation-center-line": 0.799573856657706, "driven_lanedir_consec": 0.2737818968396458}}
simulation-passed1
survival_time_max18.400000000000126
survival_time_mean18.400000000000126
survival_time_min18.400000000000126
No reset possible
8093881Bea Baselines 🐤template-tensorflowaido-LFV-sim-validationsim-0of4failednodavinci-stage-020:01:49
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 298, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 307, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Getting agent protocol
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible