Duckietown Challenges Home Challenges Submissions

Evaluator 3

ID3
evaluatornogpu-staging-spot-0-02
owneruser1000001
machinenogpu-staging-spot-0
processnogpu-staging-spot-0-02
version6.2.32
first heard
last heard
statusinactive
# evaluating
# success21 4
# timeout
# failed7 221
# error
# aborted4 205
# host-error22 25
arm0
x86_641
Mac0
gpu available0
Number of processors16
Processor frequency (MHz)0.0 GHz
Free % of processors99%
RAM total (MB)29.4 GB
RAM free (MB)7.4 GB
Disk (MB)969.3 GB
Disk available (MB)706.5 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, ETH_small_intersect
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
51870Bea Baselines 🐤template-rosaido-LFV-sim-validationsim-3of4successnonogpu-staging-spot-0-020:04:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median9.249999999999996
in-drivable-lane_median7.249999999999999
driven_lanedir_consec_median0.24754840400310285
deviation-center-line_median0.15653254911806752


other stats
deviation-center-line_max0.15653254911806752
deviation-center-line_mean0.15653254911806752
deviation-center-line_min0.15653254911806752
deviation-heading_max1.7581092013164898
deviation-heading_mean1.7581092013164898
deviation-heading_median1.7581092013164898
deviation-heading_min1.7581092013164898
distance-from-start_max2.229888428443249
distance-from-start_mean2.229888428443249
distance-from-start_median2.229888428443249
distance-from-start_min2.229888428443249
driven_any_max2.3724812416186025
driven_any_mean2.3724812416186025
driven_any_median2.3724812416186025
driven_any_min2.3724812416186025
driven_lanedir_consec_max0.24754840400310285
driven_lanedir_consec_mean0.24754840400310285
driven_lanedir_consec_min0.24754840400310285
driven_lanedir_max0.24754840400310285
driven_lanedir_mean0.24754840400310285
driven_lanedir_median0.24754840400310285
driven_lanedir_min0.24754840400310285
in-drivable-lane_max7.249999999999999
in-drivable-lane_mean7.249999999999999
in-drivable-lane_min7.249999999999999
per-episodes
details{"loop-000-ego0": {"driven_any": 2.3724812416186025, "survival_time": 9.249999999999996, "driven_lanedir": 0.24754840400310285, "in-drivable-lane": 7.249999999999999, "deviation-heading": 1.7581092013164898, "distance-from-start": 2.229888428443249, "deviation-center-line": 0.15653254911806752, "driven_lanedir_consec": 0.24754840400310285}}
simulation-passed1
survival_time_max9.249999999999996
survival_time_mean9.249999999999996
survival_time_min9.249999999999996
No reset possible
50871Bea Baselines 🐤template-rosaido-LFVI-sim-validationsim-1of4successnonogpu-staging-spot-0-020:02:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median2.3
in-drivable-lane_median0.0
driven_lanedir_consec_median0.3699312152389338
deviation-center-line_median0.1909732223612033


other stats
deviation-center-line_max0.1909732223612033
deviation-center-line_mean0.1909732223612033
deviation-center-line_min0.1909732223612033
deviation-heading_max0.6573164325690162
deviation-heading_mean0.6573164325690162
deviation-heading_median0.6573164325690162
deviation-heading_min0.6573164325690162
distance-from-start_max0.3807792714979932
distance-from-start_mean0.3807792714979932
distance-from-start_median0.3807792714979932
distance-from-start_min0.3807792714979932
driven_any_max0.38279872890564925
driven_any_mean0.38279872890564925
driven_any_median0.38279872890564925
driven_any_min0.38279872890564925
driven_lanedir_consec_max0.3699312152389338
driven_lanedir_consec_mean0.3699312152389338
driven_lanedir_consec_min0.3699312152389338
driven_lanedir_max0.3699312152389338
driven_lanedir_mean0.3699312152389338
driven_lanedir_median0.3699312152389338
driven_lanedir_min0.3699312152389338
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"udem-000-ego0": {"driven_any": 0.38279872890564925, "survival_time": 2.3, "driven_lanedir": 0.3699312152389338, "in-drivable-lane": 0.0, "deviation-heading": 0.6573164325690162, "distance-from-start": 0.3807792714979932, "deviation-center-line": 0.1909732223612033, "driven_lanedir_consec": 0.3699312152389338}}
simulation-passed1
survival_time_max2.3
survival_time_mean2.3
survival_time_min2.3
No reset possible
49573Bea Baselines 🐤template-rosaido-LF-sim-validationsim-2of4host-errornonogpu-staging-spot-0-020:03:31
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48675Bea Baselines 🐤template-rosaido-LFV_multi-sim-testingsim-2of4host-errornonogpu-staging-spot-0-020:02:36
The container "simul [...]
The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
47285Bea Baselines 🐤baseline-duckietownaido-LFVI-sim-validationsim-1of4successnonogpu-staging-spot-0-020:05:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median6.349999999999985
in-drivable-lane_median0.0
driven_lanedir_consec_median0.5776852069384593
deviation-center-line_median0.2634679505577645


other stats
deviation-center-line_max0.2634679505577645
deviation-center-line_mean0.2634679505577645
deviation-center-line_min0.2634679505577645
deviation-heading_max1.79454072279165
deviation-heading_mean1.79454072279165
deviation-heading_median1.79454072279165
deviation-heading_min1.79454072279165
distance-from-start_max0.6020795642998933
distance-from-start_mean0.6020795642998933
distance-from-start_median0.6020795642998933
distance-from-start_min0.6020795642998933
driven_any_max0.6982966458223847
driven_any_mean0.6982966458223847
driven_any_median0.6982966458223847
driven_any_min0.6982966458223847
driven_lanedir_consec_max0.5776852069384593
driven_lanedir_consec_mean0.5776852069384593
driven_lanedir_consec_min0.5776852069384593
driven_lanedir_max0.6709785362861416
driven_lanedir_mean0.6709785362861416
driven_lanedir_median0.6709785362861416
driven_lanedir_min0.6709785362861416
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"udem-000-ego0": {"driven_any": 0.6982966458223847, "survival_time": 6.349999999999985, "driven_lanedir": 0.6709785362861416, "in-drivable-lane": 0.0, "deviation-heading": 1.79454072279165, "distance-from-start": 0.6020795642998933, "deviation-center-line": 0.2634679505577645, "driven_lanedir_consec": 0.5776852069384593}}
simulation-passed1
survival_time_max6.349999999999985
survival_time_mean6.349999999999985
survival_time_min6.349999999999985
No reset possible
45785Bea Baselines 🐤baseline-duckietownaido-LFVI-sim-validationsim-1of4successnonogpu-staging-spot-0-020:06:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median7.299999999999982
in-drivable-lane_median3.4499999999999877
driven_lanedir_consec_median0.42542206098028257
deviation-center-line_median0.115755895256779


other stats
deviation-center-line_max0.115755895256779
deviation-center-line_mean0.115755895256779
deviation-center-line_min0.115755895256779
deviation-heading_max1.0549830098799686
deviation-heading_mean1.0549830098799686
deviation-heading_median1.0549830098799686
deviation-heading_min1.0549830098799686
distance-from-start_max0.4260489355604566
distance-from-start_mean0.4260489355604566
distance-from-start_median0.4260489355604566
distance-from-start_min0.4260489355604566
driven_any_max0.8243235774044736
driven_any_mean0.8243235774044736
driven_any_median0.8243235774044736
driven_any_min0.8243235774044736
driven_lanedir_consec_max0.42542206098028257
driven_lanedir_consec_mean0.42542206098028257
driven_lanedir_consec_min0.42542206098028257
driven_lanedir_max0.42542206098028257
driven_lanedir_mean0.42542206098028257
driven_lanedir_median0.42542206098028257
driven_lanedir_min0.42542206098028257
in-drivable-lane_max3.4499999999999877
in-drivable-lane_mean3.4499999999999877
in-drivable-lane_min3.4499999999999877
per-episodes
details{"udem-000-ego0": {"driven_any": 0.8243235774044736, "survival_time": 7.299999999999982, "driven_lanedir": 0.42542206098028257, "in-drivable-lane": 3.4499999999999877, "deviation-heading": 1.0549830098799686, "distance-from-start": 0.4260489355604566, "deviation-center-line": 0.115755895256779, "driven_lanedir_consec": 0.42542206098028257}}
simulation-passed1
survival_time_max7.299999999999982
survival_time_mean7.299999999999982
survival_time_min7.299999999999982
No reset possible
45281Bea Baselines 🐤template-rosaido-LFVI_multi-sim-testingsim-2of4successnonogpu-staging-spot-0-020:01:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
44782Bea Baselines 🐤baseline-duckietownaido-LFV_multi-sim-validationsim-2of4host-errornonogpu-staging-spot-0-020:02:40
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
44282Bea Baselines 🐤baseline-duckietownaido-LFV_multi-sim-validationsim-0of4host-errornonogpu-staging-spot-0-020:02:00
The container "simul [...]
The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43983Bea Baselines 🐤baseline-duckietownaido-LFP-sim-validationsim-2of4host-errornonogpu-staging-spot-0-020:02:17
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43484Bea Baselines 🐤baseline-duckietownaido-LFV-sim-validationsim-0of4successnonogpu-staging-spot-0-020:03:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median9.099999999999994
in-drivable-lane_median1.149999999999996
driven_lanedir_consec_median0.8344566121174861
deviation-center-line_median0.518370960689641


other stats
deviation-center-line_max0.518370960689641
deviation-center-line_mean0.518370960689641
deviation-center-line_min0.518370960689641
deviation-heading_max3.0680999782110314
deviation-heading_mean3.0680999782110314
deviation-heading_median3.0680999782110314
deviation-heading_min3.0680999782110314
distance-from-start_max0.7428267775131446
distance-from-start_mean0.7428267775131446
distance-from-start_median0.7428267775131446
distance-from-start_min0.7428267775131446
driven_any_max1.0634335853703605
driven_any_mean1.0634335853703605
driven_any_median1.0634335853703605
driven_any_min1.0634335853703605
driven_lanedir_consec_max0.8344566121174861
driven_lanedir_consec_mean0.8344566121174861
driven_lanedir_consec_min0.8344566121174861
driven_lanedir_max0.8344566121174861
driven_lanedir_mean0.8344566121174861
driven_lanedir_median0.8344566121174861
driven_lanedir_min0.8344566121174861
in-drivable-lane_max1.149999999999996
in-drivable-lane_mean1.149999999999996
in-drivable-lane_min1.149999999999996
per-episodes
details{"loop2-000-ego0": {"driven_any": 1.0634335853703605, "survival_time": 9.099999999999994, "driven_lanedir": 0.8344566121174861, "in-drivable-lane": 1.149999999999996, "deviation-heading": 3.0680999782110314, "distance-from-start": 0.7428267775131446, "deviation-center-line": 0.518370960689641, "driven_lanedir_consec": 0.8344566121174861}}
simulation-passed1
survival_time_max9.099999999999994
survival_time_mean9.099999999999994
survival_time_min9.099999999999994
No reset possible
42287Bea Baselines 🐤baseline-duckietownaido-LF-sim-validationsim-2of4successnonogpu-staging-spot-0-020:03:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.1266667856462005
survival_time_median18.850000000000133
deviation-center-line_median0.8263917838800863
in-drivable-lane_median3.650000000000035


other stats
deviation-center-line_max0.8263917838800863
deviation-center-line_mean0.8263917838800863
deviation-center-line_min0.8263917838800863
deviation-heading_max4.17188736178881
deviation-heading_mean4.17188736178881
deviation-heading_median4.17188736178881
deviation-heading_min4.17188736178881
distance-from-start_max0.6966187371638793
distance-from-start_mean0.6966187371638793
distance-from-start_median0.6966187371638793
distance-from-start_min0.6966187371638793
driven_any_max2.351353727203276
driven_any_mean2.351353727203276
driven_any_median2.351353727203276
driven_any_min2.351353727203276
driven_lanedir_consec_max1.1266667856462005
driven_lanedir_consec_mean1.1266667856462005
driven_lanedir_consec_min1.1266667856462005
driven_lanedir_max1.7970494826664472
driven_lanedir_mean1.7970494826664472
driven_lanedir_median1.7970494826664472
driven_lanedir_min1.7970494826664472
in-drivable-lane_max3.650000000000035
in-drivable-lane_mean3.650000000000035
in-drivable-lane_min3.650000000000035
per-episodes
details{"autolab-000-ego0": {"driven_any": 2.351353727203276, "survival_time": 18.850000000000133, "driven_lanedir": 1.7970494826664472, "in-drivable-lane": 3.650000000000035, "deviation-heading": 4.17188736178881, "distance-from-start": 0.6966187371638793, "deviation-center-line": 0.8263917838800863, "driven_lanedir_consec": 1.1266667856462005}}
simulation-passed1
survival_time_max18.850000000000133
survival_time_mean18.850000000000133
survival_time_min18.850000000000133
No reset possible
41388Bea Baselines 🐤baseline-duckietownaido-LFVI_multi-sim-validationsim-1of4host-errornonogpu-staging-spot-0-020:08:24
The container "simul [...]
The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
383103Bea Baselines 🐤baseline-duckietown-mlaido-LFV_multi-sim-testingsim-2of4host-errornonogpu-staging-spot-0-020:20:04
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1245, in run_one
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 889, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 774, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36196Bea Baselines 🐤baseline-duckietown-mlaido-LFV_multi-sim-validationsim-3of4host-errornonogpu-staging-spot-0-020:15:12
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
35096Bea Baselines 🐤baseline-duckietown-mlaido-LFV_multi-sim-validationsim-0of4host-errornonogpu-staging-spot-0-020:05:34
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
34297Bea Baselines 🐤baseline-duckietown-mlaido-LFP-sim-validationsim-0of4successnonogpu-staging-spot-0-020:08:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median0.0
deviation-center-line_median0.8948943425962775


other stats
deviation-center-line_max0.8948943425962775
deviation-center-line_mean0.8948943425962775
deviation-center-line_min0.8948943425962775
deviation-heading_max32.249113233675025
deviation-heading_mean32.249113233675025
deviation-heading_median32.249113233675025
deviation-heading_min32.249113233675025
distance-from-start_max1.1102230246251563e-16
distance-from-start_mean1.1102230246251563e-16
distance-from-start_median1.1102230246251563e-16
distance-from-start_min1.1102230246251563e-16
driven_any_max1.3322676295501878e-13
driven_any_mean1.3322676295501878e-13
driven_any_median1.3322676295501878e-13
driven_any_min1.3322676295501878e-13
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"loop2-000-ego0": {"driven_any": 1.3322676295501878e-13, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "in-drivable-lane": 0.0, "deviation-heading": 32.249113233675025, "distance-from-start": 1.1102230246251563e-16, "deviation-center-line": 0.8948943425962775, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
324104Bea Baselines 🐤baseline-duckietown-mlaido-LFP-sim-testingsim-3of4host-errornonogpu-staging-spot-0-020:08:51
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
314102Bea Baselines 🐤baseline-duckietown-mlaido-LFVI_multi-sim-validationsim-1of4host-errornonogpu-staging-spot-0-020:05:20
The container "simul [...]
The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
305104Bea Baselines 🐤baseline-duckietown-mlaido-LFP-sim-testingsim-2of4successnonogpu-staging-spot-0-020:09:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median0.0
deviation-center-line_median0.40649934386321146


other stats
deviation-center-line_max0.40649934386321146
deviation-center-line_mean0.40649934386321146
deviation-center-line_min0.40649934386321146
deviation-heading_max40.967332753486225
deviation-heading_mean40.967332753486225
deviation-heading_median40.967332753486225
deviation-heading_min40.967332753486225
distance-from-start_max8.005932084973442e-16
distance-from-start_mean8.005932084973442e-16
distance-from-start_median8.005932084973442e-16
distance-from-start_min8.005932084973442e-16
driven_any_max9.607118501967829e-13
driven_any_mean9.607118501967829e-13
driven_any_median9.607118501967829e-13
driven_any_min9.607118501967829e-13
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"autolab-000-ego0": {"driven_any": 9.607118501967829e-13, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "in-drivable-lane": 0.0, "deviation-heading": 40.967332753486225, "distance-from-start": 8.005932084973442e-16, "deviation-center-line": 0.40649934386321146, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
280110Bea Baselines 🐤baseline-duckietown-mlaido-LFVI_multi-sim-testingsim-1of4abortednonogpu-staging-spot-0-020:19:20
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 391, in main
    stats = future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 426, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
275110Bea Baselines 🐤baseline-duckietown-mlaido-LFVI_multi-sim-testingsim-0of4host-errornonogpu-staging-spot-0-020:12:17
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
26831Bea Baselines 🐤minimal-agent-stateaido-LFVI_multi-state-sim-validationsim-0of4failednonogpu-staging-spot-0-020:01:15
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
26531Bea Baselines 🐤minimal-agent-stateaido-LFVI_multi-state-sim-validationsim-1of4failednonogpu-staging-spot-0-020:01:15
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
26232Bea Baselines 🐤minimal-agent-stateaido-LFV_multi-state-sim-validation61failednonogpu-staging-spot-0-020:01:15
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
25933Bea Baselines 🐤minimal-agent-stateaido-LFP-state-sim-validationsim-1of4failednonogpu-staging-spot-0-020:01:08
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
25334Bea Baselines 🐤minimal-agent-fullaido-LFP-full-sim-validationsim-2of4successnonogpu-staging-spot-0-020:11:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median1.0999999999999377
driven_lanedir_consec_median3.4858726845389234
deviation-center-line_median4.220405841276127


other stats
deviation-center-line_max4.220405841276127
deviation-center-line_mean4.220405841276127
deviation-center-line_min4.220405841276127
deviation-heading_max2.9174896176689957
deviation-heading_mean2.9174896176689957
deviation-heading_median2.9174896176689957
deviation-heading_min2.9174896176689957
distance-from-start_max2.8928218596598874
distance-from-start_mean2.8928218596598874
distance-from-start_median2.8928218596598874
distance-from-start_min2.8928218596598874
driven_any_max3.5465963319759606
driven_any_mean3.5465963319759606
driven_any_median3.5465963319759606
driven_any_min3.5465963319759606
driven_lanedir_consec_max3.4858726845389234
driven_lanedir_consec_mean3.4858726845389234
driven_lanedir_consec_min3.4858726845389234
driven_lanedir_max3.4858726845389234
driven_lanedir_mean3.4858726845389234
driven_lanedir_median3.4858726845389234
driven_lanedir_min3.4858726845389234
in-drivable-lane_max1.0999999999999377
in-drivable-lane_mean1.0999999999999377
in-drivable-lane_min1.0999999999999377
per-episodes
details{"autolab-000-ego0": {"driven_any": 3.5465963319759606, "survival_time": 59.99999999999873, "driven_lanedir": 3.4858726845389234, "in-drivable-lane": 1.0999999999999377, "deviation-heading": 2.9174896176689957, "distance-from-start": 2.8928218596598874, "deviation-center-line": 4.220405841276127, "driven_lanedir_consec": 3.4858726845389234}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
25035Bea Baselines 🐤minimal-agent-stateaido-LF-state-sim-testingsim-3of4failednonogpu-staging-spot-0-020:01:09
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
23538Bea Baselines 🐤minimal-agent-fullaido-LFVI_multi-full-sim-validationsim-1of4successnonogpu-staging-spot-0-020:32:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.9250000000000131
driven_lanedir_consec_median3.091330139165626
deviation-center-line_median4.331903062548665


other stats
deviation-center-line_max7.34422613121448
deviation-center-line_mean4.724617132020332
deviation-center-line_min2.8904362717695165
deviation-heading_max5.663266158111513
deviation-heading_mean4.199882642983283
deviation-heading_median3.814086890191848
deviation-heading_min3.508090633437924
distance-from-start_max2.492336237280679
distance-from-start_mean2.1292805692057786
distance-from-start_median2.293970973840236
distance-from-start_min1.4368440918619636
driven_any_max3.5795949962044755
driven_any_mean3.3129663158438403
driven_any_median3.2690876797298047
driven_any_min3.134094907711277
driven_lanedir_consec_max3.539825869913338
driven_lanedir_consec_mean2.7704276818433335
driven_lanedir_consec_min1.3592245791287445
driven_lanedir_max3.559551834837159
driven_lanedir_mean3.245375313750827
driven_lanedir_median3.1929309611501084
driven_lanedir_min3.0360874978659327
in-drivable-lane_max2.0000000000000284
in-drivable-lane_mean0.9625000000000136
in-drivable-lane_min0.0
per-episodes
details{"udem-000-ego0": {"driven_any": 3.5795949962044755, "survival_time": 59.99999999999873, "driven_lanedir": 3.559551834837159, "in-drivable-lane": 0.0, "deviation-heading": 3.735588143574178, "distance-from-start": 2.342236213319778, "deviation-center-line": 3.18141954591101, "driven_lanedir_consec": 3.539825869913338}, "udem-000-ego1": {"driven_any": 3.134094907711277, "survival_time": 59.99999999999873, "driven_lanedir": 3.0360874978659327, "in-drivable-lane": 1.8500000000000263, "deviation-heading": 3.508090633437924, "distance-from-start": 2.492336237280679, "deviation-center-line": 7.34422613121448, "driven_lanedir_consec": 3.0360874774253386}, "udem-000-ego2": {"driven_any": 3.3155875122219665, "survival_time": 59.99999999999873, "driven_lanedir": 3.295483754773894, "in-drivable-lane": 0.0, "deviation-heading": 3.8925856368095175, "distance-from-start": 2.2457057343606937, "deviation-center-line": 2.8904362717695165, "driven_lanedir_consec": 3.146572800905913}, "udem-000-ego3": {"driven_any": 3.2225878472376426, "survival_time": 59.99999999999873, "driven_lanedir": 3.0903781675263233, "in-drivable-lane": 2.0000000000000284, "deviation-heading": 5.663266158111513, "distance-from-start": 1.4368440918619636, "deviation-center-line": 5.48238657918632, "driven_lanedir_consec": 1.3592245791287445}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
23239Bea Baselines 🐤minimal-agent-stateaido-LFV_multi-state-sim-testingsim-0of4failednonogpu-staging-spot-0-020:01:32
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
23039Bea Baselines 🐤minimal-agent-stateaido-LFV_multi-state-sim-testingsim-3of4host-errornonogpu-staging-spot-0-020:01:27
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
22540Bea Baselines 🐤minimal-agent-fullaido-LF-full-sim-validationsim-0of4successnonogpu-staging-spot-0-020:04:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median27.500000000000256
in-drivable-lane_median1.1000000000000156
driven_lanedir_consec_median1.4937232631899848
deviation-center-line_median2.6227097047460513


other stats
deviation-center-line_max2.6227097047460513
deviation-center-line_mean2.6227097047460513
deviation-center-line_min2.6227097047460513
deviation-heading_max2.9200558502635636
deviation-heading_mean2.9200558502635636
deviation-heading_median2.9200558502635636
deviation-heading_min2.9200558502635636
distance-from-start_max1.2100420266422207
distance-from-start_mean1.2100420266422207
distance-from-start_median1.2100420266422207
distance-from-start_min1.2100420266422207
driven_any_max1.5545963269619445
driven_any_mean1.5545963269619445
driven_any_median1.5545963269619445
driven_any_min1.5545963269619445
driven_lanedir_consec_max1.4937232631899848
driven_lanedir_consec_mean1.4937232631899848
driven_lanedir_consec_min1.4937232631899848
driven_lanedir_max1.4937232631899848
driven_lanedir_mean1.4937232631899848
driven_lanedir_median1.4937232631899848
driven_lanedir_min1.4937232631899848
in-drivable-lane_max1.1000000000000156
in-drivable-lane_mean1.1000000000000156
in-drivable-lane_min1.1000000000000156
per-episodes
details{"loop2-000-ego0": {"driven_any": 1.5545963269619445, "survival_time": 27.500000000000256, "driven_lanedir": 1.4937232631899848, "in-drivable-lane": 1.1000000000000156, "deviation-heading": 2.9200558502635636, "distance-from-start": 1.2100420266422207, "deviation-center-line": 2.6227097047460513, "driven_lanedir_consec": 1.4937232631899848}}
simulation-passed1
survival_time_max27.500000000000256
survival_time_mean27.500000000000256
survival_time_min27.500000000000256
No reset possible
22341Bea Baselines 🐤minimal-agent-stateaido-LFP-state-sim-testingsim-2of4host-errornonogpu-staging-spot-0-020:00:22
Error while running [...]
Error while running Docker Compose:

Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
22141Bea Baselines 🐤minimal-agent-stateaido-LFP-state-sim-testingsim-1of4failednonogpu-staging-spot-0-020:01:08
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21443Bea Baselines 🐤minimal-agent-fullaido-LFV_multi-full-sim-testingsim-1of4abortednonogpu-staging-spot-0-020:40:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median1.92499999999992
driven_lanedir_consec_median1.6962974377539832
deviation-center-line_median5.666007792456419


other stats
deviation-center-line_max9.920539816203728
deviation-center-line_mean6.424810701440977
deviation-center-line_min4.446687404647345
deviation-heading_max10.760072313534046
deviation-heading_mean7.75849276745697
deviation-heading_median9.16270952173033
deviation-heading_min1.9484797128331763
distance-from-start_max2.9523651654056944
distance-from-start_mean1.7043556049652504
distance-from-start_median1.5711589276694409
distance-from-start_min0.7227393991164265
driven_any_max3.524085546529734
driven_any_mean2.8239999291942173
driven_any_median3.485157582681066
driven_any_min0.8015990048850018
driven_lanedir_consec_max3.348809855286584
driven_lanedir_consec_mean1.8837011378527109
driven_lanedir_consec_min0.7933998206162931
driven_lanedir_max3.3513486271099224
driven_lanedir_mean2.6742459522341138
driven_lanedir_median3.2761176806051204
driven_lanedir_min0.7933998206162931
in-drivable-lane_max4.299999999999898
in-drivable-lane_mean2.0374999999999344
in-drivable-lane_min0.0
per-episodes
details{"zigzag-000-ego0": {"driven_any": 3.5197332868688247, "survival_time": 59.99999999999873, "driven_lanedir": 3.3513486271099224, "in-drivable-lane": 1.9999999999998863, "deviation-heading": 9.476018293244582, "distance-from-start": 2.0738235372013043, "deviation-center-line": 4.446687404647345, "driven_lanedir_consec": 2.1795590385241024}, "zigzag-000-ego1": {"driven_any": 3.450581878493307, "survival_time": 59.99999999999873, "driven_lanedir": 3.203425505923657, "in-drivable-lane": 4.299999999999898, "deviation-heading": 8.849400750216077, "distance-from-start": 1.068494318137577, "deviation-center-line": 5.8992687515673525, "driven_lanedir_consec": 1.213035836983864}, "zigzag-000-ego2": {"driven_any": 0.8015990048850018, "survival_time": 59.99999999999873, "driven_lanedir": 0.7933998206162931, "in-drivable-lane": 0.0, "deviation-heading": 1.9484797128331763, "distance-from-start": 0.7227393991164265, "deviation-center-line": 9.920539816203728, "driven_lanedir_consec": 0.7933998206162931}, "zigzag-000-ego3": {"driven_any": 3.524085546529734, "survival_time": 59.99999999999873, "driven_lanedir": 3.348809855286584, "in-drivable-lane": 1.8499999999999537, "deviation-heading": 10.760072313534046, "distance-from-start": 2.9523651654056944, "deviation-center-line": 5.432746833345485, "driven_lanedir_consec": 3.348809855286584}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
20843Bea Baselines 🐤minimal-agent-fullaido-LFV_multi-full-sim-testingsim-1of4abortednonogpu-staging-spot-0-020:32:04
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20544Bea Baselines 🐤minimal-agent-fullaido-LFVI_multi-full-sim-testingsim-3of4abortednonogpu-staging-spot-0-020:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
20045Bea Baselines 🐤minimal-agent-fullaido-LF-full-sim-testingsim-3of4successnonogpu-staging-spot-0-020:10:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median2.0000000000000284
driven_lanedir_consec_median1.6903637727595648
deviation-center-line_median5.303135750931059


other stats
deviation-center-line_max5.303135750931059
deviation-center-line_mean5.303135750931059
deviation-center-line_min5.303135750931059
deviation-heading_max5.395040966687188
deviation-heading_mean5.395040966687188
deviation-heading_median5.395040966687188
deviation-heading_min5.395040966687188
distance-from-start_max1.390965525210542
distance-from-start_mean1.390965525210542
distance-from-start_median1.390965525210542
distance-from-start_min1.390965525210542
driven_any_max3.519587646027781
driven_any_mean3.519587646027781
driven_any_median3.519587646027781
driven_any_min3.519587646027781
driven_lanedir_consec_max1.6903637727595648
driven_lanedir_consec_mean1.6903637727595648
driven_lanedir_consec_min1.6903637727595648
driven_lanedir_max3.387502194996571
driven_lanedir_mean3.387502194996571
driven_lanedir_median3.387502194996571
driven_lanedir_min3.387502194996571
in-drivable-lane_max2.0000000000000284
in-drivable-lane_mean2.0000000000000284
in-drivable-lane_min2.0000000000000284
per-episodes
details{"loop-000-ego0": {"driven_any": 3.519587646027781, "survival_time": 59.99999999999873, "driven_lanedir": 3.387502194996571, "in-drivable-lane": 2.0000000000000284, "deviation-heading": 5.395040966687188, "distance-from-start": 1.390965525210542, "deviation-center-line": 5.303135750931059, "driven_lanedir_consec": 1.6903637727595648}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
19846Bea Baselines 🐤baseline-simple-yieldaido-LF-state-sim-validationsim-0of4successnonogpu-staging-spot-0-020:08:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median0.0
deviation-center-line_median0.8948975282353262


other stats
deviation-center-line_max0.8948975282353262
deviation-center-line_mean0.8948975282353262
deviation-center-line_min0.8948975282353262
deviation-heading_max32.249113233675025
deviation-heading_mean32.249113233675025
deviation-heading_median32.249113233675025
deviation-heading_min32.249113233675025
distance-from-start_max2.2377260456559053e-16
distance-from-start_mean2.2377260456559053e-16
distance-from-start_median2.2377260456559053e-16
distance-from-start_min2.2377260456559053e-16
driven_any_max2.685271254787069e-13
driven_any_mean2.685271254787069e-13
driven_any_median2.685271254787069e-13
driven_any_min2.685271254787069e-13
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"loop2-000-ego0": {"driven_any": 2.685271254787069e-13, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "in-drivable-lane": 0.0, "deviation-heading": 32.249113233675025, "distance-from-start": 2.2377260456559053e-16, "deviation-center-line": 0.8948975282353262, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
19448Bea Baselines 🐤baseline-simple-yieldaido-LFV_multi-state-sim-validation60successnonogpu-staging-spot-0-020:19:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.9000000000000005
driven_lanedir_consec_median2.6871102139101084
deviation-center-line_median0.3688105390557702


other stats
deviation-center-line_max2.8307333506918795
deviation-center-line_mean0.9738536136871948
deviation-center-line_min0.32706002594535993
deviation-heading_max29.971365220758347
deviation-heading_mean10.040890864200705
deviation-heading_median3.726210645544219
deviation-heading_min2.7397769449560365
distance-from-start_max2.886663360191824
distance-from-start_mean1.6701921584508657
distance-from-start_median1.765770769439282
distance-from-start_min0.2625637347330755
driven_any_max5.912667289409889
driven_any_mean3.0253985422821414
driven_any_median2.9571903265894477
driven_any_min0.2745462265397819
driven_lanedir_consec_max5.186698138898711
driven_lanedir_consec_mean2.65992440019939
driven_lanedir_consec_min0.07877903407862963
driven_lanedir_max5.700046107830381
driven_lanedir_mean2.8885604232090687
driven_lanedir_median2.8877082754636327
driven_lanedir_min0.07877903407862963
in-drivable-lane_max1.850000000000001
in-drivable-lane_mean0.9125000000000004
in-drivable-lane_min0.0
per-episodes
details{"other-ego0": {"driven_any": 4.862152367300889, "survival_time": 59.99999999999873, "driven_lanedir": 4.78813476203534, "in-drivable-lane": 0.0, "deviation-heading": 3.576285706329461, "distance-from-start": 2.7872313964857303, "deviation-center-line": 0.3973380692514069, "driven_lanedir_consec": 4.386938638928291}, "other-ego1": {"driven_any": 1.0522282858780063, "survival_time": 59.99999999999873, "driven_lanedir": 0.9872817888919256, "in-drivable-lane": 0.0, "deviation-heading": 3.876135584758978, "distance-from-start": 0.7443101423928337, "deviation-center-line": 0.3402830088601335, "driven_lanedir_consec": 0.9872817888919256}, "other-ego2": {"driven_any": 5.912667289409889, "survival_time": 59.99999999999873, "driven_lanedir": 5.700046107830381, "in-drivable-lane": 1.850000000000001, "deviation-heading": 2.7397769449560365, "distance-from-start": 2.886663360191824, "deviation-center-line": 0.32706002594535993, "driven_lanedir_consec": 5.186698138898711}, "other-ego3": {"driven_any": 0.2745462265397819, "survival_time": 59.99999999999873, "driven_lanedir": 0.07877903407862963, "in-drivable-lane": 1.800000000000001, "deviation-heading": 29.971365220758347, "distance-from-start": 0.2625637347330755, "deviation-center-line": 2.8307333506918795, "driven_lanedir_consec": 0.07877903407862963}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
19046Bea Baselines 🐤baseline-simple-yieldaido-LF-state-sim-validationsim-2of4successnonogpu-staging-spot-0-020:09:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median5.278587583080871
deviation-center-line_median0.38956279425316304


other stats
deviation-center-line_max0.38956279425316304
deviation-center-line_mean0.38956279425316304
deviation-center-line_min0.38956279425316304
deviation-heading_max3.38179620399168
deviation-heading_mean3.38179620399168
deviation-heading_median3.38179620399168
deviation-heading_min3.38179620399168
distance-from-start_max3.6475491249543506
distance-from-start_mean3.6475491249543506
distance-from-start_median3.6475491249543506
distance-from-start_min3.6475491249543506
driven_any_max5.294144196737266
driven_any_mean5.294144196737266
driven_any_median5.294144196737266
driven_any_min5.294144196737266
driven_lanedir_consec_max5.278587583080871
driven_lanedir_consec_mean5.278587583080871
driven_lanedir_consec_min5.278587583080871
driven_lanedir_max5.278587583080871
driven_lanedir_mean5.278587583080871
driven_lanedir_median5.278587583080871
driven_lanedir_min5.278587583080871
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"autolab-000-ego0": {"driven_any": 5.294144196737266, "survival_time": 59.99999999999873, "driven_lanedir": 5.278587583080871, "in-drivable-lane": 0.0, "deviation-heading": 3.38179620399168, "distance-from-start": 3.6475491249543506, "deviation-center-line": 0.38956279425316304, "driven_lanedir_consec": 5.278587583080871}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
16952Bea Baselines 🐤baseline-simple-yieldaido-LFV_multi-state-sim-testingsim-2of4successnonogpu-staging-spot-0-020:19:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median4.409785371962298
deviation-center-line_median0.4624790181352636


other stats
deviation-center-line_max0.6577508498561764
deviation-center-line_mean0.46949294807266434
deviation-center-line_min0.2952629061639538
deviation-heading_max5.307512116526921
deviation-heading_mean3.4867550536958984
deviation-heading_median3.1863546180684876
deviation-heading_min2.2667988621196975
distance-from-start_max2.6935055721286703
distance-from-start_mean2.4821697133972993
distance-from-start_median2.6038435694069273
distance-from-start_min2.0274861426466715
driven_any_max5.871562159620881
driven_any_mean4.311863625352103
driven_any_median4.417167653530974
driven_any_min2.5415570347255843
driven_lanedir_consec_max5.854461252057012
driven_lanedir_consec_mean4.300680933506648
driven_lanedir_consec_min2.528691738044982
driven_lanedir_max5.854461252057012
driven_lanedir_mean4.300680933506648
driven_lanedir_median4.409785371962298
driven_lanedir_min2.528691738044982
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"autolab-000-ego0": {"driven_any": 2.5415570347255843, "survival_time": 59.99999999999873, "driven_lanedir": 2.528691738044982, "in-drivable-lane": 0.0, "deviation-heading": 3.686877873539989, "distance-from-start": 2.0274861426466715, "deviation-center-line": 0.6577508498561764, "driven_lanedir_consec": 2.528691738044982}, "autolab-000-ego1": {"driven_any": 5.291047918117968, "survival_time": 59.99999999999873, "driven_lanedir": 5.278295314265179, "in-drivable-lane": 0.0, "deviation-heading": 2.685831362596986, "distance-from-start": 2.6935055721286703, "deviation-center-line": 0.3755711412541779, "driven_lanedir_consec": 5.278295314265179}, "autolab-000-ego2": {"driven_any": 3.5432873889439804, "survival_time": 59.99999999999873, "driven_lanedir": 3.5412754296594167, "in-drivable-lane": 0.0, "deviation-heading": 5.307512116526921, "distance-from-start": 2.6561952758407625, "deviation-center-line": 0.5493868950163492, "driven_lanedir_consec": 3.5412754296594167}, "autolab-000-ego3": {"driven_any": 5.871562159620881, "survival_time": 59.99999999999873, "driven_lanedir": 5.854461252057012, "in-drivable-lane": 0.0, "deviation-heading": 2.2667988621196975, "distance-from-start": 2.5514918629730925, "deviation-center-line": 0.2952629061639538, "driven_lanedir_consec": 5.854461252057012}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
15922Bea Baselines 🐤minimal-agentaido-LFV_multi-sim-validationsim-0of4successnonogpu-staging-spot-0-020:02:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median5.949999999999987
in-drivable-lane_median4.349999999999986
driven_lanedir_consec_median0.1796243236291546
deviation-center-line_median0.13064995214824765


other stats
deviation-center-line_max0.1356245705745657
deviation-center-line_mean0.13064995214824765
deviation-center-line_min0.1256753337219296
deviation-heading_max0.9488219555580466
deviation-heading_mean0.848315713252692
deviation-heading_median0.848315713252692
deviation-heading_min0.7478094709473374
distance-from-start_max0.8124268507162297
distance-from-start_mean0.8002552992142684
distance-from-start_median0.8002552992142684
distance-from-start_min0.788083747712307
driven_any_max0.8124423795236635
driven_any_mean0.8002702946322913
driven_any_median0.8002702946322913
driven_any_min0.788098209740919
driven_lanedir_consec_max0.19955231542410212
driven_lanedir_consec_mean0.1796243236291546
driven_lanedir_consec_min0.15969633183420706
driven_lanedir_max0.19955231542410212
driven_lanedir_mean0.1796243236291546
driven_lanedir_median0.1796243236291546
driven_lanedir_min0.15969633183420706
in-drivable-lane_max4.499999999999987
in-drivable-lane_mean4.349999999999986
in-drivable-lane_min4.199999999999986
per-episodes
details{"loop2-000-ego0": {"driven_any": 0.8124423795236635, "survival_time": 5.949999999999987, "driven_lanedir": 0.19955231542410212, "in-drivable-lane": 4.199999999999986, "deviation-heading": 0.9488219555580466, "distance-from-start": 0.8124268507162297, "deviation-center-line": 0.1256753337219296, "driven_lanedir_consec": 0.19955231542410212}, "loop2-000-ego1": {"driven_any": 0.788098209740919, "survival_time": 5.949999999999987, "driven_lanedir": 0.15969633183420706, "in-drivable-lane": 4.499999999999987, "deviation-heading": 0.7478094709473374, "distance-from-start": 0.788083747712307, "deviation-center-line": 0.1356245705745657, "driven_lanedir_consec": 0.15969633183420706}}
simulation-passed1
survival_time_max5.949999999999987
survival_time_mean5.949999999999987
survival_time_min5.949999999999987
No reset possible
15224Bea Baselines 🐤minimal-agentaido-LF-sim-validationsim-2of4successnonogpu-staging-spot-0-020:02:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.19695248970180224
survival_time_median10.100000000000009
deviation-center-line_median0.1253905065384011
in-drivable-lane_median8.350000000000009


other stats
deviation-center-line_max0.1253905065384011
deviation-center-line_mean0.1253905065384011
deviation-center-line_min0.1253905065384011
deviation-heading_max0.9520924629376074
deviation-heading_mean0.9520924629376074
deviation-heading_median0.9520924629376074
deviation-heading_min0.9520924629376074
distance-from-start_max1.4926447664752684
distance-from-start_mean1.4926447664752684
distance-from-start_median1.4926447664752684
distance-from-start_min1.4926447664752684
driven_any_max1.492909772580437
driven_any_mean1.492909772580437
driven_any_median1.492909772580437
driven_any_min1.492909772580437
driven_lanedir_consec_max0.19695248970180224
driven_lanedir_consec_mean0.19695248970180224
driven_lanedir_consec_min0.19695248970180224
driven_lanedir_max0.19695248970180224
driven_lanedir_mean0.19695248970180224
driven_lanedir_median0.19695248970180224
driven_lanedir_min0.19695248970180224
in-drivable-lane_max8.350000000000009
in-drivable-lane_mean8.350000000000009
in-drivable-lane_min8.350000000000009
per-episodes
details{"autolab-000-ego0": {"driven_any": 1.492909772580437, "survival_time": 10.100000000000009, "driven_lanedir": 0.19695248970180224, "in-drivable-lane": 8.350000000000009, "deviation-heading": 0.9520924629376074, "distance-from-start": 1.4926447664752684, "deviation-center-line": 0.1253905065384011, "driven_lanedir_consec": 0.19695248970180224}}
simulation-passed1
survival_time_max10.100000000000009
survival_time_mean10.100000000000009
survival_time_min10.100000000000009
No reset possible
13729Bea Baselines 🐤minimal-agentaido-LFVI_multi-sim-testingsim-0of4successnonogpu-staging-spot-0-020:05:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median8.14999999999998
in-drivable-lane_median5.7749999999999835
driven_lanedir_consec_median0.25930808994624444
deviation-center-line_median0.24034425647007196


other stats
deviation-center-line_max0.6912492474647307
deviation-center-line_mean0.3199439942779757
deviation-center-line_min0.10783821670702835
deviation-heading_max2.3391888095272977
deviation-heading_mean1.1429380079662843
deviation-heading_median0.8294629520909831
deviation-heading_min0.5736373181558746
distance-from-start_max1.1752764064730974
distance-from-start_mean0.7619857456783958
distance-from-start_median0.7539328643495453
distance-from-start_min0.36480084754139513
driven_any_max1.175334911835754
driven_any_mean0.7620163484462134
driven_any_median0.7539616451964115
driven_any_min0.3648071915562765
driven_lanedir_consec_max0.6611857977727704
driven_lanedir_consec_mean0.32911693406395287
driven_lanedir_consec_min0.1366657585905522
driven_lanedir_max0.9897866542941732
driven_lanedir_mean0.41126714819430354
driven_lanedir_median0.25930808994624444
driven_lanedir_min0.1366657585905522
in-drivable-lane_max6.7499999999999805
in-drivable-lane_mean4.574999999999987
in-drivable-lane_min0.0
per-episodes
details{"4way-000-ego0": {"driven_any": 0.514658523987278, "survival_time": 8.14999999999998, "driven_lanedir": 0.1366657585905522, "in-drivable-lane": 6.7499999999999805, "deviation-heading": 0.9508048428349388, "distance-from-start": 0.5146479428595193, "deviation-center-line": 0.10783821670702835, "driven_lanedir_consec": 0.1366657585905522}, "4way-000-ego1": {"driven_any": 0.3648071915562765, "survival_time": 8.14999999999998, "driven_lanedir": 0.2375799978131203, "in-drivable-lane": 6.29999999999998, "deviation-heading": 0.5736373181558746, "distance-from-start": 0.36480084754139513, "deviation-center-line": 0.20320962418622704, "driven_lanedir_consec": 0.2375799978131203}, "4way-000-ego2": {"driven_any": 0.993264766405545, "survival_time": 8.14999999999998, "driven_lanedir": 0.9897866542941732, "in-drivable-lane": 0.0, "deviation-heading": 0.7081210613470273, "distance-from-start": 0.9932177858395714, "deviation-center-line": 0.6912492474647307, "driven_lanedir_consec": 0.6611857977727704}, "4way-000-ego3": {"driven_any": 1.175334911835754, "survival_time": 8.14999999999998, "driven_lanedir": 0.28103618207936854, "in-drivable-lane": 5.249999999999987, "deviation-heading": 2.3391888095272977, "distance-from-start": 1.1752764064730974, "deviation-center-line": 0.27747888875391685, "driven_lanedir_consec": 0.28103618207936854}}
simulation-passed1
survival_time_max8.14999999999998
survival_time_mean8.14999999999998
survival_time_min8.14999999999998
No reset possible
341Bea Baselines 🐤template-randomaido-LF-sim-validationsim-3of4host-errornonogpu-staging-spot-0-020:00:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
311Bea Baselines 🐤template-randomaido-LF-sim-validationsim-0of4host-errornonogpu-staging-spot-0-020:00:04
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
301Bea Baselines 🐤template-randomaido-LF-sim-validationsim-2of4host-errornonogpu-staging-spot-0-020:00:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
292Bea Baselines 🐤template-randomaido-LFP-sim-validationsim-0of4host-errornonogpu-staging-spot-0-020:00:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
282Bea Baselines 🐤template-randomaido-LFP-sim-validationsim-3of4host-errornonogpu-staging-spot-0-020:00:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
272Bea Baselines 🐤template-randomaido-LFP-sim-validationsim-2of4host-errornonogpu-staging-spot-0-020:00:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
262Bea Baselines 🐤template-randomaido-LFP-sim-validationsim-1of4host-errornonogpu-staging-spot-0-020:00:04
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
253Bea Baselines 🐤template-randomaido-LFVI_multi-sim-validationsim-0of4host-errornonogpu-staging-spot-0-020:00:24
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
44Bea Baselines 🐤template-randomaido-LFV_multi-sim-validationsim-2of4successnonogpu-staging-spot-0-020:22:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median31.750000000000316
in-drivable-lane_median25.175000000000296
driven_lanedir_consec_median0.1855246917187785
deviation-center-line_median0.49966350574004115


other stats
deviation-center-line_max0.7555299219673985
deviation-center-line_mean0.5108643440264814
deviation-center-line_min0.2886004426584447
deviation-heading_max5.677922521471409
deviation-heading_mean4.088893264090181
deviation-heading_median4.163773732737991
deviation-heading_min2.3501030694133327
distance-from-start_max0.8824046752191013
distance-from-start_mean0.7245653197432455
distance-from-start_median0.7348495479071131
distance-from-start_min0.5461575079396543
driven_any_max7.700480611528672
driven_any_mean5.058667025161513
driven_any_median5.018099001021706
driven_any_min2.4979894870739705
driven_lanedir_consec_max0.2676492332914642
driven_lanedir_consec_mean0.18563626105896863
driven_lanedir_consec_min0.10384642750685336
driven_lanedir_max0.2496385572544464
driven_lanedir_mean0.17341917865234097
driven_lanedir_median0.17223668708559292
driven_lanedir_min0.09956478318373164
in-drivable-lane_max28.300000000000303
in-drivable-lane_mean25.4625000000003
in-drivable-lane_min23.200000000000298
per-episodes
details{"autolab-000-ego0": {"driven_any": 7.700480611528672, "survival_time": 31.750000000000316, "driven_lanedir": 0.2496385572544464, "in-drivable-lane": 23.200000000000298, "deviation-heading": 5.523299075233993, "distance-from-start": 0.8824046752191013, "deviation-center-line": 0.7555299219673985, "driven_lanedir_consec": 0.2676492332914642}, "autolab-000-ego1": {"driven_any": 6.066664798227059, "survival_time": 31.750000000000316, "driven_lanedir": 0.19493515195652833, "in-drivable-lane": 23.700000000000298, "deviation-heading": 5.677922521471409, "distance-from-start": 0.7960980578537341, "deviation-center-line": 0.5957741014324851, "driven_lanedir_consec": 0.20826345296493656}, "autolab-000-ego2": {"driven_any": 3.969533203816352, "survival_time": 31.750000000000316, "driven_lanedir": 0.1495382222146575, "in-drivable-lane": 26.650000000000297, "deviation-heading": 2.8042483902419892, "distance-from-start": 0.6736010379604921, "deviation-center-line": 0.40355291004759714, "driven_lanedir_consec": 0.1627859304726205}, "autolab-000-ego3": {"driven_any": 2.4979894870739705, "survival_time": 31.750000000000316, "driven_lanedir": 0.09956478318373164, "in-drivable-lane": 28.300000000000303, "deviation-heading": 2.3501030694133327, "distance-from-start": 0.5461575079396543, "deviation-center-line": 0.2886004426584447, "driven_lanedir_consec": 0.10384642750685336}}
simulation-passed1
survival_time_max31.750000000000316
survival_time_mean31.750000000000316
survival_time_min31.750000000000316
No reset possible