Duckietown Challenges Home Challenges Submissions

Evaluator 5

ID5
evaluatornogpu-staging-spot-0-04
owneruser1000001
machinenogpu-staging-spot-0
processnogpu-staging-spot-0-04
version6.2.32
first heard
last heard
statusinactive
# evaluating
# success29 3
# timeout
# failed7 231
# error
# aborted3 204
# host-error28 8
arm0
x86_641
Mac0
gpu available0
Number of processors16
Processor frequency (MHz)0.0 GHz
Free % of processors99%
RAM total (MB)29.4 GB
RAM free (MB)7.4 GB
Disk (MB)969.3 GB
Disk available (MB)706.5 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, ETH_small_intersect
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
52674Bea Baselines 🐤template-rosaido-LFVI_multi-sim-validationsim-0of4successnonogpu-staging-spot-0-040:05:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median11.00000000000002
in-drivable-lane_median8.750000000000021
driven_lanedir_consec_median0.5311482418022926
deviation-center-line_median0.15947304487342673


other stats
deviation-center-line_max0.4398559478070397
deviation-center-line_mean0.2275398579789235
deviation-center-line_min0.1513573943618009
deviation-heading_max2.769615391682989
deviation-heading_mean1.4806180928975996
deviation-heading_median1.33352117124359
deviation-heading_min0.4858146374202303
distance-from-start_max2.7629297434228546
distance-from-start_mean1.58109725351414
distance-from-start_median1.2202552266131772
distance-from-start_min1.1209488174073508
driven_any_max3.045861237695465
driven_any_mean1.6803904576957998
driven_any_median1.263964829862916
driven_any_min1.1477709333619022
driven_lanedir_consec_max0.9924902095951228
driven_lanedir_consec_mean0.5744782414631187
driven_lanedir_consec_min0.24312627265276676
driven_lanedir_max0.9924902095951228
driven_lanedir_mean0.5744782414631187
driven_lanedir_median0.5311482418022926
driven_lanedir_min0.24312627265276676
in-drivable-lane_max9.05000000000002
in-drivable-lane_mean8.275000000000023
in-drivable-lane_min6.550000000000029
per-episodes
details{"4way-000-ego0": {"driven_any": 1.3671297069680672, "survival_time": 11.00000000000002, "driven_lanedir": 0.24312627265276676, "in-drivable-lane": 9.000000000000023, "deviation-heading": 1.721917750466321, "distance-from-start": 1.3066693091964388, "deviation-center-line": 0.15408884638986303, "driven_lanedir_consec": 0.24312627265276676}, "4way-000-ego1": {"driven_any": 3.045861237695465, "survival_time": 11.00000000000002, "driven_lanedir": 0.9924902095951228, "in-drivable-lane": 6.550000000000029, "deviation-heading": 2.769615391682989, "distance-from-start": 2.7629297434228546, "deviation-center-line": 0.4398559478070397, "driven_lanedir_consec": 0.9924902095951228}, "4way-000-ego2": {"driven_any": 1.1477709333619022, "survival_time": 11.00000000000002, "driven_lanedir": 0.6292823097554127, "in-drivable-lane": 8.500000000000021, "deviation-heading": 0.945124592020859, "distance-from-start": 1.1209488174073508, "deviation-center-line": 0.1513573943618009, "driven_lanedir_consec": 0.6292823097554127}, "4way-000-ego3": {"driven_any": 1.1607999527577648, "survival_time": 11.00000000000002, "driven_lanedir": 0.4330141738491724, "in-drivable-lane": 9.05000000000002, "deviation-heading": 0.4858146374202303, "distance-from-start": 1.1338411440299156, "deviation-center-line": 0.1648572433569904, "driven_lanedir_consec": 0.4330141738491724}}
simulation-passed1
survival_time_max11.00000000000002
survival_time_mean11.00000000000002
survival_time_min11.00000000000002
No reset possible
51670Bea Baselines 🐤template-rosaido-LFV-sim-validationsim-2of4successnonogpu-staging-spot-0-040:03:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median2.3499999999999996
in-drivable-lane_median1.0999999999999992
driven_lanedir_consec_median0.29362227329882296
deviation-center-line_median0.0812753058990936


other stats
deviation-center-line_max0.0812753058990936
deviation-center-line_mean0.0812753058990936
deviation-center-line_min0.0812753058990936
deviation-heading_max0.5339108516678389
deviation-heading_mean0.5339108516678389
deviation-heading_median0.5339108516678389
deviation-heading_min0.5339108516678389
distance-from-start_max0.4596858587324551
distance-from-start_mean0.4596858587324551
distance-from-start_median0.4596858587324551
distance-from-start_min0.4596858587324551
driven_any_max0.4609925379350057
driven_any_mean0.4609925379350057
driven_any_median0.4609925379350057
driven_any_min0.4609925379350057
driven_lanedir_consec_max0.29362227329882296
driven_lanedir_consec_mean0.29362227329882296
driven_lanedir_consec_min0.29362227329882296
driven_lanedir_max0.29362227329882296
driven_lanedir_mean0.29362227329882296
driven_lanedir_median0.29362227329882296
driven_lanedir_min0.29362227329882296
in-drivable-lane_max1.0999999999999992
in-drivable-lane_mean1.0999999999999992
in-drivable-lane_min1.0999999999999992
per-episodes
details{"autolab-000-ego0": {"driven_any": 0.4609925379350057, "survival_time": 2.3499999999999996, "driven_lanedir": 0.29362227329882296, "in-drivable-lane": 1.0999999999999992, "deviation-heading": 0.5339108516678389, "distance-from-start": 0.4596858587324551, "deviation-center-line": 0.0812753058990936, "driven_lanedir_consec": 0.29362227329882296}}
simulation-passed1
survival_time_max2.3499999999999996
survival_time_mean2.3499999999999996
survival_time_min2.3499999999999996
No reset possible
51071Bea Baselines 🐤template-rosaido-LFVI-sim-validationsim-0of4successnonogpu-staging-spot-0-040:02:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median1.6500000000000008
in-drivable-lane_median0.7000000000000005
driven_lanedir_consec_median0.1083261854411135
deviation-center-line_median0.05527658318789723


other stats
deviation-center-line_max0.05527658318789723
deviation-center-line_mean0.05527658318789723
deviation-center-line_min0.05527658318789723
deviation-heading_max0.7144337467140517
deviation-heading_mean0.7144337467140517
deviation-heading_median0.7144337467140517
deviation-heading_min0.7144337467140517
distance-from-start_max0.1627726771476448
distance-from-start_mean0.1627726771476448
distance-from-start_median0.1627726771476448
distance-from-start_min0.1627726771476448
driven_any_max0.16417291450637905
driven_any_mean0.16417291450637905
driven_any_median0.16417291450637905
driven_any_min0.16417291450637905
driven_lanedir_consec_max0.1083261854411135
driven_lanedir_consec_mean0.1083261854411135
driven_lanedir_consec_min0.1083261854411135
driven_lanedir_max0.1083261854411135
driven_lanedir_mean0.1083261854411135
driven_lanedir_median0.1083261854411135
driven_lanedir_min0.1083261854411135
in-drivable-lane_max0.7000000000000005
in-drivable-lane_mean0.7000000000000005
in-drivable-lane_min0.7000000000000005
per-episodes
details{"4way-000-ego0": {"driven_any": 0.16417291450637905, "survival_time": 1.6500000000000008, "driven_lanedir": 0.1083261854411135, "in-drivable-lane": 0.7000000000000005, "deviation-heading": 0.7144337467140517, "distance-from-start": 0.1627726771476448, "deviation-center-line": 0.05527658318789723, "driven_lanedir_consec": 0.1083261854411135}}
simulation-passed1
survival_time_max1.6500000000000008
survival_time_mean1.6500000000000008
survival_time_min1.6500000000000008
No reset possible
50172Bea Baselines 🐤template-rosaido-LFI-sim-validationsim-1of4host-errornonogpu-staging-spot-0-040:02:30
Error while running [...]
Error while running Docker Compose:

Could not run command
│    cmd: [docker-compose, -p, nogpu-staging-spot-0-04-job501-98412, up, -d]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'nogpu-staging-spot-0-04-job501-98412', 'up', '-d']' returned non-zero exit status 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48975Bea Baselines 🐤template-rosaido-LFV_multi-sim-testingsim-3of4host-errornonogpu-staging-spot-0-040:03:16
The container "simul [...]
The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48376Bea Baselines 🐤template-rosaido-LFP-sim-testingsim-0of4successnonogpu-staging-spot-0-040:01:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median2.549999999999999
in-drivable-lane_median1.5999999999999988
driven_lanedir_consec_median0.11671626493834596
deviation-center-line_median0.061597709822429215


other stats
deviation-center-line_max0.061597709822429215
deviation-center-line_mean0.061597709822429215
deviation-center-line_min0.061597709822429215
deviation-heading_max0.7141136075468724
deviation-heading_mean0.7141136075468724
deviation-heading_median0.7141136075468724
deviation-heading_min0.7141136075468724
distance-from-start_max0.4325840240100758
distance-from-start_mean0.4325840240100758
distance-from-start_median0.4325840240100758
distance-from-start_min0.4325840240100758
driven_any_max0.4352875361626187
driven_any_mean0.4352875361626187
driven_any_median0.4352875361626187
driven_any_min0.4352875361626187
driven_lanedir_consec_max0.11671626493834596
driven_lanedir_consec_mean0.11671626493834596
driven_lanedir_consec_min0.11671626493834596
driven_lanedir_max0.11671626493834596
driven_lanedir_mean0.11671626493834596
driven_lanedir_median0.11671626493834596
driven_lanedir_min0.11671626493834596
in-drivable-lane_max1.5999999999999988
in-drivable-lane_mean1.5999999999999988
in-drivable-lane_min1.5999999999999988
per-episodes
details{"loop2-000-ego0": {"driven_any": 0.4352875361626187, "survival_time": 2.549999999999999, "driven_lanedir": 0.11671626493834596, "in-drivable-lane": 1.5999999999999988, "deviation-heading": 0.7141136075468724, "distance-from-start": 0.4325840240100758, "deviation-center-line": 0.061597709822429215, "driven_lanedir_consec": 0.11671626493834596}}
simulation-passed1
survival_time_max2.549999999999999
survival_time_mean2.549999999999999
survival_time_min2.549999999999999
No reset possible
47079Bea Baselines 🐤template-rosaido-LFI-sim-testingsim-0of4successnonogpu-staging-spot-0-040:03:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median9.349999999999998
in-drivable-lane_median7.65
driven_lanedir_consec_median0.2202469231602976
deviation-center-line_median0.13469984547527072


other stats
deviation-center-line_max0.13469984547527072
deviation-center-line_mean0.13469984547527072
deviation-center-line_min0.13469984547527072
deviation-heading_max1.5972817556818208
deviation-heading_mean1.5972817556818208
deviation-heading_median1.5972817556818208
deviation-heading_min1.5972817556818208
distance-from-start_max2.175914621897351
distance-from-start_mean2.175914621897351
distance-from-start_median2.175914621897351
distance-from-start_min2.175914621897351
driven_any_max2.58659238476622
driven_any_mean2.58659238476622
driven_any_median2.58659238476622
driven_any_min2.58659238476622
driven_lanedir_consec_max0.2202469231602976
driven_lanedir_consec_mean0.2202469231602976
driven_lanedir_consec_min0.2202469231602976
driven_lanedir_max0.2202469231602976
driven_lanedir_mean0.2202469231602976
driven_lanedir_median0.2202469231602976
driven_lanedir_min0.2202469231602976
in-drivable-lane_max7.65
in-drivable-lane_mean7.65
in-drivable-lane_min7.65
per-episodes
details{"4way-000-ego0": {"driven_any": 2.58659238476622, "survival_time": 9.349999999999998, "driven_lanedir": 0.2202469231602976, "in-drivable-lane": 7.65, "deviation-heading": 1.5972817556818208, "distance-from-start": 2.175914621897351, "deviation-center-line": 0.13469984547527072, "driven_lanedir_consec": 0.2202469231602976}}
simulation-passed1
survival_time_max9.349999999999998
survival_time_mean9.349999999999998
survival_time_min9.349999999999998
No reset possible
46479Bea Baselines 🐤template-rosaido-LFI-sim-testingsim-0of4host-errornonogpu-staging-spot-0-040:02:24
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
45183Bea Baselines 🐤baseline-duckietownaido-LFP-sim-validationsim-1of4successnonogpu-staging-spot-0-040:05:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median22.90000000000019
in-drivable-lane_median0.0
driven_lanedir_consec_median2.7696253550384067
deviation-center-line_median1.4278018108387964


other stats
deviation-center-line_max1.4278018108387964
deviation-center-line_mean1.4278018108387964
deviation-center-line_min1.4278018108387964
deviation-heading_max5.077773799038486
deviation-heading_mean5.077773799038486
deviation-heading_median5.077773799038486
deviation-heading_min5.077773799038486
distance-from-start_max2.4252822229125246
distance-from-start_mean2.4252822229125246
distance-from-start_median2.4252822229125246
distance-from-start_min2.4252822229125246
driven_any_max2.895699161537162
driven_any_mean2.895699161537162
driven_any_median2.895699161537162
driven_any_min2.895699161537162
driven_lanedir_consec_max2.7696253550384067
driven_lanedir_consec_mean2.7696253550384067
driven_lanedir_consec_min2.7696253550384067
driven_lanedir_max2.7696253550384067
driven_lanedir_mean2.7696253550384067
driven_lanedir_median2.7696253550384067
driven_lanedir_min2.7696253550384067
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"zigzag-000-ego0": {"driven_any": 2.895699161537162, "survival_time": 22.90000000000019, "driven_lanedir": 2.7696253550384067, "in-drivable-lane": 0.0, "deviation-heading": 5.077773799038486, "distance-from-start": 2.4252822229125246, "deviation-center-line": 1.4278018108387964, "driven_lanedir_consec": 2.7696253550384067}}
simulation-passed1
survival_time_max22.90000000000019
survival_time_mean22.90000000000019
survival_time_min22.90000000000019
No reset possible
44383Bea Baselines 🐤baseline-duckietownaido-LFP-sim-validationsim-2of4host-errornonogpu-staging-spot-0-040:04:41
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43783Bea Baselines 🐤baseline-duckietownaido-LFP-sim-validationsim-0of4host-errornonogpu-staging-spot-0-040:04:02
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43085Bea Baselines 🐤baseline-duckietownaido-LFVI-sim-validationsim-1of4host-errornonogpu-staging-spot-0-040:02:55
The container "simul [...]
The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42386Bea Baselines 🐤baseline-duckietownaido-LFI-sim-validationsim-3of4successnonogpu-staging-spot-0-040:01:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
41686Bea Baselines 🐤baseline-duckietownaido-LFI-sim-validationsim-1of4host-errornonogpu-staging-spot-0-040:05:25
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41087Bea Baselines 🐤baseline-duckietownaido-LF-sim-validationsim-1of4host-errornonogpu-staging-spot-0-040:03:07
The container "simul [...]
The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40388Bea Baselines 🐤baseline-duckietownaido-LFVI_multi-sim-validationsim-1of4host-errornonogpu-staging-spot-0-040:04:05
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39290Bea Baselines 🐤baseline-duckietownaido-LFP-sim-testingsim-2of4host-errornonogpu-staging-spot-0-040:07:43
The container "simul [...]
The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38494Bea Baselines 🐤baseline-duckietownaido-LF-sim-testingsim-0of4successnonogpu-staging-spot-0-040:07:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.644802573514946
survival_time_median31.500000000000313
deviation-center-line_median1.366534773559904
in-drivable-lane_median12.25000000000014


other stats
deviation-center-line_max1.366534773559904
deviation-center-line_mean1.366534773559904
deviation-center-line_min1.366534773559904
deviation-heading_max7.167870371111221
deviation-heading_mean7.167870371111221
deviation-heading_median7.167870371111221
deviation-heading_min7.167870371111221
distance-from-start_max1.9194131881811023
distance-from-start_mean1.9194131881811023
distance-from-start_median1.9194131881811023
distance-from-start_min1.9194131881811023
driven_any_max4.030803948030273
driven_any_mean4.030803948030273
driven_any_median4.030803948030273
driven_any_min4.030803948030273
driven_lanedir_consec_max1.644802573514946
driven_lanedir_consec_mean1.644802573514946
driven_lanedir_consec_min1.644802573514946
driven_lanedir_max2.204991398894977
driven_lanedir_mean2.204991398894977
driven_lanedir_median2.204991398894977
driven_lanedir_min2.204991398894977
in-drivable-lane_max12.25000000000014
in-drivable-lane_mean12.25000000000014
in-drivable-lane_min12.25000000000014
per-episodes
details{"loop2-000-ego0": {"driven_any": 4.030803948030273, "survival_time": 31.500000000000313, "driven_lanedir": 2.204991398894977, "in-drivable-lane": 12.25000000000014, "deviation-heading": 7.167870371111221, "distance-from-start": 1.9194131881811023, "deviation-center-line": 1.366534773559904, "driven_lanedir_consec": 1.644802573514946}}
simulation-passed1
survival_time_max31.500000000000313
survival_time_mean31.500000000000313
survival_time_min31.500000000000313
No reset possible
358103Bea Baselines 🐤baseline-duckietown-mlaido-LFV_multi-sim-testingsim-2of4host-errornonogpu-staging-spot-0-040:15:16
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
35396Bea Baselines 🐤baseline-duckietown-mlaido-LFV_multi-sim-validationsim-2of4host-errornonogpu-staging-spot-0-040:04:31
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
34697Bea Baselines 🐤baseline-duckietown-mlaido-LFP-sim-validationsim-2of4host-errornonogpu-staging-spot-0-040:04:22
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
34199Bea Baselines 🐤baseline-duckietown-mlaido-LFVI-sim-validationsim-0of4host-errornonogpu-staging-spot-0-040:06:11
The container "simul [...]
The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
33598Bea Baselines 🐤baseline-duckietown-mlaido-LFV-sim-validationsim-0of4host-errornonogpu-staging-spot-0-040:04:24
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
33099Bea Baselines 🐤baseline-duckietown-mlaido-LFVI-sim-validationsim-2of4successnonogpu-staging-spot-0-040:01:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
323102Bea Baselines 🐤baseline-duckietown-mlaido-LFVI_multi-sim-validationsim-3of4successnonogpu-staging-spot-0-040:02:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
312102Bea Baselines 🐤baseline-duckietown-mlaido-LFVI_multi-sim-validationsim-0of4host-errornonogpu-staging-spot-0-040:06:05
The container "simul [...]
The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
303104Bea Baselines 🐤baseline-duckietown-mlaido-LFP-sim-testingsim-1of4successnonogpu-staging-spot-0-040:10:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median-2.5756538235555126e-06
deviation-center-line_median0.406520962699379


other stats
deviation-center-line_max0.406520962699379
deviation-center-line_mean0.406520962699379
deviation-center-line_min0.406520962699379
deviation-heading_max40.96736494841265
deviation-heading_mean40.96736494841265
deviation-heading_median40.96736494841265
deviation-heading_min40.96736494841265
distance-from-start_max0.0
distance-from-start_mean0.0
distance-from-start_median0.0
distance-from-start_min0.0
driven_any_max0.0
driven_any_mean0.0
driven_any_median0.0
driven_any_min0.0
driven_lanedir_consec_max-2.5756538235555126e-06
driven_lanedir_consec_mean-2.5756538235555126e-06
driven_lanedir_consec_min-2.5756538235555126e-06
driven_lanedir_max-2.5756538235555126e-06
driven_lanedir_mean-2.5756538235555126e-06
driven_lanedir_median-2.5756538235555126e-06
driven_lanedir_min-2.5756538235555126e-06
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"zigzag-000-ego0": {"driven_any": 0.0, "survival_time": 59.99999999999873, "driven_lanedir": -2.5756538235555126e-06, "in-drivable-lane": 0.0, "deviation-heading": 40.96736494841265, "distance-from-start": 0.0, "deviation-center-line": 0.406520962699379, "driven_lanedir_consec": -2.5756538235555126e-06}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
300105Bea Baselines 🐤baseline-duckietown-mlaido-LFV-sim-testingsim-1of4host-errornonogpu-staging-spot-0-040:03:56
The container "simul [...]
The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
294106Bea Baselines 🐤baseline-duckietown-mlaido-LFVI-sim-testingsim-2of4successnonogpu-staging-spot-0-040:01:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
292107Bea Baselines 🐤baseline-duckietown-mlaido-LFI-sim-testingsim-3of4successnonogpu-staging-spot-0-040:01:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
283109Bea Baselines 🐤baseline-duckietown-mlaido-LF-sim-testingsim-2of4successnonogpu-staging-spot-0-040:09:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median59.99999999999873
deviation-center-line_median0.40651342125783446
in-drivable-lane_median0.0


other stats
deviation-center-line_max0.40651342125783446
deviation-center-line_mean0.40651342125783446
deviation-center-line_min0.40651342125783446
deviation-heading_max40.967332753486225
deviation-heading_mean40.967332753486225
deviation-heading_median40.967332753486225
deviation-heading_min40.967332753486225
distance-from-start_max8.881784197001252e-16
distance-from-start_mean8.881784197001252e-16
distance-from-start_median8.881784197001252e-16
distance-from-start_min8.881784197001252e-16
driven_any_max1.0658141036401505e-12
driven_any_mean1.0658141036401505e-12
driven_any_median1.0658141036401505e-12
driven_any_min1.0658141036401505e-12
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"autolab-000-ego0": {"driven_any": 1.0658141036401505e-12, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "in-drivable-lane": 0.0, "deviation-heading": 40.967332753486225, "distance-from-start": 8.881784197001252e-16, "deviation-center-line": 0.40651342125783446, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
277109Bea Baselines 🐤baseline-duckietown-mlaido-LF-sim-testingsim-0of4successnonogpu-staging-spot-0-040:08:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median59.99999999999873
deviation-center-line_median0.4065102356188296
in-drivable-lane_median0.0


other stats
deviation-center-line_max0.4065102356188296
deviation-center-line_mean0.4065102356188296
deviation-center-line_min0.4065102356188296
deviation-heading_max40.967332753486225
deviation-heading_mean40.967332753486225
deviation-heading_median40.967332753486225
deviation-heading_min40.967332753486225
distance-from-start_max8.326672684688674e-17
distance-from-start_mean8.326672684688674e-17
distance-from-start_median8.326672684688674e-17
distance-from-start_min8.326672684688674e-17
driven_any_max9.992007221626409e-14
driven_any_mean9.992007221626409e-14
driven_any_median9.992007221626409e-14
driven_any_min9.992007221626409e-14
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"loop2-000-ego0": {"driven_any": 9.992007221626409e-14, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "in-drivable-lane": 0.0, "deviation-heading": 40.967332753486225, "distance-from-start": 8.326672684688674e-17, "deviation-center-line": 0.4065102356188296, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
274110Bea Baselines 🐤baseline-duckietown-mlaido-LFVI_multi-sim-testingsim-2of4successnonogpu-staging-spot-0-040:04:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
27030Bea Baselines 🐤minimal-agent-stateaido-LF-state-sim-validationsim-0of4failednonogpu-staging-spot-0-040:00:59
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
26631Bea Baselines 🐤minimal-agent-stateaido-LFVI_multi-state-sim-validationsim-0of4host-errornonogpu-staging-spot-0-040:01:03
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/c24d03c61f90db9a0b5e58b6261b51fba7107a938e5c4fcd9c428a80aafa3bbe/json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1245, in run_one
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 889, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 773, in inspect_container
    return self._result(
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/containers/c24d03c61f90db9a0b5e58b6261b51fba7107a938e5c4fcd9c428a80aafa3bbe/json: Not Found ("No such container: c24d03c61f90db9a0b5e58b6261b51fba7107a938e5c4fcd9c428a80aafa3bbe")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
26331Bea Baselines 🐤minimal-agent-stateaido-LFVI_multi-state-sim-validationsim-2of4failednonogpu-staging-spot-0-040:01:16
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
26032Bea Baselines 🐤minimal-agent-stateaido-LFV_multi-state-sim-validation62failednonogpu-staging-spot-0-040:01:17
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
25733Bea Baselines 🐤minimal-agent-stateaido-LFP-state-sim-validationsim-0of4failednonogpu-staging-spot-0-040:01:08
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
25434Bea Baselines 🐤minimal-agent-fullaido-LFP-full-sim-validationsim-0of4successnonogpu-staging-spot-0-040:08:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median51.949999999999186
in-drivable-lane_median3.449999999999882
driven_lanedir_consec_median2.7806693784918126
deviation-center-line_median5.983237422521915


other stats
deviation-center-line_max5.983237422521915
deviation-center-line_mean5.983237422521915
deviation-center-line_min5.983237422521915
deviation-heading_max5.351741767067896
deviation-heading_mean5.351741767067896
deviation-heading_median5.351741767067896
deviation-heading_min5.351741767067896
distance-from-start_max1.6305272648718805
distance-from-start_mean1.6305272648718805
distance-from-start_median1.6305272648718805
distance-from-start_min1.6305272648718805
driven_any_max2.95109108951375
driven_any_mean2.95109108951375
driven_any_median2.95109108951375
driven_any_min2.95109108951375
driven_lanedir_consec_max2.7806693784918126
driven_lanedir_consec_mean2.7806693784918126
driven_lanedir_consec_min2.7806693784918126
driven_lanedir_max2.7806693784918126
driven_lanedir_mean2.7806693784918126
driven_lanedir_median2.7806693784918126
driven_lanedir_min2.7806693784918126
in-drivable-lane_max3.449999999999882
in-drivable-lane_mean3.449999999999882
in-drivable-lane_min3.449999999999882
per-episodes
details{"loop2-000-ego0": {"driven_any": 2.95109108951375, "survival_time": 51.949999999999186, "driven_lanedir": 2.7806693784918126, "in-drivable-lane": 3.449999999999882, "deviation-heading": 5.351741767067896, "distance-from-start": 1.6305272648718805, "deviation-center-line": 5.983237422521915, "driven_lanedir_consec": 2.7806693784918126}}
simulation-passed1
survival_time_max51.949999999999186
survival_time_mean51.949999999999186
survival_time_min51.949999999999186
No reset possible
24436Bea Baselines 🐤minimal-agent-fullaido-LFV_multi-full-sim-validationsim-3of4successnonogpu-staging-spot-0-040:30:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median1.8249999999998965
driven_lanedir_consec_median2.089362646121109
deviation-center-line_median5.160696904244562


other stats
deviation-center-line_max5.884537088976701
deviation-center-line_mean4.608902728622453
deviation-center-line_min2.229680017023988
deviation-heading_max33.00132974561592
deviation-heading_mean11.086939889451044
deviation-heading_median5.315632499039603
deviation-heading_min0.7151648141090521
distance-from-start_max2.4007738045922906
distance-from-start_mean1.8550334398943655
distance-from-start_median1.8031549983915711
distance-from-start_min1.4130499582020293
driven_any_max3.5300951074200406
driven_any_mean2.5187726211552337
driven_any_median2.531092735855504
driven_any_min1.4828099054898876
driven_lanedir_consec_max3.437555654852413
driven_lanedir_consec_mean2.243867367124486
driven_lanedir_consec_min1.3591885214033157
driven_lanedir_max3.437555654852413
driven_lanedir_mean2.430771337654316
driven_lanedir_median2.4562457120319716
driven_lanedir_min1.3730382717009086
in-drivable-lane_max2.0000000000000284
in-drivable-lane_mean1.4124999999999552
in-drivable-lane_min0.0
per-episodes
details{"loop-000-ego0": {"driven_any": 1.5425992768884045, "survival_time": 59.99999999999873, "driven_lanedir": 1.537863345389593, "in-drivable-lane": 0.0, "deviation-heading": 0.7151648141090521, "distance-from-start": 1.5381664294601065, "deviation-center-line": 2.229680017023988, "driven_lanedir_consec": 1.537863345389593}, "loop-000-ego1": {"driven_any": 3.519586194822603, "survival_time": 59.99999999999873, "driven_lanedir": 3.3746280786743506, "in-drivable-lane": 1.9999999999998863, "deviation-heading": 6.288395459035003, "distance-from-start": 2.0681435673230366, "deviation-center-line": 5.233752335699744, "driven_lanedir_consec": 2.6408619468526253}, "loop-000-ego2": {"driven_any": 3.5300951074200406, "survival_time": 59.99999999999873, "driven_lanedir": 3.437555654852413, "in-drivable-lane": 1.6499999999999062, "deviation-heading": 4.342869539044204, "distance-from-start": 2.4007738045922906, "deviation-center-line": 5.884537088976701, "driven_lanedir_consec": 3.437555654852413}, "loop-000-ego3": {"driven_any": 1.4828099054898876, "survival_time": 59.99999999999873, "driven_lanedir": 1.3730382717009086, "in-drivable-lane": 2.0000000000000284, "deviation-heading": 33.00132974561592, "distance-from-start": 1.4130499582020293, "deviation-center-line": 5.087641472789379, "driven_lanedir_consec": 1.3591885214033157}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
24137Bea Baselines 🐤minimal-agent-stateaido-LFVI_multi-state-sim-testingsim-3of4failednonogpu-staging-spot-0-040:01:13
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
23738Bea Baselines 🐤minimal-agent-fullaido-LFVI_multi-full-sim-validationsim-3of4successnonogpu-staging-spot-0-040:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
23439Bea Baselines 🐤minimal-agent-stateaido-LFV_multi-state-sim-testingsim-3of4failednonogpu-staging-spot-0-040:01:33
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
23339Bea Baselines 🐤minimal-agent-stateaido-LFV_multi-state-sim-testingsim-2of4host-errornonogpu-staging-spot-0-040:01:26
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
23139Bea Baselines 🐤minimal-agent-stateaido-LFV_multi-state-sim-testingsim-1of4failednonogpu-staging-spot-0-040:01:16
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21842Bea Baselines 🐤minimal-agent-fullaido-LFP-full-sim-testingsim-1of4successnonogpu-staging-spot-0-040:11:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median4.249999999999972
driven_lanedir_consec_median1.5873308651520088
deviation-center-line_median5.442861880459055


other stats
deviation-center-line_max5.442861880459055
deviation-center-line_mean5.442861880459055
deviation-center-line_min5.442861880459055
deviation-heading_max11.057314601935024
deviation-heading_mean11.057314601935024
deviation-heading_median11.057314601935024
deviation-heading_min11.057314601935024
distance-from-start_max1.5192445510405312
distance-from-start_mean1.5192445510405312
distance-from-start_median1.5192445510405312
distance-from-start_min1.5192445510405312
driven_any_max3.452076592019545
driven_any_mean3.452076592019545
driven_any_median3.452076592019545
driven_any_min3.452076592019545
driven_lanedir_consec_max1.5873308651520088
driven_lanedir_consec_mean1.5873308651520088
driven_lanedir_consec_min1.5873308651520088
driven_lanedir_max3.163921596586153
driven_lanedir_mean3.163921596586153
driven_lanedir_median3.163921596586153
driven_lanedir_min3.163921596586153
in-drivable-lane_max4.249999999999972
in-drivable-lane_mean4.249999999999972
in-drivable-lane_min4.249999999999972
per-episodes
details{"zigzag-000-ego0": {"driven_any": 3.452076592019545, "survival_time": 59.99999999999873, "driven_lanedir": 3.163921596586153, "in-drivable-lane": 4.249999999999972, "deviation-heading": 11.057314601935024, "distance-from-start": 1.5192445510405312, "deviation-center-line": 5.442861880459055, "driven_lanedir_consec": 1.5873308651520088}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
21642Bea Baselines 🐤minimal-agent-fullaido-LFP-full-sim-testingsim-3of4successnonogpu-staging-spot-0-040:04:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median13.200000000000053
in-drivable-lane_median0.0
driven_lanedir_consec_median0.7222898911877365
deviation-center-line_median0.6263425139703792


other stats
deviation-center-line_max0.6263425139703792
deviation-center-line_mean0.6263425139703792
deviation-center-line_min0.6263425139703792
deviation-heading_max1.4170853186472243
deviation-heading_mean1.4170853186472243
deviation-heading_median1.4170853186472243
deviation-heading_min1.4170853186472243
distance-from-start_max0.7201981002422111
distance-from-start_mean0.7201981002422111
distance-from-start_median0.7201981002422111
distance-from-start_min0.7201981002422111
driven_any_max0.7295990048299275
driven_any_mean0.7295990048299275
driven_any_median0.7295990048299275
driven_any_min0.7295990048299275
driven_lanedir_consec_max0.7222898911877365
driven_lanedir_consec_mean0.7222898911877365
driven_lanedir_consec_min0.7222898911877365
driven_lanedir_max0.7222898911877365
driven_lanedir_mean0.7222898911877365
driven_lanedir_median0.7222898911877365
driven_lanedir_min0.7222898911877365
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"loop-000-ego0": {"driven_any": 0.7295990048299275, "survival_time": 13.200000000000053, "driven_lanedir": 0.7222898911877365, "in-drivable-lane": 0.0, "deviation-heading": 1.4170853186472243, "distance-from-start": 0.7201981002422111, "deviation-center-line": 0.6263425139703792, "driven_lanedir_consec": 0.7222898911877365}}
simulation-passed1
survival_time_max13.200000000000053
survival_time_mean13.200000000000053
survival_time_min13.200000000000053
No reset possible
21343Bea Baselines 🐤minimal-agent-fullaido-LFV_multi-full-sim-testingsim-3of4abortednonogpu-staging-spot-0-040:31:51
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 391, in main
    stats = future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 426, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20644Bea Baselines 🐤minimal-agent-fullaido-LFVI_multi-full-sim-testingsim-1of4abortednonogpu-staging-spot-0-040:32:42
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20444Bea Baselines 🐤minimal-agent-fullaido-LFVI_multi-full-sim-testingsim-2of4abortednonogpu-staging-spot-0-040:01:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
19647Bea Baselines 🐤baseline-simple-yieldaido-LFVI_multi-state-sim-validationsim-0of4successnonogpu-staging-spot-0-040:24:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median1.6319459545511148
deviation-center-line_median0.6761894074746656


other stats
deviation-center-line_max2.128497010687399
deviation-center-line_mean0.9118070399743456
deviation-center-line_min0.16635233426065268
deviation-heading_max30.712966764077652
deviation-heading_mean12.071833772307716
deviation-heading_median8.102353395533342
deviation-heading_min1.369661534086524
distance-from-start_max2.326127210132977
distance-from-start_mean1.26739642763406
distance-from-start_median1.371729250201632
distance-from-start_min1.1102230246251563e-16
driven_any_max5.706014592237314
driven_any_mean2.2451654021001595
driven_any_median1.6373235080815955
driven_any_min1.3322676295501878e-13
driven_lanedir_consec_max5.700993068656802
driven_lanedir_consec_mean2.241221244439758
driven_lanedir_consec_min0.0
driven_lanedir_max5.7009930700123315
driven_lanedir_mean2.2412212447786404
driven_lanedir_median1.6319459545511148
driven_lanedir_min0.0
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"4way-000-ego0": {"driven_any": 2.57110973235146, "survival_time": 59.99999999999873, "driven_lanedir": 2.5636131933572193, "in-drivable-lane": 0.0, "deviation-heading": 6.067035502566059, "distance-from-start": 2.0967710836100393, "deviation-center-line": 0.5795480514789713, "driven_lanedir_consec": 2.5636131933572193}, "4way-000-ego1": {"driven_any": 1.3322676295501878e-13, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "in-drivable-lane": 0.0, "deviation-heading": 30.712966764077652, "distance-from-start": 1.1102230246251563e-16, "deviation-center-line": 2.128497010687399, "driven_lanedir_consec": 0.0}, "4way-000-ego2": {"driven_any": 5.706014592237314, "survival_time": 59.99999999999873, "driven_lanedir": 5.7009930700123315, "in-drivable-lane": 0.0, "deviation-heading": 1.369661534086524, "distance-from-start": 2.326127210132977, "deviation-center-line": 0.16635233426065268, "driven_lanedir_consec": 5.700993068656802}, "4way-000-ego3": {"driven_any": 0.703537283811731, "survival_time": 59.99999999999873, "driven_lanedir": 0.7002787157450103, "in-drivable-lane": 0.0, "deviation-heading": 10.137671288500623, "distance-from-start": 0.6466874167932248, "deviation-center-line": 0.77283076347036, "driven_lanedir_consec": 0.7002787157450103}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
18847Bea Baselines 🐤baseline-simple-yieldaido-LFVI_multi-state-sim-validationsim-1of4successnonogpu-staging-spot-0-040:24:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median4.6376162564286725
deviation-center-line_median0.2460849300744901


other stats
deviation-center-line_max0.3027462438305415
deviation-center-line_mean0.2285455619078385
deviation-center-line_min0.11926614365183257
deviation-heading_max9.946396239005722
deviation-heading_mean3.855173781568858
deviation-heading_median1.9276149321059517
deviation-heading_min1.6190690230578058
distance-from-start_max2.9886233557369257
distance-from-start_mean2.0365138251139387
distance-from-start_median2.578715972359414
distance-from-start_min4.440892098500626e-16
driven_any_max6.0684701780856685
driven_any_mean4.013770844417987
driven_any_median4.993306599792874
driven_any_min5.329070518200751e-13
driven_lanedir_consec_max6.055252305833061
driven_lanedir_consec_mean3.8326212046726016
driven_lanedir_consec_min0.0
driven_lanedir_max6.055252305833061
driven_lanedir_mean4.007057326167881
driven_lanedir_median4.986488499419231
driven_lanedir_min0.0
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"4way-001-ego0": {"driven_any": 5.329070518200751e-13, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "in-drivable-lane": 0.0, "deviation-heading": 9.946396239005722, "distance-from-start": 4.440892098500626e-16, "deviation-center-line": 0.3027462438305415, "driven_lanedir_consec": 0.0}, "4way-001-ego1": {"driven_any": 5.9151446829206735, "survival_time": 59.99999999999873, "driven_lanedir": 5.907112719490209, "in-drivable-lane": 0.0, "deviation-heading": 1.6190690230578058, "distance-from-start": 2.694991871881108, "deviation-center-line": 0.2170574921856774, "driven_lanedir_consec": 5.503953288263572}, "4way-001-ego2": {"driven_any": 4.071468516665073, "survival_time": 59.99999999999873, "driven_lanedir": 4.0658642793482525, "in-drivable-lane": 0.0, "deviation-heading": 1.8176869322712883, "distance-from-start": 2.9886233557369257, "deviation-center-line": 0.11926614365183257, "driven_lanedir_consec": 3.7712792245937727}, "4way-001-ego3": {"driven_any": 6.0684701780856685, "survival_time": 59.99999999999873, "driven_lanedir": 6.055252305833061, "in-drivable-lane": 0.0, "deviation-heading": 2.037542931940615, "distance-from-start": 2.462440072837721, "deviation-center-line": 0.2751123679633027, "driven_lanedir_consec": 6.055252305833061}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
18249Bea Baselines 🐤baseline-simple-yieldaido-LFP-state-sim-validationsim-0of4successnonogpu-staging-spot-0-040:02:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median8.999999999999993
in-drivable-lane_median0.0
driven_lanedir_consec_median0.7940788932080864
deviation-center-line_median0.09395078397363615


other stats
deviation-center-line_max0.09395078397363615
deviation-center-line_mean0.09395078397363615
deviation-center-line_min0.09395078397363615
deviation-heading_max0.9391470252254364
deviation-heading_mean0.9391470252254364
deviation-heading_median0.9391470252254364
deviation-heading_min0.9391470252254364
distance-from-start_max0.6761226695999434
distance-from-start_mean0.6761226695999434
distance-from-start_median0.6761226695999434
distance-from-start_min0.6761226695999434
driven_any_max0.801037708816156
driven_any_mean0.801037708816156
driven_any_median0.801037708816156
driven_any_min0.801037708816156
driven_lanedir_consec_max0.7940788932080864
driven_lanedir_consec_mean0.7940788932080864
driven_lanedir_consec_min0.7940788932080864
driven_lanedir_max0.7940788932080864
driven_lanedir_mean0.7940788932080864
driven_lanedir_median0.7940788932080864
driven_lanedir_min0.7940788932080864
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"loop2-000-ego0": {"driven_any": 0.801037708816156, "survival_time": 8.999999999999993, "driven_lanedir": 0.7940788932080864, "in-drivable-lane": 0.0, "deviation-heading": 0.9391470252254364, "distance-from-start": 0.6761226695999434, "deviation-center-line": 0.09395078397363615, "driven_lanedir_consec": 0.7940788932080864}}
simulation-passed1
survival_time_max8.999999999999993
survival_time_mean8.999999999999993
survival_time_min8.999999999999993
No reset possible
17550Bea Baselines 🐤baseline-simple-yieldaido-LF-state-sim-testingsim-2of4successnonogpu-staging-spot-0-040:09:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median5.73984349568068
deviation-center-line_median0.34563700639694345


other stats
deviation-center-line_max0.34563700639694345
deviation-center-line_mean0.34563700639694345
deviation-center-line_min0.34563700639694345
deviation-heading_max2.5694983917919236
deviation-heading_mean2.5694983917919236
deviation-heading_median2.5694983917919236
deviation-heading_min2.5694983917919236
distance-from-start_max2.7200533759378014
distance-from-start_mean2.7200533759378014
distance-from-start_median2.7200533759378014
distance-from-start_min2.7200533759378014
driven_any_max5.757169705884872
driven_any_mean5.757169705884872
driven_any_median5.757169705884872
driven_any_min5.757169705884872
driven_lanedir_consec_max5.73984349568068
driven_lanedir_consec_mean5.73984349568068
driven_lanedir_consec_min5.73984349568068
driven_lanedir_max5.73984349568068
driven_lanedir_mean5.73984349568068
driven_lanedir_median5.73984349568068
driven_lanedir_min5.73984349568068
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"autolab-000-ego0": {"driven_any": 5.757169705884872, "survival_time": 59.99999999999873, "driven_lanedir": 5.73984349568068, "in-drivable-lane": 0.0, "deviation-heading": 2.5694983917919236, "distance-from-start": 2.7200533759378014, "deviation-center-line": 0.34563700639694345, "driven_lanedir_consec": 5.73984349568068}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
16653Bea Baselines 🐤baseline-simple-yieldaido-LFP-state-sim-testingsim-0of4successnonogpu-staging-spot-0-040:04:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median28.700000000000273
in-drivable-lane_median0.0
driven_lanedir_consec_median2.466265024742011
deviation-center-line_median0.20118057030596695


other stats
deviation-center-line_max0.20118057030596695
deviation-center-line_mean0.20118057030596695
deviation-center-line_min0.20118057030596695
deviation-heading_max1.8119531955727164
deviation-heading_mean1.8119531955727164
deviation-heading_median1.8119531955727164
deviation-heading_min1.8119531955727164
distance-from-start_max1.5216486851402575
distance-from-start_mean1.5216486851402575
distance-from-start_median1.5216486851402575
distance-from-start_min1.5216486851402575
driven_any_max2.4774336491708535
driven_any_mean2.4774336491708535
driven_any_median2.4774336491708535
driven_any_min2.4774336491708535
driven_lanedir_consec_max2.466265024742011
driven_lanedir_consec_mean2.466265024742011
driven_lanedir_consec_min2.466265024742011
driven_lanedir_max2.466265024742011
driven_lanedir_mean2.466265024742011
driven_lanedir_median2.466265024742011
driven_lanedir_min2.466265024742011
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"loop2-000-ego0": {"driven_any": 2.4774336491708535, "survival_time": 28.700000000000273, "driven_lanedir": 2.466265024742011, "in-drivable-lane": 0.0, "deviation-heading": 1.8119531955727164, "distance-from-start": 1.5216486851402575, "deviation-center-line": 0.20118057030596695, "driven_lanedir_consec": 2.466265024742011}}
simulation-passed1
survival_time_max28.700000000000273
survival_time_mean28.700000000000273
survival_time_min28.700000000000273
No reset possible
14226Bea Baselines 🐤minimal-agentaido-LFV_multi-sim-testingsim-3of4successnonogpu-staging-spot-0-040:06:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median17.650000000000116
in-drivable-lane_median11.725000000000108
driven_lanedir_consec_median0.24314588875643384
deviation-center-line_median0.492512294919456


other stats
deviation-center-line_max0.8126244209313231
deviation-center-line_mean0.5002146341831466
deviation-center-line_min0.2032095259623513
deviation-heading_max11.126245949311896
deviation-heading_mean4.236373829435642
deviation-heading_median2.6228060251373986
deviation-heading_min0.5736373181558746
distance-from-start_max2.7459838926308224
distance-from-start_mean1.675681764213373
distance-from-start_median1.9783715821113348
distance-from-start_min2.220446049250313e-16
driven_any_max2.746678396584005
driven_any_mean1.6759923664295098
driven_any_median1.9786455345669784
driven_any_min7.838174553853605e-14
driven_lanedir_consec_max1.2107394186161191
driven_lanedir_consec_mean0.4242577990322467
driven_lanedir_consec_min0.0
driven_lanedir_max1.2107394186161191
driven_lanedir_mean0.4242577990322467
driven_lanedir_median0.24314588875643384
driven_lanedir_min0.0
in-drivable-lane_max15.800000000000114
in-drivable-lane_mean9.812500000000082
in-drivable-lane_min0.0
per-episodes
details{"loop-000-ego0": {"driven_any": 2.746678396584005, "survival_time": 17.650000000000116, "driven_lanedir": 0.24871177969974267, "in-drivable-lane": 14.200000000000086, "deviation-heading": 3.18255248934304, "distance-from-start": 2.7459838926308224, "deviation-center-line": 0.33528240826838673, "driven_lanedir_consec": 0.24871177969974267}, "loop-000-ego1": {"driven_any": 1.72526901924623, "survival_time": 17.650000000000116, "driven_lanedir": 0.237579997813125, "in-drivable-lane": 15.800000000000114, "deviation-heading": 0.5736373181558746, "distance-from-start": 1.725159137636758, "deviation-center-line": 0.2032095259623513, "driven_lanedir_consec": 0.237579997813125}, "loop-000-ego2": {"driven_any": 2.232022049887727, "survival_time": 17.650000000000116, "driven_lanedir": 1.2107394186161191, "in-drivable-lane": 9.250000000000131, "deviation-heading": 2.0630595609317575, "distance-from-start": 2.2315840265859115, "deviation-center-line": 0.6497421815705252, "driven_lanedir_consec": 1.2107394186161191}, "loop-000-ego3": {"driven_any": 7.838174553853605e-14, "survival_time": 17.650000000000116, "driven_lanedir": 0.0, "in-drivable-lane": 0.0, "deviation-heading": 11.126245949311896, "distance-from-start": 2.220446049250313e-16, "deviation-center-line": 0.8126244209313231, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max17.650000000000116
survival_time_mean17.650000000000116
survival_time_min17.650000000000116
No reset possible
13828Bea Baselines 🐤minimal-agentaido-LF-sim-testingsim-2of4successnonogpu-staging-spot-0-040:02:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.13666552017194578
survival_time_median7.349999999999982
deviation-center-line_median0.10783809919009717
in-drivable-lane_median5.9499999999999815


other stats
deviation-center-line_max0.10783809919009717
deviation-center-line_mean0.10783809919009717
deviation-center-line_min0.10783809919009717
deviation-heading_max0.9508048428349386
deviation-heading_mean0.9508048428349386
deviation-heading_median0.9508048428349386
deviation-heading_min0.9508048428349386
distance-from-start_max1.043144324612823
distance-from-start_mean1.043144324612823
distance-from-start_median1.043144324612823
distance-from-start_min1.043144324612823
driven_any_max1.0431971841733916
driven_any_mean1.0431971841733916
driven_any_median1.0431971841733916
driven_any_min1.0431971841733916
driven_lanedir_consec_max0.13666552017194578
driven_lanedir_consec_mean0.13666552017194578
driven_lanedir_consec_min0.13666552017194578
driven_lanedir_max0.13666552017194578
driven_lanedir_mean0.13666552017194578
driven_lanedir_median0.13666552017194578
driven_lanedir_min0.13666552017194578
in-drivable-lane_max5.9499999999999815
in-drivable-lane_mean5.9499999999999815
in-drivable-lane_min5.9499999999999815
per-episodes
details{"autolab-000-ego0": {"driven_any": 1.0431971841733916, "survival_time": 7.349999999999982, "driven_lanedir": 0.13666552017194578, "in-drivable-lane": 5.9499999999999815, "deviation-heading": 0.9508048428349386, "distance-from-start": 1.043144324612823, "deviation-center-line": 0.10783809919009717, "driven_lanedir_consec": 0.13666552017194578}}
simulation-passed1
survival_time_max7.349999999999982
survival_time_mean7.349999999999982
survival_time_min7.349999999999982
No reset possible
161Bea Baselines 🐤template-randomaido-LF-sim-validationsim-3of4host-errornonogpu-staging-spot-0-040:00:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
151Bea Baselines 🐤template-randomaido-LF-sim-validationsim-1of4host-errornonogpu-staging-spot-0-040:00:04
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
141Bea Baselines 🐤template-randomaido-LF-sim-validationsim-0of4host-errornonogpu-staging-spot-0-040:00:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
131Bea Baselines 🐤template-randomaido-LF-sim-validationsim-2of4host-errornonogpu-staging-spot-0-040:00:04
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
122Bea Baselines 🐤template-randomaido-LFP-sim-validationsim-3of4host-errornonogpu-staging-spot-0-040:00:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
112Bea Baselines 🐤template-randomaido-LFP-sim-validationsim-1of4host-errornonogpu-staging-spot-0-040:00:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102Bea Baselines 🐤template-randomaido-LFP-sim-validationsim-0of4host-errornonogpu-staging-spot-0-040:00:04
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
92Bea Baselines 🐤template-randomaido-LFP-sim-validationsim-2of4host-errornonogpu-staging-spot-0-040:00:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
83Bea Baselines 🐤template-randomaido-LFVI_multi-sim-validationsim-0of4host-errornonogpu-staging-spot-0-040:00:23
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
34Bea Baselines 🐤template-randomaido-LFV_multi-sim-validationsim-0of4successnonogpu-staging-spot-0-040:11:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median29.000000000000277
in-drivable-lane_median20.025000000000247
driven_lanedir_consec_median0.2300486386994012
deviation-center-line_median0.7278799269429306


other stats
deviation-center-line_max0.8194317074657906
deviation-center-line_mean0.7278799269429306
deviation-center-line_min0.6363281464200704
deviation-heading_max6.73261632177244
deviation-heading_mean6.633830808409963
deviation-heading_median6.633830808409963
deviation-heading_min6.535045295047486
distance-from-start_max0.8626308120701062
distance-from-start_mean0.8293644349619178
distance-from-start_median0.8293644349619178
distance-from-start_min0.7960980578537296
driven_any_max7.22251940565216
driven_any_mean6.586737153678611
driven_any_median6.586737153678611
driven_any_min5.950954901705061
driven_lanedir_consec_max0.2657607447138941
driven_lanedir_consec_mean0.2300486386994012
driven_lanedir_consec_min0.19433653268490825
driven_lanedir_max0.24775126076977605
driven_lanedir_mean0.21437974622313805
driven_lanedir_median0.21437974622313805
driven_lanedir_min0.18100823167650004
in-drivable-lane_max20.30000000000025
in-drivable-lane_mean20.025000000000247
in-drivable-lane_min19.750000000000245
per-episodes
details{"loop2-000-ego0": {"driven_any": 5.950954901705061, "survival_time": 29.000000000000277, "driven_lanedir": 0.24775126076977605, "in-drivable-lane": 19.750000000000245, "deviation-heading": 6.73261632177244, "distance-from-start": 0.7960980578537296, "deviation-center-line": 0.8194317074657906, "driven_lanedir_consec": 0.2657607447138941}, "loop2-000-ego1": {"driven_any": 7.22251940565216, "survival_time": 29.000000000000277, "driven_lanedir": 0.18100823167650004, "in-drivable-lane": 20.30000000000025, "deviation-heading": 6.535045295047486, "distance-from-start": 0.8626308120701062, "deviation-center-line": 0.6363281464200704, "driven_lanedir_consec": 0.19433653268490825}}
simulation-passed1
survival_time_max29.000000000000277
survival_time_mean29.000000000000277
survival_time_min29.000000000000277
No reset possible