Evaluator 1

sim-0of4

success

aido-LFI-sim-validation

2021-10-18 23:36:41+00:00

2021-10-18 23:39:21+00:00

0:02:40

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	7.349999999999982
in-drivable-lane_median	5.599999999999984
driven_lanedir_consec_median	0.1672873755073363
deviation-center-line_median	0.1271097357365434

other stats

deviation-center-line_max	0.1271097357365434
deviation-center-line_mean	0.1271097357365434
deviation-center-line_min	0.1271097357365434
deviation-heading_max	1.6700830083320053
deviation-heading_mean	1.6700830083320053
deviation-heading_median	1.6700830083320053
deviation-heading_min	1.6700830083320053
distance-from-start_max	1.777179504647827
distance-from-start_mean	1.777179504647827
distance-from-start_median	1.777179504647827
distance-from-start_min	1.777179504647827
driven_any_max	1.824530991374534
driven_any_mean	1.824530991374534
driven_any_median	1.824530991374534
driven_any_min	1.824530991374534
driven_lanedir_consec_max	0.1672873755073363
driven_lanedir_consec_mean	0.1672873755073363
driven_lanedir_consec_min	0.1672873755073363
driven_lanedir_max	0.1672873755073363
driven_lanedir_mean	0.1672873755073363
driven_lanedir_median	0.1672873755073363
driven_lanedir_min	0.1672873755073363
in-drivable-lane_max	5.599999999999984
in-drivable-lane_mean	5.599999999999984
in-drivable-lane_min	5.599999999999984
per-episodes	details {"loop2-000-ego0": {"driven_any": 1.824530991374534, "survival_time": 7.349999999999982, "driven_lanedir": 0.1672873755073363, "in-drivable-lane": 5.599999999999984, "deviation-heading": 1.6700830083320053, "distance-from-start": 1.777179504647827, "deviation-center-line": 0.1271097357365434, "driven_lanedir_consec": 0.1672873755073363}}
simulation-passed	1
survival_time_max	7.349999999999982
survival_time_mean	7.349999999999982
survival_time_min	7.349999999999982

No reset possible

511

72

Bea Baselines 🐤

template-ros

sim-1of4

success

aido-LFVI_multi-sim-validation

2021-10-18 23:34:06+00:00

2021-10-18 23:36:21+00:00

0:02:15

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	2.3499999999999996
in-drivable-lane_median	1.1999999999999993
driven_lanedir_consec_median	0.1988796151850476
deviation-center-line_median	0.07270560080119294

other stats

deviation-center-line_max	0.07270560080119294
deviation-center-line_mean	0.07270560080119294
deviation-center-line_min	0.07270560080119294
deviation-heading_max	0.6205061311705431
deviation-heading_mean	0.6205061311705431
deviation-heading_median	0.6205061311705431
deviation-heading_min	0.6205061311705431
distance-from-start_max	0.3836452552534402
distance-from-start_mean	0.3836452552534402
distance-from-start_median	0.3836452552534402
distance-from-start_min	0.3836452552534402
driven_any_max	0.39002844671480263
driven_any_mean	0.39002844671480263
driven_any_median	0.39002844671480263
driven_any_min	0.39002844671480263
driven_lanedir_consec_max	0.1988796151850476
driven_lanedir_consec_mean	0.1988796151850476
driven_lanedir_consec_min	0.1988796151850476
driven_lanedir_max	0.1988796151850476
driven_lanedir_mean	0.1988796151850476
driven_lanedir_median	0.1988796151850476
driven_lanedir_min	0.1988796151850476
in-drivable-lane_max	1.1999999999999993
in-drivable-lane_mean	1.1999999999999993
in-drivable-lane_min	1.1999999999999993
per-episodes	details {"udem1-000-ego0": {"driven_any": 0.39002844671480263, "survival_time": 2.3499999999999996, "driven_lanedir": 0.1988796151850476, "in-drivable-lane": 1.1999999999999993, "deviation-heading": 0.6205061311705431, "distance-from-start": 0.3836452552534402, "deviation-center-line": 0.07270560080119294, "driven_lanedir_consec": 0.1988796151850476}}
simulation-passed	1
survival_time_max	2.3499999999999996
survival_time_mean	2.3499999999999996
survival_time_min	2.3499999999999996

No reset possible

492

74

Bea Baselines 🐤

template-ros

sim-0of4

host-error

2021-10-18 23:28:42+00:00

2021-10-18 23:33:58+00:00

0:05:16

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

482

76

Bea Baselines 🐤

template-ros

sim-3of4

success

2021-10-18 23:26:16+00:00

2021-10-18 23:28:22+00:00

0:02:06

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	8.549999999999986
in-drivable-lane_median	6.499999999999981
driven_lanedir_consec_median	0.24783714366018209
deviation-center-line_median	0.12337466679710549

other stats

deviation-center-line_max	0.12337466679710549
deviation-center-line_mean	0.12337466679710549
deviation-center-line_min	0.12337466679710549
deviation-heading_max	1.6305856009146025
deviation-heading_mean	1.6305856009146025
deviation-heading_median	1.6305856009146025
deviation-heading_min	1.6305856009146025
distance-from-start_max	1.9971303615810716
distance-from-start_mean	1.9971303615810716
distance-from-start_median	1.9971303615810716
distance-from-start_min	1.9971303615810716
driven_any_max	2.1826203707495946
driven_any_mean	2.1826203707495946
driven_any_median	2.1826203707495946
driven_any_min	2.1826203707495946
driven_lanedir_consec_max	0.24783714366018209
driven_lanedir_consec_mean	0.24783714366018209
driven_lanedir_consec_min	0.24783714366018209
driven_lanedir_max	0.24783714366018209
driven_lanedir_mean	0.24783714366018209
driven_lanedir_median	0.24783714366018209
driven_lanedir_min	0.24783714366018209
in-drivable-lane_max	6.499999999999981
in-drivable-lane_mean	6.499999999999981
in-drivable-lane_min	6.499999999999981
per-episodes	details {"loop-000-ego0": {"driven_any": 2.1826203707495946, "survival_time": 8.549999999999986, "driven_lanedir": 0.24783714366018209, "in-drivable-lane": 6.499999999999981, "deviation-heading": 1.6305856009146025, "distance-from-start": 1.9971303615810716, "deviation-center-line": 0.12337466679710549, "driven_lanedir_consec": 0.24783714366018209}}
simulation-passed	1
survival_time_max	8.549999999999986
survival_time_mean	8.549999999999986
survival_time_min	8.549999999999986

No reset possible

479

77

Bea Baselines 🐤

template-ros

aido-LFV-sim-testing

sim-1of4

host-error

2021-10-18 23:25:08+00:00

2021-10-18 23:26:09+00:00

0:01:01

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 95, in docker_pull
    for ps in yield_updates(pulling):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 207, in yield_updates
    raise ZValueError(msg, all_lines=all_lines)
zuper_commons.types.exceptions.ZValueError: Weird problem - I could not get the result of the pull
all_lines:
list[30]
#0 dict[2]
   │ status: Pulling from andrea/duckietown-challenges
   │ id: registry-stage2.duckietown.org/andrea/duckietown-challenges@sha256:da87badf9f998d9ebc096fb9e42d905d43b1e62eb8a2b567864c27cddaee8109
#1 {status: Already exists, progressDetail: {} , id: e4c3d3e4f7b0}
#2 {status: Already exists, progressDetail: {} , id: 101c41d0463b}
#3 {status: Already exists, progressDetail: {} , id: 8275efcd805f}
#4 {status: Already exists, progressDetail: {} , id: 751620502a7a}
#5 {status: Already exists, progressDetail: {} , id: 0a5e725150a2}
#6 {status: Already exists, progressDetail: {} , id: 397dba5694db}
#7 {status: Already exists, progressDetail: {} , id: b453803265bc}
#8 {status: Already exists, progressDetail: {} , id: a49354af7e01}
#9 {status: Already exists, progressDetail: {} , id: 7f7c8879a275}
#10 {status: Already exists, progressDetail: {} , id: c8a1e36a7f2c}
#11 {status: Already exists, progressDetail: {} , id: f4c5789bacce}
#12 {status: Already exists, progressDetail: {} , id: 1f2a893a8dea}
#13 {status: Already exists, progressDetail: {} , id: 9b63f8e29332}
#14 {status: Already exists, progressDetail: {} , id: 4e026d7e16b0}
#15 {status: Already exists, progressDetail: {} , id: a95583779b63}
#16 {status: Already exists, progressDetail: {} , id: 8fe9a42de7fb}
#17 {status: Already exists, progressDetail: {} , id: 17002f2d0e30}
#18 {status: Already exists, progressDetail: {} , id: adfd14f3a59a}
#19 {status: Already exists, progressDetail: {} , id: b2679d954f86}
#20 {status: Already exists, progressDetail: {} , id: 6e17e5a10070}
#21 {status: Already exists, progressDetail: {} , id: a834518edbc8}
#22 {status: Already exists, progressDetail: {} , id: 729f9fdfe243}
#23 {status: Already exists, progressDetail: {} , id: 168637db4042}
#24 {status: Already exists, progressDetail: {} , id: d552cb9d0c64}
#25 {status: Already exists, progressDetail: {} , id: c740b0f1eb84}
#26 {status: Already exists, progressDetail: {} , id: b3eaf99e39d8}
#27 {status: Already exists, progressDetail: {} , id: 9ddf7b64604b}
#28 {status: Pulling fs layer, progressDetail: {} , id: 3dc32f31018b}
#29 dict[2]
    │ errorDetail:
    │ dict[1]
    │ │ message: error pulling image configuration: Get https://registry-stage2.duckietown.org/v2/andrea/duckietown-challenges/blobs/sha256:4096cd509cb5acb8dbc716f4380c6aff390ef8b0cbeacd61698793086ec7c5ad: dial tcp: lookup registry-stage2.duckietown.org: Temporary failure in name resolution
    │ error: error pulling image configuration: Get https://registry-stage2.duckietown.org/v2/andrea/duckietown-challenges/blobs/sha256:4096cd509cb5acb8dbc716f4380c6aff390ef8b0cbeacd61698793086ec7c5ad: dial tcp: lookup registry-stage2.duckietown.org: Temporary failure in name resolution

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

471

78

Bea Baselines 🐤

template-ros

aido-LFVI-sim-testing

sim-1of4

host-error

2021-10-18 23:21:18+00:00

2021-10-18 23:24:42+00:00

0:03:24

The container "simul [...]

The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

462

80

Bea Baselines 🐤

template-ros

sim-2of4

host-error

2021-10-18 23:17:52+00:00

2021-10-18 23:21:05+00:00

0:03:13

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

454

83

Bea Baselines 🐤

baseline-duckietown

sim-2of4

success

aido-LFVI-sim-validation

2021-10-18 23:13:12+00:00

2021-10-18 23:17:12+00:00

0:04:00

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	11.60000000000003
in-drivable-lane_median	0.0
driven_lanedir_consec_median	1.2994805803856493
deviation-center-line_median	0.8336339241280859

other stats

deviation-center-line_max	0.8336339241280859
deviation-center-line_mean	0.8336339241280859
deviation-center-line_min	0.8336339241280859
deviation-heading_max	3.938954882155482
deviation-heading_mean	3.938954882155482
deviation-heading_median	3.938954882155482
deviation-heading_min	3.938954882155482
distance-from-start_max	1.109816036716749
distance-from-start_mean	1.109816036716749
distance-from-start_median	1.109816036716749
distance-from-start_min	1.109816036716749
driven_any_max	1.3887619925172674
driven_any_mean	1.3887619925172674
driven_any_median	1.3887619925172674
driven_any_min	1.3887619925172674
driven_lanedir_consec_max	1.2994805803856493
driven_lanedir_consec_mean	1.2994805803856493
driven_lanedir_consec_min	1.2994805803856493
driven_lanedir_max	1.2994805803856493
driven_lanedir_mean	1.2994805803856493
driven_lanedir_median	1.2994805803856493
driven_lanedir_min	1.2994805803856493
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"autolab-000-ego0": {"driven_any": 1.3887619925172674, "survival_time": 11.60000000000003, "driven_lanedir": 1.2994805803856493, "in-drivable-lane": 0.0, "deviation-heading": 3.938954882155482, "distance-from-start": 1.109816036716749, "deviation-center-line": 0.8336339241280859, "driven_lanedir_consec": 1.2994805803856493}}
simulation-passed	1
survival_time_max	11.60000000000003
survival_time_mean	11.60000000000003
survival_time_min	11.60000000000003

No reset possible

444

85

Bea Baselines 🐤

baseline-duckietown

sim-1of4

host-error

aido-LFVI-sim-validation

2021-10-18 23:08:08+00:00

2021-10-18 23:12:53+00:00

0:04:45

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

438

85

Bea Baselines 🐤

baseline-duckietown

sim-1of4

host-error

aido-LFI-sim-validation

2021-10-18 23:04:41+00:00

2021-10-18 23:08:00+00:00

0:03:19

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

424

86

Bea Baselines 🐤

baseline-duckietown

sim-1of4

success

aido-LFI-sim-validation

2021-10-18 22:58:32+00:00

2021-10-18 23:04:21+00:00

0:05:49

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	27.15000000000025
in-drivable-lane_median	3.2000000000000357
driven_lanedir_consec_median	2.738223726880766
deviation-center-line_median	2.3010233920668317

other stats

deviation-center-line_max	2.3010233920668317
deviation-center-line_mean	2.3010233920668317
deviation-center-line_min	2.3010233920668317
deviation-heading_max	6.967377665958814
deviation-heading_mean	6.967377665958814
deviation-heading_median	6.967377665958814
deviation-heading_min	6.967377665958814
distance-from-start_max	2.090879368710883
distance-from-start_mean	2.090879368710883
distance-from-start_median	2.090879368710883
distance-from-start_min	2.090879368710883
driven_any_max	3.4533227557500545
driven_any_mean	3.4533227557500545
driven_any_median	3.4533227557500545
driven_any_min	3.4533227557500545
driven_lanedir_consec_max	2.738223726880766
driven_lanedir_consec_mean	2.738223726880766
driven_lanedir_consec_min	2.738223726880766
driven_lanedir_max	2.738223726880766
driven_lanedir_mean	2.738223726880766
driven_lanedir_median	2.738223726880766
driven_lanedir_min	2.738223726880766
in-drivable-lane_max	3.2000000000000357
in-drivable-lane_mean	3.2000000000000357
in-drivable-lane_min	3.2000000000000357
per-episodes	details {"udem1-000-ego0": {"driven_any": 3.4533227557500545, "survival_time": 27.15000000000025, "driven_lanedir": 2.738223726880766, "in-drivable-lane": 3.2000000000000357, "deviation-heading": 6.967377665958814, "distance-from-start": 2.090879368710883, "deviation-center-line": 2.3010233920668317, "driven_lanedir_consec": 2.738223726880766}}
simulation-passed	1
survival_time_max	27.15000000000025
survival_time_mean	27.15000000000025
survival_time_min	27.15000000000025

No reset possible

419

86

Bea Baselines 🐤

baseline-duckietown

sim-3of4

host-error

aido-LFV_multi-sim-testing

2021-10-18 22:55:15+00:00

2021-10-18 22:58:00+00:00

0:02:45

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

409

89

Bea Baselines 🐤

baseline-duckietown

sim-3of4

host-error

2021-10-18 22:49:07+00:00

2021-10-18 22:54:05+00:00

0:04:58

The container "simul [...]

The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

404

87

Bea Baselines 🐤

baseline-duckietown

sim-2of4

host-error

2021-10-18 22:45:15+00:00

2021-10-18 22:48:55+00:00

0:03:40

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

393

90

Bea Baselines 🐤

baseline-duckietown

sim-0of4

host-error

2021-10-18 22:37:56+00:00

2021-10-18 22:45:01+00:00

0:07:05

The container "simul [...]

The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

390

90

Bea Baselines 🐤

baseline-duckietown

sim-1of4

success

2021-10-18 22:34:42+00:00

2021-10-18 22:37:49+00:00

0:03:07

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	8.399999999999984
in-drivable-lane_median	0.39999999999999947
driven_lanedir_consec_median	0.6426193141851129
deviation-center-line_median	0.7480848838142601

other stats

deviation-center-line_max	0.7480848838142601
deviation-center-line_mean	0.7480848838142601
deviation-center-line_min	0.7480848838142601
deviation-heading_max	4.21014512866558
deviation-heading_mean	4.21014512866558
deviation-heading_median	4.21014512866558
deviation-heading_min	4.21014512866558
distance-from-start_max	0.552701493751132
distance-from-start_mean	0.552701493751132
distance-from-start_median	0.552701493751132
distance-from-start_min	0.552701493751132
driven_any_max	0.970395930035684
driven_any_mean	0.970395930035684
driven_any_median	0.970395930035684
driven_any_min	0.970395930035684
driven_lanedir_consec_max	0.6426193141851129
driven_lanedir_consec_mean	0.6426193141851129
driven_lanedir_consec_min	0.6426193141851129
driven_lanedir_max	0.7504705516314369
driven_lanedir_mean	0.7504705516314369
driven_lanedir_median	0.7504705516314369
driven_lanedir_min	0.7504705516314369
in-drivable-lane_max	0.39999999999999947
in-drivable-lane_mean	0.39999999999999947
in-drivable-lane_min	0.39999999999999947
per-episodes	details {"zigzag-000-ego0": {"driven_any": 0.970395930035684, "survival_time": 8.399999999999984, "driven_lanedir": 0.7504705516314369, "in-drivable-lane": 0.39999999999999947, "deviation-heading": 4.21014512866558, "distance-from-start": 0.552701493751132, "deviation-center-line": 0.7480848838142601, "driven_lanedir_consec": 0.6426193141851129}}
simulation-passed	1
survival_time_max	8.399999999999984
survival_time_mean	8.399999999999984
survival_time_min	8.399999999999984

No reset possible

381

91

Bea Baselines 🐤

baseline-duckietown

aido-LFV-sim-testing

sim-2of4

host-error

2021-10-18 22:28:46+00:00

2021-10-18 22:34:35+00:00

0:05:49

The container "simul [...]

The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

374

93

Bea Baselines 🐤

baseline-duckietown

aido-LFI-sim-testing

sim-1of4

host-error

2021-10-18 22:25:00+00:00

2021-10-18 22:28:28+00:00

0:03:28

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

359

97

Bea Baselines 🐤

baseline-duckietown-ml

sim-1of4

aborted

aido-LFVI_multi-sim-testing

2021-10-18 22:13:54+00:00

2021-10-18 22:24:48+00:00

0:10:54

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 391, in main
    stats = future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 426, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

352

95

Bea Baselines 🐤

baseline-duckietown

sim-2of4

host-error

2021-10-18 22:09:05+00:00

2021-10-18 22:13:47+00:00

0:04:42

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

340

97

Bea Baselines 🐤

baseline-duckietown-ml

sim-3of4

host-error

2021-10-18 21:58:19+00:00

2021-10-18 22:08:59+00:00

0:10:40

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

339

97

Bea Baselines 🐤

baseline-duckietown-ml

sim-0of4

host-error

aido-LFVI-sim-validation

2021-10-18 21:55:31+00:00

2021-10-18 21:58:12+00:00

0:02:41

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1245, in run_one
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 889, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 774, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

329

99

Bea Baselines 🐤

baseline-duckietown-ml

sim-0of4

host-error

aido-LFVI_multi-sim-validation

2021-10-18 21:51:03+00:00

2021-10-18 21:55:06+00:00

0:04:03

The container "simul [...]

The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

320

102

Bea Baselines 🐤

baseline-duckietown-ml

sim-0of4

host-error

2021-10-18 21:46:36+00:00

2021-10-18 21:50:42+00:00

0:04:06

The container "simul [...]

The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

307

104

Bea Baselines 🐤

baseline-duckietown-ml

sim-3of4

host-error

2021-10-18 21:37:16+00:00

2021-10-18 21:46:17+00:00

0:09:01

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 862, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 48, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 275, in upload
    shutil.copy(realfile0, realfile)
  File "/usr/lib/python3.8/shutil.py", line 418, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/aido-LFP-sim-testing/submission104/sim-3of4-nogpu-staging-spot-0-05-job307-a-wd/challenge-evaluation-output/episodes/loop-000/ego0/camera.mp4-active.avi'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

281

109

Bea Baselines 🐤

baseline-duckietown-ml

sim-0of4

success

2021-10-18 21:14:30+00:00

2021-10-18 21:22:55+00:00

0:08:25

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.0
survival_time_median	59.99999999999873
deviation-center-line_median	0.4065102356188296
in-drivable-lane_median	0.0

other stats

deviation-center-line_max	0.4065102356188296
deviation-center-line_mean	0.4065102356188296
deviation-center-line_min	0.4065102356188296
deviation-heading_max	40.967332753486225
deviation-heading_mean	40.967332753486225
deviation-heading_median	40.967332753486225
deviation-heading_min	40.967332753486225
distance-from-start_max	8.326672684688674e-17
distance-from-start_mean	8.326672684688674e-17
distance-from-start_median	8.326672684688674e-17
distance-from-start_min	8.326672684688674e-17
driven_any_max	9.992007221626409e-14
driven_any_mean	9.992007221626409e-14
driven_any_median	9.992007221626409e-14
driven_any_min	9.992007221626409e-14
driven_lanedir_consec_max	0.0
driven_lanedir_consec_mean	0.0
driven_lanedir_consec_min	0.0
driven_lanedir_max	0.0
driven_lanedir_mean	0.0
driven_lanedir_median	0.0
driven_lanedir_min	0.0
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"loop2-000-ego0": {"driven_any": 9.992007221626409e-14, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "in-drivable-lane": 0.0, "deviation-heading": 40.967332753486225, "distance-from-start": 8.326672684688674e-17, "deviation-center-line": 0.4065102356188296, "driven_lanedir_consec": 0.0}}
simulation-passed	1
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

279

109

Bea Baselines 🐤

baseline-duckietown-ml

sim-3of4

host-error

2021-10-18 21:06:19+00:00

2021-10-18 21:13:57+00:00

0:07:38

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 862, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 48, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 275, in upload
    shutil.copy(realfile0, realfile)
  File "/usr/lib/python3.8/shutil.py", line 418, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/aido-LF-sim-testing/submission109/sim-3of4-nogpu-staging-spot-0-05-job279-a-wd/challenge-evaluation-output/episodes/loop-000/ego0/camera.mp4-active.avi'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

276

109

Bea Baselines 🐤

baseline-duckietown-ml

sim-2of4

host-error

aido-LF-state-sim-validation

2021-10-18 21:02:50+00:00

2021-10-18 21:06:06+00:00

0:03:16

The container "simul [...]

The container "simulator" exited with code 137.


Error code 137 means an out-of-memory error. The container exceeded the memory available to it.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

271

30

Bea Baselines 🐤

minimal-agent-state

sim-2of4

failed

aido-LF-state-sim-validation

2021-10-18 21:01:31+00:00

2021-10-18 21:02:24+00:00

0:00:53

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

267

30

Bea Baselines 🐤

minimal-agent-state

sim-3of4

failed

aido-LFVI_multi-state-sim-validation

2021-10-18 21:00:12+00:00

2021-10-18 21:01:11+00:00

0:00:59

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

264

31

Bea Baselines 🐤

minimal-agent-state

sim-3of4

failed

aido-LFV_multi-state-sim-validation

2021-10-18 20:58:24+00:00

2021-10-18 20:59:41+00:00

0:01:17

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

261

32

Bea Baselines 🐤

minimal-agent-state

failed

aido-LFP-state-sim-validation

2021-10-18 20:56:57+00:00

2021-10-18 20:58:12+00:00

0:01:15

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

258

33

Bea Baselines 🐤

minimal-agent-state

sim-2of4

failed

aido-LFP-state-sim-validation

2021-10-18 20:55:42+00:00

2021-10-18 20:56:51+00:00

0:01:09

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

256

33

Bea Baselines 🐤

minimal-agent-state

sim-3of4

failed

aido-LFV_multi-full-sim-validation

2021-10-18 20:54:28+00:00

2021-10-18 20:55:35+00:00

0:01:07

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

246

36

Bea Baselines 🐤

minimal-agent-full

sim-1of4

success

aido-LFVI_multi-state-sim-testing

2021-10-18 20:15:42+00:00

2021-10-18 20:52:59+00:00

0:37:17

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	59.99999999999873
in-drivable-lane_median	3.899999999999888
driven_lanedir_consec_median	2.5771471249064906
deviation-center-line_median	6.473819463827759

other stats

deviation-center-line_max	6.674142000428059
deviation-center-line_mean	6.181505966921849
deviation-center-line_min	5.104242939603817
deviation-heading_max	10.109778463057902
deviation-heading_mean	8.40091693810159
deviation-heading_median	8.144703245422594
deviation-heading_min	7.204482798503264
distance-from-start_max	2.1709881263228397
distance-from-start_mean	1.5634886461508375
distance-from-start_median	1.6003159105512157
distance-from-start_min	0.8823346371780796
driven_any_max	3.4820813210699852
driven_any_mean	3.46412233740923
driven_any_median	3.462662697063404
driven_any_min	3.4490826344401264
driven_lanedir_consec_max	3.283654781667601
driven_lanedir_consec_mean	2.5120311189354596
driven_lanedir_consec_min	1.6101754442612572
driven_lanedir_max	3.283654781667601
driven_lanedir_mean	3.242098497224624
driven_lanedir_median	3.2385172744696966
driven_lanedir_min	3.2077046582915023
in-drivable-lane_max	4.349999999999895
in-drivable-lane_mean	3.8499999999999073
in-drivable-lane_min	3.249999999999958
per-episodes	details {"zigzag-000-ego0": {"driven_any": 3.4715894450565132, "survival_time": 59.99999999999873, "driven_lanedir": 3.283654781667601, "in-drivable-lane": 3.599999999999884, "deviation-heading": 7.204482798503264, "distance-from-start": 1.6768936305156847, "deviation-center-line": 6.674142000428059, "driven_lanedir_consec": 3.283654781667601}, "zigzag-000-ego1": {"driven_any": 3.4537359490702957, "survival_time": 59.99999999999873, "driven_lanedir": 3.2159917018244157, "in-drivable-lane": 4.199999999999893, "deviation-heading": 8.20099581155267, "distance-from-start": 2.1709881263228397, "deviation-center-line": 6.659565442186463, "driven_lanedir_consec": 3.2159917018244157}, "zigzag-000-ego2": {"driven_any": 3.4820813210699852, "survival_time": 59.99999999999873, "driven_lanedir": 3.261042847114978, "in-drivable-lane": 3.249999999999958, "deviation-heading": 10.109778463057902, "distance-from-start": 1.5237381905867462, "deviation-center-line": 5.104242939603817, "driven_lanedir_consec": 1.6101754442612572}, "zigzag-000-ego3": {"driven_any": 3.4490826344401264, "survival_time": 59.99999999999873, "driven_lanedir": 3.2077046582915023, "in-drivable-lane": 4.349999999999895, "deviation-heading": 8.08841067929252, "distance-from-start": 0.8823346371780796, "deviation-center-line": 6.288073485469056, "driven_lanedir_consec": 1.9383025479885652}}
simulation-passed	1
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

242

37

Bea Baselines 🐤

minimal-agent-state

sim-0of4

failed

aido-LFVI_multi-state-sim-testing

2021-10-18 20:14:00+00:00

2021-10-18 20:15:15+00:00

0:01:15

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

239

37

Bea Baselines 🐤

minimal-agent-state

sim-1of4

failed

aido-LF-full-sim-validation

2021-10-18 20:12:18+00:00

2021-10-18 20:13:34+00:00

0:01:16

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

229

40

Bea Baselines 🐤

minimal-agent-full

sim-2of4

success

aido-LFP-state-sim-testing

2021-10-18 20:03:08+00:00

2021-10-18 20:11:59+00:00

0:08:51

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	59.99999999999873
in-drivable-lane_median	1.899999999999892
driven_lanedir_consec_median	3.40640170732006
deviation-center-line_median	5.159547195129483

other stats

deviation-center-line_max	5.159547195129483
deviation-center-line_mean	5.159547195129483
deviation-center-line_min	5.159547195129483
deviation-heading_max	7.059107454008588
deviation-heading_mean	7.059107454008588
deviation-heading_median	7.059107454008588
deviation-heading_min	7.059107454008588
distance-from-start_max	2.2176127613608925
distance-from-start_mean	2.2176127613608925
distance-from-start_median	2.2176127613608925
distance-from-start_min	2.2176127613608925
driven_any_max	3.532579606984142
driven_any_mean	3.532579606984142
driven_any_median	3.532579606984142
driven_any_min	3.532579606984142
driven_lanedir_consec_max	3.40640170732006
driven_lanedir_consec_mean	3.40640170732006
driven_lanedir_consec_min	3.40640170732006
driven_lanedir_max	3.40640170732006
driven_lanedir_mean	3.40640170732006
driven_lanedir_median	3.40640170732006
driven_lanedir_min	3.40640170732006
in-drivable-lane_max	1.899999999999892
in-drivable-lane_mean	1.899999999999892
in-drivable-lane_min	1.899999999999892
per-episodes	details {"autolab-000-ego0": {"driven_any": 3.532579606984142, "survival_time": 59.99999999999873, "driven_lanedir": 3.40640170732006, "in-drivable-lane": 1.899999999999892, "deviation-heading": 7.059107454008588, "distance-from-start": 2.2176127613608925, "deviation-center-line": 5.159547195129483, "driven_lanedir_consec": 3.40640170732006}}
simulation-passed	1
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

226

41

Bea Baselines 🐤

minimal-agent-state

sim-2of4

failed

aido-LFP-full-sim-testing

2021-10-18 20:01:21+00:00

2021-10-18 20:02:35+00:00

0:01:14

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v1
│         k: observations
│        v1: DB20ObservationsPlusState
│        v2: DB20ObservationsOnlyState
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "camera"
│            │ |  of type <class 'aido_schemas.protocol_simulator.JPGImageWithTimestamp'>
│            │ |  but <class 'aido_schemas.schemas.DB20ObservationsOnlyState'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│ p2_inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 249, in main
    aci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 129, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {seed: int, get_commands: GetCommands, observations: DB20ObservationsPlusState, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs: {observations: DB20ObservationsOnlyState, seed: int, get_commands: GetCommands, episode_start: EpisodeStart}
│     │ outputs: {commands: Duckiebot1Commands}
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 254, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Could not get protocol for agent 'ego0'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

217

42

Bea Baselines 🐤

minimal-agent-full

sim-2of4

success

aido-LFVI_multi-full-sim-testing

2021-10-18 19:50:06+00:00

2021-10-18 20:00:47+00:00

0:10:41

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	59.99999999999873
in-drivable-lane_median	1.0999999999999377
driven_lanedir_consec_median	3.4858726845389234
deviation-center-line_median	4.220405841276127

other stats

deviation-center-line_max	4.220405841276127
deviation-center-line_mean	4.220405841276127
deviation-center-line_min	4.220405841276127
deviation-heading_max	2.9174896176689957
deviation-heading_mean	2.9174896176689957
deviation-heading_median	2.9174896176689957
deviation-heading_min	2.9174896176689957
distance-from-start_max	2.8928218596598874
distance-from-start_mean	2.8928218596598874
distance-from-start_median	2.8928218596598874
distance-from-start_min	2.8928218596598874
driven_any_max	3.5465963319759606
driven_any_mean	3.5465963319759606
driven_any_median	3.5465963319759606
driven_any_min	3.5465963319759606
driven_lanedir_consec_max	3.4858726845389234
driven_lanedir_consec_mean	3.4858726845389234
driven_lanedir_consec_min	3.4858726845389234
driven_lanedir_max	3.4858726845389234
driven_lanedir_mean	3.4858726845389234
driven_lanedir_median	3.4858726845389234
driven_lanedir_min	3.4858726845389234
in-drivable-lane_max	1.0999999999999377
in-drivable-lane_mean	1.0999999999999377
in-drivable-lane_min	1.0999999999999377
per-episodes	details {"autolab-000-ego0": {"driven_any": 3.5465963319759606, "survival_time": 59.99999999999873, "driven_lanedir": 3.4858726845389234, "in-drivable-lane": 1.0999999999999377, "deviation-heading": 2.9174896176689957, "distance-from-start": 2.8928218596598874, "deviation-center-line": 4.220405841276127, "driven_lanedir_consec": 3.4858726845389234}}
simulation-passed	1
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

212

44

Bea Baselines 🐤

minimal-agent-full

sim-0of4

aborted

aido-LFV_multi-full-sim-testing

2021-10-18 19:16:33+00:00

2021-10-18 19:49:22+00:00

0:32:49

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 391, in main
    stats = future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 426, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

209

43

Bea Baselines 🐤

minimal-agent-full

sim-3of4

aborted

aido-LF-full-sim-testing

2021-10-18 18:48:04+00:00

2021-10-18 19:16:27+00:00

0:28:23

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 862, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 48, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 275, in upload
    shutil.copy(realfile0, realfile)
  File "/usr/lib/python3.8/shutil.py", line 418, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/aido-LFV_multi-full-sim-testing/submission43/sim-3of4-nogpu-staging-spot-0-05-job209-a-wd/challenge-evaluation-output/episodes/loop-000/ui_image.mp4.palette.tmp.png'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

202

45

Bea Baselines 🐤

minimal-agent-full

sim-0of4

success

aido-LF-full-sim-testing

2021-10-18 18:39:10+00:00

2021-10-18 18:47:24+00:00

0:08:14

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	59.99999999999873
in-drivable-lane_median	4.349999999999895
driven_lanedir_consec_median	2.09163409663777
deviation-center-line_median	5.917240924506971

other stats

deviation-center-line_max	5.917240924506971
deviation-center-line_mean	5.917240924506971
deviation-center-line_min	5.917240924506971
deviation-heading_max	8.389433016031823
deviation-heading_mean	8.389433016031823
deviation-heading_median	8.389433016031823
deviation-heading_min	8.389433016031823
distance-from-start_max	1.030783103523257
distance-from-start_mean	1.030783103523257
distance-from-start_median	1.030783103523257
distance-from-start_min	1.030783103523257
driven_any_max	3.4490819837243887
driven_any_mean	3.4490819837243887
driven_any_median	3.4490819837243887
driven_any_min	3.4490819837243887
driven_lanedir_consec_max	2.09163409663777
driven_lanedir_consec_mean	2.09163409663777
driven_lanedir_consec_min	2.09163409663777
driven_lanedir_max	3.2040229365911816
driven_lanedir_mean	3.2040229365911816
driven_lanedir_median	3.2040229365911816
driven_lanedir_min	3.2040229365911816
in-drivable-lane_max	4.349999999999895
in-drivable-lane_mean	4.349999999999895
in-drivable-lane_min	4.349999999999895
per-episodes	details {"loop2-000-ego0": {"driven_any": 3.4490819837243887, "survival_time": 59.99999999999873, "driven_lanedir": 3.2040229365911816, "in-drivable-lane": 4.349999999999895, "deviation-heading": 8.389433016031823, "distance-from-start": 1.030783103523257, "deviation-center-line": 5.917240924506971, "driven_lanedir_consec": 2.09163409663777}}
simulation-passed	1
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

199

45

Bea Baselines 🐤

minimal-agent-full

sim-1of4

success

aido-LFV_multi-state-sim-validation

2021-10-18 18:26:00+00:00

2021-10-18 18:39:04+00:00

0:13:04

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	59.99999999999873
in-drivable-lane_median	2.149999999999878
driven_lanedir_consec_median	3.364933059128951
deviation-center-line_median	5.251516924575086

other stats

deviation-center-line_max	5.251516924575086
deviation-center-line_mean	5.251516924575086
deviation-center-line_min	5.251516924575086
deviation-heading_max	8.784883477445915
deviation-heading_mean	8.784883477445915
deviation-heading_median	8.784883477445915
deviation-heading_min	8.784883477445915
distance-from-start_max	2.151743265759587
distance-from-start_mean	2.151743265759587
distance-from-start_median	2.151743265759587
distance-from-start_min	2.151743265759587
driven_any_max	3.5150892362964297
driven_any_mean	3.5150892362964297
driven_any_median	3.5150892362964297
driven_any_min	3.5150892362964297
driven_lanedir_consec_max	3.364933059128951
driven_lanedir_consec_mean	3.364933059128951
driven_lanedir_consec_min	3.364933059128951
driven_lanedir_max	3.364933059128951
driven_lanedir_mean	3.364933059128951
driven_lanedir_median	3.364933059128951
driven_lanedir_min	3.364933059128951
in-drivable-lane_max	2.149999999999878
in-drivable-lane_mean	2.149999999999878
in-drivable-lane_min	2.149999999999878
per-episodes	details {"zigzag-000-ego0": {"driven_any": 3.5150892362964297, "survival_time": 59.99999999999873, "driven_lanedir": 3.364933059128951, "in-drivable-lane": 2.149999999999878, "deviation-heading": 8.784883477445915, "distance-from-start": 2.151743265759587, "deviation-center-line": 5.251516924575086, "driven_lanedir_consec": 3.364933059128951}}
simulation-passed	1
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

193

48

Bea Baselines 🐤

baseline-simple-yield

success

aido-LF-state-sim-validation

2021-10-18 18:04:07+00:00

2021-10-18 18:25:48+00:00

0:21:41

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	59.99999999999873
in-drivable-lane_median	0.0
driven_lanedir_consec_median	5.52890795225499
deviation-center-line_median	0.3906395987118805

other stats

deviation-center-line_max	0.5366645647154024
deviation-center-line_mean	0.3952829247945452
deviation-center-line_min	0.2631879370390176
deviation-heading_max	4.218577913674367
deviation-heading_mean	3.437285272646963
deviation-heading_median	3.4534573481652857
deviation-heading_min	2.6236484805829137
distance-from-start_max	3.0439420716312515
distance-from-start_mean	2.274568172275226
distance-from-start_median	2.2828603191065455
distance-from-start_min	1.488609979256562
driven_any_max	5.920501221938654
driven_any_mean	5.79949756817544
driven_any_median	5.780047946557019
driven_any_min	5.717393157649067
driven_lanedir_consec_max	5.747121905004368
driven_lanedir_consec_mean	5.508285549833637
driven_lanedir_consec_min	5.228204389820196
driven_lanedir_max	5.752685282976317
driven_lanedir_mean	5.713982482752739
driven_lanedir_median	5.718870070593104
driven_lanedir_min	5.665504506848428
in-drivable-lane_max	1.6000000000000008
in-drivable-lane_mean	0.4000000000000002
in-drivable-lane_min	0.0
per-episodes	details {"udem1-ego0": {"driven_any": 5.752109532071804, "survival_time": 59.99999999999873, "driven_lanedir": 5.69061823618184, "in-drivable-lane": 0.0, "deviation-heading": 3.2539462096775833, "distance-from-start": 1.488609979256562, "deviation-center-line": 0.32130068454730965, "driven_lanedir_consec": 5.69061823618184}, "udem1-ego1": {"driven_any": 5.807986361042234, "survival_time": 59.99999999999873, "driven_lanedir": 5.752685282976317, "in-drivable-lane": 0.0, "deviation-heading": 3.652968486652988, "distance-from-start": 1.9661790515292268, "deviation-center-line": 0.4599785128764513, "driven_lanedir_consec": 5.228204389820196}, "udem1-ego2": {"driven_any": 5.717393157649067, "survival_time": 59.99999999999873, "driven_lanedir": 5.665504506848428, "in-drivable-lane": 0.0, "deviation-heading": 2.6236484805829137, "distance-from-start": 2.599541586683864, "deviation-center-line": 0.2631879370390176, "driven_lanedir_consec": 5.36719766832814}, "udem1-ego3": {"driven_any": 5.920501221938654, "survival_time": 59.99999999999873, "driven_lanedir": 5.747121905004368, "in-drivable-lane": 1.6000000000000008, "deviation-heading": 4.218577913674367, "distance-from-start": 3.0439420716312515, "deviation-center-line": 0.5366645647154024, "driven_lanedir_consec": 5.747121905004368}}
simulation-passed	1
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

191

46

Bea Baselines 🐤

baseline-simple-yield

sim-3of4

success

aido-LFV_multi-state-sim-testing

2021-10-18 17:55:42+00:00

2021-10-18 18:03:59+00:00

0:08:17

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	59.99999999999873
in-drivable-lane_median	0.0
driven_lanedir_consec_median	0.0
deviation-center-line_median	0.8948988630736971

other stats

deviation-center-line_max	0.8948988630736971
deviation-center-line_mean	0.8948988630736971
deviation-center-line_min	0.8948988630736971
deviation-heading_max	32.249113233675025
deviation-heading_mean	32.249113233675025
deviation-heading_median	32.249113233675025
deviation-heading_min	32.249113233675025
distance-from-start_max	4.965068306494546e-16
distance-from-start_mean	4.965068306494546e-16
distance-from-start_median	4.965068306494546e-16
distance-from-start_min	4.965068306494546e-16
driven_any_max	5.95808196779335e-13
driven_any_mean	5.95808196779335e-13
driven_any_median	5.95808196779335e-13
driven_any_min	5.95808196779335e-13
driven_lanedir_consec_max	0.0
driven_lanedir_consec_mean	0.0
driven_lanedir_consec_min	0.0
driven_lanedir_max	0.0
driven_lanedir_mean	0.0
driven_lanedir_median	0.0
driven_lanedir_min	0.0
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"loop-000-ego0": {"driven_any": 5.95808196779335e-13, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "in-drivable-lane": 0.0, "deviation-heading": 32.249113233675025, "distance-from-start": 4.965068306494546e-16, "deviation-center-line": 0.8948988630736971, "driven_lanedir_consec": 0.0}}
simulation-passed	1
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

167

52

Bea Baselines 🐤

baseline-simple-yield

sim-1of4

success

2021-10-18 17:34:33+00:00

2021-10-18 17:55:29+00:00

0:20:56

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	59.99999999999873
in-drivable-lane_median	0.0
driven_lanedir_consec_median	5.282946112107654
deviation-center-line_median	0.4487548000354511

other stats

deviation-center-line_max	0.5181038839457253
deviation-center-line_mean	0.43710587322751226
deviation-center-line_min	0.3328100088934215
deviation-heading_max	4.144049089921387
deviation-heading_mean	3.38976709217754
deviation-heading_median	3.3606085552415066
deviation-heading_min	2.69380216830576
distance-from-start_max	3.30368373695706
distance-from-start_mean	2.9649553103990725
distance-from-start_median	3.0412965599033734
distance-from-start_min	2.4735443848324823
driven_any_max	5.719883933788911
driven_any_mean	5.36507623085502
driven_any_median	5.2985881926024785
driven_any_min	5.143244604426211
driven_lanedir_consec_max	5.689084887688807
driven_lanedir_consec_mean	5.343802572164705
driven_lanedir_consec_min	5.120233176754706
driven_lanedir_max	5.689084887688807
driven_lanedir_mean	5.343802572164705
driven_lanedir_median	5.282946112107654
driven_lanedir_min	5.120233176754706
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"zigzag-000-ego0": {"driven_any": 5.719883933788911, "survival_time": 59.99999999999873, "driven_lanedir": 5.689084887688807, "in-drivable-lane": 0.0, "deviation-heading": 4.144049089921387, "distance-from-start": 3.30368373695706, "deviation-center-line": 0.5181038839457253, "driven_lanedir_consec": 5.689084887688807}, "zigzag-000-ego1": {"driven_any": 5.423492923013815, "survival_time": 59.99999999999873, "driven_lanedir": 5.404021617503196, "in-drivable-lane": 0.0, "deviation-heading": 3.134586559255208, "distance-from-start": 3.006847502058396, "deviation-center-line": 0.420082730869773, "driven_lanedir_consec": 5.404021617503196}, "zigzag-000-ego2": {"driven_any": 5.173683462191141, "survival_time": 59.99999999999873, "driven_lanedir": 5.16187060671211, "in-drivable-lane": 0.0, "deviation-heading": 2.69380216830576, "distance-from-start": 3.07574561774835, "deviation-center-line": 0.3328100088934215, "driven_lanedir_consec": 5.16187060671211}, "zigzag-000-ego3": {"driven_any": 5.143244604426211, "survival_time": 59.99999999999873, "driven_lanedir": 5.120233176754706, "in-drivable-lane": 0.0, "deviation-heading": 3.586630551227805, "distance-from-start": 2.4735443848324823, "deviation-center-line": 0.4774268692011292, "driven_lanedir_consec": 5.120233176754706}}
simulation-passed	1
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

158

23

Bea Baselines 🐤

minimal-agent

sim-0of4

success

2021-10-18 17:32:19+00:00

2021-10-18 17:34:25+00:00

0:02:06

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	12.700000000000044
in-drivable-lane_median	9.40000000000005
driven_lanedir_consec_median	0.33122977762117056
deviation-center-line_median	0.26973544501818203

other stats

deviation-center-line_max	0.26973544501818203
deviation-center-line_mean	0.26973544501818203
deviation-center-line_min	0.26973544501818203
deviation-heading_max	2.53835057480462
deviation-heading_mean	2.53835057480462
deviation-heading_median	2.53835057480462
deviation-heading_min	2.53835057480462
distance-from-start_max	1.9240493840830368
distance-from-start_mean	1.9240493840830368
distance-from-start_median	1.9240493840830368
distance-from-start_min	1.9240493840830368
driven_any_max	1.9241954407136792
driven_any_mean	1.9241954407136792
driven_any_median	1.9241954407136792
driven_any_min	1.9241954407136792
driven_lanedir_consec_max	0.33122977762117056
driven_lanedir_consec_mean	0.33122977762117056
driven_lanedir_consec_min	0.33122977762117056
driven_lanedir_max	0.33122977762117056
driven_lanedir_mean	0.33122977762117056
driven_lanedir_median	0.33122977762117056
driven_lanedir_min	0.33122977762117056
in-drivable-lane_max	9.40000000000005
in-drivable-lane_mean	9.40000000000005
in-drivable-lane_min	9.40000000000005
per-episodes	details {"loop2-000-ego0": {"driven_any": 1.9241954407136792, "survival_time": 12.700000000000044, "driven_lanedir": 0.33122977762117056, "in-drivable-lane": 9.40000000000005, "deviation-heading": 2.53835057480462, "distance-from-start": 1.9240493840830368, "deviation-center-line": 0.26973544501818203, "driven_lanedir_consec": 0.33122977762117056}}
simulation-passed	1
survival_time_max	12.700000000000044
survival_time_mean	12.700000000000044
survival_time_min	12.700000000000044

No reset possible

154

24

Bea Baselines 🐤

minimal-agent

sim-3of4

success

2021-10-18 17:30:09+00:00

2021-10-18 17:31:51+00:00

0:01:42

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.19694617160956465
survival_time_median	6.199999999999986
deviation-center-line_median	0.12539055697768176
in-drivable-lane_median	4.449999999999985

other stats

deviation-center-line_max	0.12539055697768176
deviation-center-line_mean	0.12539055697768176
deviation-center-line_min	0.12539055697768176
deviation-heading_max	0.9520924629376074
deviation-heading_mean	0.9520924629376074
deviation-heading_median	0.9520924629376074
deviation-heading_min	0.9520924629376074
distance-from-start_max	0.8501561575907637
distance-from-start_mean	0.8501561575907637
distance-from-start_median	0.8501561575907637
distance-from-start_min	0.8501561575907637
driven_any_max	0.8502902342720419
driven_any_mean	0.8502902342720419
driven_any_median	0.8502902342720419
driven_any_min	0.8502902342720419
driven_lanedir_consec_max	0.19694617160956465
driven_lanedir_consec_mean	0.19694617160956465
driven_lanedir_consec_min	0.19694617160956465
driven_lanedir_max	0.19694617160956465
driven_lanedir_mean	0.19694617160956465
driven_lanedir_median	0.19694617160956465
driven_lanedir_min	0.19694617160956465
in-drivable-lane_max	4.449999999999985
in-drivable-lane_mean	4.449999999999985
in-drivable-lane_min	4.449999999999985
per-episodes	details {"loop-000-ego0": {"driven_any": 0.8502902342720419, "survival_time": 6.199999999999986, "driven_lanedir": 0.19694617160956465, "in-drivable-lane": 4.449999999999985, "deviation-heading": 0.9520924629376074, "distance-from-start": 0.8501561575907637, "deviation-center-line": 0.12539055697768176, "driven_lanedir_consec": 0.19694617160956465}}
simulation-passed	1
survival_time_max	6.199999999999986
survival_time_mean	6.199999999999986
survival_time_min	6.199999999999986

No reset possible

144

28

Bea Baselines 🐤

minimal-agent

sim-0of4

success

aido-LFVI_multi-sim-testing

2021-10-18 17:27:20+00:00

2021-10-18 17:29:38+00:00

0:02:18

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.13666480491623578
survival_time_median	10.85000000000002
deviation-center-line_median	0.10783802485852027
in-drivable-lane_median	9.45000000000002

other stats

deviation-center-line_max	0.10783802485852027
deviation-center-line_mean	0.10783802485852027
deviation-center-line_min	0.10783802485852027
deviation-heading_max	0.9508048428349386
deviation-heading_mean	0.9508048428349386
deviation-heading_median	0.9508048428349386
deviation-heading_min	0.9508048428349386
distance-from-start_max	1.6182798682575137
distance-from-start_mean	1.6182798682575137
distance-from-start_median	1.6182798682575137
distance-from-start_min	1.6182798682575137
driven_any_max	1.6183892771013826
driven_any_mean	1.6183892771013826
driven_any_median	1.6183892771013826
driven_any_min	1.6183892771013826
driven_lanedir_consec_max	0.13666480491623578
driven_lanedir_consec_mean	0.13666480491623578
driven_lanedir_consec_min	0.13666480491623578
driven_lanedir_max	0.13666480491623578
driven_lanedir_mean	0.13666480491623578
driven_lanedir_median	0.13666480491623578
driven_lanedir_min	0.13666480491623578
in-drivable-lane_max	9.45000000000002
in-drivable-lane_mean	9.45000000000002
in-drivable-lane_min	9.45000000000002
per-episodes	details {"loop2-000-ego0": {"driven_any": 1.6183892771013826, "survival_time": 10.85000000000002, "driven_lanedir": 0.13666480491623578, "in-drivable-lane": 9.45000000000002, "deviation-heading": 0.9508048428349386, "distance-from-start": 1.6182798682575137, "deviation-center-line": 0.10783802485852027, "driven_lanedir_consec": 0.13666480491623578}}
simulation-passed	1
survival_time_max	10.85000000000002
survival_time_mean	10.85000000000002
survival_time_min	10.85000000000002

No reset possible

132

29

Bea Baselines 🐤

minimal-agent

sim-3of4

success

2021-10-18 17:24:15+00:00

2021-10-18 17:26:47+00:00

0:02:32

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

other stats

simulation-passed	1
skipped	1

No reset possible

40

1

Bea Baselines 🐤

template-random

sim-3of4

host-error

2021-10-17 16:43:15+00:00

2021-10-17 16:43:17+00:00

0:00:02

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24

1

Bea Baselines 🐤

template-random

sim-1of4

success

2021-10-17 16:33:29+00:00

2021-10-17 16:43:07+00:00

0:09:38

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.2732907192401193
survival_time_median	49.74999999999931
deviation-center-line_median	0.8940598330260405
in-drivable-lane_median	37.69999999999921

other stats

deviation-center-line_max	0.8940598330260405
deviation-center-line_mean	0.8940598330260405
deviation-center-line_min	0.8940598330260405
deviation-heading_max	9.653852545002117
deviation-heading_mean	9.653852545002117
deviation-heading_median	9.653852545002117
deviation-heading_min	9.653852545002117
distance-from-start_max	1.1376277719266692
distance-from-start_mean	1.1376277719266692
distance-from-start_median	1.1376277719266692
distance-from-start_min	1.1376277719266692
driven_any_max	12.450153382790834
driven_any_mean	12.450153382790834
driven_any_median	12.450153382790834
driven_any_min	12.450153382790834
driven_lanedir_consec_max	0.2732907192401193
driven_lanedir_consec_mean	0.2732907192401193
driven_lanedir_consec_min	0.2732907192401193
driven_lanedir_max	0.26112935377465085
driven_lanedir_mean	0.26112935377465085
driven_lanedir_median	0.26112935377465085
driven_lanedir_min	0.26112935377465085
in-drivable-lane_max	37.69999999999921
in-drivable-lane_mean	37.69999999999921
in-drivable-lane_min	37.69999999999921
per-episodes	details {"zigzag-000-ego0": {"driven_any": 12.450153382790834, "survival_time": 49.74999999999931, "driven_lanedir": 0.26112935377465085, "in-drivable-lane": 37.69999999999921, "deviation-heading": 9.653852545002117, "distance-from-start": 1.1376277719266692, "deviation-center-line": 0.8940598330260405, "driven_lanedir_consec": 0.2732907192401193}}
simulation-passed	1
survival_time_max	49.74999999999931
survival_time_mean	49.74999999999931
survival_time_min	49.74999999999931

No reset possible

23

1

Bea Baselines 🐤

template-random

sim-2of4

host-error

2021-10-17 16:33:13+00:00

2021-10-17 16:33:16+00:00

0:00:03

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

22

1

Bea Baselines 🐤

template-random

sim-0of4

host-error

2021-10-17 16:32:53+00:00

2021-10-17 16:32:56+00:00

0:00:03

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

21

2

Bea Baselines 🐤

template-random

sim-0of4

host-error

2021-10-17 16:32:32+00:00

2021-10-17 16:32:35+00:00

0:00:03

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

20

2

Bea Baselines 🐤

template-random

sim-3of4

host-error

2021-10-17 16:32:15+00:00

2021-10-17 16:32:18+00:00

0:00:03

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

19

2

Bea Baselines 🐤

template-random

sim-1of4

host-error

2021-10-17 16:31:59+00:00

2021-10-17 16:32:02+00:00

0:00:03

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

18

2

Bea Baselines 🐤

template-random

sim-2of4

host-error

aido-LFVI_multi-sim-validation

2021-10-17 16:31:35+00:00

2021-10-17 16:31:39+00:00

0:00:04

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

17

3

Bea Baselines 🐤

template-random

sim-0of4

host-error

aido-LFV_multi-sim-validation

2021-10-17 16:30:59+00:00

2021-10-17 16:31:22+00:00

0:00:23

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http+docker://localhost/v1.41/auth

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    do_login_for_registry(client, credentials, br.registry)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 78, in do_login_for_registry
    docker_login(client, registry, credentials[registry]["username"], credentials[registry]["secret"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_logging_in.py", line 35, in docker_login
    res = client.login(username=docker_username, password=docker_password, reauth=True, registry=registry)
  File "/usr/local/lib/python3.8/dist-packages/docker/client.py", line 199, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/daemon.py", line 152, in login
    return self._result(response, json=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 401 Client Error for http+docker://localhost/v1.41/auth: Unauthorized ("Get https://registry-stage2.duckietown.org/v2/: unauthorized: authentication required")

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

2

4

Bea Baselines 🐤

template-random

sim-1of4

success