Submission 44

Submission

44

Competing

user retired

Challenge

aido-LFVI_multi-full-sim-testing

User

Bea Baselines 🐤

Date submitted

2021-10-18 17:24:36+00:00

Last status update

2021-10-19 09:44:23+00:00

Complete

not complete

Details

Jobs in evaluation.

Sisters

Result

Jobs

sim-0of4: 637 sim-1of4: 633 sim-2of4: 635 sim-3of4: 636

User label

minimal-agent-full

Admin priority

Blessing

n/a

User priority

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs

Job ID

step

status

up to date

date started

date completed

duration

message

637

sim-0of4

success

2021-10-19 09:29:39+00:00

2021-10-19 09:44:17+00:00

0:14:38

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	59.99999999999873
in-drivable-lane_median	0.0
driven_lanedir_consec_median	3.133472185041853
deviation-center-line_median	3.569274924383762

other stats

deviation-center-line_max	5.185171597412875
deviation-center-line_mean	3.7385983910856466
deviation-center-line_min	2.6306721181621873
deviation-heading_max	8.12390911081905
deviation-heading_mean	4.904445209175695
deviation-heading_median	3.9916054099782254
deviation-heading_min	3.510660905927283
distance-from-start_max	2.230934169509707
distance-from-start_mean	1.8422550062375431
distance-from-start_median	1.83762780792641
distance-from-start_min	1.4628302395876458
driven_any_max	3.579587679138324
driven_any_mean	3.408211238652509
driven_any_median	3.5593312753396225
driven_any_min	2.934594724792467
driven_lanedir_consec_max	3.4288085645897515
driven_lanedir_consec_mean	2.8688896364826095
driven_lanedir_consec_min	1.7798056112569802
driven_lanedir_max	3.5615676792313065
driven_lanedir_mean	3.351579262728021
driven_lanedir_median	3.467420011836784
driven_lanedir_min	2.909909348007209
in-drivable-lane_max	1.3499999999999446
in-drivable-lane_mean	0.33749999999998614
in-drivable-lane_min	0.0
per-episodes	details {"udem-000-ego0": {"driven_any": 2.934594724792467, "survival_time": 59.99999999999873, "driven_lanedir": 2.909909348007209, "in-drivable-lane": 0.0, "deviation-heading": 4.334701504286536, "distance-from-start": 1.573887683474244, "deviation-center-line": 5.185171597412875, "driven_lanedir_consec": 2.902990889136124}, "udem-000-ego1": {"driven_any": 3.539079376082112, "survival_time": 59.99999999999873, "driven_lanedir": 3.3772467891565436, "in-drivable-lane": 1.3499999999999446, "deviation-heading": 8.12390911081905, "distance-from-start": 1.4628302395876458, "deviation-center-line": 4.1496605050413296, "driven_lanedir_consec": 1.7798056112569802}, "udem-000-ego2": {"driven_any": 3.579587679138324, "survival_time": 59.99999999999873, "driven_lanedir": 3.5615676792313065, "in-drivable-lane": 0.0, "deviation-heading": 3.510660905927283, "distance-from-start": 2.230934169509707, "deviation-center-line": 2.6306721181621873, "driven_lanedir_consec": 3.4288085645897515}, "udem-000-ego3": {"driven_any": 3.5795831745971327, "survival_time": 59.99999999999873, "driven_lanedir": 3.5575932345170247, "in-drivable-lane": 0.0, "deviation-heading": 3.648509315669915, "distance-from-start": 2.101367932378576, "deviation-center-line": 2.9888893437261945, "driven_lanedir_consec": 3.363953480947582}}
simulation-passed	1
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

636

sim-3of4

success

2021-10-19 09:29:19+00:00

2021-10-19 09:30:37+00:00

0:01:18

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

other stats

simulation-passed	1
skipped	1

No reset possible

635

sim-2of4

success

2021-10-19 09:29:16+00:00

2021-10-19 09:30:36+00:00

0:01:20

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

other stats

simulation-passed	1
skipped	1

No reset possible

633

sim-1of4

aborted

2021-10-19 09:27:23+00:00

2021-10-19 09:52:03+00:00

0:24:40

KeyboardInterrupt: T [...]

KeyboardInterrupt:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1237, in run_one
    heartbeat()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in heartbeat
    raise KeyboardInterrupt(msg_)
KeyboardInterrupt: The server told us to abort the job because: The submission has been retired.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

593

sim-0of4

success

2021-10-19 06:34:32+00:00

2021-10-19 07:01:18+00:00

0:26:46

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

592

sim-2of4

success

2021-10-19 06:34:31+00:00

2021-10-19 06:37:10+00:00

0:02:39

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

590

sim-3of4

success

2021-10-19 06:34:31+00:00

2021-10-19 06:37:32+00:00

0:03:01

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

589

sim-1of4

success

2021-10-19 06:34:30+00:00

2021-10-19 06:50:59+00:00

0:16:29

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

215

sim-1of4

aborted

2021-10-18 19:17:29+00:00

2021-10-18 19:57:04+00:00

0:39:35

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

212

sim-0of4

aborted

2021-10-18 19:16:33+00:00

2021-10-18 19:49:22+00:00

0:32:49

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 391, in main
    stats = future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 689, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 426, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

206

sim-1of4

aborted

2021-10-18 18:44:05+00:00

2021-10-18 19:16:47+00:00

0:32:42

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

205

sim-3of4

aborted

2021-10-18 18:43:28+00:00

2021-10-18 18:44:44+00:00

0:01:16

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

204

sim-2of4

aborted

2021-10-18 18:42:30+00:00

2021-10-18 18:43:52+00:00

0:01:22

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

203

sim-0of4

aborted

2021-10-18 18:42:30+00:00

2021-10-18 19:17:22+00:00

0:34:52

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1141, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible