Duckietown Challenges Home Challenges Submissions

Submission 143

Submission143
Competinguser retired
Challengeaido-LFV_multi-full-sim-validation
UserBea Baselines 🐤
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 1727 sim-1of4: 1730 sim-2of4: 1732 sim-3of4: 1729
Next
User labelminimal-agent-full
Admin priority50
Blessingn/a
User priority50

1732

Click the images to see detailed statistics about the episode.

loop-000

1730

Click the images to see detailed statistics about the episode.

autolab-000

1729

Click the images to see detailed statistics about the episode.

zigzag-000

1727

Click the images to see detailed statistics about the episode.

loop2-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
1732sim-2of4successno0:14:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median1.8249999999998965
driven_lanedir_consec_median2.08936264612111
deviation-center-line_median5.160696904244562


other stats
deviation-center-line_max5.8845370889767015
deviation-center-line_mean4.6089027286224535
deviation-center-line_min2.229680017023988
deviation-heading_max33.00132974561592
deviation-heading_mean11.086939889451044
deviation-heading_median5.3156324990396016
deviation-heading_min0.7151648141090521
distance-from-start_max2.4007738045922906
distance-from-start_mean1.8550334398943655
distance-from-start_median1.8031549983915711
distance-from-start_min1.4130499582020293
driven_any_max3.5300951074200486
driven_any_mean2.518772621155235
driven_any_median2.531092735855501
driven_any_min1.4828099054898884
driven_lanedir_consec_max3.437555654852413
driven_lanedir_consec_mean2.243867367124487
driven_lanedir_consec_min1.359188521403316
driven_lanedir_max3.437555654852413
driven_lanedir_mean2.430771337654317
driven_lanedir_median2.4562457120319725
driven_lanedir_min1.373038271700909
in-drivable-lane_max2.0000000000000284
in-drivable-lane_mean1.4124999999999552
in-drivable-lane_min0.0
per-episodes
details{"loop-000-ego0": {"driven_any": 1.5425992768884036, "survival_time": 59.99999999999873, "driven_lanedir": 1.537863345389593, "in-drivable-lane": 0.0, "deviation-heading": 0.7151648141090521, "distance-from-start": 1.538166429460106, "deviation-center-line": 2.229680017023988, "driven_lanedir_consec": 1.537863345389593}, "loop-000-ego1": {"driven_any": 3.519586194822599, "survival_time": 59.99999999999873, "driven_lanedir": 3.3746280786743523, "in-drivable-lane": 1.9999999999998863, "deviation-heading": 6.2883954590350015, "distance-from-start": 2.0681435673230366, "deviation-center-line": 5.233752335699744, "driven_lanedir_consec": 2.6408619468526267}, "loop-000-ego2": {"driven_any": 3.5300951074200486, "survival_time": 59.99999999999873, "driven_lanedir": 3.437555654852413, "in-drivable-lane": 1.6499999999999062, "deviation-heading": 4.3428695390442025, "distance-from-start": 2.4007738045922906, "deviation-center-line": 5.8845370889767015, "driven_lanedir_consec": 3.437555654852413}, "loop-000-ego3": {"driven_any": 1.4828099054898884, "survival_time": 59.99999999999873, "driven_lanedir": 1.373038271700909, "in-drivable-lane": 2.0000000000000284, "deviation-heading": 33.00132974561592, "distance-from-start": 1.4130499582020293, "deviation-center-line": 5.087641472789379, "driven_lanedir_consec": 1.359188521403316}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
1730sim-1of4successno0:15:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median1.4749999999999552
driven_lanedir_consec_median3.4002822157660937
deviation-center-line_median5.992377953630342


other stats
deviation-center-line_max7.516029088900661
deviation-center-line_mean6.078065420662326
deviation-center-line_min4.811476686487961
deviation-heading_max12.16868520862779
deviation-heading_mean6.880886780073767
deviation-heading_median6.2173939902337825
deviation-heading_min2.920073931199714
distance-from-start_max3.090134830032883
distance-from-start_mean2.5082472337131936
distance-from-start_median2.7478842626588915
distance-from-start_min1.4470855795021071
driven_any_max3.5495824572427233
driven_any_mean3.5349643358858858
driven_any_median3.535345617441779
driven_any_min3.51958365141726
driven_lanedir_consec_max3.48565390432292
driven_lanedir_consec_mean3.0923220304183836
driven_lanedir_consec_min2.083069785818428
driven_lanedir_max3.48565390432292
driven_lanedir_mean3.409680797064197
driven_lanedir_median3.4002822157660937
driven_lanedir_min3.352504852401682
in-drivable-lane_max2.0000000000000284
in-drivable-lane_mean1.4874999999999703
in-drivable-lane_min0.9999999999999432
per-episodes
details{"autolab-000-ego0": {"driven_any": 3.5465963269625824, "survival_time": 59.99999999999873, "driven_lanedir": 3.48565390432292, "in-drivable-lane": 1.1000000000000156, "deviation-heading": 2.920073931199714, "distance-from-start": 2.876631299535302, "deviation-center-line": 7.516029088900661, "driven_lanedir_consec": 3.48565390432292}, "autolab-000-ego1": {"driven_any": 3.5240949079209765, "survival_time": 59.99999999999873, "driven_lanedir": 3.4260604158492853, "in-drivable-lane": 1.8499999999998948, "deviation-heading": 3.5163301581559163, "distance-from-start": 2.619137225782481, "deviation-center-line": 6.510873598064545, "driven_lanedir_consec": 3.4260604158492853}, "autolab-000-ego2": {"driven_any": 3.5495824572427233, "survival_time": 59.99999999999873, "driven_lanedir": 3.374504015682902, "in-drivable-lane": 0.9999999999999432, "deviation-heading": 12.16868520862779, "distance-from-start": 3.090134830032883, "deviation-center-line": 4.811476686487961, "driven_lanedir_consec": 3.374504015682902}, "autolab-000-ego3": {"driven_any": 3.51958365141726, "survival_time": 59.99999999999873, "driven_lanedir": 3.352504852401682, "in-drivable-lane": 2.0000000000000284, "deviation-heading": 8.918457822311648, "distance-from-start": 1.4470855795021071, "deviation-center-line": 5.4738823091961395, "driven_lanedir_consec": 2.083069785818428}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
1729sim-3of4successno0:16:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median3.899999999999888
driven_lanedir_consec_median2.5771471236493877
deviation-center-line_median6.473819463882826


other stats
deviation-center-line_max6.674142000428096
deviation-center-line_mean6.18150596694939
deviation-center-line_min5.104242939603816
deviation-heading_max10.1097784630579
deviation-heading_mean8.400916938079984
deviation-heading_median8.144703245379382
deviation-heading_min7.20448279850327
distance-from-start_max2.1709881263292004
distance-from-start_mean1.563488646152428
distance-from-start_median1.600315910551216
distance-from-start_min0.8823346371780796
driven_any_max3.482081321069974
driven_any_mean3.4641223374092447
driven_any_median3.4626626970634384
driven_any_min3.4490826344401277
driven_lanedir_consec_max3.2836547816684893
driven_lanedir_consec_mean2.5120311183071315
driven_lanedir_consec_min1.6101754442612604
driven_lanedir_max3.2836547816684893
driven_lanedir_mean3.242098496596279
driven_lanedir_median3.2385172732125596
driven_lanedir_min3.2077046582915076
in-drivable-lane_max4.349999999999895
in-drivable-lane_mean3.8499999999999073
in-drivable-lane_min3.249999999999958
per-episodes
details{"zigzag-000-ego0": {"driven_any": 3.4715894450565083, "survival_time": 59.99999999999873, "driven_lanedir": 3.2836547816684893, "in-drivable-lane": 3.599999999999884, "deviation-heading": 7.20448279850327, "distance-from-start": 1.6768936305156854, "deviation-center-line": 6.674142000428096, "driven_lanedir_consec": 3.2836547816684893}, "zigzag-000-ego1": {"driven_any": 3.4537359490703685, "survival_time": 59.99999999999873, "driven_lanedir": 3.2159916993102144, "in-drivable-lane": 4.199999999999893, "deviation-heading": 8.200995811466246, "distance-from-start": 2.1709881263292004, "deviation-center-line": 6.659565442296598, "driven_lanedir_consec": 3.2159916993102144}, "zigzag-000-ego2": {"driven_any": 3.482081321069974, "survival_time": 59.99999999999873, "driven_lanedir": 3.2610428471149047, "in-drivable-lane": 3.249999999999958, "deviation-heading": 10.1097784630579, "distance-from-start": 1.5237381905867466, "deviation-center-line": 5.104242939603816, "driven_lanedir_consec": 1.6101754442612604}, "zigzag-000-ego3": {"driven_any": 3.4490826344401277, "survival_time": 59.99999999999873, "driven_lanedir": 3.2077046582915076, "in-drivable-lane": 4.349999999999895, "deviation-heading": 8.08841067929252, "distance-from-start": 0.8823346371780796, "deviation-center-line": 6.288073485469053, "driven_lanedir_consec": 1.9383025479885607}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
1728sim-2of4host-errorno0:03:38
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1247, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 422, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1139, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
1727sim-0of4successno0:07:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median3.1249999999999662
driven_lanedir_consec_median1.44565035204534
deviation-center-line_median5.216571375721109


other stats
deviation-center-line_max5.528679809125458
deviation-center-line_mean5.216571375721109
deviation-center-line_min4.904462942316762
deviation-heading_max8.487354063916152
deviation-heading_mean8.116239601526146
deviation-heading_median8.116239601526146
deviation-heading_min7.74512513913614
distance-from-start_max1.242743835742773
distance-from-start_mean1.2243627584318864
distance-from-start_median1.2243627584318864
distance-from-start_min1.2059816811209998
driven_any_max3.486583257271078
driven_any_mean3.485833636753743
driven_any_median3.485833636753743
driven_any_min3.485084016236408
driven_lanedir_consec_max1.4708900970780951
driven_lanedir_consec_mean1.44565035204534
driven_lanedir_consec_min1.4204106070125853
driven_lanedir_max3.290495971975133
driven_lanedir_mean3.2881483739964987
driven_lanedir_median3.2881483739964987
driven_lanedir_min3.285800776017864
in-drivable-lane_max3.1499999999999666
in-drivable-lane_mean3.1249999999999662
in-drivable-lane_min3.099999999999966
per-episodes
details{"loop2-000-ego0": {"driven_any": 3.485084016236408, "survival_time": 59.99999999999873, "driven_lanedir": 3.290495971975133, "in-drivable-lane": 3.1499999999999666, "deviation-heading": 7.74512513913614, "distance-from-start": 1.2059816811209998, "deviation-center-line": 4.904462942316762, "driven_lanedir_consec": 1.4204106070125853}, "loop2-000-ego1": {"driven_any": 3.486583257271078, "survival_time": 59.99999999999873, "driven_lanedir": 3.285800776017864, "in-drivable-lane": 3.099999999999966, "deviation-heading": 8.487354063916152, "distance-from-start": 1.242743835742773, "deviation-center-line": 5.528679809125458, "driven_lanedir_consec": 1.4708900970780951}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible