Duckietown Challenges Home Challenges Submissions

Submission 475

Submission475
Competinguser retired
Challengeaido-LFVI-state-sim-validation
UserBea Baselines 🐤
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 5270 sim-1of4: 5268 sim-2of4: 5269 sim-3of4: 5267
Next
User labelbaseline-simple-yield
Admin priority50
Blessingn/a
User priority50

5270

Click the images to see detailed statistics about the episode.

udem-001

5269

Click the images to see detailed statistics about the episode.

4way-000

5268

Click the images to see detailed statistics about the episode.

udem-000

5267

Click the images to see detailed statistics about the episode.

udem-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
5270sim-0of4successno0:09:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median5.331344237139176
deviation-center-line_median0.14341993721660595


other stats
deviation-center-line_max0.14341993721660595
deviation-center-line_mean0.14341993721660595
deviation-center-line_min0.14341993721660595
deviation-heading_max1.3020944572743824
deviation-heading_mean1.3020944572743824
deviation-heading_median1.3020944572743824
deviation-heading_min1.3020944572743824
distance-from-start_max2.952739272534191
distance-from-start_mean2.952739272534191
distance-from-start_median2.952739272534191
distance-from-start_min2.952739272534191
driven_any_max5.629455349358906
driven_any_mean5.629455349358906
driven_any_median5.629455349358906
driven_any_min5.629455349358906
driven_lanedir_consec_max5.331344237139176
driven_lanedir_consec_mean5.331344237139176
driven_lanedir_consec_min5.331344237139176
driven_lanedir_max5.62600246220405
driven_lanedir_mean5.62600246220405
driven_lanedir_median5.62600246220405
driven_lanedir_min5.62600246220405
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"udem-001-ego0": {"driven_any": 5.629455349358906, "survival_time": 59.99999999999873, "driven_lanedir": 5.62600246220405, "in-drivable-lane": 0.0, "deviation-heading": 1.3020944572743824, "distance-from-start": 2.952739272534191, "deviation-center-line": 0.14341993721660595, "driven_lanedir_consec": 5.331344237139176}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5269sim-2of4successno0:08:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median0.0
deviation-center-line_median0.8949116056299468


other stats
deviation-center-line_max0.8949116056299468
deviation-center-line_mean0.8949116056299468
deviation-center-line_min0.8949116056299468
deviation-heading_max32.249113233675025
deviation-heading_mean32.249113233675025
deviation-heading_median32.249113233675025
deviation-heading_min32.249113233675025
distance-from-start_max4.440892098500626e-16
distance-from-start_mean4.440892098500626e-16
distance-from-start_median4.440892098500626e-16
distance-from-start_min4.440892098500626e-16
driven_any_max5.329070518200751e-13
driven_any_mean5.329070518200751e-13
driven_any_median5.329070518200751e-13
driven_any_min5.329070518200751e-13
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"4way-000-ego0": {"driven_any": 5.329070518200751e-13, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "in-drivable-lane": 0.0, "deviation-heading": 32.249113233675025, "distance-from-start": 4.440892098500626e-16, "deviation-center-line": 0.8949116056299468, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5268sim-1of4successno0:09:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median0.3684768253684605
deviation-center-line_median0.26341057846251453


other stats
deviation-center-line_max0.26341057846251453
deviation-center-line_mean0.26341057846251453
deviation-center-line_min0.26341057846251453
deviation-heading_max2.2732184433064777
deviation-heading_mean2.2732184433064777
deviation-heading_median2.2732184433064777
deviation-heading_min2.2732184433064777
distance-from-start_max0.3686479402497791
distance-from-start_mean0.3686479402497791
distance-from-start_median0.3686479402497791
distance-from-start_min0.3686479402497791
driven_any_max0.3741734844735202
driven_any_mean0.3741734844735202
driven_any_median0.3741734844735202
driven_any_min0.3741734844735202
driven_lanedir_consec_max0.3684768253684605
driven_lanedir_consec_mean0.3684768253684605
driven_lanedir_consec_min0.3684768253684605
driven_lanedir_max0.3684768253684605
driven_lanedir_mean0.3684768253684605
driven_lanedir_median0.3684768253684605
driven_lanedir_min0.3684768253684605
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"udem-000-ego0": {"driven_any": 0.3741734844735202, "survival_time": 59.99999999999873, "driven_lanedir": 0.3684768253684605, "in-drivable-lane": 0.0, "deviation-heading": 2.2732184433064777, "distance-from-start": 0.3686479402497791, "deviation-center-line": 0.26341057846251453, "driven_lanedir_consec": 0.3684768253684605}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5267sim-3of4successno0:15:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median0.3685223689035335
deviation-center-line_median0.2644910695042564


other stats
deviation-center-line_max0.2644910695042564
deviation-center-line_mean0.2644910695042564
deviation-center-line_min0.2644910695042564
deviation-heading_max2.2636034166978196
deviation-heading_mean2.2636034166978196
deviation-heading_median2.2636034166978196
deviation-heading_min2.2636034166978196
distance-from-start_max0.36864756052127134
distance-from-start_mean0.36864756052127134
distance-from-start_median0.36864756052127134
distance-from-start_min0.36864756052127134
driven_any_max0.374173471565073
driven_any_mean0.374173471565073
driven_any_median0.374173471565073
driven_any_min0.374173471565073
driven_lanedir_consec_max0.3685223689035335
driven_lanedir_consec_mean0.3685223689035335
driven_lanedir_consec_min0.3685223689035335
driven_lanedir_max0.3685223689035335
driven_lanedir_mean0.3685223689035335
driven_lanedir_median0.3685223689035335
driven_lanedir_min0.3685223689035335
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"udem-000-ego0": {"driven_any": 0.374173471565073, "survival_time": 59.99999999999873, "driven_lanedir": 0.3685223689035335, "in-drivable-lane": 0.0, "deviation-heading": 2.2636034166978196, "distance-from-start": 0.36864756052127134, "deviation-center-line": 0.2644910695042564, "driven_lanedir_consec": 0.3685223689035335}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
4360sim-2of4successno0:15:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4359sim-1of4successno0:14:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4339sim-1of4host-errorno0:16:24
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 862, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 48, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 275, in upload
    shutil.copy(realfile0, realfile)
  File "/usr/lib/python3.8/shutil.py", line 419, in copy
    copymode(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 307, in copymode
    st = stat_func(src)
  File "/usr/lib/python3.8/shutil.py", line 229, in _stat
    return fn.stat() if isinstance(fn, os.DirEntry) else os.stat(fn)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/aido-LFVI-state-sim-validation/submission475/sim-1of4-nogpu-staging-spot-0-03-job4339-a-wd/challenge-evaluation-output/episodes/4way-001/log.gs2.cbor.tmp'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4338sim-2of4host-errorno0:17:34
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 862, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 48, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 275, in upload
    shutil.copy(realfile0, realfile)
  File "/usr/lib/python3.8/shutil.py", line 419, in copy
    copymode(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 307, in copymode
    st = stat_func(src)
  File "/usr/lib/python3.8/shutil.py", line 229, in _stat
    return fn.stat() if isinstance(fn, os.DirEntry) else os.stat(fn)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/aido-LFVI-state-sim-validation/submission475/sim-2of4-nogpu-staging-spot-0-05-job4338-a-wd/challenge-evaluation-output/episodes/udem-001/log.gs2.cbor.tmp'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4316sim-1of4host-errorno0:19:54
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/connection.py", line 73, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/botocore/httpsession.py", line 345, in send
    urllib_response = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 507, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 358, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x7f2335c9a820>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 862, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 48, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 291, in upload
    aws_object.load()
  File "/usr/local/lib/python3.8/dist-packages/boto3/resources/factory.py", line 505, in do_action
    response = action(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(*args, **params)
  File "/usr/local/lib/python3.8/dist-packages/botocore/client.py", line 388, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.8/dist-packages/botocore/client.py", line 694, in _make_api_call
    http, parsed_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/botocore/client.py", line 714, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/usr/local/lib/python3.8/dist-packages/botocore/endpoint.py", line 102, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/local/lib/python3.8/dist-packages/botocore/endpoint.py", line 136, in _send_request
    while self._needs_retry(attempts, operation_model, request_dict,
  File "/usr/local/lib/python3.8/dist-packages/botocore/endpoint.py", line 252, in _needs_retry
    responses = self._event_emitter.emit(
  File "/usr/local/lib/python3.8/dist-packages/botocore/hooks.py", line 357, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/usr/local/lib/python3.8/dist-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/usr/local/lib/python3.8/dist-packages/botocore/retryhandler.py", line 183, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/usr/local/lib/python3.8/dist-packages/botocore/retryhandler.py", line 250, in __call__
    should_retry = self._should_retry(attempt_number, response,
  File "/usr/local/lib/python3.8/dist-packages/botocore/retryhandler.py", line 277, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/lib/python3.8/dist-packages/botocore/retryhandler.py", line 316, in __call__
    checker_response = checker(attempt_number, response,
  File "/usr/local/lib/python3.8/dist-packages/botocore/retryhandler.py", line 222, in __call__
    return self._check_caught_exception(
  File "/usr/local/lib/python3.8/dist-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
  File "/usr/local/lib/python3.8/dist-packages/botocore/endpoint.py", line 199, in _do_get_response
    http_response = self._send(request)
  File "/usr/local/lib/python3.8/dist-packages/botocore/endpoint.py", line 268, in _send
    return self.http_session.send(request)
  File "/usr/local/lib/python3.8/dist-packages/botocore/httpsession.py", line 374, in send
    raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://duckietown-ai-driving-olympics-1.s3.us-east-2.amazonaws.com/v3/staging/by-value/sha256/527aa9f431539da8e151d5434d1d5e611d973f601d8e970790882624554146b0"
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4315sim-3of4successno0:09:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4314sim-2of4host-errorno0:20:08
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 862, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 48, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 275, in upload
    shutil.copy(realfile0, realfile)
  File "/usr/lib/python3.8/shutil.py", line 419, in copy
    copymode(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 307, in copymode
    st = stat_func(src)
  File "/usr/lib/python3.8/shutil.py", line 229, in _stat
    return fn.stat() if isinstance(fn, os.DirEntry) else os.stat(fn)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/aido-LFVI-state-sim-validation/submission475/sim-2of4-nogpu-staging-spot-0-01-job4314-a-wd/challenge-evaluation-output/episodes/udem-001/log.gs2.cbor.tmp'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4312sim-0of4successno0:09:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible