Submission 805

Submission

805

Competing

yes

Challenge

aido-hello-sim-validation

User

Bea Baselines 🐤

Date submitted

2021-11-06 20:39:10+00:00

Last status update

2022-11-29 13:15:37+00:00

Complete

complete

Details

Evaluation is complete.

Sisters

Result

💚

Jobs

sanity-check: 17042

User label

challenge-aido_LF-baseline-duckietown:daffy-staging:06423b9f834b346c04889a28f919db3673a8d2c1

Admin priority

Blessing

n/a

User priority

17042

Click the images to see detailed statistics about the episode.

scenarios

Evaluation jobs for this submission

Show only up-to-date jobs

Job ID

step

status

up to date

date started

date completed

duration

message

17042

sanity-check

success

yes

2022-11-29 13:14:27+00:00

2022-11-29 13:15:37+00:00

0:01:10

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	0.49999999999999994
in-drivable-lane_median	0.0
driven_lanedir_consec_median	0.013228475740935774
deviation-center-line_median	0.006545593960936848

other stats

deviation-center-line_max	0.006545593960936848
deviation-center-line_mean	0.006545593960936848
deviation-center-line_min	0.006545593960936848
deviation-heading_max	0.26446454042588047
deviation-heading_mean	0.26446454042588047
deviation-heading_median	0.26446454042588047
deviation-heading_min	0.26446454042588047
distance-from-start_max	0.015078142643126045
distance-from-start_mean	0.015078142643126045
distance-from-start_median	0.015078142643126045
distance-from-start_min	0.015078142643126045
driven_any_max	0.015084395507555193
driven_any_mean	0.015084395507555193
driven_any_median	0.015084395507555193
driven_any_min	0.015084395507555193
driven_lanedir_consec_max	0.013228475740935774
driven_lanedir_consec_mean	0.013228475740935774
driven_lanedir_consec_min	0.013228475740935774
driven_lanedir_max	0.013228475740935774
driven_lanedir_mean	0.013228475740935774
driven_lanedir_median	0.013228475740935774
driven_lanedir_min	0.013228475740935774
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"scenarios-ego0": {"driven_any": 0.015084395507555193, "survival_time": 0.49999999999999994, "driven_lanedir": 0.013228475740935774, "in-drivable-lane": 0.0, "deviation-heading": 0.26446454042588047, "distance-from-start": 0.015078142643126045, "deviation-center-line": 0.006545593960936848, "driven_lanedir_consec": 0.013228475740935774}}
simulation-passed	1
survival_time_max	0.49999999999999994
survival_time_mean	0.49999999999999994
survival_time_min	0.49999999999999994

No reset possible

15903

sanity-check

success

2022-11-25 12:56:52+00:00

2022-11-25 12:58:18+00:00

0:01:26

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

14812

sanity-check

success

2022-11-24 07:30:32+00:00

2022-11-24 07:31:58+00:00

0:01:26

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

13368

sanity-check

error

2022-11-22 22:46:08+00:00

2022-11-22 22:47:12+00:00

0:01:04

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

12668

sanity-check

error

2022-11-22 18:52:04+00:00

2022-11-22 18:53:44+00:00

0:01:40

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

11777

sanity-check

success

2022-11-17 22:41:29+00:00

2022-11-17 22:43:34+00:00

0:02:05

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

10031

sanity-check

host-error

2022-11-16 21:55:47+00:00

2022-11-16 21:55:48+00:00

0:00:01

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 841, in get_cr
    mkdirs_thread_safe(wd)
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 18, in mkdirs_thread_safe
    os.mkdir(dst, 0o777)
OSError: [Errno 28] No space left on device: '/tmp/ubuntu/aido-hello-sim-validation'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

10023

sanity-check

host-error

2022-11-16 21:55:28+00:00

0:00:00

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 841, in get_cr
    mkdirs_thread_safe(wd)
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 18, in mkdirs_thread_safe
    os.mkdir(dst, 0o777)
OSError: [Errno 28] No space left on device: '/tmp/ubuntu/aido-hello-sim-validation'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

10018

sanity-check

host-error

2022-11-16 21:55:09+00:00

2022-11-16 21:55:10+00:00

0:00:01

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 841, in get_cr
    mkdirs_thread_safe(wd)
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 18, in mkdirs_thread_safe
    os.mkdir(dst, 0o777)
OSError: [Errno 28] No space left on device: '/tmp/ubuntu/aido-hello-sim-validation'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

10009

sanity-check

host-error

2022-11-16 21:54:46+00:00

0:00:00

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 841, in get_cr
    mkdirs_thread_safe(wd)
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 18, in mkdirs_thread_safe
    os.mkdir(dst, 0o777)
OSError: [Errno 28] No space left on device: '/tmp/ubuntu/aido-hello-sim-validation'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

10006

sanity-check

host-error

2022-11-16 21:54:27+00:00

2022-11-16 21:54:28+00:00

0:00:01

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 841, in get_cr
    mkdirs_thread_safe(wd)
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 18, in mkdirs_thread_safe
    os.mkdir(dst, 0o777)
OSError: [Errno 28] No space left on device: '/tmp/ubuntu/aido-hello-sim-validation'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

8985

sanity-check

host-error

2022-11-16 20:46:13+00:00

0:00:00

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 840, in get_cr
    mkdirs_thread_safe(wd)
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 18, in mkdirs_thread_safe
    os.mkdir(dst, 0o777)
OSError: [Errno 28] No space left on device: '/tmp/ubuntu/aido-hello-sim-validation'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

8973

sanity-check

host-error

2022-11-16 20:44:58+00:00

2022-11-16 20:44:59+00:00

0:00:01

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 840, in get_cr
    mkdirs_thread_safe(wd)
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 16, in mkdirs_thread_safe
    mkdirs_thread_safe(cast(DirPath, head))
  File "/usr/local/lib/python3.8/dist-packages/zuper_commons/fs/zc_mkdirs.py", line 18, in mkdirs_thread_safe
    os.mkdir(dst, 0o777)
OSError: [Errno 28] No space left on device: '/tmp/ubuntu/aido-hello-sim-validation'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

7487

sanity-check

success

2021-11-06 20:40:41+00:00

2021-11-06 20:42:08+00:00

0:01:27

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

7486

sanity-check

success

2021-11-06 20:40:00+00:00

2021-11-06 20:43:27+00:00

0:03:27

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible