Duckietown Challenges Home Challenges Submissions

Submission 528

Submission528
Competingyes
Challengeaido-LFV_multi-sim-validation
UserBea Baselines 🐤
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💔
Jobssim-0of4: 17113 sim-1of4: 17111 sim-2of4: 17108 sim-3of4: 17110
Next
User labelbaseline-dagger-pytorch
Admin priority50
Blessingn/a
User priority50

17111

Click the images to see detailed statistics about the episode.

loop-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
17113sim-0of4failedyes0:01:21
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17111sim-1of4successyes0:17:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median55.99999999999896
in-drivable-lane_median1.1500000000000163
driven_lanedir_consec_median4.653481495174014
deviation-center-line_median3.5453797379995837


other stats
deviation-center-line_max3.9824961601847417
deviation-center-line_mean2.911832862112948
deviation-center-line_min0.5740758122678834
deviation-heading_max18.01691450160684
deviation-heading_mean10.956447853270433
deviation-heading_median11.07683077000424
deviation-heading_min3.6552153714664217
distance-from-start_max2.8170465898074104
distance-from-start_mean2.3466513922443113
distance-from-start_median2.3466034401448157
distance-from-start_min1.8763520988802036
driven_any_max9.80554921064579
driven_any_mean5.633608827719401
driven_any_median5.013841314886996
driven_any_min2.7012034704578247
driven_lanedir_consec_max8.88402852401974
driven_lanedir_consec_mean4.84361781652413
driven_lanedir_consec_min1.1834797517287534
driven_lanedir_max8.88402852401974
driven_lanedir_mean4.84361781652413
driven_lanedir_median4.653481495174014
driven_lanedir_min1.1834797517287534
in-drivable-lane_max47.74999999999896
in-drivable-lane_mean12.512499999999749
in-drivable-lane_min0.0
per-episodes
details{"loop-000-ego0": {"driven_any": 9.80554921064579, "survival_time": 55.99999999999896, "driven_lanedir": 8.88402852401974, "in-drivable-lane": 0.8000000000000114, "deviation-heading": 18.01691450160684, "distance-from-start": 2.782681283913034, "deviation-center-line": 3.310825501210877, "driven_lanedir_consec": 8.88402852401974}, "loop-000-ego1": {"driven_any": 4.969918757785727, "survival_time": 55.99999999999896, "driven_lanedir": 4.339192951063964, "in-drivable-lane": 1.500000000000021, "deviation-heading": 11.41085818813874, "distance-from-start": 2.8170465898074104, "deviation-center-line": 3.9824961601847417, "driven_lanedir_consec": 4.339192951063964}, "loop-000-ego2": {"driven_any": 2.7012034704578247, "survival_time": 55.99999999999896, "driven_lanedir": 1.1834797517287534, "in-drivable-lane": 47.74999999999896, "deviation-heading": 3.6552153714664217, "distance-from-start": 1.8763520988802036, "deviation-center-line": 0.5740758122678834, "driven_lanedir_consec": 1.1834797517287534}, "loop-000-ego3": {"driven_any": 5.057763871988264, "survival_time": 55.99999999999896, "driven_lanedir": 4.967770039284064, "in-drivable-lane": 0.0, "deviation-heading": 10.742803351869735, "distance-from-start": 1.9105255963765968, "deviation-center-line": 3.7799339747882903, "driven_lanedir_consec": 4.967770039284064}}
simulation-passed1
survival_time_max55.99999999999896
survival_time_mean55.99999999999896
survival_time_min55.99999999999896
No reset possible
17110sim-3of4failedyes0:01:20
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17108sim-2of4failedyes0:01:22
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
15973sim-0of4failedno0:01:10
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
15972sim-2of4successno0:09:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
15971sim-3of4successno0:22:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
15969sim-1of4successno0:17:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14899sim-2of4failedno0:01:07
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14898sim-0of4successno0:20:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14897sim-1of4successno0:21:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14896sim-3of4successno0:22:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13763sim-2of4errorno0:01:30
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13761sim-3of4errorno0:01:15
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13760sim-1of4errorno0:01:15
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13759sim-0of4errorno0:01:17
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12415sim-2of4successno0:09:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12413sim-3of4successno0:23:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12412sim-1of4successno0:20:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12411sim-0of4successno0:19:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7363sim-3of4successno0:16:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7361sim-0of4failedno0:01:14
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7360sim-2of4successno0:14:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7359sim-1of4successno0:13:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6133sim-2of4successno0:14:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6132sim-1of4failedno0:01:13
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6131sim-0of4successno0:08:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6130sim-3of4successno0:16:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5118sim-1of4successno0:20:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5116sim-0of4successno0:11:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4541sim-3of4failedno0:01:34
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 720, in run_episode
    msg = await loop.run_in_executor(executor, f)
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 243, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 333, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 360, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego0" aborted:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 413, in loop
              ||     handle_message_node(parsed, receiver0, context0)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 586, in handle_message_node
              ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 74, in on_received_get_commands
              ||     velocity, omega = self.compute_action(self.current_image)
              ||   File "solution.py", line 70, in compute_action
              ||     action = self.model.predict(observation)
              ||   File "/workspace/model.py", line 125, in predict
              ||     is_speed_up, steering_angle = self.forward(images)
              ||   File "/workspace/model.py", line 108, in forward
              ||     features = self.feature_extractor(images)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
              ||     return forward_call(*input, **kwargs)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/container.py", line 141, in forward
              ||     input = module(input)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
              ||     return forward_call(*input, **kwargs)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/conv.py", line 446, in forward
              ||     return self._conv_forward(input, self.weight, self.bias)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/conv.py", line 442, in _conv_forward
              ||     return F.conv2d(input, weight, bias, self.stride,
              || RuntimeError: cuDNN error: CUDNN_STATUS_INTERNAL_ERROR
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4539sim-3of4abortedno0:01:07
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.40/containers/224dbd2dc83d1e63fc2d414daf2b7b52d9949be5da1ee18b8bb617b3bc8706a8/json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1245, in run_one
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 897, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 771, in inspect_container
    return self._result(
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.40/containers/224dbd2dc83d1e63fc2d414daf2b7b52d9949be5da1ee18b8bb617b3bc8706a8/json: Not Found ("No such container: 224dbd2dc83d1e63fc2d414daf2b7b52d9949be5da1ee18b8bb617b3bc8706a8")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4537sim-2of4failedno0:01:12
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4536sim-1of4abortedno0:07:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4534sim-0of4abortedno0:08:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible