Duckietown Challenges Home Challenges Submissions

Submission 541

Submission541
Competingyes
Challengeaido-LF-sim-testing
UserBea Baselines 🐤
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 17059 sim-1of4: 17063 sim-2of4: 17061 sim-3of4: 17065
Next
User labelbaseline-dagger-pytorch
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

The highlights are available only to the owner and the admins.

The highlights are available only to the owner and the admins.

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
17065sim-3of4successyes0:12:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median8.615883420707828
survival_time_median59.99999999999873
deviation-center-line_median3.3817505795370923
in-drivable-lane_median0.9999999999999432


other stats
deviation-center-line_max3.3817505795370923
deviation-center-line_mean3.3817505795370923
deviation-center-line_min3.3817505795370923
deviation-heading_max14.112191151388236
deviation-heading_mean14.112191151388236
deviation-heading_median14.112191151388236
deviation-heading_min14.112191151388236
distance-from-start_max2.8951004509544163
distance-from-start_mean2.8951004509544163
distance-from-start_median2.8951004509544163
distance-from-start_min2.8951004509544163
driven_any_max9.22016015355508
driven_any_mean9.22016015355508
driven_any_median9.22016015355508
driven_any_min9.22016015355508
driven_lanedir_consec_max8.615883420707828
driven_lanedir_consec_mean8.615883420707828
driven_lanedir_consec_min8.615883420707828
driven_lanedir_max8.615883420707828
driven_lanedir_mean8.615883420707828
driven_lanedir_median8.615883420707828
driven_lanedir_min8.615883420707828
in-drivable-lane_max0.9999999999999432
in-drivable-lane_mean0.9999999999999432
in-drivable-lane_min0.9999999999999432
per-episodes
details{"zigzag-000-ego0": {"driven_any": 9.22016015355508, "survival_time": 59.99999999999873, "driven_lanedir": 8.615883420707828, "in-drivable-lane": 0.9999999999999432, "deviation-heading": 14.112191151388236, "distance-from-start": 2.8951004509544163, "deviation-center-line": 3.3817505795370923, "driven_lanedir_consec": 8.615883420707828}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
17063sim-1of4successyes0:12:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median9.141200429181076
survival_time_median59.99999999999873
deviation-center-line_median3.257594718676204
in-drivable-lane_median2.6999999999998465


other stats
deviation-center-line_max3.257594718676204
deviation-center-line_mean3.257594718676204
deviation-center-line_min3.257594718676204
deviation-heading_max17.87731460516444
deviation-heading_mean17.87731460516444
deviation-heading_median17.87731460516444
deviation-heading_min17.87731460516444
distance-from-start_max3.2695546403215916
distance-from-start_mean3.2695546403215916
distance-from-start_median3.2695546403215916
distance-from-start_min3.2695546403215916
driven_any_max10.49053225758392
driven_any_mean10.49053225758392
driven_any_median10.49053225758392
driven_any_min10.49053225758392
driven_lanedir_consec_max9.141200429181076
driven_lanedir_consec_mean9.141200429181076
driven_lanedir_consec_min9.141200429181076
driven_lanedir_max9.141200429181076
driven_lanedir_mean9.141200429181076
driven_lanedir_median9.141200429181076
driven_lanedir_min9.141200429181076
in-drivable-lane_max2.6999999999998465
in-drivable-lane_mean2.6999999999998465
in-drivable-lane_min2.6999999999998465
per-episodes
details{"loop-000-ego0": {"driven_any": 10.49053225758392, "survival_time": 59.99999999999873, "driven_lanedir": 9.141200429181076, "in-drivable-lane": 2.6999999999998465, "deviation-heading": 17.87731460516444, "distance-from-start": 3.2695546403215916, "deviation-center-line": 3.257594718676204, "driven_lanedir_consec": 9.141200429181076}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
17061sim-2of4successyes0:03:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.2345125869984837
survival_time_median18.00000000000012
deviation-center-line_median0.5377327590901083
in-drivable-lane_median9.250000000000124


other stats
deviation-center-line_max0.5377327590901083
deviation-center-line_mean0.5377327590901083
deviation-center-line_min0.5377327590901083
deviation-heading_max3.8276106247929023
deviation-heading_mean3.8276106247929023
deviation-heading_median3.8276106247929023
deviation-heading_min3.8276106247929023
distance-from-start_max1.6836505396487214
distance-from-start_mean1.6836505396487214
distance-from-start_median1.6836505396487214
distance-from-start_min1.6836505396487214
driven_any_max3.0202360125560412
driven_any_mean3.0202360125560412
driven_any_median3.0202360125560412
driven_any_min3.0202360125560412
driven_lanedir_consec_max1.2345125869984837
driven_lanedir_consec_mean1.2345125869984837
driven_lanedir_consec_min1.2345125869984837
driven_lanedir_max1.2345125869984837
driven_lanedir_mean1.2345125869984837
driven_lanedir_median1.2345125869984837
driven_lanedir_min1.2345125869984837
in-drivable-lane_max9.250000000000124
in-drivable-lane_mean9.250000000000124
in-drivable-lane_min9.250000000000124
per-episodes
details{"loop2-000-ego0": {"driven_any": 3.0202360125560412, "survival_time": 18.00000000000012, "driven_lanedir": 1.2345125869984837, "in-drivable-lane": 9.250000000000124, "deviation-heading": 3.8276106247929023, "distance-from-start": 1.6836505396487214, "deviation-center-line": 0.5377327590901083, "driven_lanedir_consec": 1.2345125869984837}}
simulation-passed1
survival_time_max18.00000000000012
survival_time_mean18.00000000000012
survival_time_min18.00000000000012
No reset possible
17059sim-0of4successyes0:11:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median9.224086897557136
survival_time_median59.99999999999873
deviation-center-line_median2.816030773064013
in-drivable-lane_median1.249999999999929


other stats
deviation-center-line_max2.816030773064013
deviation-center-line_mean2.816030773064013
deviation-center-line_min2.816030773064013
deviation-heading_max12.623518823609292
deviation-heading_mean12.623518823609292
deviation-heading_median12.623518823609292
deviation-heading_min12.623518823609292
distance-from-start_max2.8983115115864795
distance-from-start_mean2.8983115115864795
distance-from-start_median2.8983115115864795
distance-from-start_min2.8983115115864795
driven_any_max9.794819266071189
driven_any_mean9.794819266071189
driven_any_median9.794819266071189
driven_any_min9.794819266071189
driven_lanedir_consec_max9.224086897557136
driven_lanedir_consec_mean9.224086897557136
driven_lanedir_consec_min9.224086897557136
driven_lanedir_max9.224086897557136
driven_lanedir_mean9.224086897557136
driven_lanedir_median9.224086897557136
driven_lanedir_min9.224086897557136
in-drivable-lane_max1.249999999999929
in-drivable-lane_mean1.249999999999929
in-drivable-lane_min1.249999999999929
per-episodes
details{"autolab-000-ego0": {"driven_any": 9.794819266071189, "survival_time": 59.99999999999873, "driven_lanedir": 9.224086897557136, "in-drivable-lane": 1.249999999999929, "deviation-heading": 12.623518823609292, "distance-from-start": 2.8983115115864795, "deviation-center-line": 2.816030773064013, "driven_lanedir_consec": 9.224086897557136}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
15925sim-0of4successno0:06:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
15924sim-2of4successno0:02:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
15923sim-1of4successno0:04:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
15922sim-3of4successno0:06:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14851sim-1of4successno0:06:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14850sim-0of4successno0:06:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14849sim-2of4successno0:05:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14846sim-3of4successno0:06:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13715sim-3of4errorno0:00:57
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13714sim-0of4errorno0:00:56
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13713sim-2of4errorno0:00:53
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13711sim-1of4errorno0:00:57
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 540, in run_episode
    dynamics=robot_conf.dynamics,
AttributeError: 'ScenarioRobotSpec' object has no attribute 'dynamics'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 430, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12352sim-1of4successno0:07:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12351sim-2of4successno0:06:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12350sim-0of4successno0:06:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12349sim-3of4successno0:07:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7275sim-2of4successno0:05:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7274sim-0of4successno0:06:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7255sim-1of4successno0:06:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7253sim-3of4successno0:06:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6066sim-1of4successno0:06:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6065sim-2of4successno0:05:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6064sim-3of4successno0:06:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6061sim-0of4successno0:05:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5123sim-3of4successno0:05:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5120sim-1of4successno0:11:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5119sim-2of4successno0:11:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4460sim-3of4abortedno0:25:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4458sim-2of4abortedno0:06:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4457sim-1of4abortedno0:05:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4454sim-0of4failedno0:00:59
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 720, in run_episode
    msg = await loop.run_in_executor(executor, f)
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 243, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 333, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 360, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego0" aborted:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 413, in loop
              ||     handle_message_node(parsed, receiver0, context0)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 586, in handle_message_node
              ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 74, in on_received_get_commands
              ||     velocity, omega = self.compute_action(self.current_image)
              ||   File "solution.py", line 70, in compute_action
              ||     action = self.model.predict(observation)
              ||   File "/workspace/model.py", line 125, in predict
              ||     is_speed_up, steering_angle = self.forward(images)
              ||   File "/workspace/model.py", line 108, in forward
              ||     features = self.feature_extractor(images)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
              ||     return forward_call(*input, **kwargs)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/container.py", line 141, in forward
              ||     input = module(input)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
              ||     return forward_call(*input, **kwargs)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/conv.py", line 446, in forward
              ||     return self._conv_forward(input, self.weight, self.bias)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/conv.py", line 442, in _conv_forward
              ||     return F.conv2d(input, weight, bias, self.stride,
              || RuntimeError: cuDNN error: CUDNN_STATUS_INTERNAL_ERROR
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible