Duckietown Challenges Home Challenges Submissions

Evaluator 127

ID127
evaluatordavinci-stage-04-6.2.52
owneruser1000001
machinedavinci-stage
processdavinci-stage-04-6.2.52
version6.2.52
first heard
last heard
statusinactive
# evaluating
# success
# timeout1 16678
# failed
# error2 16668
# aborted
# host-error
arm0
x86_641
Mac0
gpu available1
Number of processors64
Processor frequency (MHz)2.8 GHz
Free % of processors86%
RAM total (MB)251.6 GB
RAM free (MB)244.3 GB
Disk (MB)491.1 GB
Disk available (MB)171.3 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, ETH_small_intersect
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
166781700Bea Baselines 🐤minimal-agent-fullaido-LFVI-full-sim-testingsim-1of4timeoutyesdavinci-stage-04-6.2.52----No reset possible
166731712Bea Baselines 🐤minimal-agent-stateaido-LFV_multi-state-sim-validationsim-0of3erroryesdavinci-stage-04-6.2.520:01:28
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/zuper_ipce/conv_object_from_ipce.py", line 360, in object_from_ipce_dataclass_instance
    attrs[k] = object_from_ipce_(v, et_k, ieds=ieds, iedo=iedo)
  File "/usr/local/lib/python3.8/dist-packages/zuper_ipce/conv_object_from_ipce.py", line 177, in object_from_ipce_
    return object_from_ipce_dict(mj, K, ieds=ieds, iedo=iedo)
  File "/usr/local/lib/python3.8/dist-packages/zuper_ipce/conv_object_from_ipce.py", line 461, in object_from_ipce_dict
    attrs[k] = object_from_ipce_(v, et_V, ieds=ieds, iedo=iedo)
  File "/usr/local/lib/python3.8/dist-packages/zuper_ipce/conv_object_from_ipce.py", line 185, in object_from_ipce_
    return object_from_ipce_dataclass_instance(mj, K, ieds=ieds, iedo=iedo)
  File "/usr/local/lib/python3.8/dist-packages/zuper_ipce/conv_object_from_ipce.py", line 391, in object_from_ipce_dataclass_instance
    raise ZValueError(msg, anns=anns, T=T, known=sorted(mj), f=f)
zuper_commons.types.exceptions.ZValueError: Cannot find field 'dynamics' in data for class ScenarioRobotSpec and no default available
│  anns: dict[7]
│        │ configuration: RobotConfiguration
│        │ color: str
│        │ description: str
│        │ controllable: bool
│        │ protocol: Optional[ProtocolDesc ]
│        │ dynamics: str
│        │ dynamics_params: Dict[str,Any ]
│     T: str
│ known: [color, configuration, controllable, description, protocol]
│     f: Field(name='dynamics',type=<class 'str'>,default=<dataclasses._MISSING_TYPE object at 0x7f836a312e20>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f836a312e20>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),_field_type=_FIELD)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/zuper_ipce/conv_object_from_ipce.py", line 74, in object_from_ipce
    res = object_from_ipce_(mj, expect_type, ieds=ieds, iedo=iedo)
  File "/usr/local/lib/python3.8/dist-packages/zuper_ipce/conv_object_from_ipce.py", line 185, in object_from_ipce_
    return object_from_ipce_dataclass_instance(mj, K, ieds=ieds, iedo=iedo)
  File "/usr/local/lib/python3.8/dist-packages/zuper_ipce/conv_object_from_ipce.py", line 366, in object_from_ipce_dataclass_instance
    raise ZValueError(
zuper_commons.types.exceptions.ZValueError: Cannot deserialize attribute 'robots' of Scenario.
│ K_annotations: dict[6]
│                │ scenario_name: str
│                │ environment: Any
│                │ player_robots: List[str]
│                │ robots: Dict[str,ScenarioRobotSpec]
│                │ duckies: Dict[str,ScenarioDuckieSpec]
│                │ payload_yaml: str
│   expect_type: Dict[str,ScenarioRobotSpec]
│         ann_K: Dict[str,ScenarioRobotSpec]
│        K_name: Scenario

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 167, in main
    episodes0 = get_episodes_from_dirs(config.scenarios)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 833, in get_episodes_from_dirs
    scenario = cast(Scenario, object_from_ipce(config_, Scenario))
  File "/usr/local/lib/python3.8/dist-packages/zuper_ipce/conv_object_from_ipce.py", line 93, in object_from_ipce
    raise ZValueError(msg, expect_type=expect_type, mj=mj) from e
zuper_commons.types.exceptions.ZValueError: Cannot deserialize object
│ expect_type: Scenario
│          mj: dict[6]
│              │ payload_yaml: ''
│              │ player_robots: [ego0, ego1, ego2, ego3]
│              │ scenario_name: LFV_multi-state-4way-000
│              │ robots:
│              │ dict[4]
│              │ │ ego0:
│              │ │ dict[5]
│              │ │ │ color: red
│              │ │ │ protocol: PROTOCOL_STATE
│              │ │ │ description: Playable robot ego0
│              │ │ │ controllable: True
│              │ │ │ configuration: {pose: {x: 3.19, y: 1.276, theta_deg: 180.0} , velocity: {x: 0.0, y: 0.0, theta_deg: 0.0} }
│              │ │ ego1:
│              │ │ dict[5]
│              │ │ │ color: green
│              │ │ │ protocol: PROTOCOL_STATE
│              │ │ │ description: Playable robot ego1
│              │ │ │ controllable: True
│              │ │ │ configuration: {pose: {x: 2.75, y: 2.175, theta_deg: 90.0} , velocity: {x: 0.0, y: 0.0, theta_deg: 0.0} }
│              │ │ ego2:
│              │ │ dict[5]
│              │ │ │ color: blue
│              │ │ │ protocol: PROTOCOL_STATE
│              │ │ │ description: Playable robot ego2
│              │ │ │ controllable: True
│              │ │ │ configuration: {pose: {x: 2.75, y: 3.074, theta_deg: 90.0} , velocity: {x: 0.0, y: 0.0, theta_deg: 0.0} }
│              │ │ ego3:
│              │ │ dict[5]
│              │ │ │ color: red
│              │ │ │ protocol: PROTOCOL_STATE
│              │ │ │ description: Playable robot ego3
│              │ │ │ controllable: True
│              │ │ │ configuration: {pose: {x: 1.914, y: 2.7492, theta_deg: 0.0} , velocity: {x: 0.0, y: 0.0, theta_deg: 0.0} }
│              │ duckies: {}
│              │ environment:
│              │ |# 5x5 counter-clockwise loop with a 4-way intersection in the middle
│              │ |# Designed to test the 4-way intersection tile and traffic light
│              │ |tiles:
│              │ |- [floor, floor, floor, floor, floor, floor, floor]
│              │ |- [floor, curve_left/W , straight/W, 3way_left/W   , straight/W, curve_left/N, floor]
│              │ |- [floor, straight/S   , asphalt   , straight/N    , asphalt   , straight/N, floor]
│              │ |- [floor, 3way_left/S  , straight/W, 4way          , straight/E, 3way_left/N, floor]
│              │ |- [floor, straight/S   , asphalt   , straight/S    , asphalt   , straight/N, floor]
│              │ |- [floor, curve_left/S , straight/E, 3way_left/E   , straight/E, curve_left/E, floor]
│              │ |- [floor, floor, floor, floor, floor, floor, floor]
│              │ |
│              │ |objects:
│              │ |  trafficlight:
│              │ |    kind: trafficlight
│              │ |    place:
│              │ |      tile: [3,3]
│              │ |      relative:
│              │ |        ~SE2Transform:
│              │ |          p: [-0.18,-0.18]
│              │ |          theta_deg: 135
│              │ |    height: 0.4
│              │ |    optional: true
│              │ |
│              │ |tile_size: 0.585
│              │ |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
166681712Bea Baselines 🐤minimal-agent-stateaido-LFV_multi-state-sim-validationsim-1of3erroryesdavinci-stage-04-6.2.520:01:29
The result file [...]
    The result file is not found in working dir /tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd:

    File '/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/challenge-results/challenge_results.yaml' does not exist.

    This usually means that the evaluator did not finish and some times that there was an import error.
    Check the evaluator log to see what happened.

List of all files:

 -/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/docker-compose.yaml
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/runner
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/docker-compose.original.yaml
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/solution-ego1/online.log
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/solution-ego1/online.html
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/solution-ego3/online.log
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/solution-ego3/online.html
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/simulator/online.log
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/simulator/online.html
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/challenges-runner/stdout.log
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/challenges-runner/stderr.log
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/solution-ego2/online.log
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/solution-ego2/online.html
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/evaluator/online.log
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/evaluator/online.html
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/solution-ego0/online.log
-/tmp/andrea/evaluations/davinci-stage/davinci-stage-04-6.2.52/aido-LFV_multi-state-sim-validation/submission-1712/job-16668-step-sim-1of3-a-wd/logs/solution-ego0/online.html
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible