Nablaaa/ppo-Pyramids is a PPO (Proximal Policy Optimization) agent trained with Unity ML-Agents on the Pyramids environment. The policy maps a 172-dim vector observation (56+56+56+4) to a discrete 5-way single-branch action.
51afdc57fa90b739589ca4b9082ac738191eb606Concat 172 -> Linear 172×512 Swish -> Linear 512×512 Swish -> Linear 512×5 + masked Softmax + ArgMax/Multinomialobs_0 56 + obs_1 56 + obs_2 56 + obs_3 4 = 172 float32 (no normalizer)action_masks all-ones, deterministic = argmax)Pyramids/Pyramids-3000009.pt (3,000,009 steps, also available as Pyramids-3000009.onnx)This repository adapts the real PPO policy to run on a single Ascend NPU (npu:0). All model parameters and every observation/action tensor are placed on npu:0; inference.py loads the real weights from HF cache and produces real logits/probs and deterministic discrete actions.
| Field | Value |
|---|---|
| Route | offline-rl / classic-rl-policy (ML-Agents PPO Pyramids discrete) |
| Observation dim | 172 (56+56+56+4) |
| Action dim | 5 discrete (0..4 single branch) |
| Hidden | 512×2, Swish = x·sigmoid(x) |
| Output | logits[5], probs=softmax(masked_logits), deterministic=argmax |
| Horizon | 1 decision per forward |
| Masks | [1,1,1,1,1] |
npu:0, CANN 8.5.1)Ascend910_9362, torch.npu.is_available() = Truepip install -r requirements.txtpython inference.py --device npu
# or CPU
python inference.py --device cpu
# auto-select NPU if available
python inference.pyWeights are loaded from /data/adapt-npu-agent/hf/hub/models--Nablaaa--ppo-Pyramids/snapshots/51afdc57fa90b739589ca4b9082ac738191eb606/Pyramids/Pyramids-3000009.pt if cached, otherwise via huggingface_hub (HF_ENDPOINT=https://hf-mirror.com).
Executed on npu:0 (Ascend910_9362) with synchronized timing (torch.npu.synchronize() around forward):
[0.498,-0.523,-0.251,-1.055](1,5) device npu:0 dtype float32: [-1.94938, 19.62939, -14.29364, -7.64803, -2.16591][4.25e-10, 1.0, 1.85e-15, 1.42e-12, 3.42e-10][1]Inputs: 5 varied observations ([0.1]*172 etc., see validate_npu.py)
[-1.63797, 2.07002, -3.21489, 0.55549, 0.40300] -> det [1,4,1,4,1][-1.63797, 2.07002, -3.21489, 0.55549, 0.40300] -> det [1,4,1,4,1]| Metric | Value |
|---|---|
| Max abs diff (logits) | 2.15e-06 |
| Mean abs diff | 5.50e-07 |
| Tolerance | 1e-4 + 1e-3*max_ref = 6.89e-03 |
| Finite | True |
| Actions match | True (all 5 decisions) |
| Result | PASS |
Evidence screenshots rendered via xterm.js from real logs:
assets/agent_workflow.png – workflow, model discovery, CPU/NPU validation stepsassets/npu_device_call.png – npu-smi info + torch.npu.is_available() + device proofassets/model_result.png – python inference.py --device npu real output on npu:0
