2301_76761127/dolphin-2.9.1-yi-1.5-34b-npu
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

dphn/dolphin-2.9.1-yi-1.5-34b derived FP8 store on two Ascend NPUs

This migration runs the official model's main function: complete autoregressive Yi/Llama text generation. The official snapshot is BF16 and contains 34,388,917,248 parameters (68,777,898,032 weight bytes). It publishes no official FP8 checkpoint. The runtime artifact described here is therefore explicitly DERIVED_FROM_OFFICIAL_BF16: each fixed-revision official shard is verified and deterministically converted to an FP8 E4M3FN storage shard with one F32 scale per eligible projection tensor. It must never be described as official FP8.

At load time, checkpoint-dependent FP8 bytes are decoded through a 256-entry byte LUT and multiplied by their scales on NPU, yielding BF16 runtime weights. Layers 0-29 are hydrated on npu:0 and layers 30-59 on npu:1; embeddings remain on npu:0, while final norm and untied LM head remain on npu:1. CPU is limited to safetensors/index I/O and tokenization. CPU/disk/meta offload and CPU/CUDA model or dequantization compute are fail-closed.

Fixed official source and precision

  • Official AI GitCode mirror: https://ai.gitcode.com/hf_mirrors/dphn/dolphin-2.9.1-yi-1.5-34b
  • Actual download endpoint: https://gitcode.com/hf_mirrors/dphn/dolphin-2.9.1-yi-1.5-34b.git
  • Revision: 0141cba238d0faad09bc240ea7af14c9ea5aec44
  • Official precision: complete BF16 snapshot
  • Runtime storage precision: self-derived FP8 E4M3FN plus per-tensor F32 scale
  • Derivation label: DERIVED_FROM_OFFICIAL_BF16 (not an official/public FP8 source)
  • Required weights: all 15 shards referenced by model.safetensors.index.json
  • Locked selected-file inventory: audit/p4r16-official-bf16-selected.json
  • Derived manifest contract: audit/p4r16-derived-fp8-manifest.schema.json

The exact-model FP8 source audit rejected a Yi-1.5-34B-Chat-16K FP8 candidate because it is a different checkpoint, tokenizer, and context configuration. GGUF, GPTQ, AWQ, EXL2, MLX, and NVFP4 artifacts are also not substitutes.

git clone https://gitcode.com/hf_mirrors/dphn/dolphin-2.9.1-yi-1.5-34b.git model
git -C model checkout 0141cba238d0faad09bc240ea7af14c9ea5aec44
git -C model lfs pull --include='config.json,generation_config.json,tokenizer*,special_tokens_map.json,model.safetensors.index.json,model-*.safetensors'

Deterministic streaming derivation and manifest gate

tools/p4r16_stream_fp8.py converts one independently verified official shard at a time. The converter keeps embeddings, LM head, norms and buffers in BF16, and quantizes only the seven locked Yi projection matrices in each of 60 layers. Its audit persists every source key, shape and dtype before an official shard may be released. Unknown 2-D tensors fail instead of being guessed.

python3 tools/p4r16_stream_fp8.py \
  model/model-00001-of-00015.safetensors \
  derived/model-00001-of-00015.safetensors \
  --expected-sha256 9a10cac63db6cee2460c3a0a46731598feafc6248e8ad7ad8529d80be2be6cc7

Repeat in the locked 15-shard order from audit/p4r16-official-bf16-selected.json. Release a BF16 shard only after its output and .audit.json identities are durable. Copy the unchanged official model.safetensors.index.json and metadata into derived/, then finalize and re-verify the global gate:

python3 tools/p4r16_fp8_e2e.py finalize \
  --official-metadata model \
  --derived derived \
  --plan audit/p4r16-official-bf16-selected.json \
  --output derived/p4r16-derived-fp8-manifest.json

python3 tools/p4r16_fp8_e2e.py verify \
  --official-metadata model \
  --derived derived \
  --manifest derived/p4r16-derived-fp8-manifest.json

The global gate requires exact official-index parity; complete key, shape and source-dtype coverage; FP8/retained-BF16 dtype policy; one scalar F32 scale for every quantized tensor; all 15 source/output identities; and a 30/30 layer map. Before real hydration, each card must have at least its exact decoded-BF16 parameter bytes plus the configured 8 GiB runtime reserve free.

Verified environment and dependency rebuild

The hosted matrix is CANN 8.5.1, PyTorch 2.9.0, and torch-npu 2.9.0.post1. Keep that platform stack and rebuild only the fixed user-space closure from the Tsinghua mirror into an empty target:

python3 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple \
  --target "$PWD/pydeps" --no-deps --no-build-isolation -r requirements.txt
test ! -e "$PWD/pydeps/torch"
test ! -e "$PWD/pydeps/torch_npu"

Run

The atomic derived lane is /data/atomcode-lanes/heavy19-20260820/p4r16-dolphin-fp8. Do not start until that directory has both WEIGHT_READY and its final derived manifest. The official tokenizer/config/index metadata remains in the fixed-revision checkout under .one-bf16-shard/.source/checkout; its tiny LFS pointer files are not runtime weights.

The runtime constructs PreTrainedTokenizerFast directly from the complete tokenizer.json plus the pinned special-token/chat-template fields. It does not use the metadata-only LFS pointer stored as tokenizer.model.

Both NPUs must be idle and admitted before loading. The manifest records the exact decoded-BF16 parameter bytes assigned to each card; the loader requires that exact amount plus an 8 GiB reserve independently on npu:0 and npu:1. This is a hard pre-load gate, not an estimate.

lane=/data/atomcode-lanes/heavy19-20260820/p4r16-dolphin-34b
derived=/data/atomcode-lanes/heavy19-20260820/p4r16-dolphin-fp8
repo="$lane/repo"
metadata="$derived/.one-bf16-shard/.source/checkout"
mkdir -p "$lane/audit-final"
source /usr/local/Ascend/cann-8.5.1/set_env.sh
PYTHONNOUSERSITE=1 TORCH_DEVICE_BACKEND_AUTOLOAD=0 \
HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
PYTHONPATH="$lane/pydeps:$repo/tools${PYTHONPATH:+:$PYTHONPATH}" \
python3 "$repo/audit/run-p4r16-derived-two-npu.py" \
  --repo "$repo" \
  --derived-lane "$derived" \
  --official-metadata "$metadata" \
  --output "$lane/audit-final/result.json" \
  --max-new-tokens 8 \
  --hold-seconds 600

status=PASS requires the immutable manifest gate, the HBM gate, every hydrated parameter on one of the two NPUs, the exact 30/30 split, NPU input/output, and complete autoregressive generation. CPU/disk/meta offload and CUDA fallback are forbidden. The key/shape/dtype/scale/index checks are checkpoint construction integrity gates, not an accuracy or parameter-count score.

Evidence contract

The public repository carries exactly three hosted PNGs after the final run:

  • assets/agent_workflow.png: AtomCode plan-only workflow for the official-BF16 to derived-FP8 stream, immutable manifest gate, and two-NPU execution.
  • assets/npu_device_call.png: the successful held PID visible on both NPUs.
  • assets/model_result.png: the same run's JSON showing artifact_identity=DERIVED_FROM_OFFICIAL_BF16, official_fp8_available=false, manifest identity, the 30/30 split, and NPU input/output.

AtomCode migration plan

Two-NPU process

Full generation result

Public fresh-clone gate

Only after the same successful held PID has supplied the device and result images may this package be pushed. An anonymous fresh clone must match the pushed main commit, remain clean, contain the README/inference/requirements triad, pass py_compile for the inference, runner and all three tools, parse all audit JSON/schema files, and contain exactly the three PNGs above with three distinct SHA-256 values and all three README references. It must contain no official or derived weight shards, __pycache__, .pyc, partial payload, or nested .git directory.

本仓为 2301_76761127 队伍交付件(与备份账号合并提交)。