This migration runs the official model's main function: complete autoregressive
Yi/Llama text generation. The official snapshot is BF16 and contains 34,388,917,248
parameters (68,777,898,032 weight bytes). It publishes no official FP8
checkpoint. The runtime artifact described here is therefore explicitly
DERIVED_FROM_OFFICIAL_BF16: each fixed-revision official shard is verified and
deterministically converted to an FP8 E4M3FN storage shard with one F32 scale per
eligible projection tensor. It must never be described as official FP8.
At load time, checkpoint-dependent FP8 bytes are decoded through a 256-entry byte
LUT and multiplied by their scales on NPU, yielding BF16 runtime weights. Layers
0-29 are hydrated on npu:0 and layers 30-59 on npu:1; embeddings remain on
npu:0, while final norm and untied LM head remain on npu:1. CPU is limited to
safetensors/index I/O and tokenization. CPU/disk/meta offload and CPU/CUDA model or
dequantization compute are fail-closed.
https://ai.gitcode.com/hf_mirrors/dphn/dolphin-2.9.1-yi-1.5-34bhttps://gitcode.com/hf_mirrors/dphn/dolphin-2.9.1-yi-1.5-34b.git0141cba238d0faad09bc240ea7af14c9ea5aec44DERIVED_FROM_OFFICIAL_BF16 (not an official/public FP8 source)model.safetensors.index.jsonaudit/p4r16-official-bf16-selected.jsonaudit/p4r16-derived-fp8-manifest.schema.jsonThe exact-model FP8 source audit rejected a Yi-1.5-34B-Chat-16K FP8 candidate because it is a different checkpoint, tokenizer, and context configuration. GGUF, GPTQ, AWQ, EXL2, MLX, and NVFP4 artifacts are also not substitutes.
git clone https://gitcode.com/hf_mirrors/dphn/dolphin-2.9.1-yi-1.5-34b.git model
git -C model checkout 0141cba238d0faad09bc240ea7af14c9ea5aec44
git -C model lfs pull --include='config.json,generation_config.json,tokenizer*,special_tokens_map.json,model.safetensors.index.json,model-*.safetensors'tools/p4r16_stream_fp8.py converts one independently verified official shard at
a time. The converter keeps embeddings, LM head, norms and buffers in BF16, and
quantizes only the seven locked Yi projection matrices in each of 60 layers. Its
audit persists every source key, shape and dtype before an official shard may be
released. Unknown 2-D tensors fail instead of being guessed.
python3 tools/p4r16_stream_fp8.py \
model/model-00001-of-00015.safetensors \
derived/model-00001-of-00015.safetensors \
--expected-sha256 9a10cac63db6cee2460c3a0a46731598feafc6248e8ad7ad8529d80be2be6cc7Repeat in the locked 15-shard order from
audit/p4r16-official-bf16-selected.json. Release a BF16 shard only after
its output and .audit.json identities are durable. Copy the unchanged official
model.safetensors.index.json and metadata into derived/, then finalize and
re-verify the global gate:
python3 tools/p4r16_fp8_e2e.py finalize \
--official-metadata model \
--derived derived \
--plan audit/p4r16-official-bf16-selected.json \
--output derived/p4r16-derived-fp8-manifest.json
python3 tools/p4r16_fp8_e2e.py verify \
--official-metadata model \
--derived derived \
--manifest derived/p4r16-derived-fp8-manifest.jsonThe global gate requires exact official-index parity; complete key, shape and source-dtype coverage; FP8/retained-BF16 dtype policy; one scalar F32 scale for every quantized tensor; all 15 source/output identities; and a 30/30 layer map. Before real hydration, each card must have at least its exact decoded-BF16 parameter bytes plus the configured 8 GiB runtime reserve free.
The hosted matrix is CANN 8.5.1, PyTorch 2.9.0, and torch-npu
2.9.0.post1. Keep that platform stack and rebuild only the fixed user-space
closure from the Tsinghua mirror into an empty target:
python3 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple \
--target "$PWD/pydeps" --no-deps --no-build-isolation -r requirements.txt
test ! -e "$PWD/pydeps/torch"
test ! -e "$PWD/pydeps/torch_npu"The atomic derived lane is
/data/atomcode-lanes/heavy19-20260820/p4r16-dolphin-fp8. Do not start until
that directory has both WEIGHT_READY and its final derived manifest. The
official tokenizer/config/index metadata remains in the fixed-revision checkout
under .one-bf16-shard/.source/checkout; its tiny LFS pointer files are not
runtime weights.
The runtime constructs PreTrainedTokenizerFast directly from the complete
tokenizer.json plus the pinned special-token/chat-template fields. It does not use
the metadata-only LFS pointer stored as tokenizer.model.
Both NPUs must be idle and admitted before loading. The manifest records the
exact decoded-BF16 parameter bytes assigned to each card; the loader requires
that exact amount plus an 8 GiB reserve independently on npu:0 and npu:1.
This is a hard pre-load gate, not an estimate.
lane=/data/atomcode-lanes/heavy19-20260820/p4r16-dolphin-34b
derived=/data/atomcode-lanes/heavy19-20260820/p4r16-dolphin-fp8
repo="$lane/repo"
metadata="$derived/.one-bf16-shard/.source/checkout"
mkdir -p "$lane/audit-final"
source /usr/local/Ascend/cann-8.5.1/set_env.sh
PYTHONNOUSERSITE=1 TORCH_DEVICE_BACKEND_AUTOLOAD=0 \
HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
PYTHONPATH="$lane/pydeps:$repo/tools${PYTHONPATH:+:$PYTHONPATH}" \
python3 "$repo/audit/run-p4r16-derived-two-npu.py" \
--repo "$repo" \
--derived-lane "$derived" \
--official-metadata "$metadata" \
--output "$lane/audit-final/result.json" \
--max-new-tokens 8 \
--hold-seconds 600status=PASS requires the immutable manifest gate, the HBM gate, every hydrated
parameter on one of the two NPUs, the exact 30/30 split, NPU input/output, and
complete autoregressive generation. CPU/disk/meta offload and CUDA fallback are
forbidden. The key/shape/dtype/scale/index checks are checkpoint construction
integrity gates, not an accuracy or parameter-count score.
The public repository carries exactly three hosted PNGs after the final run:
assets/agent_workflow.png: AtomCode plan-only workflow for the official-BF16
to derived-FP8 stream, immutable manifest gate, and two-NPU execution.assets/npu_device_call.png: the successful held PID visible on both NPUs.assets/model_result.png: the same run's JSON showing
artifact_identity=DERIVED_FROM_OFFICIAL_BF16, official_fp8_available=false,
manifest identity, the 30/30 split, and NPU input/output.


Only after the same successful held PID has supplied the device and result
images may this package be pushed. An anonymous fresh clone must match the
pushed main commit, remain clean, contain the README/inference/requirements
triad, pass py_compile for the inference, runner and all three tools, parse all
audit JSON/schema files, and contain exactly the three PNGs above with three
distinct SHA-256 values and all three README references. It must contain no
official or derived weight shards, __pycache__, .pyc, partial payload, or
nested .git directory.
本仓为 2301_76761127 队伍交付件(与备份账号合并提交)。