DCT-Net（Domain-Calibrated Translation for Portrait Stylization）是阿里巴巴达摩院与北京大学合作发表于SIGGRAPH 2022的人像卡通化模型，其核心优势在于仅需约100张风格样本即可通过域校准机制实现真实人像到多种艺术风格的高质量迁移，模型架构采用生成对抗网络并结合Stable Diffusion增强风格适应性，能有效处理遮挡、配饰等复杂面部细节，广泛应用于个性化头像生成、电商虚拟试穿及数字内容创作等创意娱乐与设计场景。
版本说明：
url=https://github.com/menyifang/DCT-Net.git
commit_id=a075d94be0ade99be6c27e7a050c30426fdf6dc5
model_name=DCT-Net该模型需要以下插件与驱动
表 1 版本配套表
| 配套 | 版本 | 环境准备指导 |
|---|---|---|
| 固件与驱动 | 25.3.rc1.2 | Pytorch框架推理环境准备 |
| CANN | 8.3.RC1 | CANN版本兼容性 8.5的cann也没问题 |
| Python | 3.7.16 | - |
| TensorFlow | 2.6.5 | - |
| Ascend Extension TensorFlow | 2.6.5 | - |
| 说明：Atlas 800I A2 推理卡请以CANN版本选择实际固件与驱动版本。 | \ | \ |
获取TensorFlow源码
git clone https://gitcode.com/ascend/ModelZoo-PyTorch.git
cd ModelZoo-PyTorch/ACL_PyTorch/built-in/cv/DCT-Net
git clone https://github.com/menyifang/DCT-Net.git安装依赖
# ubuntu系统，openeuler系统请使用yum命令
apt-get install -y --allow-downgrades udev
apt install -y build-essential autoconf automake libass-dev libfreetype6-dev \
libsdl2-dev libtheora-dev libtool libva-dev libvdpau-dev libvorbis-dev \
libxcb1-dev libxcb-shm0-dev libxcb-xfixes0-dev pkg-config texinfo zlib1g-dev \
yasm libx264-dev libfdk-aac-dev libmp3lame-dev libopus-dev libvpx-dev
wget https://ffmpeg.org/releases/ffmpeg-4.2.10.tar.bz2 --no-check-certificate
tar -xjvf ffmpeg-4.2.10.tar.bz2
cd ffmpeg-4.2.10
./configure --enable-shared --prefix=/usr/local/ffmpeg --enable-pic --enable-gpl --enable-nonfree \
--enable-libx264 --enable-libfdk-aac --enable-libmp3lame --enable-libvpx \
--enable-libvorbis --enable-libopus --enable-libtheora
# 编译并安装
make -j$(nproc)
make install
ln -s /usr/local/ffmpeg/bin/ffmpeg /usr/local/bin/ffmpeg
which ffmpeg
ffmpeg -version
cd ..# 解决`pkg-config` 无法找到 FFmpeg 相关的库文件
which ffmpeg
find /usr/local/ffmpeg -name libsw*
export PKG_CONFIG_PATH=/usr/local/ffmpeg/lib/pkgconfig:$PKG_CONFIG_PATH
pkg-config --modversion libavcodec
pkg-config --list-all | grep -i ffmpeg# 编译安装arm机器的三方库
wget https://files.pythonhosted.org/packages/36/78/3e941ff7ec082b27e6751134ab96f7dcd246589db8745164e52cb64567df/PyMCubes-0.1.3.tar.gz
tar -zxf PyMCubes-0.1.3.tar.gz
cd PyMCubes-0.1.3
python setup.py install
cd ..
pip install cymem==2.0.11 preshed==3.0.2 murmurhash==1.0.13
wget https://files.pythonhosted.org/packages/79/fd/13cdec8c1a3a2d57f5161e183639367d9d002965dfc893f222293fb1cc04/spacy-2.2.0.tar.gz
tar -zxf spacy-2.2.0.tar.gz
cd spacy-2.2.0
python setup.py install
cd ..
cd /root/miniconda3/envs/dctnet37/lib/python3.7/site-packages/npu_device/convert_tf2npu
wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/MindX/OpenSource/python/packages/tensorflow-2.6.5-cp37-cp37m-manylinux2014_aarch64.whl#sha256=daf14490ef2b9334c7472d25ca66640e2546bc8e446c6bc1d88e5d4855e75db6
pip install tensorflow-2.6.5-cp37-cp37m-manylinux2014_aarch64.whl
wget https://gitee.com/ascend/tensorflow/releases/tag/tfa_v0.0.44_8.3.RC1
pip install npu_device-2.6.5-py3-none-manylinux2014_aarch64.whl
pip install "modelscope[cv]==1.3.2" -f http://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html --trusted-host modelscope.oss-cn-beijing.aliyuncs.com
pip install "modelscope[multi-modal]==1.3.2" -f http://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html --trusted-host modelscope.oss-cn-beijing.aliyuncs.com
workdir=$(pwd)
source_path=$(pip show tensorflow | grep Location | awk '{print $2}')
cd ${source_path}/npu_device/convert_tf2npu
python3 main.py -i ${workdir}/DCT-Net -m ${workdir}/DCT-Net/run.py
cd output_npu_20260318092139/DCT-Net_npu_20260318092139 # 这里的日期后缀因人而异，里面就是迁移后的脚本
patch source/cartoonize.py < ${workdir}/cartoonize_changes.patch # 修改源码，使能npu
patch run.py < ${workdir}/run_changes.patch # 修改自动迁移后的脚本python download.py权重目录大致结构为：
damo/
|-- cv_unet_person-image-cartoon-3d_compound-models
| |-- README.md
| |-- alpha.jpg
| |-- cartoon_bg.pb
| |-- cartoon_h.pb
| |-- configuration.json
| |-- description
| | |-- demo.gif
| | `-- network.png
| |-- detector.pb
| `-- keypoints.pb
|-- cv_unet_person-image-cartoon-artstyle_compound-models
| |-- README.md
| |-- alpha.jpg
| |-- cartoon_bg.pb
| |-- cartoon_h.pb
| |-- configuration.json
| |-- description
| | |-- demo.gif
| | `-- network.png
| |-- detector.pb
| `-- keypoints.pb
|-- cv_unet_person-image-cartoon-handdrawn_compound-models
| |-- README.md
| |-- alpha.jpg
| |-- cartoon_bg.pb
| |-- cartoon_h.pb
| |-- configuration.json
| |-- description
| | |-- demo.gif
| | `-- network.png
| |-- detector.pb
| `-- keypoints.pb
|-- cv_unet_person-image-cartoon-sd-design_compound-models
| |-- README.md
| |-- alpha.jpg
| |-- cartoon_bg.pb
| |-- cartoon_h.pb
| |-- configuration.json
| |-- description
| | |-- demo.gif
| | `-- network.png
| |-- detector.pb
| `-- keypoints.pb
|-- cv_unet_person-image-cartoon-sd-illustration_compound-models
| |-- README.md
| |-- alpha.jpg
| |-- cartoon_bg.pb
| |-- cartoon_h.pb
| |-- configuration.json
| |-- description
| | |-- demo.gif
| | `-- network.png
| |-- detector.pb
| `-- keypoints.pb
|-- cv_unet_person-image-cartoon-sketch_compound-models
| |-- README.md
| |-- alpha.jpg
| |-- cartoon_bg.pb
| |-- cartoon_h.pb
| |-- configuration.json
| |-- description
| | |-- demo.gif
| | `-- network.png
| |-- detector.pb
| `-- keypoints.pb
`-- cv_unet_person-image-cartoon_compound-models
|-- README.md
|-- alpha.jpg
|-- cartoon_anime_bg.pb
|-- cartoon_anime_h.pb
|-- cartoon_bg.pb
|-- cartoon_h.pb
|-- configuration.json
|-- description
| |-- demo.gif
| |-- network.png
| |-- sim1.png
| |-- sim2.png
| |-- sim3.png
| |-- sim4.png
| |-- sim5.png
| |-- sim_3d.png
| |-- sim_anime.png
| |-- sim_artstyle.png
| |-- sim_design.png
| |-- sim_handdrawn.png
| |-- sim_illu.png
| `-- sim_sketch.png
|-- detector.pb
|-- keypoints.pb
|-- tf_ckpts
| |-- model-16999.data-00000-of-00001
| `-- model-16999.index
`-- vgg19.npy
15 directories, 80 filespython3 run.py在OmniDocBench数据集上的性能数据如下：
| 模型 | 芯片 | 第一张性能（s） | 第二张性能（s） |
|---|---|---|---|
| DCT-Net | A100 | 1.2 | 0.454 |
| DCT-Net | 800I A2 64G | 32.3 | 0.0295 |
NPU结果图如下所示
在NPU与GPU的run.py文件中分别添加以下代码，以生成两个.npy文件
import numpy as np
...
np.save('xpu_result.npy', result)进行余弦相似度的计算如下所示
python accuracy_comparison.py npu_result.npy gpu_result.npy最终结果应该是1.00，如下表所示
| 模型 | 芯片 | 输入 | 余弦相似度 |
|---|---|---|---|
| DCT-Net | A100 && 800I A2 64G | input.png | 1.00 |