cugan_tf发布

发布于 2023-09-16  285 次阅读


ACG2vec系列之CUGAN_TF——运行在浏览器上的动漫超分工具(Real-CUGAN的tensorflow实现)

介绍

当前最优秀的动漫领域超分模型之一Real-CUGAN的tensorflow实现,依赖tfjs框架完成自适应后端的能运行在浏览器上的动漫超分工具。

原版实现分为切块后超分与整图超分,两种都以实现,但切块超分版本转为tfjs模型后在网页运行不正常,已向tfjs仓库提交issue。目前预览版本是整图超分版本,由于内存限制,限制了原始图片大小(512x512以内),后续issue解决将发布切块超分,大概率将不会有限制。

This module is an implementation of one of the current leading anime super-resolution models in the field, Real-CUGAN, using TensorFlow. It relies on the tfjs framework to create an adaptive backend, enabling it to run as an anime super-resolution tool in web browsers.

The original implementation offers two versions: chunk-based super-resolution and full-image super-resolution, both of which have been implemented. However, the chunk-based super-resolution version encounters issues when converted into a tfjs model and run in a web browser. An issue has been raised in the tfjs repository to address this problem.

The current preview version focuses on full-image super-resolution but has limitations on the original image size (up to 512x512) due to memory constraints. Once the issue is resolved, the chunk-based super-resolution version is likely to be released without such limitations.

预览

image-20230916210759548

效果

输入

110897028_p0_square1200_副本

无降噪输出

image (5)_副本

保守输出

image (4)

降噪输出

cugan_jz

pytorch模型迁移到tensorflow应该注意的点

  • 图片处理默认维度顺序: tensorflow为nhwc,pytorch为nchw,卷积权重维度顺序也不相同
  • tensorflow转置卷积无法自定义padding: Conv2DTranspose层padding设为0,后续使用slice手动crop输出
  • tf.pad无法接受负数:使用 tf.slice作为替代
  • 多尺寸输入导致无法batch :无解
  • 延迟设置输入尺寸运行时获取size:这是tensorflow图模型的限制
  • python操作逻辑最好翻译为tensorflow 分支选择api
  • TensorArray:TensorArray是图模式中python list替代品, TensorArray在eager模式TensorArray.write(i, x)可以直接生效,而在graph模式时需要将引用赋值给自身

面向ACG编程