STB-VMM: Swin Transformer based Video Motion Magnification

Ricard Lado-Roigé, Marco A. Pérez*

*Corresponding author for this work

Research output: Indexed journal article Articlepeer-review

10 Citations (Scopus)

Abstract

The goal of video motion magnification techniques is to magnify small motions in a video to reveal previously invisible or unseen movement. Its uses extend from bio-medical applications and deepfake detection to structural modal analysis and predictive maintenance. However, discerning small motion from noise is a complex task, especially when attempting to magnify very subtle, often sub-pixel movement. As a result, motion magnification techniques generally suffer from noisy and blurry outputs. This work presents a new state-of-the-art model based on the Swin Transformer, which offers better tolerance to noisy inputs as well as higher-quality outputs that exhibit less noise, blurriness, and artifacts than prior-art. Improvements in output image quality will enable more precise measurements for any application reliant on magnified video sequences, and may enable further development of video motion magnification techniques in new technical fields.

Original languageEnglish
Article number110493
Number of pages10
JournalKnowledge-Based Systems
Volume269
DOIs
Publication statusPublished - 7 Jun 2023

Keywords

  • Computer vision
  • Deep learning
  • Image quality assessment
  • Motion magnification
  • Swin Transformer

Fingerprint

Dive into the research topics of 'STB-VMM: Swin Transformer based Video Motion Magnification'. Together they form a unique fingerprint.

Cite this