TY - JOUR
T1 - STB-VMM
T2 - Swin Transformer based Video Motion Magnification
AU - Lado-Roigé, Ricard
AU - Pérez, Marco A.
N1 - Funding Information:
The authors would like to gratefully acknowledge the support and funding of the Catalan Agency for Business Competitiveness (ACCIÓ) through the project INNOTEC ISAPREF 2021. Furthermore, the first author would like to acknowledge a Doctoral Scholarship from IQS . Finally, the authors would like to thank Dr. Eduardo Blanco from the University of Arizona and Dr. Ariadna Chueca de Bruijn for their help.
Funding Information:
The authors would like to gratefully acknowledge the support and funding of the Catalan Agency for Business Competitiveness (ACCIÓ) through the project INNOTEC ISAPREF 2021. Furthermore, the first author would like to acknowledge a Doctoral Scholarship from IQS. Finally, the authors would like to thank Dr. Eduardo Blanco from the University of Arizona and Dr. Ariadna Chueca de Bruijn for their help.
Publisher Copyright:
© 2023 Elsevier B.V.
PY - 2023/6/7
Y1 - 2023/6/7
N2 - The goal of video motion magnification techniques is to magnify small motions in a video to reveal previously invisible or unseen movement. Its uses extend from bio-medical applications and deepfake detection to structural modal analysis and predictive maintenance. However, discerning small motion from noise is a complex task, especially when attempting to magnify very subtle, often sub-pixel movement. As a result, motion magnification techniques generally suffer from noisy and blurry outputs. This work presents a new state-of-the-art model based on the Swin Transformer, which offers better tolerance to noisy inputs as well as higher-quality outputs that exhibit less noise, blurriness, and artifacts than prior-art. Improvements in output image quality will enable more precise measurements for any application reliant on magnified video sequences, and may enable further development of video motion magnification techniques in new technical fields.
AB - The goal of video motion magnification techniques is to magnify small motions in a video to reveal previously invisible or unseen movement. Its uses extend from bio-medical applications and deepfake detection to structural modal analysis and predictive maintenance. However, discerning small motion from noise is a complex task, especially when attempting to magnify very subtle, often sub-pixel movement. As a result, motion magnification techniques generally suffer from noisy and blurry outputs. This work presents a new state-of-the-art model based on the Swin Transformer, which offers better tolerance to noisy inputs as well as higher-quality outputs that exhibit less noise, blurriness, and artifacts than prior-art. Improvements in output image quality will enable more precise measurements for any application reliant on magnified video sequences, and may enable further development of video motion magnification techniques in new technical fields.
KW - Computer vision
KW - Deep learning
KW - Image quality assessment
KW - Motion magnification
KW - Swin Transformer
UR - http://www.scopus.com/inward/record.url?scp=85151378111&partnerID=8YFLogxK
U2 - 10.1016/j.knosys.2023.110493
DO - 10.1016/j.knosys.2023.110493
M3 - Article
AN - SCOPUS:85151378111
SN - 0950-7051
VL - 269
JO - Knowledge-Based Systems
JF - Knowledge-Based Systems
M1 - 110493
ER -