How To Improve Photo Quality by AI | Noise Reduction, Super-resolution Tutorial

Video installation、Composite media
10’40”
2022

如何使用AI提升照片品質 | Noise Reduction、Super Resolution教學

錄像裝置、複合媒材
10’40”
2022
　

︎ 2023 Taipei Art Awards Honorable Mention
︎ 2023 臺北美術獎優選

《如何使用AI提升照片品質 | Noise Reduction、Super Resolution教學》展場作品照

《如何使用AI提升照片品質 | Noise Reduction、Super Resolution教學》展場作品照

With the booming technology of communication, multimedia and audiovisual occupy people’s vision through streaming media in which various genres of self-media privatize image files of the poor image, further propelling the creative works of video content towards privatization.

Of all the genres tutorial videos have become one of the significant ways of learning for people. Schools and institutions no longer circumscribe spaces for learning, and people can have technologies difficult for self-learners such as AI (artificial intelligence) at their disposal.

The introduction of AI technology opens up a new chapter in optimization for photos. AI learning takes place between millions of photos before and after adding noise, or between different resolutions of the same photo, enabling a better result in noise reduction and image super-resolution.

In 1990, the spacecraft Voyager 1 on the verge of the solar system photographed the Pale Blue Dot photo in the direction towards Earth upon the astronomer Carl Sagan’s request. In this photo filled with noise, Earth occupies merely a white dot in 0.12 pixels, barely noticeable, as if easily erasable as part of the noise.

“Consider again at that dot,” Carl Sagan depicts the photo with a poetic revelation that evinces the humble existence of humans and the delusion of human privilege in the universe. This shiny, tiny blue dot challenges such delusion and unravels the human perception of distance extended by scientific images.

“That's here. That's home. That's us. On it everyone you love, everyone you know, everyone you ever heard of, every human being who ever was, lived out their lives. The aggregate of our joy and suffering, thousands of confident religions, ideologies, and economic doctrines, every hunter and forager, every hero and coward, every creator and destroyer of civilization, every king and peasant, every young couple in love…lived there - on a mote of dust suspended in a sunbeam…Our posturings, our imagined self-importance, the delusion that we have some privileged position in the Universe…”

A large amount of training data may reduce the impact of exceptions in AI models, thus strengthening the adaptability (robustness) of the AI models. However, the AI noise reduction model, trained under this seemingly reasonable logic, perceives Earth in the Pale Blue Dot photo as noise and erases it with the rest of the photo noises - are the 12 new resolutions in the AI super-resolution model based on a universe without us and Earth?

I imitate the style and method of the popular online tutorials in the setting and filming, where the tutor sits behind a desk with editing equipment in a colorful, atmospheric background. The tutor demonstrates the AI image editing software with the Pale Blue Dot photo as teaching materials. While the tutor explains the steps in handling noise reduction and super-resolution and eventually produces a high-quality Pale Blue Dot photo, Carl Sagan’s speech interpolates through the tutorial. In the tutorial video, it employs the stock photos and video clips used in self-media. In the exhibition site, the tutorial video is matched with lighting and a kinetic installation of laser grids, bridging the imagination of traveling interstellar space with the metaphor of the universe perceived in resolution.

隨著通訊技術的蓬勃發展，多媒體影音透過串流（streaming media）佔據了人們的視線，各種類型的自媒體（self-media or we media）把弱影像（poor image）中的影像檔案私有化，更進一步推向了影像內容創作的私有化。

其中的教學類型影片（tutorial video），已經成為人們重要的學習途徑之一，讓學習的場域不再受限於學校、機構，也讓如AI等等原本不易自學的技術變得唾手可得。

AI技術的導入開啟了照片品質優化的另一個篇章。AI學習數百萬張加入雜訊前後的照片或解析度不同的相同影像之間，讓降低雜訊（noise reduction）與影像超解析度（image super resolution）有了更好的效果。

1990年，將要飛出太陽系的「航海家 1 號」太空船，應天文學家卡爾・薩根（Carl Sagan）的要求，在64億公里外，往地球方向拍攝了「暗淡藍點」（Pale Blue Dot）這張照片。照片中充滿了雜訊，地球只是佔整張照片0.12像素的一顆白點，看起來並不顯眼，彷彿隨時都會被當作是雜訊忽視掉。

卡爾・薩根對這張照片發表了如詩詞般的啟示，以「讓我們再看一眼這個小圓點」為開頭，描寫了人類的渺小以及人類在宇宙中常常會有特權的錯覺。這顆泛著藍光的小點挑戰著這個錯覺，同時也揭示了人類感官透過科學影像延伸的距離。

　　「就在這裡。這就是家。這就是我們。在這個小點上，每一個你愛的人，每一個你認識的人，每一個你聽說過的人，每一個人，無論他是誰，都在這度過了他自己的一生。我們所有的快樂和掙扎，數以千萬自傲的宗教信仰、思想體系觀念意識，以及經濟學原理教義，每一個獵人或征服者，每一位勇士或是懦夫，每一個文明的締造者或摧毀者，每一位君王或農夫，每一對陷入愛河的年輕伴侶......我們的一切一切，全部都存在於這樣一粒懸浮在一束陽光中的塵埃上......我們的故作深沉，我們想像出來的自我重要性，我們以為自己在宇宙裡有什麼特權的錯覺...... 」

左：暗淡藍點原圖
右：暗淡藍點經過AI降噪、超解析處理後

大量的訓練資料可以盡可能的減少特例對AI模型的影響，進而提高AI模型的適應力（穩健性，robustness），然而在如此合理的邏輯之下所訓練出來的AI降躁模型，卻把「暗淡藍點」裡的地球感知成雜訊，連同其他雜訊一起去除了。而AI超解析度模型所感知延伸的12個新像素，是否便是建立在沒有地球、沒有我們的宇宙之下？

我仿擬網路上常見的教學影片風格、手法進行陳設與拍攝，教學者在桌前搭配著剪輯設備，身後佈置多彩氛圍。教學者以「暗淡藍點」照片當做示範素材，使用AI影像編修軟體，講解照片降噪與超解析度的處理步驟，最終產出一張高品質的「暗淡藍點」。教學過程中穿插卡爾・薩根的語音，搭配在自媒體影片中常使用的素材庫裡所挑選的影片素材，並同時搭配展場空間的光線變化及雷射方格動力裝置，連結遨遊星際的想像以及像素感知宇宙的隱喻。