"Zoom and Enhance" to Become a Reality Thanks to Machine Learning

[btarunr]

The one phrase from television that makes IT people and creative professionals cringe the most is "zoom and enhance" - the notion that you zoom into a digital image and, at the push of a button, it converts a pixellated image into something with details - which lets CSI catch the bad guys. Up until now, this has been laughably impossible. Images are made up of dots called pixels, and the more pixels you have, the more details you can have in your image (resolution). Zooming into images eventually shows you a colorful checkerboard that's proud of its identity.

Google is tapping into machine-learning, in an attempt to change this. The company has reportedly come up with a machine-learning technique that attempts to reconstruct details in low-resolution images. Google is calling this RAISR (rapid and accurate image super-resolution). The technology works with the software learning "edges" of a picture (portions of the image with drastic changes in color and brightness gradients), and attempts to reconstruct them. What makes this different from conventional super-sampling methods is its machine-learning component. A low-resolution image is studied by the machine to invent an upscaling method most effective for the image, in-situ. While its application in law-enforcement is tricky, and will only become a reality when a reasonably high court of law sets a spectacular precedent; this technology could have commercial applications in up-scaling low-resolution movies to new formats such as 4K Ultra HD, and perhaps even 8K.



View at TechPowerUp Main Site

 

[Steevo]

Meh, what happens when the machine is only as good as humans as juxtaposing what might be there, will we still place faith in what a machine with AI does when all it knows is averages, and puts an average face on a pixelated "cleaned up" image and gets someone innocent thrown in jail?

 

[slozomby]

Meh, what happens when the machine is only as good as humans as juxtaposing what might be there, will we still place faith in what a machine with AI does when all it knows is averages, and puts an average face on a pixelated "cleaned up" image and gets someone innocent thrown in jail?

i'd trust a cleaned up image far more than i trust the average human to remember what happened at a crime scene. if we're convicting people based on a single image then we're already screwed.

 

[Xzibit]

 

[R-T-B]

i'd trust a cleaned up image far more than i trust the average human to remember what happened at a crime scene. if we're convicting people based on a single image then we're already screwed.

Except we're not talking "cleaned up." We're talking outright creation of new data from adjacent pixels to fill in what should rightfully be nothingness.

Given that, frankly, I wouldn't trust it at all.

 

[LAN_deRf_HA]

There's already ways to do this now. The latest "waifu2x" has a photo mode and the output looks just like that. Results are a real hit or miss but it works surprisingly well sometimes.

 

[RejZoR]

If you've been living under a rock, this is something "new". If you know this tech from years ago, it's called S-Spline filter, developed by Benvista and used in their PhotoZoom.

http://www.benvista.com/photozoompro/examples

I've also used their S-Spline filter to optimize System Shock 2 cutscenes...

https://rejzor.wordpress.com/2015/05/26/system-shock-2-high-definition-cutscenes/

Clearly, you can't pull non existent information out of your ass, but advanced prediction algorithms can certainly elevate the image details far beyond high quality filters like Bicubic or Lanczos.

 

[Rowsol]

That third picture reminds me of what texture scalers do on old sprite based video games. https://www.google.com/search?q=xbr...UICSgC&biw=1745&bih=895#imgrc=ikO5rBw2Ki8E9M:

 

[natr0n]

I bet emulation forum users are shitting the floor with this news.

 

[newconroer]

Meh, what happens when the machine is only as good as humans as juxtaposing what might be there, will we still place faith in what a machine with AI does when all it knows is averages, and puts an average face on a pixelated "cleaned up" image and gets someone innocent thrown in jail?

I believe that's why he mentions that a spectacular precedent would have to set.

 

[Vario]

 

[RejZoR]

If I had the tools to quickly process textures, I'd use PhotoZoom on old games and scale them up to higher resolutions (if game engines support higher res).

 

[CAT-THE-FIFTH]

 

[Prima.Vera]

I always got cracked up of that scene, lol. The resolution of that film must have been, what, 512K? in order to pull that kind of detail.

 

[marios15]

OMG hahahahahahaha this is gold and true in some years(?)

 

[jigar2speed]

Guess Tarun again forgot to mention tiny detail on who Google has tapped for
Accelerating Machine Learning - AMD.

Reference - http://www.forbes.com/sites/aaronti...g-machine-learning-in-the-cloud/#1dd64f7e4181

 

[cdawall]

Guess Tarun again forgot to mention tiny detail on who Google has tapped for
Accelerating Machine Learning - AMD.

Reference - http://www.forbes.com/sites/aaronti...g-machine-learning-in-the-cloud/#1dd64f7e4181

It figures they would use amd GPUs here they are more powerful when utilized properly.

 

[btarunr]

Guess Tarun again forgot to mention tiny detail on who Google has tapped for
Accelerating Machine Learning - AMD.

Reference - http://www.forbes.com/sites/aaronti...g-machine-learning-in-the-cloud/#1dd64f7e4181

I did not forget because it does not figure in the source article.

 

[jigar2speed]

I did not forget because it does not figure in the source article.

Agreed, your source link doesn't mention AMD at all...

 

[$ReaPeR$]

its so nice to live in an age with such technological achievements!

 

[BiggieShady]

That third picture reminds me of what texture scalers do on old sprite based video games. https://www.google.com/search?q=xbrz&source=lnms&tbm=isch&sa=X&ved=0ahUKEwi60Oj0ocfQAhWK4SYKHYLdA6oQ_AUICSgC&biw=1745&bih=895#imgrc=ikO5rBw2Ki8E9M:

Oh yes, I have (re)played Day Of The Tentacle and other SCUMM games upscaled in ScummVM emulator using similar filters ...

 

[arterius2]

Oh yes, I have (re)played Day Of The Tentacle and other SCUMM games upscaled in ScummVM emulator using similar filters ...
View attachment 81412

AdvMAME3X looks pretty good.

 

[DeathtoGnomes]

8k resolution, is that like Jetsons 3D tv?

 

[geon2k2]

nice one

 

[arterius2]

This is so sad, filmmakers, of all people should understand this considering they work with visual media.