StyleGAN2

117 points by rolux 6 years ago | 34 comments

Veedrac 6 years ago |

I set up this super simple ‘Which Face Is Real?’ (http://www.whichfaceisreal.com/) style challenge. Click the row to show the answers. You might need to zoom out.

https://veedrac.github.io/stylegan2-real-or-fake/game.html

There's a harder version as well, where the image is zoomed in.

https://veedrac.github.io/stylegan2-real-or-fake/game_croppe...

I get 100% reliably with the first link (game.html), and got 4/5 on the cropped version (game_cropped.html) so far.

ebg13 6 years ago | |

> I get 100% reliably with the first link, and got 4/5 on the cropped version so far.

Looking at whichfaceisreal, How much time do you have to spend on each decision, and would your success rate change if you didn't know in advance that exactly 1 of 2 was generated? It's easy to say 100% reliable, but I find myself really having to dig deep with my eyes to search for small tells* , which you have to know to do up front before you actually do it.

* - Often the tells are as minuscule as some ringing around the hair, which could just as easily be compression artifacts on a real photo.

Veedrac 6 years ago | | |

For the first link (game.html), normally 5-10s, but much longer or shorter for some of them. For the second (game_cropped.html), it takes much longer, like a minute or so, except when the real image contains something distinctive StyleGAN2 can't do. For whichfaceisreal.com, which uses the original StyleGAN and only offers two options, none hand-picked, it takes me a second or two.

echelon 6 years ago | |

> I set up this super simple ‘Which Face Is Real?’ (http://www.whichfaceisreal.com/) style challenge.

GANs still don't get teeth right. If any artificial face smiles, the teeth are a dead giveaway.

Veedrac 6 years ago | | |

I thought StyleGAN2's teeth were one of the biggest upgrades to the face proper. The original's were pretty bad, but removing the phase artefact issue seems to have made a huge difference.

Here are some examples of teeth I found pretty decent:

https://veedrac.github.io/stylegan2-real-or-fake/PSI%201.0/0... https://veedrac.github.io/stylegan2-real-or-fake/PSI%200.5/0... https://veedrac.github.io/stylegan2-real-or-fake/PSI%200.5/0... https://veedrac.github.io/stylegan2-real-or-fake/Curated/ffh...

ImminentFate 6 years ago | |

On your site I can consistently get 100% by looking at the backgrounds since they generate in somewhat inorganic patterns.

nuccy 6 years ago | | |

Also after watching the video from the StyleGAN2 team https://drive.google.com/file/d/1f_gbKW6FUUHKkUxciJ_lQx29mCq... now I know that original StyleGAN, images from which are apparently used for this "game", produces faces with a "water droplet" and phase artifacts, so I was able to spot few fakes just by looking for those things.

ebg13 6 years ago | | |

That's speculative at best. What's inorganic about the background in https://veedrac.github.io/stylegan2-real-or-fake/Curated/ffh... ?

TomVDB 6 years ago | | |

When not the ears, it's the background.

If the real pictures had the background removed, I'd have a very hard time scoring 100%.

gwd 6 years ago |

Only watched the video, but one of the interesting things is the potential method to tell a generated image from a real one: namely, if you take a generated image, it's possible to find parameters which will generate exactly the same image. But if you take a real image, it's generally not possible to get exactly the same image, but only a similar one.

The exact point in the video:

https://youtu.be/c-NJtV9Jvp0?t=208

gwd 6 years ago | |

Also, looks like https://thispersondoesnotexist.com/ has been updated to use the new generator.

mzs 6 years ago | | |

Phew I looked at three and they all had toothy smiles where the teeth grew out out of the lips and one had a floating tooth.

TeeWEE 6 years ago | |

This is only possible if you have access to the model neural net... If you dont you cant tell the difference.

TeeWEE 6 years ago | | |

Actually i was wrong. Generative Adverserial Nets often work accross Machine Learning models...

resiros 6 years ago |

The demo in the official video is mind blowing. https://www.youtube.com/watch?v=c-NJtV9Jvp0 I wonder when we will see full movies unrecognizable from real ones made from deep learning.

globuous 6 years ago | |

Insane the part where they get multiple angles from the same generated face

alexcnwy 6 years ago |

The part of the video showing the location bias in phase artifacts (straight teeth on angled faces) is really interesting and very clear in retrospect if you look at StyleGAN v1 outputs.

Their “new method for finding the latent code that reproduces a given image” is really interesting and I’m curious to see if it plays a role in the new $1 million Kaggle DeepFakes Detection competition.

It feels like we’re almost out of the uncanny valley. It’s interesting to place this in context and think about where this technology will be a few years from now - see this Tweet by Ian Goodfellow on 4.5 years of GAN progress for face generation: https://twitter.com/goodfellow_ian/status/108497359623614464...

anonfunction 6 years ago |

I'm surprised to see Nvidia hosting[1] the pre-trained networks on google drive which has already been blocked for going over the quota:

> Google Drive download quota exceeded -- please try again later

1. https://github.com/NVlabs/stylegan2#using-pre-trained-networ...

anyzen 6 years ago |

A bit off-topic - the license [0] is interesting. IIUC, if anyone who is using this code decides to sue NVidia, the grants are revoked, and they can sue back for copyright infringement?

Also, interesting that even with such "short" licences there are trivial mistakes in it (section 2.2 is missing, though it is referenced from 3.4 and 3.6 - I wonder what it was...)

[0] https://nvlabs.github.io/stylegan2/license.html

supermatt 6 years ago | |

Patent grants.

Its a butchered Amazon Software License: https://aws.amazon.com/asl/

bonoboTP 6 years ago | | |

I've never understood why it's allowed to give up "suing rights" in contracts. It is in the interest of the public that any law infringement gets investigated and the infringers punished.

In principle a lawsuit is just asking a neutral party to judge whether there indeed was a law breaking where I suspect there was one. Ideally, this is not a inherently hostile action that should be met with any negative consequences.

I know criminal and civil law are different beasts, but still the situation is analogous to renting out a room to someone in exchange for them promising not to report me to the police if I beat them up, else I can kick them out without notice.

It should be an inalienable right of anyone to report/sue for any wrongdoing against them. It should not be conditional on losing some (any) beneficial things.

"I agree I will not sue you even if I later find out that you did something illegal against me" should not be legal to be in a contract.

tiborsaas 6 years ago |

Imagine when these faces start talking, tracking objects with their eyes with a perfectly synthesized voice all, generated in real time.

gdubs 6 years ago |

Of course we’ll hit a wall at some point, but when this repo dropped the other night and I saw the rotating faces in the video, it made me realize that in the future, VR experiences might be generated with nets rather than modeled with traditional CG.

sails 6 years ago |

Any good resources for using GANs to generate synthetic tabular data?

narsk 6 years ago |

ctfu at the car images. I made a twitter bot to tweet them out with fake make/model names https://twitter.com/bustletonauto

nalllar 6 years ago |

urgh, custom CUDA ops now.

Original StyleGAN worked on AMD cards, this won't without porting those.

jdkdnfndnfjd 6 years ago |

It makes me feel ill to see computers doing things like this. Aidungeon was difficult to stomach as well. GANs were invented on a whim by a single person. Nobody thought it would work when applied to this kind of problem. It came out of nowhere. Pretty soon someone will try something on a higher order task and it’s going to work. We are opening Pandora’s box and I’m not sure that we should do that.

allenbrunson 6 years ago | |

okay, i'll be the one to stick my neck out ...

i read a few of the AI Dungeon transcripts. i think it's worthless. you type in a command, like you would with an actual infocom-style adventure game, and it spits out a bunch of flowery language and gobbledygook, the likes of which you can get in abundance from any self-help guru. there is no state, no way to win, no way to lose. to compare it to actual adventure games is ludicrous.

likewise, i have yet to see anything having to do with StyleGAN photo manipulation that goes very far beyond the level of a parlor trick.

this stuff is going to appeal to the same people who love cryptocurrencies, and will have about the same level of real-world effect.