The Magical Tech Behind Paper For iPad's Color-Mixing

The Magical Tech Behind Paper For iPad's Color-Mixing(fastcompany.com)

163 points by julianwa 13 years ago | 42 comments

gmaslov 13 years ago |

I was disappointed to read that they gave up the principled modeling approach (the 80-year old German paper mentioned early on in the article) and instead collected 100 data points that "looked good" and interpolated somehow.

The article says that the Kubelka-Munk model achieved color mixing that was "too realistic" and therefore difficult to use for the purpose of creating a distinctive brand palette. What this tells me is that they weren't looking for a model of how colors mix together, but a model of what colors look good together.

> "We know red and yellow should yield orange, or that red and blue should make purple--but there isn't any way to arrive at these colors no matter what color-space you use."

Well, that's clearly not true. In the HSL color wheel [1], orange is halfway between red and yellow, and purple is halfway between red and blue. Should have picked a better example.

[1] http://www.workwithcolor.com/hsl-color-picker-01.htm

jules 13 years ago | |

Look at it another way. What is the goal of their product? A person should be able to sit down, and draw something. That person has a model in his head of what the drawing should look like. He picks and mixes colors to achieve this according to his intuition. So the goal of the program is to match human intuition, not to match reality. Why would real world paint be the optimal painting tool? The best way to match human intuition is to base the behavior of the program on data collected from human intuition.

julianwa 13 years ago | |

Kubelka-Munk is fantastic for modeling the behavior of real world pigments as they're applied in thin layers on a canvas. However, since we were not trying to simulate the act of using physical watercolor, oil paints, etc., it was not only overkill but actually produced unintuitive results.

Our problem is more abstract. We want to appeal to the user's notions of how pigments blend together, but in such a way that the operation is predictable and can be hand tuned to avoid unpleasant cases. Ultimately, the mixing behavior that we released is inspired by the physical world, but is tuned by our designers to produce what they consider pleasing results.

julianwa 13 years ago | |

You're correct in that, for example, purple lies halfway between read an blue in terms of hue. However, an interpolation between those colors based on hue that does not adjust saturation, will look quite ugly, as it passes through magenta, etc. The question is, how should one adjust the saturation?

There's nothing wrong with any given color space. It was designed with particular goals in mind, and those goals generally did not include attractive blending through linear interpolation. So, yes, we developed a means of hand-tuning the interpolation such that it produces colors that look good together, which is what our users really cares about.

andrewingram 13 years ago |

I'm blown away by the effort put into a single feature, a feature which isn't even unlocked in the basic install of the app. Seriously impressive dedication to getting something right.

charlesmchen 13 years ago | |

We're lucky enough to have founders that are designers and engineers, and we have a design-driven process and a commitment to high quality standards. It's a pleasure to know you'll get the chance to get things right.

rsl7 13 years ago |

Style over substance.

Mixing colors would be a neat addition if the rest of the app was geared toward actual creative work. Drawing is laggy, there's no zoom, the UI enforces landscape mode, you can't select your own drawing order or see more than one drawing at a time with their fancy pants book page flip animation.

gstar 13 years ago |

The URL in the screenshot still works, you can play with it in the browser:

http://dl.dropbox.com/u/6432940/paper/mixingWell/24/index.ht...

AceJohnny2 13 years ago | |

no longer.

stevenwei 13 years ago |

Make no mistake, this level of attention to detail is why this app has remained consistently in the top charts since its release.

alayne 13 years ago | |

It's on the charts because it's "free". The overall rating is only 4/5 stars on itunes. They use the annoying model of making you buy tools like color as in app purchases.

Notability, which has some similarities, sells for 0.99 without in app purchase nonsense and is rated higher.

glhaynes 13 years ago | | |

I don't get the hate for in-app purchases of this kind. It's just like the shareware model that used to be popular: lots of people get absolutely free usage/enjoyment out of the base product and then, those who specifically want more, can pay—in this case a very small fee—for it. One constantly hears people clamoring for more choice, for example, to pay for just the cable channels they want. So what makes this model "annoying nonsense"?

mnicole 13 years ago | | |

The in-app purchases are perfect for this. You get to demo the brushes before you use them to figure out if you even need or like them. They're asking what amounts to what -- $10? for a beautiful app that looks and works worlds better than most in the same arena. I much prefer paying nothing up front and being able to try all of the features and figure out which ones I want to buy. I ended up getting them all simply to support the concept. You cite Notability but I don't want an app that does 20 things, with an ugly/more complex interface to boot. I have other apps that excel at those other things if I need them. I just want a drawing app, and I'm willing to pay a premium for one that offers the best experience for what I'm trying to accomplish with it.

I'm tired of paying for apps up-front only to find that they weren't what I was looking for or what was advertised. Good reviews don't necessarily mean that it is the right tool for the task, and forcing the user to then have to request a refund instead of just having a trial period or a free version they can simply delete is not user-friendly and results in all of the "DO NOT PAY FOR THIS APP!!!!1" and "I WANT A REFUND!" reviews from people who know none-the-wiser (although Apple doesn't make it very clear/easy to do so to begin with) and cause more people to turn away before giving it a shot. It also allows for the developer to be able to persuade initially unsatisfied users with later [demo] releases.

leephillips 13 years ago |

This quote from the article:

“Compare red nail polish to red ink: both are red, but the nail polish will be visible on black paper because it reflects light. The ink won’t be, because it absorbs light.”

doesn't seem right to me, but I'm not an expert. The ink doesn't reflect light? If you put a drop of red ink on a sheet of glass, it won't look red? Isn't the difference because the ink will soak in to the paper while the polish will sit on top of it?

maggit 13 years ago | |

The nail polish is opaque, which means it reflects or absorbs light that hits it. Red nail polish reflects red light and absorbs the rest. Shine white light on it, and it will look red.

Ink is transparent; It lets through or absorbs light that hits it. Red ink lets though red light and absorbs the rest. Shine white light through it, and it will look red.

Shining white light on a black piece of paper with red ink on it will look black. What light is not absorbed by the ink is absorbed instead by the paper.

Of course red ink reflects light, as you say, but that is just because of reality. It is not essential to its function. If it were possible to make a 100% non-reflective ink, it would work perfectly well.

I hope this clears it up :)

leephillips 13 years ago | | |

It does. Thanks to you and the other excellent replies, I now finally understand how ink works. Apparently it's a filter, and we see the light after it has been twice filtered by the ink, down through the ink into the paper and reflected from the paper back up through the ink to our eyes. In contrast to paint or polish, which reflects from its surface.

sp332 13 years ago | |

Ink mainly changes the color of light that passes through it. Nail polish absorbs and reflects light at its surface. If you look at a deep pool of red ink, it looks pretty close to black because it mostly absorbs light instead of reflecting it. The wording could be clearer though.

arscan 13 years ago | |

It won't look red if you put that sheet of glass over a black piece of paper.

Ink is semi-transparent. So think of white paper as a light source, and the red ink absorbs all colors but red coming off the paper before it reaches your eye.

Nail polish isn't transparent, but it still looks red because it simply reflects red light. That light never reaches the paper -- it bounces off the nail polish and comes back to your eye.

mitjak 13 years ago |

I'm curious as to why the article felt it necessary to highlight multiple times that the German scientists are "dead"…

dspillett 13 years ago | |

They're not dead. They're pinin' for the fjords.

achille 13 years ago |

The examples shown are neat and the screenshots shown are usable on your ipad:

http://dl.dropbox.com/u/6432940/paper/mixingWell/4/index.htm...

http://dl.dropbox.com/u/6432940/paper/mixingWell/22/index.ht...

http://dl.dropbox.com/u/6432940/paper/mixingWell/24/index.ht...

Interesting they use Processing.JS and CoffeeScript.

This is an interesting technical post, but why is it being hosted on fastcompany.com. This belongs on their blog.

charlesmchen 13 years ago | |

These prototypes were developed by designer Andrew Allen https://twitter.com/asallen working with prototyping engineer Amit Pitaru https://twitter.com/pitaru

If you'd like to discuss our approach to color blending, I can be reached at https://twitter.com/cmchen

achille 13 years ago | | |

Just curious, why'd you take down the prototypes? I'm assuming the actual app doesn't share any of that source.

aristus 13 years ago | |

Because FastCompany is consciously trying to publish more in-depth technical articles, in the manner of "Old Wired". I've chatted with the author of this article off an on for a couple of months now and I'm pretty happy to see that these articles are starting to hit publication.

0x0 13 years ago |

I wonder if CMYK would be appropriate? The article didn't mention anything about it (even though it talks about how they tried a bunch of other color spaces by name)

aw3c2 13 years ago | |

CMYK is an subtractive color model, it is not very understandable and definitely not intuitive for human color perception. HSL is much better for that. It all is not too revelant in the end result I think because those are just models and their software exposes certain aspects of colors.

sp332 13 years ago | | |

Mixing pigments (ink or paint) can be done in CMYK because it is a subtractive process. It would have been realistic and something that anyone who has played with finger paints or crayons has experienced. (Although I agree that it's not very relevant to the end result, it would have been nice to see it in the discussion.)

charlesmchen 13 years ago | |

We looked at a number of color blending algorithms that operate in the CMYK colorspace. None of them work well. Ultimately, colorspaces are designed to solve the problem of representing colors in isolation, not color blending.

duopixel 13 years ago | |

CMYK is more intuitive than RGB but still not quite there. It is difficult to grasp that mixing a saturated pinkish color (magenta) with yellow will produce red. On top of that, if you add blue (cyan) it will produce black!

Printer cartridges are highly calibrated for this kind of mixing, if you attempt to do it with acrylics or oils you will get a dirty dark grayish color. This is because different inks have different opacity and reflect light in different ways (as the German research points out).

namank 13 years ago |

I love that you discarded all solutions that didn't meet your criteria.

It's the only way to make great things.

aw3c2 13 years ago |

I find that hard to read, might be all the breathless adjectives? It does seem quite interesting though. They used some 80 year old paper on color blending/mixing. I wonder if they patented their derivative implementation.

kmfrk 13 years ago |

Another way to get an appreciation of the skill of the creators is to watch their amazing promotional videos.

https://vimeo.com/fiftythree/videos

t_hozumi 13 years ago |

This article really made me rethink importance of investing a great deal of energy in core problem. More importantly, we hardly notice that there is such a deep problem we should solve.

tammer 13 years ago |

Anyone know any desktop or web apps with similar functionality? I don't have any iThings and am bored with conventional color pickers.

duopixel 13 years ago | |

Color picker for data applies some of the principles mentioned in the article, it's more of a palette generator than a color picker put it produces very good results:

http://tristen.ca/hcl-picker/

archagon 13 years ago | |

Corel Painter has paint mixing, but I don't know how it compares to Paper's approach.

jcl 13 years ago | | |

ArtRage also has a paint layering/mixing model, although I'm guessing it's using neither Kubelka-Munk nor Paper's survey-based approach.

http://www.artrage.com/artrage-studio-features.html

julianwa 13 years ago | | |

Krita has (or had) an interesting implementation for paint mixing that uses Kubelka-Munk: http://www.youtube.com/watch?v=lyLPZDVdQiQ