I grew up in the ‘80s/’90s in a small ranch in a suburb of Boston with several nicely framed high-end reproductions of paintings from the Boston Museum of Fine Arts in my living room. Sure they were just replicas, but I loved them as they were a great way to bring some of the magic of my favorite museum into our home where we could live with them in the context of our everyday lives. They even had a texture to them to suggest brush strokes. But let’s face it - it was clearly not the same as seeing the actual paintings in the museum.
Fast forward 30 years and just down the street from the Boston MFA in Cambridge, a group researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) has just released a paper on a new approach for replicating paintings that they claim is 4x more accurate than existing models at recreating exact color shades.
Traditional printing that most of us are familiar with uses just four inks: cyan, magenta, yellow, and black (also known as CMYK). The team from MIT/CSAIL are using ten inks layered in a stack to achieve more accurate results. To do this, the team developed a deep learning model to identify the ideal mix of colors for the stack.
They also take advantage of the older technique of halftoning (employing spatial modulation - think dot patterns in Roy Lichtenstein paintings) and contoning (combining thin layers of inks) which improved accuracy and provides a smooth appearance.
One of the major advantages to the new technique is that the replica reacts to different lighting situations in a similar manner to the original painting. Previous techniques relied on “colorimetric color reproduction“ which would sample colors from a painting under a single lighting condition. But as the paper points out, this can lead to “metamerism, a well-known problem in color reproduction wherein a good reproduction is obtained under one light source, but not under another.”
According to Changil Kim, co-author of the paper and a postdoctoral fellow at MIT CSAIL.
If you just reproduce the color of a painting as it looks in the gallery, it might look different in your home. Our system works under any lighting condition, and shows a far greater color reproduction capability…
The team plans to make a data set publicly available which contains:
20,878 contone ink stack spectra and layouts, spectrally captured oil paintings, together with their optimized layouts using our ink library, and photographs of our printed reproductions under multiple illuminations.
The team at CSAIL believes the system could be used to protect originals from wear-and-tear in museums while also making it possible for people to view replicated versions in their own homes or multiple museums around the globe. According to mechanical engineer Mike Foshey:
The value of fine art has rapidly increased in recent years, so there's an increased tendency for it to be locked up in warehouses away from the public eye. We're building the technology to reverse this trend, and to create inexpensive and accurate reproductions that can be enjoyed by all.
As someone who thinks a lot about art and tech, I think it is good that we are advancing our technology for replicating works. But the romantic in me still wants to believe that there is something magical about the canvas that has been worked by the hand of the artist that will never be replicated.
Left : A photograph of one of the team’s printed sample patches. Each color square is 1 mm × 1 mm. Right: Their spectral acquisition setup.
While the early results are very promising, the new approach is still in its early development and has some significant limitations. At the moment, the time-intensive nature of 3D printing means that the reproductions are much smaller than the originals and it struggles with certain pigments like cobalt blue.
In the future they plan to expand this library, as well as create a painting-specific algorithm for selecting inks. They also can hope to achieve better detail to account for aspects like surface texture and reflection, so that they can achieve specific effects such as glossy and matte finishes. Given the current limitations we likely won’t need to worry about it being used for forgeries right now, but it is easy to imagine it becoming problematic in the future.