2020 Art Market Predictions

We are in the middle of a generational culture war, and the entire art market is about to be turned upside down by an emerging class of collectors looking to reshape the art world in their own image.

As we enter a new decade, my 2020 art market predictions will make the case that the art market will be unrecognizable ten years from now (in 2030). I build this case on three core beliefs that I will outline and argue in this article:

• Artist popularity is more important in driving the price of artworks than any qualities intrinsic to the art

• Millennials/Gen-Zers are culturally antithetical to Boomers

• Boomer wealth will change hands over the next two decades and Millenials/Gen-Zers will use it to invest in diversity in the arts

I've spent the last three months researching the literature on price prediction and valuation for the art market for an article I submitted to an academic journal. I won't re-write that article here, but I will share that I came away believing we are about to see a massive shift in which artists are propped up by the art market.  Here's why:

Art Prices are Largely a Popularity Contest

A Mark Rothko forgery that turned out to be by Pei-Shen Qian

In 2004, Domenico De Sole, former CEO of Gucci and current chairman of Sotheby's, paid $8.3M to the Knoedler & Co. gallery for a painting thought to be by Mark Rothko. The work was one of many painted by art forger Pei-Shen Qian and sold through the gallery to unsuspecting collectors. De Sole sued the Knoedler gallery in 2016 for $25M in damages. When asked if the knowledge that the painting was no longer by Mark Rothko changed its value, De Sole exclaimed, "I think so!" adding, "It's worthless."

Nothing physically changed about the painting — it had the same appearance and the same craftsmanship and quality. Yet it went from being worth millions of dollars to worthless when discovered to be by Pei-Shen Qian instead of Mark Rothko. This extreme devaluation of works discovered to be forgeries happens all the time. Why? Because the popularity of an artist is more important than any qualities having to do with the artwork itself in establishing its price. Mark Rothko is more popular than Pei-Shen Qian, so his work is more expensive.

We’d like to think an artist’s popularity and success is tied to their talent and skill. History shows that gender, skin color, and according to one recent study, where you were born and the level of access you have to a handful of prestigious institutions, are the most important factors in developing a successful career as an artist. Talent-wise, you might be the next Mark Rothko, but if you are not popular with the right group of people, you might as well be Pei-Shen Qian.  

This dependency on artist popularity means art prices are especially vulnerable to changes in values from one generation to the next. Millennials and Gen-Zers value systems are measurably at odds with Boomers and we should anticipate unprecedented market upheaval in the coming decades.

Millenials/Gen-Zers are Culturally Antithetical to Boomers

Climate activist Greta Thunberg staring down Donald Trump

If 7-Up is the Uncola, it might be helpful to think of Millenials/Gen-Zers as the "Unboomer." They broadly define themselves in opposition to the ideals of the Boomer generation. They've even popularized the catch phrase "OK Boomer" to efficiently dismiss a generation whose values they see as closed-minded and incompatible with their own. Likewise, Boomers frequently refer to Millennials/Gen-Zers as the "Snowflake Generation" for believing that they are each unique and special, seeking too much attention, and being over-emotional and sensitive. 

How big is the gap in values between generations, and where do they differ? Polls show children and grandchildren of Boomers are far more tolerant on critical issues like diversity and gender norms. They are also much more concerned about the climate crisis.

• 62% of Gen-Zers/Millennials believe increasing diversity is good for society vs. 48% (less than half) of Boomers

• 68% of Millennials prefer movies and TV shows with diverse casts vs. 32% of Boomers

• 70% of Millennials support gay and lesbian marriage vs. 39% of Boomers

• 56% of Millennials see a link between human activity and climate change vs. 45% of Boomers

How do these polls play out in real-life decision making? No generation is monolithic, but you can tell a lot about a group's values by who they elect as their leader and which artists benefit most from their patronage.

In 2016, more than half of the Boomers who voted in the United States voted for Donald Trump as their president. Trump's best-known mantras are "your fired" and "build a wall." Both are messages of exclusivity and subjugation designed to bully and humiliate those in less powerful positions. Trump's persona is a caricature of the 1980s playboy built on public displays of extravagance and wealth. His attitudes towards women inspired the largest single-day protest march in US history just one day after his inauguration. Trump's administration has not been friendly to LGBTQ rights. And finally, Trump is anti-science and does not agree with experts about climate change, calling it a Chinese hoax.

Obviously, Trump embodies some firmly held Boomer ideas about leadership, or else the majority of them wouldn't have elected him as their president. But is Trump an anomaly or a true reflection of Boomer values? Let's follow the money and look at which artists most benefited from Boomer patronage to see if there's a trend.

Jeff Koons and Damien Hirst have generally ranked near the top of the list for being the wealthiest living artists over the last several decades. Like Trump, both are white, male, and rich. They both made hundreds of millions of dollars by outsourcing the art-making process to craftspeople and then laid off most of those people in many cases with little cause or notice. Like Trump, both have built their personas around conspicuous consumption and ostentation.

Damien Hirst - For the Love of God, 2007

Hirst, who has said money is important as love and death, is known for creating a diamond-encrusted skull that fetched $100M in a private sale in 2007. Koons’ stainless steel sculpture of a cheap inflatable rabbit sold for $91M last year, setting a new auction record for a living artist.  

As with Trump, Koons and Hirst have mastered the art of using shock tactics to keep their names in the press and to stay visible and relevant. Koons by hiring adult film star Ilona Staller to have sex with him for a pornography shoot and then selling photo-realistic paintings and tacky sculptures of the sex acts as his art.

Jeff Koons, Dirty – Jeff On Top, 1991.

Hirst kept his name in the news by killing animals, cutting them in half, and floating them in large tanks of formaldehyde. In 2017, Artnet estimated nearly a million (913,450) animals had given their lives for Hirst's art.

Damien Hirst, The Prodigal Son, 1994

In the notoriously illiquid art market, it often takes a generation before an artwork is resold. Will Millenials and Gen-Zers who grew up in the #metoo movement and losing sleep over climate change be interested in the artists that were propped up by Boomer wealth?

Millennials/Gen-Zers Will Shape the Art Market in Their Own Image

Data from the Pew Research Center

In the next 25 years an estimated forty-five million US households will pass down an unprecedented $68 trillion to their children and grandchildren, according to a report from Cerulli Associates.

With this wealth transfer will come a new generation of freshly minted collectors eager to build an art market in their image. As the most diverse generation in the history of the United States, we should anticipate a greater variety in the gender and race of the artists who will benefit at the highest levels from their patronage. We should also expect that this younger generation of collectors will be much more concerned about climate change and ecological instability. And lastly, Millenials/Gen-Zers are already far more likely than Boomers to buy online. 93% of high net worth Millennials had purchased art online vs. less than half of high net worth Boomers, according to a 2019 UBS report.

We are already seeing some progress for diversity in the art market. In the last six years, art by women increased in value by 72.9% vs. 8% for art by male artists as tracked by Sotheby's Mei Moses index. But as the stats below indicate, we are nowhere near parity.

• Only 2% of auction sales between 2008 and 2018 were by women

• Just 11% of all work acquired by US museums in the last decade was by women

• Just 14% of museum exhibitions in US featured female artists

The art world's track record for supporting artists of color is equally poor.  Again, the stats show we are nowhere near parity:

• Just 2.3% of all acquisitions by museums were of art by African-American artists from 2008 - 2018

• Only 7.7% of exhibitions at 30 prominent US museums were by African-American artists from 2008 - 2018

• Just 1.2% of art in American museums is by African-Americans

• Just 2.8% of art in American museums is by Latino Artists

Museums are racing to atone for past discretions, and many are literally de-accessioning paintings by well-known white male artists to fund the acquisition of works by women and artists of color. We should expect similar corrections to private collections and in the art market.

Which Artists Will Benefit Most From a Generation Eager to Highlight Diversity?

Jean Michel Basquiat - Untitled, 1982

The $63.7B-a-year art market is notoriously top heavy, with just 1% of artists accounting for 64% of sales value. And the vast majority of that 1% have two things in common: They are white and they are male. This should change in the next two decades.

Expect several female artists and artists of color to break into the highest echelon of the market, joining the likes of Van Gogh, Monet, Picasso, Rothko, Warhol, etc. We have already seen Jean Michel Basquiat break through the ceiling with his Untitled, selling for $110M in 2017. I believe artist Alma Woodsey Thomas also has a good shot at breaking into the 1% Club.  

Alma Thomas at the opening for her show at the Whitney

Thomas is an incredibly strong painter. Find a museum showing her work and see for yourself how her paintings hold up against all the other ab-ex and color-field titans of the 20th century. From a talent perspective, Thomas is clearly one of the most important colorists of all time, but we know talent is only part of the story — and not even the largest part.

Alma Thomas, A Fantastic Sunset, 1970

Since we know the price of art often has more to do with popularity of the artist than the quality of the artwork, it is essential to look at Thomas' bona fides, which are nothing short of remarkable — especially given the all of the discrimination she faced.

• First African- American women to graduate with an art degree

• Earned a Master of Arts degree from Columbia University in 1934

• First African- American woman to receive a solo exhibition at the Whitney Museum (1974)

• Resounding reviews in The New York Times in an era of prejudice and bias

• Selected by the Obamas for display in the dining room of the White House

Alma Thomas’ Ressurection hanging in the White House dining room.

Thomas' market is already a rocketship. In 2017, a new record was set for her work with the sale of Spring Flowers for $387,500. That record fell again in 2019 when A Fantastic Sunset sold for $2,655,000. The ceiling on her work effectively increased 580% in just two years, and there is still plenty of room to grow. We’ve known that Thomas is among our greatest painters for a very long time. We just needed to wait for a generation excited to embrace and celebrate artists of all colors and genders to come along before we could give her the proper recognition she deserves.

Not all the artists whose success was limited due to discrimination will break into the 1% Club. But we will see the fingerprints of Millennials/Gen-Zers correcting for centuries of bias across all levels, not just the top of the market. For example, important works by Agnes Denes and Vera Molnar, pioneers in environmental art and generative art, respectively, can still be collected for under $10K. I expect the market for both these category-defining artists to heat up quite a bit over the next ten to twenty years.

Denes, who currently has a retrospective at The Shed in NYC, is positioned to break out into the mainstream and establish her legacy as the leading figure in environmental and ecological art.

Agnes Denes, Wheatfield—A Confrontation, 1982

Denes is best known for her work Wheatfield—A Confrontation, which featured two acres of wheat planted and harvested by the artist on the Battery Park landfill in Manhattan during the Summer of 1982. Denes explains that Wheatfield "referred to mismanagement, waste, world hunger, and ecological concerns. It called attention to our misplaced priorities." Wheatfield, with its dual focus on the environment and economic inequality, fits perfectly into the post-Boomer zeitgeist.

Among Denes’ many significant accomplishments:

• Created the first site-specific performance piece with ecological concerns

• Founding member of A.I.R. Gallery, the first all-female artists’ cooperative gallery in the United States

• Created the first human-made virgin forest in her 1992-1996 reclamation project Tree Mountain, a legally protected site for the next 400 years.

The grand scale and conceptual nature of Denes’ works does make them challenging to collect. However, she also created many “philosophical drawings” and prints, exploring isometric systems and complex map projections. These drawings can be found in the MoMA and other important collections and periodically come up at auction for very reasonable prices.

Agnes Denes, Map projection: the snail, 1976

Denes' work and views on the environment will only feel more prescient and relevant as time moves on. As the next generation of collectors scour history to find artists who gave us early warnings that we needed to take better care of our planet and the people who inhabit it, Denes' name will regularly show up at the top of the list of those few who did.

In addition to their concerns over climate change, Millenials/Gen-Zers are also the most technologically fluent generation we have seen. Given their interest and comfort with tech, I believe that the market for generative art — art generally created in code using autonomous, automated systems -- will explode in the next two decades. As one of the only early generative art pioneers to come to computers in the 1960s with decades of experience as a working artist, I believe Vera Molnar is perhaps the most important generative artist of all time.

Vera Molnar

At the age of 96, Vera Molnar has now created cutting-edge art that explore algorithms with and without computers across eight decades. Molnar helped bridge the gap between computers, which were thought to be cold and antagonistic to human creativity, and traditional artistic practice. She once said that “ultimately the intuition of an artist is the ‘random walk’ of the computer.” Her work paved the way for a generation of influential generative artists and will continue to indefinitely into the future.

Even the art world, slow to embrace women and slower to embrace tech, is finally starting to take notice of Molnar’s work and achievements:

• Art in America featured Molnar in its January, 2020, edition focusing on generative art

• Artnet listed Molnar as one of nine artists who will break out in 2020

• At 96 years old, she is still working and actively exhibiting work around the world

Vera Molnar - (Dés)Ordres, 1974

If I had tens of thousands of dollars to spend on art, all of it would be going to build a collection of important works by Vera Molnar. However, my meager art budget requires that I look for work that costs a few hundred dollars at most, not tens of millions or even tens of thousands. In many ways, I prefer this. I see collecting as an act that can be just as creative as making art or curating an exhibition. If I were spending tens or hundreds of thousands of dollars, I’d be thinking in terms of a financial investment instead of just looking for art I truly love by artists I want to support and see succeed.

Osinachi - Can’t Sleep, 2019

Lately I have been buying as much work by artist Prince Jacon Osinachi Igwe as I can get my hands on. For me, Osinachi reflects the best of what the coming generation of artists have to offer.

Osinachi’s work stands out in part because he is self-taught and creates his work using Microsoft Word, a common word processing tool few would ever think to use for making art. His signature use of color and pattern create sophisticated compositions that are dynamic, but flat like a collage, unusual for artists working digitally. His work is bold, distinct, authentic, sincere, and addresses his generation’s desire for equality, diversity, and environmentalism in a way that is direct without being too on the nose.

Osinachi signing a print of Nduka's Wedding Day, 2019

Osinachi is an intellectual. His writing is as thoughtful and candid as his art. His 2015 story A Man and His Breasts is a sensitive portrayal of a boy growing up in Nigeria with gynecomastia. It gets to the same core elements present in his visual art — the pain and suffering all of us feel at times for simply living and being who we are in a world full of rules and systems designed to make us feel shame and fear. Or, as Osinachi describes it, “visible existence as protest.”

Osinachi lives and works in Nigeria, a country which has some of the most brutal anti-LGBTQ laws in the world. By law, members of the LGBTQ community can be whipped or face up to 14 years in jail for showing public affection to same-sex partners. The laws have become so strict that simply being accused of having a “gay” hairstyle or clothing can get you arrested.

Osinachi - Nduka's Wedding Day, 2019

In this environment, Osinachi bravely celebrates people from the LGBTQ community in his work, showing them as everyday people flourishing in their lives. This can be seen in his wonderful work Becoming Sochukwuma, inspired by a 2014 essay I Will Call Him Sochukwuma: Nigeria’s Anti-Gay Problem, written by Chimamanda Adichie in reaction to Nigeria's anti-gay law. It can also be seen in the celebratory Nduka's Wedding Day featuring a male bride on his wedding day holding a bouquet.

Prince Osinachi - Becoming Sochukwuma, 2019

Osinachi’s work covers many other topics challenging stereotypes around single motherhood, highlighting the need for diversity, and protecting the environment. And some of his work is just plain fun.

Osinachi - NWANYỊ AHỤ NA-AKWA AKWA, 2019

In a world where artists resort to making porn or sawing animals in half to get attention, Osinachi is bravely fighting for people to be able to live as they are. It’s a welcome use of art and its potential to positively impact society.

Measurable Predictions for 2030

OK, not going to let myself finish without making some bold and measurable art market predictions for you to hold me accountable for. Based on the extreme shift in values and the ensuing wealth transfer, I predict that by 2030:

• Works by five non-white and or non-male artists will sell for more than $50M each

• Women move from 2% of sales at auction to at least 10%

• Women shift from 11% of all work acquired for permanent collections to at least 25%

• Women will go from being featured in just 14% of exhibitions to 30%

• Acquisitions of art by African-Americans by major US museums will go from 2.3% to 10%

• Art in US museums will go from 2.5% by Latino Artists to 5%

In summary, art’s price is built around popularity. What was popular with the Boomers is mostly unpopular with Millenials and Gen-Zers. Specifically, these younger generations are more interested in diversity, more concerned with climate change, and more comfortable with technology. We should expect that artists propped up by Boomer wealth may have a hard time finding buyers in ten or twenty years — specifically, artists who sexualized or objectified women or treated animals or the environment poorly. By contrast, many talented female artists and artists of color will see their work increase in value -- not because of their gender or skin color, but because they have always been great and only needed a more open-minded generation to be fully recognized for their talents.

Artnome - 2019 Year in Review

What a year for Artnome! An art exhibition in China, two more in Switzerland, an invitation from the Princess of Bahrain to moderate panels on AI, and presentations at both Christie’s and Sotheby’s.

We also had 28 blog posts, including guest posts by artists Casey Reas and David Young, as well as an interview with John Maeda on his new book How to Speak Machine. We formed two business partnerships, both with companies founded by strong female entrepreneurs. The first partnership is with Kate Vass Galerie to promote up-and-coming digital artists and the second to fight art forgery using machine learning with Art Recognition.

2019 was Artnome’s best year by far, but 2020 is already giving it a run for its money. We are thrilled to share that we have officially kicked off the year by writing the cover story for the January edition of Art in America!

Art in America, January 2020. Photo and cover art by Sougwan Chung

How did all this happen? I keep waiting for things to slow down so I can document it and make sense of it all, but the time never comes. So this past week I woke up at 4:00 a.m. instead of my usual 5:00 a.m. to find the time to write it all down to share with you, because without you, there is no Artnome!

Part I - Quit My Job With No Plan And Fly to Bahrain

McDonalds in the Manama Airport, Bahrain

In January of 2019, after four incredible years at Tamr, one of the hottest machine learning startups in Boston, I decided it was time to take a leap and pursue Artnome more aggressively. Leaving a fantastic job with a great team and with no idea how I would make money felt crazy. It was like jumping out of a moving car into the middle of traffic just to see what it feels like to get run over. To make things more interesting, I had just been diagnosed with Lyme disease for a second time right before striking out on my own and my legs all but stopped working with both knees swollen to the size of cantaloupes.

I was scared to death, but I wasted no time in taking advantage of my newfound freedom. I received an email inviting me to fly to Manama as a guest of the Princess of Bahrain to moderate several panels on AI and art. This invitation was exactly the type of out-of-the-blue opportunity I had been turning down in the past to hold down my nine-to-five office job. But along with the anxiety of having no job came the flexibility to jump on these opportunities.

Map on the back of the seat in front of me on the flight to Bahrain

I'd never traveled to the Middle East and didn't know what to expect. I received a lot of well-intended but misguided warnings from my fellow Americans. They told me that Bahrain was dangerous and that I needed to be careful. I should dress modestly and pack only long-sleeved shirts and pants as not to show any excess skin. I'm a beer guy, and more than a few people warned me that there would be no beer and that I'd get arrested if I drank any alcohol. I received bad info.

An hour after landing in Manama, I was partying at a bar on the balcony of a five-star hotel. It was a lax atmosphere, and nobody was dressed modestly except for me. I met a Bahraini that night at the bar who had just been to my home state Massachusetts to take his daughter to Boston Children's Hospital for a rare medical issue. It reminded me that we are all brothers and sisters just trying to do the best for our families. The world is a small place.

Bahrain was beautiful, and its people were warm and immediately welcoming. My hosts could not have been more accommodating and arranged excursions to museums and cultural sites, as well as fantastic and memorable meals. The best part was bonding not only with the locals, but the leading thinkers in art in tech whom they flew in from around the world. I built friendships there that have already changed the course of my life.

Left to right: Mario Klingemann, Robbie Barat, Aparajita Jain

The first panel I moderated in Bahrain included two of the most important artists working with machine learning today: Mario Klingemann and Robbie Barrat.

Mario Klingemann had just sold his work Memories of Passersby at Sotheby’s in London before flying to Bahrain. Klingeman has long been at the forefront of exploring generative art, and his dark and challenging works are finally getting some of the recognition they deserve. In addition to selling his work at Sotheby’s, Klingemann’s work The Butcher’s Son won the Lumin Prize in 2018.

American AI artist Robbie Barrat had just come off of a successful show in Paris collaborating with French painter Ronan Barrot. Barrat, a generative art prodigy, made headlines when it had became clear that the French artist collective Obvious sold a work at Christie’s for $450,00 that was largely based on Barrat’s work and code. Undeterred, Barrat continues to innovate at a rapid pace, regularly producing new and cutting-edge work.

Also participating on the artistic creation panel was the visionary Nature Morte Gallery director and co-owner, Aparajita Jain. Jain, who curated one of the first AI art shows (Gradient Descent) in her gallery in New Delhi, shared her thoughts on the new challenges and opportunities that AI presents to traditional art galleries and the art market. Rounding out the panel was Attorney Ed Klaris, CEO of KlarisIP & Klaris Law. Klaris delved into the many concerns and considerations around copyright and IP that are associated with this new medium.

Left to right: Jason Bailey, Cristobal Valenzuela, Andrew Shum, Zike Wu, Robert Norton

The second panel featured an equally impressive group of panelists, this time focused on AI and art solutions. It kicked off with a demo of Cristóbal Valenzuela’s machine learning tool for artists and designers, RunwayML. Runway ML puts the same tools used by artists like Mario Klingemann and Robbie Barrat into the hands of less technical artists and designers, almost like a machine learning version of Adobe Photoshop. I was also thrilled to have Andrew Shum, CEO of Thread Genius, on the panel. Shum shared some of the strategy and thinking behind his Spotify-like approach to recommending artworks to Sotheby’s collector base. Co-founder of ArtVenture Zike Wu and CEO of Verisart Robert Norton rounded out the panel and added their expertise to the engaging conversation.

Part 2 - “Automat und Mensch”: How I Became an International Art Curator

Left to right: Robbie Barrat, Herbert W. Franke, Mario Klingemann, Kate Vass, Jason Bailey, Georg Bak

I have long dreamed of attending an art exhibition that presented the full range of generative art. Ideally it would start with early analog works of the late 1950s and range all the way up to the new work being done with machine learning in the last few years. To my knowledge, no such show has ever been curated. Just to attend such a show would be a dream come true for me.

So when the Kate Vass Galerie proposed that I co-curate a show on the history of generative art, I thought I had died and gone to heaven. While I love early generative art, especially artists like Vera Molnar and Frieder Nake, my passion is really centered around contemporary generative art. So pairing up with my good friend Georg Bak, expert in early generative photography, was the perfect match. Georg brings an unmatched passion and detailed understanding of early generative art that firmly plants this show in a deep and rich tradition that many have yet to learn about.

Herbert W. Franke and Jason Bailey

I'm incredibly proud of Automat und Mensch. I believe it is one of the most comprehensive surveys of generative art ever held.

The show featured over thirty artists, ranging from the earliest pioneers to the most exciting artists working today. To give you an idea of the range in ages, we had both 19-year-old generative art prodigy Robbie Barrat and 91-year-old generative photography pioneer Herbert W. Franke at the opening. Thirty is an enormous amount of artists for a relatively small gallery, but we were able to pull it off because of the tireless work of both Kate Vass and her assistant Alessia Realis. They handled all the logistics and hosted a fantastic opening that was well attended and will not soon be forgotten. I couldn’t have asked for a better team.

Seedorf Switzerland. wins the award for the most beautiful/peaceful place I traveled in 2019

I snuck in a short trip with my wife through the Swiss countryside after the opening of Automat und Mensch, and we absolutely fell in love with it. The photo above is the view from our dinner table in Seedorf. I would love to find a way to get back there at some point.

If you are interested in more details about the Automat un Mensch exhibition, you can learn more in this Artnome post and see all the works that were included in the show on the Kate Vass Galerie website.

Part 3 - Four trips to NYC in One Month Followed by Christie’s Art & Tech Summit

CADAF NYC - Left to right: Georg Bak, Jonathan Watkinson, David Young, Shirin Anlen, Kevin Abosch

I'm not great at planning. I'm even worse at saying no. So somehow, I ended up agreeing to four speaking engagements in New York City in May — the problem being that I live four hours away in Massachusetts. I could not have predicted at the time that I would also have a tube sticking out of my arm for most of that month so that I could take intravenous antibiotics for my Lyme disease (though this gave me the use of my legs back for the most part).

Moniker Art Fair - From left to right: Jason Bailey, Steven Schindler, Katherine Wilson-Milne

My engagements for May included:

• 05/01/19 - Moniker Art Fair, Technologizing Fine Art

• 05/05/19 - CADAF NYC, Art and AI

• 05/10/19 - Ethereal, Innovation and Blockchain Art

• 05/18/19 - Rare Digital Art Fair, Rare AF 2, Return of the Rare

All of the speaking engagements for May went great, and they could not have been more different from each other. First, I presented with two lawyers next to a furnace in a basement at a street art fair. Then I moderated a panel in a room with projections from floor to ceiling on all four walls. On the next panel I hosted, they told us we had half as much time as we expected as we walked on stage, so I moderated in fast-forward. Finally, there was the keynote at Rare AF, which is always like a homecoming with so many very close friends there. I even got to carpool in to NYC for Rare AF with Joe Looney, the godfather of cryptoart. It was a welcome break after three solo trips to NYC and back in one month!

Art+Tech Summit at Christie’s: The A.I. Revolution

Jason Bailey and Robbie Barrat

I've been lucky enough to speak at Christie's Art+Tech Summit two years in a row now. Both times the events ran flawlessly, featured terrific presenters, and were excellent for networking. Both years, Artnome advisor Anne Bracegirdle (now at PaceX) played a central role in organizing the events and was kind enough to include me.

For the 2019 summit, they gave me the freedom to chose my own topic. I decided to present on my passion for fighting forgery and improving the quality and quantity of data on art.

I was also given the opportunity to interview artist and good friend Robbie Barrat at the summit. Christie’s felt bad about overlooking Barrat’s work when they promoted work that relied heavily on his ideas and code. This gave him an opportunity to share his work and his thoughts. Robbie is incredibly well spoken as an artist and many people shared with me that our interview was the highlight of the summit.

The Art+Tech Summit has become one of the most important art and tech events of the year, and I hope Christie’s continues to host it. I'd happily present there again.

Part 4 - Taking a Break and Recharging

Lake St. George, Liberty Maine

2019 was terrific, but my schedule was and is still crazy. In addition to all of my Artnome commitments, I started a tech marketing consultancy to keep the lights on at home. To my surprise, it quickly flourished and I had more work than I knew what to do with. The reason I can accomplish as much as I do is because I have an incredibly understanding and supportive wife. Erin makes tremendous sacrifices so that I can pursue my crazy dreams with Artnome.

My girls wondering when I am going to stop working and do something fun.

But by midsummer, it was catching up with me, and I needed to get into the woods. I decided to give myself a few months off from speaking engagements to recover a little and to focus on writing and my growing consulting practice. I was able to spend some time up in Maine working from my "mobile office," a hammock slung between two trees just feet from Lake St. George. In between floating in the lake and resting in the hammock, my family spoiled me by making me delicious meals.

Mobile office, Lake St. George, Liberty Maine

I also used this time to plan my next moves for Artnome. I am addicted to finding new and interesting digital art, but sometimes months go by where all the work I see starts to look the same. I was in that kind of a slump when I discovered artist Espen Kluge and his phenomenal generative portraits. I immediately fell in love, as did Kate, and we reached out to Espen for an interview and to offer him a solo show in November.

I had also been speaking on and off with my close graduate school friend Fan Xiang who teaches design at Parsons in NYC and at Tsinghua University in Beijing. Fan had been following Artnome and wanted to arrange a speaking tour for me in China along with an exhibition of generative art. She ended up getting too busy, but introduced me to Professor CJ Chen from Nanjing University. The two of us become best friends right away. We decided to do an exhibition in November.

So coming out of my midsummer break in the woods, I had committed to two more exhibitions: one in Switzerland and one in China. I also accepted a speaking engagement at Sotheby’s Institute to address the incoming class on the topic of art and tech. I agreed to write an article for Art in America. I then agreed to moderate a panel on machine learning at Art Basel Miami. The second half of my year was looking even busier than the first half.

Part 5 - Back-to-Back Shows in Switzerland and China

Left to right: Kate Vass, Jason Bailey, Espen Kluge

I think of Switzerland as my home away from home now. I was excited to come back in late October for the second time in a year to see my friends Kate and Alessia. I knew from my interview that Espen Kluge and I would quickly become brothers. It was just a three-day stay, but I enjoyed every minute of it and spent a great deal of time getting to know Espen and his equally creative and amazing wife Ida.

Espen Kluge, ash when he arrives, every time - 2019

The opening was a hit. People really responded to Espen’s powerful portraits at the show. It was also a good time for Kate and I to connect in person and plan out 2020. It was a short stay, but a very pleasant one. From there it was back on a plane to fly to Nanjing for the second opening in less than one week.

Augmenting Creativity - Nanjing University

Augmenting Creativity - Nanjing China. Left to right: Jared Tarbel Substrate - 2003, Robbie Barret Balenciaga -2018, Sougwan Chung The Limitless, The Absolute -2017

As with Bahrain, I was given all kinds of bad advice about China from people who have never been there. I was told to watch out because they would try to steal my laptop or my wallet. It turns out that Nanjing is a bustling, densely populated city, but there is almost zero crime. I offered to walk the students home when we were out late one night for dinner. They laughed and explained that the city is so safe that there was no need to worry.

Left to right: Machi, Moon, Han, CJ

The students treated me like a rock star from the moment I got off the plane. I was even given the nickname “Treasure,” which was later updated to “Sparkling Goat Goat Treasure.” But it was CJ who was the real rock star and her students who are the real treasures. They did all the heavy lifting to organize and set up the show, and it came out phenomenally. People came from all around China for the opening, and we even made the Chinese national news.

Left to right: Han, Moon, CJ, Manchi

The show was extended to the Nanjing University of the Arts, and one of CJ’s former students, Manchi Li, did a wonderful job walking through the exhibition in the video below. You can also learn more about the artists who were included and read the exhibition essay in this post.

I spent almost ten days in China and had the great honor of presenting in Shanghai at China’s largest library on the Artnome database. There, as everywhere else I went in China, I was treated with respect and gratitude and asked many deep and thoughtful questions about my work and research in art and tech. The head of the library took CJ and I, as well as our new friends from Europe who were also lecturing at the university, out to one of the nicest restaurants in Shanghai.

The whole trip was amazing. If I can find a way, I’d like to move to Nanjing for a few years at some point to learn, teach, and lecture. I connected with people in China in a way I typically don’t in the US. I found everyone to be incredibly thoughtful, kind, and generous. I miss the professors and students I got to spend time with, and I think about them often and the unparalleled hospitality they showed me. They are family now.

Artnome 2020

It is shaping up to be an another exciting year! In addition to the Art in America feature, I just recently completed a second commissioned piece for a publication (to be named later) that I really respect! I’ve already signed up to curate two shows: one in Montreal in March and one likely in Berlin in July (details forthcoming). I’ll be at SXSW (South by Southwest) in Austin in March with some amazing panelists to share how digital art can save the art market. It also looks like I will likely be presenting on new research I have been doing on art price prediction in NYC in the fall.

Thank you for all the support, and hopefully you found my recap of 2019 enjoyable! I get asked sometimes how all of these opportunities open up for me, and almost all of it starts with this blog. If you are passionate about something, I’d encourage you to write about it. You may not think you have time, but wake up an hour earlier each morning and you’d be surprised how much you can write. Don’t expect amazing things to happen right away — it takes the better part of a year before anyone will notice — but once they do, the world opens up to you and becomes your extended family. Cheers and best of luck in 2020 to you all!

Drawing In Code: Pencils, Pixels, Process

When I graduated with my MFA from the Dynamic Media Institute at Massachusetts College of Art and Design in 2010, we were required to design a printed book outlining our projects and our thesis. I thought I had lost the files for my thesis but recently checked with Blurb, where I had it published over a decade ago and was excited to find they still had a copy.

Some of the ideas in my thesis are a bit cringeworthy ten years on, but it is still the most singular representation of who am I am and how and why I think the way I do as a creative. A lot of Artnome readers know me as a writer, but my thesis focuses on my most active years as a designer and generative artist (2000-2010). It also references the many years I spent before graduate school focused on traditional studio art like painting, printmaking, and sculpture (1990-2000). While it is not a short thesis, I made sure to write it in my voice and to avoid traditional academic-speak. It is more autobiographical than academic, and If you are interested in the creative process and the digital transformation artists and designers have gone through in the last several decades, you might find it a worthwhile read. 

I had told myself I'd publish my thesis on Artnome if I ever had a month where I was too busy to write a new article. This month I have been lucky enough to be writing commissioned pieces for two well-known publications I respect and admire (which I'm psyched to share soon) and did not get a chance to publish anything fresh to Artnome. Not wanting to break an almost three-year streak, I decided it was time to share my master's thesis, ten years on from when I originally finished it. Enjoy!

Drawing in Code: Pencils, Pixels, Process

“We must expect great innovations to transform the entire technique of the arts, thereby affecting artistic invention itself and perhaps even bringing about an amazing change in our very notion of art.”

Paul Valéry, Pièces sur L’Art, 1931

Introduction

I have spent my whole life using drawing as a means of understanding and clarifying my environment, synthesizing that understanding, and sharing it with others.

Sometimes drawing has meant trying to accurately recreate what I see around me, such as creating a still life of some interesting objects on a table or a portrait of a family member. This experience provided me an opportunity to examine my world in much closer detail than I otherwise do in daily life.

In my day job drawing has also meant taking highly technical information, like the complex physics involved in a car accident, and boiling it down into an easily understood graphic so that others might understand it better; or illustrating statistical data, like the variable temperature in a data center, and turning an overwhelming list of numbers into a clear visual story.

Drawing has also been about journey and discovery for me, often starting without clearly defined objectives and letting the process unfold and dictate the outcome and not always drawing the reality that is in front of me but creating a new reality. Paul Klee described this process elegantly when he stated: "Art does not reproduce the visible; rather, it makes visible. " (Chipp, 182) It is this last aspect of drawing that I have always enjoyed the most.

This mode of drawing allows me to find questions that I wouldn’t otherwise think to ask. The process has always delivered me better ideas than simply brainstorming or even writing.

Throughout my life, I have changed the tools I use for drawing, starting at a young age with markers, crayons, and pencil, eventually learning to paint with oils and watercolors. Painting is a process that I still consider drawing, only with a brush. In my undergraduate studies, I learned to make linoleum prints and then intaglio prints (etchings, drypoint, etc.). And then it happened... about eight years ago I started using the computer as a drawing tool. Now I use it almost exclusively. It wasn't until the last year of my graduate program that I asked myself, "Why?"

At first, I thought I used computers to draw because that was the only way I could make a career of doing what I have always loved most: drawing. There was a good argument for this, as I don't know anyone personally who lives off of drawing without using a computer (with the exception of a family friend who is employed full-time as a political cartoonist). There had to be more to this permanent switch in tools, so I asked myself what computers are good at and came up with a list of six key things.

I decided computers are good at illustrating space and time, facilitating interaction by managing input and output, storing and presenting data, running and storing complex algorithms, and distributing information to a large audience at little or no cost.

When I thought about this list relative to the history of drawing, I made a dramatic realization that these issues have long been of primary interest to artists using analog drawing tools. The computer was not just a fancy calculator for crunching numbers, but also a natural extension of the history of drawing.

For example, Marcel Duchamp's painting Nude Descending a Staircase explores several themes from my list. In this painting, Duchamp is playing with the Cubist algorithms for the deconstruction of space, reducing the figure into simplified geometry and displaying it from several angles. He goes beyond space and tries to represent time by showing the figure in multiple positions on the same static 2D canvas, which also suggests motion. Unlike more representational paintings, Duchamp's painting required the user to interpret or decode the visual system of the painting to see the movement of the figure and better understand the complex space it occupies.

Duchamp was also one of the first artists to draw attention to distribution in an increasingly mass-produced society. His ready-made sculptures, like Fountain, an everyday urinal, challenged the viewer to question the value placed on art as craftsmanship and one-of-a-kind artifacts by executing the conceptual act of placing a machined, mass-produced object in a gallery.

The more time I spent thinking back to art history and the history of my drawing process, the more confident I became that the affordances of the computer were a match for the historical objectives of drawing.

Using the computer, I am able to build and display models of 3D space on a 2D plane, generate drawings that change over time, turn data into drawings, write algorithms to generate near-infinite visual iterations of my work, distribute my work to a larger audience, and allow for greater interaction from that audience.

The computer also provides me a means for pushing drawing in new directions beyond its analog origins. When digitized, all media share a common language of ones and zeros, allowing for uniquely digital, cross-pollination of media. For example, I can now make drawings with data taken from sound or compose music with data from video. My thesis is of a series of stories describing the process behind several of my drawing projects, both digital and analog. I have tried to capture the discoveries I made about the computer, how I use it as a drawing tool, and how it has changed the way I work as an artist.

What I learned from my projects is that it is best to involve people at every stage of development and that taking this approach is not only more effective but more fun. This was counter to my instincts of going off and developing things in isolation based on theories of interactions. Sharing the process was more important than sharing the results. For this reason, I decided to write primarily about my process in my thesis. I wanted to tell stories, stories of successes, and stories of failures, in the hope that others may learn from my experiences as I have from theirs.

Space

Study #2, Processing - Jason Bailey (2009)

I grew up Mormon and spent a lot of time drawing during the long Sunday services. I remember my dad being about as disinterested in the service as I was. The upside to this was that he would show me drawing tricks to keep me busy.

On one such occasion, he drew a square and then a second square diagonally offset from the first square. He connected the corners, and something magical happened. The two squares became a cube that seemed to jump off the page. Seeing how impressed I was, he followed the first trick by connecting two triangles and then two circles. Each time he made a new shape, I was more impressed. For a kid who spent as much time drawing flat airplanes, flat horses, and flat ninjas as I did, this was a pretty amazing trick. Not only was it possible to describe 3D space on a 2D plane, but it also appeared to be incredibly simple using the tricks he showed me.

As with most people learning to draw, I spent a lot of time trying to figure out the perspective and how to properly recreate the three-dimensional world around me on a two-dimensional plane. As an undergraduate student, I spent hundreds of hours drawing. This was when I realized that drawing complex shapes with proper perspective was a time-consuming and challenging task. Even drawing something as simple as an empty dorm room with a couple of shelves, some books, and a few bricks of Ramen noodles could take several hours to get right. Organic shapes like fruits or faces were even more difficult.

Learning to use 3D graphics programs on the computer restored the magic to the illusion of creating three-dimensional space on a two-dimensional plane.  This is what I had seen when my father showed me how to make that first cube. With the computer, I could simply push and pull surfaces in any direction I wanted, and the proper perspective would be calculated by the computer and applied to the scene. The idea that I can create a single model and show it from any angle without restarting the drawing process from scratch remains intriguing to me.

3D Study #1, Processing, Jason Bailey (2009)

3D Brush Tool

As the colorful and repeating sequence of rotating 3D shapes followed the cursor I was controlling with my mouse across my computer screen, I thought to myself, is this just more eye candy, or is there something here?

From over my shoulder, I heard my classmate Simon let out a coo of joy, followed by a giggle. We had both signed up to take a summer course in the programming language Processing with guest lecturer Peter Kirn, a guru media programmer from New York City. Our second class was about to start. I had spent the first week exploring the 3D capabilities of the Processing programming language. Several classmates and professors told me that Processing had many advantages over Flash when it came to working with 3D, including access to the graphics card, which would allow for more complex shapes as well as a simplified way to generate basic shapes like spheres and cubes. Hearing Simon's positive response to the drawing program I wrote and seeing that he immediately wanted to try it reassured me that there was some potential for an engaging project here.

Study #4, Processing - Jason Bailey (2009)

I had some light experience in programming languages and a background in 3D graphics, but had tried and failed several times to marry the two. Knowing that I would want more than just the simple boxes and spheres that Processing provided by default, I looked into methods for bringing more complex models from Maya into Processing for further manipulation and interaction. After finding a library of code to support the transfer of files from Maya to Processing, I began experimenting with building and animating the models first in Maya, and then manipulating them in Processing. Initially, I created models of real-life objects, such as a table, lamp, etc. I wanted to make an interactive space, populated with realistic 3D models, similar to those in video games. Pretty quickly, I realized that even Processing would have difficulty rendering lifelike objects with textures and lighting in real-time. I decided instead to experiment with animating some abstract geometric shapes. I set up a processing sketch that would rotate my animated 3D meshes as the user dragged them around the screen. The resulting images looked pretty unique. It felt like a novel direction to move in.

I was excited about the 3D brush project but equally excited about the next several projects in the Processing class, so I put the 3D brush project on the back burner. Knowing that I wanted to revisit the 3D brush project, I signed up for an independent study in the fall of 2009. I had been looking at the algorithmic work of other designers, such as the work of Jared Tarbell. Tarbell's programming and design process resonated with my process. In describing the creation of his algorithmic drawing, Substrate, he emphasizes the experimentation and discovery involved in programming drawings: "Before writing the program, I only had a vague idea of what it might look like. It wasn't until the first couple of bug-free executions that I realized something incredible was happening. The resulting form was much more complex than the originating algorithm. This particular quality of software is what keeps me interested." (Fry, 157)

Tarbell's programmed sketches were intriguing to me for the painterly effects he achieved with them. Although generated entirely from code, Tarbell's images had a feeling of human gesture that was less present or not present at all in the work of other designers creating algorithm-driven artwork. I started thinking about designing a tool that would balance out the automated but organic results of Tarbell's algorithmic drawings with some degree of control from the user.

Torus Brush, Processing. Jason Bailey (2009)

The first shape that I had used for my drawing tool was a torus or donut shape that appeared to dance as its mesh deformed throughout the 50 frames. The donut shape repeated its animated dance in an endless loop and left a trail of the previous frames in its wake. I spent several days experimenting with this shape, using wireframes, different lighting scenarios, color randomization, and increasing and decreasing the scale. I eventually settled on a scale that seemed appropriate for a drawing application, not so large that it immediately filled up the screen yet not so small that the details of the ever-evolving 3D mesh would get lost for the user. After becoming more comfortable with using the tool to draw, I learned how to balance out the idiosyncratic rhythm of the evolving shape with the control I did have over the motion of the brush.

It struck me during the development of this drawing program that my programming process and my drawing process were very similar. Casey Reas, co-creator of the Processing language, describes the similarities of the two processes: ”As with other types of art, software may be written through a process of intuitive exploration, and it can be written precisely to meet a goal.” (Reas) I would often approach the code by making a small adjustment, sitting back and evaluating its impact on the overall image, and then going in and fine-tuning the code. As with drawing, I did not have complete control over the outcome, as I allowed for experimentation and surprises to influence both the code and the drawing.

Time

Home Office, Processing. Jason Bailey (2009)

Taking Mike Golembewski’s class in creative projects made the underpinnings of computer graphics seem less elusive.

Mike was serving as adjunct faculty in the spring semester of my second year. In the first two weeks of Mike's creative projects class, we worked in groups. Our first assignment was to combine and rethink two objects, which would be selected randomly from two lists. The first list included objects that function as containers, including drawer, backpack, closet, etc. The second list included technological objects such as scanner, webcam, digital screen. My partner Elaine Froehlich and I were assigned backpack and webcam.

We had a great time brainstorming different ideas and ultimately decided that the webcam would pull in live video from the front of the bag as the user walked around and display the video on the outside flap when the bag was placed on the floor, playing back the journey.

Gallery Sketch, Processing, Jason Bailey (2009)

We also thought filtering the video would make the displayed video more interesting, so we messed around with some webcam video in After Effects to simulate how it might look. Seeing that we were interested in filtering video, Mike set aside some class time to explain how digital video is drawn to the screen, and how real-time filters can be written in Processing to affect the display. I have worked with digital video editing software for about ten years now, cutting and pasting clips, adding music and titles, and occasionally adding filters for color correction or transitions between clips. Software like Adobe Premiere and Final Cut Pro has made it possible to edit video without ever understanding how it is pulled into the camera and drawn to the screen.

As Mike explained to us, video is drawn to the screen, many frames per second, pixel-by-pixel, left to right. It starts in the upper left corner of the screen, moves to the right corner, when it hits the right side of the screen, it then moves down a row, similar to the motion of a typewriter. Despite the speed at which this all happens, we can ask each pixel for information about its position on the screen, its color value, and its brightness. More amazing was discovering that these values can be changed on the fly.

With this new information, video had become a much more exciting area to explore. The next week in Mike's class, we were discussing how you can measure the difference between two different colors using a vector math function and the RGB values for each color. Although I still don't understand precisely how vector math works, I recognized this function as the same one I used to find the distance between XYZ points in my 3D programming projects. I had an epiphany that if I could use the same function to measure a series of three numbers for color values as I did for three points in space, and there might be some interesting visual mapping opportunities.

Gallery Sequence, Processing, Jason Bailey (2009)

I wrote a program that would read every frame of a movie, get the average color of the pixels for that frame, and then correlate the R value to the X position, the G value to the Y position, and the B value to the Z position on the screen. Often the color for a frame would average out to some shade of gray, and the frames from the video would cluster around the center of the screen. Every once in a while, a dominantly red frame would move off to the right, a green frame moved to the top of the screen, or a blue frame would move forward in Z space. I experimented with using the position of the brightest pixel in each frame to dictate the position of each frame as well, trying to get better separation. After experimenting with setting different thresholds for color and adding in some random motion, I had a pretty exciting program.

As I put different color objects in front of the camera or pointed the camera at different light sources, a line of still frames was drawn across the screen. My mapping wasn't perfect, but this idea of drawing in space and time using video frames was turning up some exciting images and was fun to work on.

Wave Sketch, Processing. Jason Bailey (2019)

About a week later, I decided to exhibit the partial project at the Doran Gallery at Massachusetts College of Art. Although not complete, I thought I could benefit from watching people interact with my project.

I’m not sure who he was, but one gentleman at the opening leaned over with curiosity and stared into the lens of my webcam, which lay resting on the white pedestal in the gallery. Clearly restraining himself from touching the camera (after all, it was in a gallery and it can be hard to tell what you are allowed to touch and what is off-limits), his eyeball was only about 3 inches away from the lens of the camera when he finally backed away and looked at the projection on the wall. There it was, the still image of his eyeball, projected as a plane in space, on the gallery wall, next to hundreds of other images taken from the camera during the previous 30 or so minutes. He went back to the camera, put his hand in front of it, stepped back again, and searched for the image of his hand on the wall, smiled, and then walked away. 

Collage Tool

I believe it was a motorcycle. I had been cutting up comic strips from the Sunday edition of The Boston Globe and blowing them up into large abstract shapes on the Xerox machine in the Whittemore Library at Framingham State College.

It was 1997. Scanners probably existed then, but I did not have access to one and would likely not have the technical skill required to operate one. The stylized engine and exhaust pipe of the cartoon motorcycle had these long elegant lines that I knew would work nicely in my collage. I was working on one of the many inspiring art projects assigned by professor John Anderson in my 2D design class. The assignment required us to use the photocopy machine to blow up the simplified, familiar images into several abstract compositions exploring the balance of negative and positive space and variation of line weight. I had never done anything like this before, and developing a library of shapes that I could freely move around in combinations to create a new composition was enjoyable.

The forms that emerged were complex and exciting in ways that I would not have discovered if I were creating a preconceived abstract drawing. The element of chance and the speed with which I could experiment with new compositions by sifting through and juxtaposing images made this a memorable experience for me.

This project was memorable enough that over ten years later, it served as inspiration for a project at the Dynamic Media Institute. It was the process of creating these collages and not the results that were so memorable and engaging for me. I decided to try and capture that process with a digital interface. This would require me to think about time and motion, two elements that were not present in the static drawings, paintings, and illustrations I had created in the past. I put together a proposal for a time-based, interactive collage tool: "The tool would be based on the juxtaposition of simple 2D shapes to make more complex shapes. The interface would save the compositions created by the users into a database and play them back as inspiration for future users. When you approach the interface, the shapes would move about the screen, occasionally assuming the compositions created by previous users. The current user could freeze the screen at any time and then rearrange the shapes, controlling position, scale, and rotation to form more complex imagery. Ideally, the results would express a meaningful and creative co-authorship between user and toolmaker."

I had been researching creativity and ways of facilitating it in users through the design of the interface. One person who fascinated me in my research was Will Wright, the inventor of Sim City, The Sims, and Spore. At the time, I was trying to figure out if engaging people in the creative process was something that just happened or if there were design decisions that could increase the likelihood of success. Wright was of particular interest to me because he had developed wildly successful and engaging interfaces that allowed users to combine existing elements already present in the game creatively. I figured there had to be some type of formula for creative engagement. I was determined to figure it out.

In reading everything I could find about Wright, I found this quote from him: "What is the simplest possible system that I can build, that for you is going to decompress into the most elaborate set of possibilities?" (Moggridge, 381) This elegant statement made a lot of sense to me. I had thought that interfaces that facilitated creative experiences came from complexity, and that complexity came from a wide variety of available actions and components upon which to act. What I was missing was that complexity can stem from simplicity. I realized that I wanted to try and design a simple system that could unfold over time into a more complex series of interactions. I thought back to the simple collage project that I had worked on in my 2D design class and how it had constraints for components and had simple rules but also served as an engaging system for the creative process. I decided to use it as the inspiration for a digital collage interface. I needed to find some interesting shapes that people could use in combination to create more interesting shapes. I decided to vectorize several artworks from 20th Century artists, including Jean Dubuffet, Jean Arp, and the collage work of Henri Mattise, and break them apart to provide a library of shapes. These artists appealed to me for this project because their work includes simple, graphic, hard-edged forms.

I knew I wanted people to be able to combine these forms, rotate them freely, and change the scale, just as I had in my Xerox collage project, only this time with a computer. I was worried that this experience might not be as fun in a digital environment, so before I started coding up the interface for scaling and rotating the shapes, I played with the shapes in Illustrator, making several compositions. Convinced that this was worth exploring, I moved forward with my programming.

While experimenting with Illustrator, I realized the steps required to move, rotate, and scale the shapes were quite complex. Brainstorming ways to reduce this complexity, I came up with an idea for a single-click interface that would cycle through functionality but use the same gestures. I centralized the code so that each shape was an object with the same behavior; I could have one shape or a thousand shapes using the same amount of code. When you clicked on the shape, it went into the first stage of interaction, and you could drag it around the screen, juxtaposing it with other images. A faster gesture with the mouse would throw the shape across the screen, bouncing off the edges until it decelerated to a complete stop. A second click allowed the user to rotate the shape by moving the mouse to the left or to the right; changing the scale was accomplished by moving the mouse up or down.

After many months of programming, I had an interface with which I was happy. I had solved several technical problems, devised a novel method for scaling and rotating shapes, and the motion felt entirely organic.

However, I was reluctant to have anyone use my interface. What if all of this work resulted in a dud? I eventually brought a variation of the interface into class and had several classmates try it out. While they had fun with it, none of them created the collage-type images I had in mind. I had worked in isolation too long and made too many assumptions. Developing a single formula for facilitating creative engagement was starting to feel like a pretty weak idea, but it didn't stop me from further researching this idea.

Studying Will Wright led me to more research on complexity derived from simple systems. Wright had mentioned Conway's Game of Life and cellular automata as a source of inspiration, so I decided to investigate this further. Cellular automata are "grid-based systems that vaguely resemble game boards." The squares or cells can become occupied or vacant based on a simple set of rules, which often lead to complex, emergent behavior. Conway's Game of life, created in 1970 by mathematician John Conway, is probably the best-known example of cellular automata. His rules dictated that an occupied cell would be "alive," and vacant cells would be "dead."

When set into motion, each cell would change based over time based on the state of its neighbors. The fantastic part about these simple systems is the variety and complexity of patterns that emerge. (Salen, 161) I was starting to understand why Conway was so inspirational to Will Wright in creating his games. Because Wright used only simple rules and few components, his games were easy to learn, but his understanding of systems meant the games had an extremely diverse set of outcomes. This again started to sound like a formula that could be applied to a tool for image-making, whether it be drawing, collage, or building.

I became more interested in systems and simulations as a means of creative engagement. A classmate suggested I pick up a copy of Mitchel Resnick's book Turtles, Termites & Trafic Jams. In his book, Resnick describes the development of Starlogo, a program Resnick wrote for students to model and simulate the behavior of decentralized systems like traffic jams. Before StarLogo, Resnick had been working with students to learn about decentralized systems using one of his other projects, Lego/Logo, and realized it was not the ideal environment. He explains: "To explore phenomena like these, I decided to go back to the computer screen, to work with virtual creatures in virtual worlds. Virtual worlds offer many advantages. In virtual worlds, it is easy to create large numbers of creatures. It is easy to give new sensory capabilities to the creatures. And it is easy to set up and control precise experimental conditions." (Resnick, 31) Like Will Wright's Sim City, Resnick's StarLogo interested me for the variety of potential outcomes the user could arrive at while manipulating only a small number of components. At this stage in my studies, I was doing more reading than making. After reading Turtles, Termites & Trafic Jams, I was hungry for more reading on systems. In browsing the bookshelves at the local book store, I came across Noah Waldrip-Fruin's book Expressive Processing. In the introduction, he states: "...computational processes are an increasingly significant means of expression for authors. Rather than define the sequence of the words for a book or images for a film, today's authors are increasingly defining rules for system behavior." (Fruin, 3) I was now more convinced than ever that all the important people were creating systems, authoring experiences through the development of rules.

It was the beginning of my last summer at DMI, and I started feeling the pressure to come up with a thesis project. Inspired by everything I had been learning about simulations and systems, I started a second attempt at creating an interface that would allow the user to set simple components into motion with the hope that these components would evolve into interesting compositions over time. I had a decent understanding of the Processing programming language, and I was impressed with its 3D capabilities. Inspired by the simplicity of the Lego building system and my research on systems and cellular automata, I set out to design a simpler version of the collage project in a 3D environment. This was an unusual approach for me to prove a project idea based on reading and having a detailed plan of what the project would be before starting to make it. After several weeks of programming, reading tutorials, and "borrowing" code, I had an empty environment full of gray cubes that would highlight in red when selected. Once selected, you could move the cubes around the screen and combine them to make more complex shapes. After you built up a model with the simple cubes, you could apply simulated physics to make the model fall apart. This was my attempt at further applying the idea of simplicity leading to complexity as a means of generating an interface that could facilitate creativity in the user.

I had my friends and classmates try the interface. The general response was that it was impressive from a programming standpoint, but the glitches it did have prevented people from wanting to build and explore with the interface for very long. While this was a significant success for me from a technical perspective and a vast improvement on past projects, it was still missing something. It was not engaging the user. I also felt like working with such a predetermined outcome was unnatural for me. It felt more forced and less enjoyable, not as open to chance and discovery. I was trying too hard to push my process to mirror the research I had done on simulation and systems.

Interaction

"Art teachers are the biggest bullshit artists on the planet!" Oh, my God, I thought, what have I done? It was my first day as a high school student teacher.

I was hardly a year or two older than most of the high school students in the classroom, and the student who just called me a wannabe “bullshit artist” looked like he was a year or two older than me and wanted to beat my face in. I forget his name, but he had the same stupid glower of the seemingly infinite number of jocks that had made my own experience as a high school student so miserable.

I had voluntarily gone back to hell. Somewhere in my head, I thought that being a high school art teacher would allow me to help struggling students find purpose and shelter like my art teacher had done for me. I had forgotten that art is a requirement for all high school students to graduate, and the majority of those students would rather be playing kickball or doing just about anything other than sitting in art class. I don't mean to imply that they weren't good kids; it's just that not everybody is that into the whole art thing.

Some of the students enjoyed being there and enjoyed the projects they were working on. There was the kid with bushy black hair riddled with dandruff drawing an army of robots in compromising positions. The girl with no eyelashes and cigarette burns on her arms, drawing a beautiful still-life with colored pencils and cray-pas. And the painfully stereotypical teacher's pet: a bright, smiling Chinese girl with a Dartmouth College sweatshirt. She had single-handedly resuscitated the ceramic kiln from near extinction and proudly displayed a long row of elegantly crafted organic vessels. Two years earlier, I would have been hanging out with these three kids after school, perhaps even starting a crappy garage band with them. They were my people.

For the majority of the semester, I just watched and took notes as Mrs. B. directed her classroom. She was a great teacher, but I couldn't help but notice how all the students worked in isolation. Their projects seemed too precious to them. They were afraid to try new things for fear of failure and judgment. I remembered these observations when Mrs. B. gave me an opportunity to create an assignment of my own, and thus, full control over the class.

There was a graffiti problem at school. They sandblasted one side of the building several times that semester. I remember thinking about how the anonymity of that act of graffiti must have been freeing for a high school student under constant surveillance and scrutiny. Inspired by these anonymous acts of graffiti and a desire to get the students to collaborate, I decided that my assignment would involve each student drawing on a large piece of paper for one minute and then passing it to the person on their right. They would do this until they received the drawing they had initiated.

I brought in my cheap dorm room CD player so I could play inspiring music for them while they worked. Before playing a selection from some crazy experimental sound band from the '90s (Praxis, I think), I laid down the ground rules of the assignment. I told them the only rules were to draw for that one minute and then pass to the right. I didn't care what they drew. I told them I would not censor them. "Serious? We can draw anything?" they asked before letting out a collective giggle. "Anything you want," I said. I started the music, set a timer, and said: "go."

Everyone started drawing with an urgency. Some drew hearts and smiley faces; others drew abstract squiggles; some chose to write words. It was awesome. One student decided to draw the same image of two stick figures having sex over and over again. Another student further down the line was offended by the X-rated stick figures and would turn them into lowers, rainbows, or just cross them out. In the end, there were 30 unique compositions created by the class as a whole. We were able to critique them openly because there was no single clear author.

Mrs. B. hung them side by side like a giant graffiti wall in the hall outside the art room. She told me she would be doing that assignment again and even asked for the name of the CD I had played.

I have re-purposed this drawing exercise several times since my student teaching days, most recently in response to an assignment on anticipation at DMI and again as a visiting lecturer at Framingham State College (my alma mater). Turning drawing into an exercise in socialization and interaction, instead of isolation, appeals to me. It puts more emphasis on process, the part I value most, and less on the quality of the artifact or finished piece.

Entering into my final year at DMI, I realized I had worked on several projects that I intended to be platforms for collaboration, but always hesitated to bring them to the level of user testing for fear that they would fall flat. Initially, I thought an interface with a blank screen that allowed users to collaborate in 3D was the way to go. I had tried Second Life and several other spinoffs and found them to be too cluttered for intense collaboration.

In my second semester at DMI, I decided to try and create a minimalist collaborative environment where people could develop models and make drawings together in three dimensions. The environment would combine 3D software like Autodesk Maya and Google Sketchup, with the networking capabilities and social interaction of Second Life. It would be a blank canvas without artificial terrain and over-the-top avatars, or so I hoped.

Knowing my limitations in programming, I sought out a platform to build upon that would already have resolved the 3D elements such as cameras, texture mapping, and navigation. I eventually settled on Croquet, a program that Alan Kay developed along with several other computer graphics luminaries. According to the Croquet consortium website, Croquet is “a powerful open source software technology that, in the form of the Croquet Software Developer’s Kit (Croquet SDK), can be used by experienced software developers to create and deploy deeply collaborative multiuser online virtual world applications on and across multiple operating systems and devices.” (Croquet)

I must have glossed over the part about “used by experienced programmers” because I spent the better part of a semester trying to resolve technical challenges to get a prototype of my empty 3D world up and running. At the end of the semester, I presented a blank screen with a simple grid on which several people could log in as 3D chicken avatars and draw in 3D space. I planned on replacing the chicken avatars with floating 2D planes mapped with live webcam video, but was not able to make the changes in time for the presentation.

Croquet collaborative drawing prototype

The presentation was a flop. The silly-looking chickens on the minimalist background combined with the awkward drawing tools, and the inability to demo live without crashing, made it difficult to express my intentions. Because getting the platform up and running that semester used up the bulk of my time, the majority of the project relied on existing tools and functionality already built into Croquet. It took another two semesters before my classmates stopped picking on me for my chicken drawing collaborative interface.

Although I had been thinking of this collaborative 3D environment as a potential thesis project, I abandoned it as there were too many technical challenges, and I had sensed little to no interest from people in using such a tool.

I decided that I needed to do more research on collaboration and creativity to make my future projects more successful. I read book after book about systems, game design, interaction design, design patterns, and the philosophy of creativity in the hope that they would lead me to a set of best practices for designing interfaces that would facilitate creative interaction. By this point in my thesis development, I had realized that I wanted to focus on drawing as an interface for creativity. Still, I also realized that my approach to researching in books for a set of rules for designing creative interactive interfaces was becoming counterproductive.

Luckily, at the start of my final year at school, I realized what was missing from my previous collaborative projects: collaboration. I had come up with these projects in isolation. I worked by myself at the computer for long hours with a predetermined vision of a finished piece and lots of false assumptions about how people would interact with my projects.

It was around this time that I had a conversation with Dennis Ludvino, a classmate of mine who was working on a collaborative writing environment that had a lot of crossover with my interests in creating a collaborative drawing environment. I approached him about “collaborating” on making a shared canvas together. He thought it was a great idea, so we started working out the details. It turned out to be one of the best decisions I made while at DMI.

We started by using a pre-built Processing sketch that would allow two people to share a screen through a network. We immediately ran into limitations. The software would crash, it only supported two users, and we needed to adjust the code every time our computer's IP address changed. These types of technical problems usually pop up at the beginning of my projects, but it was already less intimidating having someone else to work on the issues with.

With zero networking experience between the two of us, we went off and researched the different problems, checking in periodically with different solutions that could potentially advance our progress. By sharing the work, we were able to make several breakthroughs on the technical side in a relatively short amount of time. The first breakthrough was learning about and implementing Dynamic DNS on the server-side of our application. This allowed us to use my home computer as the server and not worry about if and when the IP address was going to change, which would break our network. The Dynamic DNS would check for this change and automatically update without requiring us to change our code.

First Generation Collaborative Drawing Platform

The breakthrough we are most proud of was finding a way to recognize multiple users. We had a theory that if we were already passing the X and Y positions of our mouse back and forth, why not give a new identification number to let the other users know who the information came from? Then we could write code to recognize each user individually as the data went back and forth and add many more users. This may sound simple, but having come up with this solution entirely on our own, with such little programming experience, left us thinking there was very little chance that it would work. It was extremely gratifying to see our homegrown solution expand the demo code we had started with into a truly multiuser interface.

After solving the main technical issues, the collaboration only got stronger. We felt like Alexander Graham Bell and Thomas Watson when we were finally able to get the shared canvas up and running with Dennis drawing from his home in Lowell, Massachusetts, and I, many miles away, drawing from my home in Ashland, Massachusetts. The first drawing came together rather quickly. One of us, I don't remember who, started drawing a horse, the other added a single horn, turning it into a unicorn, and then we both collaborated on a rainbow. The drawing was hideous, but the experience was fantastic. At some point early on, we made a mistake in the code, and there was a line being drawn between the two users, in addition to the lines created by the users mouse. This changed the drawing process quite a bit as it filled up the screen at a much faster rate. It also made some very intriguing shapes. We were pretty sure we understood what was going on, but we invited Dennis's fiancé Krista and my wife Erin to join us, creating a four-person drawing to confirm our assumptions. We all had a great time making strange abstract webs of network lines. After we made the drawing, we combined all four drawings, one from each computer, into Photoshop layers, where we could study the relationship between them.

Working with Dennis kept this project moving in a fun direction. We both felt like there was something cool about drawing at the same time as someone else, in different locations, on the same canvas. The more people we let use it, the more confident we became that we did not need to justify the purpose of the interface. It was fun and engaging, and every time we came together to work on it we discovered something new.

The drawings we created using the program were genuinely collaborative: often juvenile in their subject matter (robots, unicorns, middle fingers), but always in response to one another. Typically, someone would draw a circle or a box, to which the other users would add a face or body. Psychologist Manfredo Masseroni devised a similar exercise whereby the reader would create a simple scribble and transform it into a bird by simply adding a small circle O and beak < shape. Colin Ware describes how the "deliberately 'random' scribble is often used by artists and designers as a way of liberating their creative process from stereotyped visual thinking. The perceptual mechanisms are exquisitely tuned to find meaning often from slender evidence, and this is why meaning is so easily found in meaningless scribbles." (Ware, 154) Our meaningless scribbles would transform into a variety of characters, sometimes appearing as Santa Claus, an alien, or a portrait of one of the users. Inevitably, someone would start to create a landscape for these characters to live in, drawing a simple line for the ground, stylized trees, or mountains in the distance.

Space Bus, created on our custom collaborative drawing platform written in Processing (2009)

As the screen filled up with lines and became more chaotic, the subject matter would also shift toward chaos. Someone would draw weapons for the characters, guns, or axes attacking the other characters on the screen. This aggression would spill over from the battle of the characters on the screen to personal attacks between users. These personal attacks were in the form of someone writing a comment about someone else's mother, or a series of drawings of hands giving the middle finger. In our last semester, we decided that we would continue to work on our collaborative drawing project together, but this time we would bring in even more people to help. Instead of thinking of the people we shared our project with as user testers, we thought of them as coauthors for our project. We would use the project with them and implement the feedback they gave us on the fly.

For example, we created a series of drawings over the network with Scott Murray, a classmate living in San Francisco. As we made our drawings, Scott mentioned that the canvas seemed to ill up too rapidly, making it hard to continue the drawing. He suggested we have the drawing fade over time, and the three of us implemented the code on the spot.

In another session with once classmate and now teacher, Colin Owens, we discussed ways to decrease the delay between user gesture and the image appearing on the screen. Colin wrote a version of the code that helped us speed up the lag making the program closer to real-time. We used Colin's code in combination with some code we worked on with my younger brother Matt, a software engineer by trade. The new code pushed lived on the server-side, further reducing delays and allowing for additional users without having to adjust the code, improving the user experience.

Collaborative drawing created on our custom coded platform (2009)

Instead of fearing the feedback of the users, we viewed them as assets to help improve our project. We discovered that most people were excited to participate in the making process, instead of just the using process, and the more people we included, the more fun we all had.

Working together turned a stressful semester of trying to finish my thesis into an enjoyable time of collaboration between friends, classmates, and family. I am not sure if we will continue to work on this project after graduation, but I am convinced that the experience has changed the way I will work in the future. Working in isolation, guessing at how people will interact, and performing staged user testing was not working for me. Getting together with other creative people and working through ideas together from the beginning is a lot more fun and a much better approach for designing interactive projects.

Data  

Data Visualization of the first 5 seconds of John Coltranes Giant Steps - Jason Bailey (2009)

A week before my first thesis meeting with my advisor, I went on an annual getaway with my father and brothers we call the "Bailey boy weekend" camping trip. It was the last weekend before I started my final year at DMI.

After three days of drinking beer and tequila, and incorrectly playing a partial copy of the board game Risk, my brothers, my father, and I sat around the ire to continue the 31-year-plus-long conversation about software, hardware, and technology. As the least technical member of the group, my role in these conversations has generally been as a listener.

Strangely, the conversation shifted to a debate about analog vs. digital, which is something we frequently debate at DMI. But to hear three engineers talk about it made it seem like a different conversation. My father played the role of the defender of all things analog, saying, "There is no such thing as a unique digital concept." My dad claimed anything digital was created in analog format first, and when you convert something from analog to digital, you inevitably lose information in the process. My brothers Matt and Brian, who argue for sport, had a rare surrender, agreeing with this concept of there being no unique digital experience and analog's superiority.

While the unfamiliarity of consensus briefly paralyzed them, I saw a rare window of opportunity to say something without being talked over or shut down. I disagreed with their outcome. I started explaining but couldn't get the ideas in my head to come out of my mouth. Damn you, Jose Cuervo!

Despite the tequila haze that followed me for the next several days, I had locked onto this concept of a unique digital experience. I thought about the projects I had made at DMI. Why were they digital? Just because I was in a digital program at school? More directly, what was I doing with a computer that I couldn't do utilizing analog tools, and why?

I eventually had one of those "eureka" moments: most traditional analog media have entirely different inputs and outputs. For example, a saxophone player blows air across a wooden reed (input); outcomes the sound of vibrating air, which we call music (output). This is dramatically different from a film camera in which film is exposed to light and treated with chemicals (input) to create a sequence of images projected in succession (output). Equally different from the two previous examples are drawing and painting, which both employ pigment and a stylus (input) transferred to what is usually a flat surface as a composed static image (output). I decided one of the unique opportunities of digital media is that all traditional media, which had entirely different inputs and outputs, now share a stage of development where they are all data. The result is not a crude approximation of an analog experience but a uniquely digital and entirely new experience.

Drawing With Data

After having the epiphany about reducing all media to a common language of ones and zeros, or "data," I thought back to past projects I had worked on that took advantage of this. One project I created would draw lines in space using frames from live video. A second project I created that summer used object tracking algorithms to create musical compositions from video data taken of different playdough shapes. The user could change the shapes and adjust the sound, or move the shapes left and right to affect the panning. In both of these cases, I was driving one media using the input from another. In the first case, video input was translated into a 3D line drawing output. In the second case, the sculptural form of the Play-dough was recognized and translated through video as an input and then output as sound.

While these projects were already taking advantage of the cross-pollination that is made possible by digital media, I wanted to make several smaller studies with this idea in mind. At the time, I had also been thinking about the role of gesture in drawing. In my programmatic drawings, the physical gesture became irrelevant. The way I pressed the keys on the keyboard did not affect the resulting drawing. I started thinking about the mouse gesture and how I had thought of it more as a limitation for drawing than anything else. I eventually asked myself, Why I was writing off the mouse gesture as irrelevant? After all, I spend more time gesturing with my mouse than I do, making just about any other type of gesture; it's the nature of working with computers. What if I captured the data from the everyday gestures of my mouse and turned them into a drawing? What would it look like?

Normally, I would spend a lot of time trying to write a program to capture my mouse position over time, but I knew I wanted this to be a short, experimental project. So instead, I found some free software that would track the X and Y coordinates of my mouse and save them to a text file. I turned on the mouse tracker and generated thousands of lines of data as I went about my regular computer-related tasks. I tried not to change my behavior as I was interested in capturing the unmodified gestures of my daily mouse-driven activities. I realized pretty quickly that I would have more data than I knew what to do with, so I grabbed a five-minute chunk of the data and put it into a format I could use to import it into Processing.

Data Visualization of my mouse movements at work. Jason Bailey (2008)

Once I was able to get the mouse data into Processing, I started playing around with drawing different colored shapes and lines to represent the data. At first, I thought of this exercise as a data visualization project. I tried to make it clear where the mouse started and stopped by adding circles at these departure points. I lost interest in that approach and started to tweak variables, and add new code to add randomization to the color and location of the lines, testing the drawing each time I made a change. The drawing transformed very quickly from a visualization of my mouse gestures to a more abstract and appealing image. The dense network of colorful lines spanning the screen created a complex dimensional space. Often my goal in creating drawings is not to recreate what I see, but rather to create an entirely new visual space comparable in intricacy to the world around me, less familiar yet still inviting. Initially, it seemed ironic that breaking from the data acquired from actual gestures had made the composition more gestural in its visual qualities. Unlike drawing with a pencil or brush, my programmed drawings relied more on conceptual decisions than physical gestures. My analog drawings required a learned physical dexterity and facility developed over time. That process was different yet strangely similar to drawing with programming.

My analog drawing process, particularly when creating more abstract work, involves making a few marks, stepping back, and evaluating. Often I spend more time assessing than making marks. After that period of contemplation, I usually feel like I know where that next mark belongs.

Frequently when drawing, something outside of my control will happen, an accidental drip of paint, a smudge of charcoal, etc. These elements have become a welcome part of the creative process for me and frequently take the composition in new and interesting directions. When programming drawings, I have a very similar process. I start playing with code, checking in periodically to see what the resulting image looks like. More often than not, I am surprised by what comes on the screen when I run the program. Sometimes I like what I see, and sometimes I don't. One of the great advantages of programming drawings is that my accidents are repeatable. If I tweak the code and like what I see, I can reuse that code in new combinations and turn what was initially an accident into a tool.

After completing the mouse drawing series, I realized I had a tool that could make drawings out of any data with a similar two-column format by mapping the data to the X and Y coordinates and creating lines. Thinking back to my ideas on driving one media with another, I started thinking about using this same code to make drawings out of songs. I knew that the music I played on my computer was just a sequence of numbers at a base level, regardless of whether it was a .mp3 file, a .wav file, or any other audio format. I experimented with different formats trying to find one that would produce data that I could use with my mouse position drawing program. It turned out that exporting music to the ASCII format made precisely the type of data that I needed. The patterns in music would translate to numeric patterns when digitized, which would become visible when processed trough my drawing application. I have always enjoyed John Coltrane's music and decided to use his composition Giant Steps for my experiment with drawing music. I tried to export the entire song into ASCII format and crashed my computer. Realizing that it was just too much data, I cut it in half and crashed my computer again. I proceeded to cut the song into smaller and smaller chunks until I found a manageable piece: the first five seconds. I dropped the data into my drawing program, and a beautiful composition came out.

I knew that five seconds was likely not enough time to establish much of a pattern in the song. Still, it didn't matter to me as I had proved to myself that I could write a song drawing tool by taking advantage of the common language across digital media of ones and zeroes.

Harmonica Data Visualization. Jason Bailey (2008)

While I was happy to have visualized the data from the John Coltrane song, I wanted something that I had more control over less of a visualization project, and more of a drawing project. What if I produced the music live and created the drawing with real-time audio data?

Harmonica Data Visualization. Jason Bailey (2008)

I play several instruments equally poorly. At different times in my life, I have tried to learn the clarinet, drums, and harmonica with varying degrees of success. For my live music drawing program, I settled on the harmonica, as it was the most portable. I had a little experience messing around with krister.ees, an audio library for Processing, so I decided to utilize it for this project. I hooked up my microphone, started writing code, and eventually worked through a tutorial that created real-time waveforms responding to the data captured from my harmonica.

Harmonica Data Visualization. Jason Bailey (2008)

While I was pleased to have completed the loop of the real-time audio input to visual output, the visualization itself was still just a boring, run-of-the-mill waveform. So I started to adjust the code to try and make the visuals feel like the audio sounded. I made adjustments to color, motion, line thickness, and every other variable I could find until the visualization organically evolved into a visual equivalent for my sounds. No longer just a series of waveforms, I was now drawing with my harmonica.

Algorithm

I've always found libraries difficult to navigate, and to this day, I don't understand the Dewey Decimal System. As a child, I would just wander the stacks, reading the spines, and then extrapolate the theme of any given section.

Because I could not understand the card catalog, it became essential for me to memorize where the good sections were. To this day, I have a pretty crisp image of the two shelves of books that lined the back left corner of the Billerica Public Library. One row deep on the right side and one shelf up from the floor were the children's drawing books. I remember the drawing books sandwiched between other books about juggling, ventriloquism, magic tricks, and karate. I am sure there was some Dewey-type-logic governing this seemingly disparate cluster of books, but to a 6-year-old it was just the "awesome section."

Among this awesome assortment of books, none was more awesome than the drawing books of Ed Emberly. What child could resist books with titles like Make a World and The Drawing Book of Weirdos? The covers of these books were littered with a menagerie of curious, colorful creatures all created from just a few simple shapes.

Although I could not yet read, I was able to intuitively understand the visual instructions, the step-by-step accumulation of simple visual components, combining a line here and a circle there, creating a much larger and more exciting whole. Emberly used slow disclosure, simplicity, and structural transparency to inspire millions of children to draw. The real lesson of Emberly was not in memorizing the specific algorithm of how to draw a witch or dog, but in understanding that you could draw anything, no matter how complex, by disassembling and abstracting it to its fundamental parts.

Like most 6-year-old boys, I was also excited by the idea of superheroes and action figures. In addition to crayons and drawing books, I received G.I Joe and He-Man figures for birthdays and Christmases. These action figures allowed me to stage epic fantasy battles, but I still wanted to generate characters of my own. So I created a team of superheroes called the Wee Wee Men. I know this sounds ridiculous, but I was six, and this seemed like a perfectly proper name at the time.

Initially, there were three Wee Wee Men, each one comprised of two simple overlapping shapes. Think of a square in your mind. Now remove the top line. It should look like a staple. Now duplicate this shape and lip it vertically and move it down slightly. It forms a rectangle with two lines coming off the bottom and two lines protruding from the top. The lines extending from the top were arms, stretched straight up in the air, the lines shooting downward were the legs. In the middle, where the horizontal rectangle formed, I would add two dots for eyes and a crescent for a mouth. The other two Wee Wee Men were of similar construction but used triangles and half-circles.

I eventually realized that by cross-pollinating half-circle with square, square with triangle, etc., I could expand to a much larger cast of characters. I devised an algorithm for drawing superheroes. I spent many hours drawing the scenery with mountains, plains, and rivers, and then populating them with Wee Wee Men societies playing out different situations, usually gory battles where someone would lose a leg or an arm.

Later in life, I made my way into the work world and found little time to create artwork for pleasure. In one of those rare moments where I would create my work for pleasure, I started a series of algorithmic drawings in Maya 3D software. I had taught myself MEL script, the native programming language for Maya Software. The desire to explore how I could apply my new scripting knowledge to create intricate, abstract 3D drawings was consuming me. I would wait until after my wife and I went to bed, get back up, and write 3D drawing algorithms. I was addicted to the surprises of this process. The first surprise would come after I created a simple algorithm and ran the routine. Although driven by math, these drawings were strangely organic, each one a surprise. The second surprise would come after setting up the virtual camera and lights on the model and rendering out the high-quality image. Depending on the complexity of the model, the render could take several hours before finishing. Sometimes the results were stunning. Other times, they were relatively unremarkable, but it was the variety of results and the lack of complete control that made the surprises so rewarding.

Case Study: Player Visualizer

Stat-based data visualization of NBA players. Jason Bailey (2008)

The sweat from my face poured down onto the keyboard. Even in the shade of the wooden cyber-shack, the heat of St. John in early September was unbearable. I had snuck away from my new bride on our honeymoon to research any off-season trades that may have affected my fantasy basketball draft strategy. The season didn't even start for another month. We had decided to spend the week at a campground tucked into the hills on the shore of a beautiful Virgin Islands beach, and here I was paying a dollar a minute for Internet access to catch up on basketball news. The pleasure I received from basketball statistics eclipsed the joy of watching basketball. This was strange on many levels. Math was responsible for the majority of my scholastic low-points, including failure to place into a credit-bearing course in college despite spending a month in summer school re-taking algebra. This obsession with statistics could only be one thing: sibling rivalry.

Like most siblings, my younger brother and I have always been pretty competitive with each other. Unlike most siblings, our competitions almost always played out in some obscure form: Frisbee golf, eating hot foods, or statistical analysis, for example. We spent a lot of time reading and sharing literature on basketball statistical analysis, including Basketball on Paper by Dean Oliver and Dave Berri's Wages of Wins. Our annual fantasy basketball league was our chance to put our knowledge and theories against one another; it had become an obsession for us both.

Statistics were interesting to me for more than just bragging rights. As a designer creating courtroom graphics for an accident investigation company, I tried to represent complex data in a graphical form that would be intuitive for a jury to understand. This job helped me realize that I enjoyed the process of distilling complex information into simple graphics. That career provided me a never-ending stream of data to work with: crush-data derived from vehicle collisions, thermal maps of building fires, and the fatigue-testing data of microelectromechanical systems, to name a few. But after leaving that job, there was a shortage of new data to visualize. Basketball statistics filled that void for me.

All sports are algorithmic, some more than others. For example, statisticians love baseball because it is arguably the most algorithmic of the more popular sports, making it the easiest to model. Baseball has the longest season, 162 games, which provides a large statistical sample. The action is slow and isolated, making it easy to document. Success and failure are measured in discrete steps based on which base a player gets to: first, second, third, or home. Statistically speaking, basketball represents a much more dynamic space than baseball. It features continuous and decentralized action, making it a challenging sport to document and analyze statistically. So when DMI guest lecturer Peter Kirn asked us to choose a data set to visualize for a summer course in Processing, basketball statistics were the obvious choice.

The first step was to figure out how to import data. I took the information for a single player, in a single game, cleaned it up in Excel, and figured out how to import it into Processing. After tracing the data to the screen, I figured out how to map it to the length of a rectangle, approximating a crude bar graph. I was confident that I understood how the process of importing data worked, but I wanted to make something more interesting than a bar graph. I thought about what my visualization would look like if I stayed with this bar graph approach, but tried to represent every statistical category for each player for the entire previous season. I realized that if I continued on this path, the resulting visualization would be overwhelming and illegible. This is precisely the kind of visualization problem I relish. I started mapping the same data to other shapes: circles, triangles, lines. Then, inspiration struck: I could write an algorithm to draw caricatures of the players using their statistics. It was a similar process to the one I used to illustrate the Wee Wee men superheroes. I could combine simple shapes to develop a wide variety of characters. What made this especially appealing was that many statistics are tied to human physical attributes. For example, a player with great passing ability has good eyes, a rebounder typically is taller with longer legs, shot blockers often have longer arms, etc.

This brought on the type of excitement that gets me past the technical challenges of learning a new programming language. I worked feverishly for several nights, perfecting my caricature drawing algorithm for a single player. When I felt like I had a decent likeness of that player, I imported the data for the rest of the 432 players taken from the 82 games played that season and ran the program. I realized, like never before, the power of the computer as a drawing tool. With a few simple code tweaks, I had gone from drawing a single player to drawing 432, and many of them looked like the actual person they represented. More importantly, it was incredibly easy to make quick visual comparisons between the players and even the teams and extract the statistical differences. Instincts provide us with an ability to read the human form incredibly fast and notice subtle differences without difficulty. Although I knew this somewhat instinctively, I confirmed it when my classmate Scott sent me information on Herman Chernoff.

In 1973, Chernoff invented a visualization technique called the "Chernoff Face," which acts on this same basic principle using subtle changes in simple human caricatures to represent large data sets. Drawing just one of these basketball caricatures with statistical accuracy utilizing a pencil or even Adobe Illustrator would have taken me an hour. Now, as if by magic, I had a machine that could generate thousands.

Distribution

Data visualization of all shots taken in the NBA by location and volume. Jason Bailey & Matt Bailey (2008)

The geek culture blog Boing Boing receives more than ten million page views per month. On May 11th of 2009, Boing Boing Gadget, a sister blog to Boing Boing, published an interactive data visualization I created with my younger brother.

The visualization displayed a 3D bar graph of every shot taken in the NBA for the 2008 season, positioned relative to the location on the court at which the shot was attempted. We had seen a heat map graphic that used color to depict the information in 2D but felt like it did not communicate the discrepancy of shot volume between the locations on a basketball court from which the shot was attempted. (Witus) We decided that a navigable 3D interface could enhance the discrepancies between shots while increasing the clarity of shot location. The visualization had been on my blog for seven months. I had no reason to believe it would be treated any differently than any other post. Typically one or two of my classmates would read my posts and occasionally, my brother, but that was about it. Being the lazy blogger that I am, I did not realize that Boing Boing had picked up my post until several months after I posted the article to my blog.

I was surprised and delighted to find that many other blogs and websites discovered our visualization interesting. There was Alark Joshi, the post-doctoral student from Yale who was doing work on open-source medical imaging and had developed ground-breaking visualizations of hurricane data. Alark posted a large image of our graphic on his "visualization blog" with the comment, "An awesome visualization of all the shots from the NBA 2007-2008 season." (Joshi)

AERS, a company specializing in "advanced transactional data processing," was extremely complimentary. They posted the visualization to their website with the title "Fantastic 3D visualization in Sports" and finished the article by stating, "Kudos to Jason… I hope your phone is ringing off the hook with NBA and NHL teams." (AERS) No one ever called from the NBA or NHL. But as of January 27, 2010, the AERS reposting of our visualization project had 1,478 views.

Not all the feedback was complimentary. Among the comments on Boing Boing’s original posting were, “It’s a tongue. I can’t be the only one who sees this, “ and, “That image reminds me, very strongly, of the boss fight with Andross in StarFox.” Unbeknownst to us, our project had developed a life of its own.

(Detail) Data visualization of all shots taken in the NBA by location and volume. Jason Bailey & Matt Bailey (2008)

Of all the drawings, paintings, and illustrations that I have created in my life, I never thought that a chart of three-point shots would get the most exposure and generate the most interest. This is a uniquely digital phenomenon with little or no analog equivalent. Before working on the computer, like most artists and designers, I would create work, show my friends and family, and every once in awhile, put work in a local gallery where maybe 50 to 100 people would see it. The Internet allowed us to put our work in front of more eyes and inspired more feedback than all my other work combined.

Not to imply that it is simple to get people to care about your work. The visualization had laid dormant on my blog for months before being referenced by a more popular blog, Boing Boing. The Internet, with its 16 billion users, gives anyone with a computer and modem access to an enormous audience. The chances of finding several thousand people who care about basketball visualizations in my everyday life are pretty slim. However, on the Internet, subcultures of people who share my fascinations become a readily available audience.

Conclusion 

Self Portrait, Age 5. Jason Bailey

Every fall it is a DMI tradition for current and past DMI students to share “pearls of wisdom” with incoming students. In the past two years, I have given the same advice both times: “get to know your classmates as fast as possible.”

The followup to this "pearl" is to "go out after class with them and drink beer and eat pizza." I think this second part is just as crucial as the first, or perhaps it is just the best way I know for people to get comfortable with each other. The DMI faculty have consistently done a great job of choosing an eclectic group of design professionals. The students include architects, filmmakers, ceramicists, writers, usability experts, performance artists, and industrial designers from all around the globe. I made it a point to meet every student that entered the program, learn their name, their background, and invite them out for beer and pizza. I wasn't just nice. I was selfish. Every time I met a new student, I learned something new about design, and I made a new friend.

One of the primary reasons for applying to graduate school for me was the hope that I would become part of a group of creative professionals who would share my interests and help me develop new ones. My biggest regret for my time spent at DMI is not having chosen to work on projects as a group more often. Programming, like drawing, is often an isolated activity. I spent 90 percent of my three years at DMI locked up in my home office, working into the night by myself. There were certainly times where I enjoyed working by myself. drawing and more recently, programming has provided valued "me time."

About halfway through my time in the program, I was working on a project that involved blowing on a balloon tied to a Wii-mote to navigate Google Earth. I spent a lot of time thinking about how to make this fun for the user. At one point, my wife and I tied 30 helium balloons to our Wii mote and blew it around the backyard with a leaf blower. The balloons carried the Wii mote high up into the air, sailing toward the neighbor's yard and eventually into a tree. Although this approach could be seen as a failure, we had a fantastic time documenting it and trying to get the Wii mote down from the tree. Several months later, I tried to give a live demo with a single balloon carefully placed on a balancing Wii mote such that it would rock back and forth steering through Google earth when the user blew on the balloon. During the end of the semester reviews, I asked everyone to move into the room next door to interact with my demo. When I tried to start my demo, the laptop had fallen asleep, and the demo would not work. Even if the demo had worked, it was not that interesting. It was too predictable, too controlled, too much of a canned experience. The making process was more fun with other people and ultimately more fun than interacting with my final project.

The semester after the failed balloon demo, I took Gunta Kaza's design as an experience class. It was the right timing. Gunta's assignments put more emphasis on the experience, interaction, and collaboration than they did on the artifacts resulting from experience. In one project, we needed to design a solution for characters in conflict from one of several selected stories. I selected an excerpt from The Life of PI detailing several days in the lives of a starving man and a tiger stranded on a raft in the ocean. This seemed like an absurd situation, so I came up with a crazy solution. I decided that it could be a friendly tiger, but the inability to communicate would keep them from realizing they meant each other no harm. I designed a helmet that projected cartoon speech bubbles, allowing them to share in a way that would be clear and non-threatening. I thought I was done with the project, but Gunta asked us to push further. Have solved the immediate crisis, I decided they needed a way to pass the time. Keeping with the absurdist solutions, I decided that Pi and the tiger would write a book of haiku together, which they could then share with the world when they were saved. I had wanted to involve more people in my projects, so I decided to ask my family, friends, and classmates to write a haiku from the perspective of one of the characters. I was surprised by how willing everyone was and how much fun everyone had, and was further inspired to involve others in the creation of future projects.

A few weeks before finishing this thesis book, I had a realization. I had learned many new things at DMI, done a massive amount of research, and developed many new skills, but in the end, I returned to drawing. I thought about the other DMI graduates and realized that most of them had returned to their areas of expertise as well. Whether returning to writing, architecture, music, etc., we almost always come back to our areas of expertise, but with an entirely new viewpoint. I would recommend to future DMI graduates who are struggling as I did to find that big thesis idea to remember where they came from. I think that is where the thesis lives.

Neural Style Transfer And Inpainting For Artists

I work for myself and I am cruel boss so I don’t take too much time off. But I make an exception on Thanksgiving so yesterday I let myself get creative and play around a bit with some machine learning models. The end result of playing around is the image above which composites a still from the movie JAWS into a painting of a shark attack from 1778 by John Singleton Copley called Watson and the Shark. You can see the two source images below.

Watson and the Shark, John Singleton Copley - 1778

Still from the 1975 Steven Spielberg movie JAWS

Like lots of people, sharks freak me out. Copley’s “Watson and the Shark” is a large painting (72 1/4 x 90 3/8) and it made an equally large impression on me as a child visiting the museum. Learning that the person getting attacked by the shark in the painting was a 14 year old boy didn’t make me feel much better at the time. Interestingly, the boy (Brook Watson) survived minus one leg and eventually became Lord Mayor of London and an original committee member of Llyods of London eventually becoming an early chairman. It was Watson who commissioned the epic painting from Boston born painter John Singleton Copley.

Around the same time I saw this painting I also watched JAWs for the first time. The two experiences have been intermingled in my mind ever since. So I thought it would be fun to try and use a combination of neural style transfer and inpainting to combine them.

First step was to find a film still from JAWS that looked like it would fit with the image. I lucked out and the first image from Google that came up felt like it could be a good fit. I then opened the images in a software package called Runway ML created by my friend Cristóbal Valenzuela to “bring the power of artificial intelligence to your creative projects with an intuitive and simple visual interface.” In Runway I selected the Ada-IN Style-Transfer model and loaded the JAWS film still first and then loaded the Copley painting second as a source for the style transfer. The results were pretty solid right off the bat!

Ada-IN Style-Transfer

I then opened the original image of “Watson And The Shark” in Photoshop and extended the canvas space on both sides (which was initially blank) and crudely placed the two men on the left side and the shark on the right. Though as you can see from the screen shot of Runway ML the style transfer did a great job, I tweaked the levels and saturation a bit to get it even closer to the painting - but honestly not very much.

Photoshop aligning the pieces

Then I used Photoshop’s “content aware fill” tool which is essentially an inpainting algorithm that pulls information from surrounding areas to fill in holes or areas where there is no content. To do this you just mask off the areas you want to fill using inpainting and then activate the content aware fill tools. I like to mask the seams between images to smooth out the transitions.

Masking for the content aware fill tool.

And that is basically it! Actually, I ran the whole thing through the style transfer one more time with my new collaged image as the destination and the original painting as the style source, this helped a lot with making the whole image look and feel unified covering up color and texture differences. Final image below.

I did some other cool/fun stuff with Runway and Photoshop including this layered Inpainting which is created from a few dozen layers and creates a really cool output image.

I don’t usually do tutorials but if you dig this let me know and maybe I will do some more (or bug my friends who are real experts to see if they’d be willing to share a tip or two).

Speaking Machine, Art, and Design With John Maeda

I’m thrilled to share that John Maeda, a personal hero of mine, agreed to let me interview him about his new book How to Speak Machine: Computational Thinking For the Rest of Us which, at the time of this interview, has been out just 24 hours. I think most folks who read Artnome are already very familiar with John, but by way of a short introduction, John is an accomplished engineer, artist, and designer who served as associate director at MIT Media Lab, president at RISD, and currently serves as chief experience officer at Publicis Sapient. John’s art, writing, and presentations have shaped my own thinking on art, design, technology, and diversity over the last two decades, and it is a real honor to share our conversation below:

Jason Bailey: Hi John, thanks for writing your new book How to Speak Machine and for the opportunity to ask you some questions about it! As an artist who grew up computationally challenged in a family of engineers (and then worked for two decades in tech in a non-technical capacity), you have been a personal hero of mine for a long time. I believe your new book takes some really important steps in helping people understand the language of computation which is increasingly governing the world we live in despite only a small number of people speaking it with any fluency.

John Maeda: Thanks. I have enjoyed reading your writings and now hearing about your background, it makes sense to me how you mix computational fluency with the arts. It's a rare capability -- but I've noticed that it's slowly increasing. Yes. The book is in reaction to realizing how obscure the computational world really is to someone who never spent a long time immersed in how the machines work. So it's not their fault to be disengaged because the hurdle to really comprehending what's happening in the invisible consciousness of the cloud today is simply hard to believe.

JB: I was pleasantly surprised to see how much space in the book you dedicated to talking about art and design. My first question is about your personal distinction between art and design. I noticed you don’t really spell out a clear differentiation between art and design in the book, but you talk about them very differently. You speak of art and artists with reverence, and I appreciate and share your reverence. A few quotes from the book: 

• Art is the science of enjoying life

• An artist’s perspective keeps you humanly curious

• The earth without art is just eh

• The question inducing power of art 

• Artists excel at making unlikely connections all the time 

As a reader, it feels like you treat art almost like an antidote to a life that can swing too far towards tech. 

JM: I have a terrible short-term memory. So as a habit I try to remind myself what is important by writing it down. My hope is that when I come back to it later, I might remember to take it seriously. LOL. The arts are high on my list. 

JB: By contrast, you describe the temple of design as being a bit more fickle and commercial, with a handful of superstars dictating new ideas that quickly cycle into and out of fashion. 

JM: I would lob a similar criticism about the contemporary arts world, actually. 

JB: You also suggest that this model of the sole genius or design star crafting a finished product gets disrupted in a computational world where designs are always in flux, never finished, and informed by continuous feedback from users. The quote that hammered your opinion of traditional design home for me was:

“The standards that elite institutions uphold as the cultural compass for the classical design world are underwritten by subjective decisions and invisible wealth networks that facilitate what gets remembered vs forgotten. The temple of design’s narrative of the genius designer is not a reliable path to success. It’s seductive, but it’s stupid.”

Do you think there is a clear difference between art and design, and, if so, what do you see as the main differences in how they are impacted by an increasingly computational and automated world? 

JM: I've always thought of design as making solutions. And art is making questions. Both art and design have access to computation. And so it's possible to make different kinds of solutions and different kinds of questions that were not previously possible.

JB: Will art, like design, shift away from the idea of the sole genius and move toward continuous development and group design, or is the idea of the individual genius too critical to preserving the “question-inducing power of art”? 

JM: I think both art and design are grounded in the idea of the lone genius. It's a romantic idea. I lived it for quite a long time. And then I came to realize that teamwork is much more interesting. Should art be created by teams? If the questions to ponder are really big, then yes.

JB: My second question is about artificial intelligence and machine learning. These are hot topics these days, and I think the public has a lot of fear and anxiety based on misinformation they have received from movies and the media about AI/ML being sentient and taking over our jobs. 

JM: I agree. And a mood of fear gives the perfect opportunity for politicians and other kinds of leaders to exert power over others with the promise of protecting them. That is concerning to me.

JB: I would have had these same concerns about AI/ML if I hadn’t spent four years at a machine learning startup seeing how difficult it is to get AI/ML to automate even the most basic tasks. So I don’t blame people for being afraid, it seems like there needs to be more education. 

JM: I agree. Unfortunately, it's really hard to understand how computation works. It took me many years, just like you. I was wondering if I could make some kind of shortcut. How To Speak Machine is my attempt to do so.

JB: I was thrilled to read your quote from Andrew Ng early in the book, “If you're trying to understand AI's near-term impact, don't think ‘sentience.’ Instead think ‘automation on steroids,’” which aligns with how most experts I know would describe AI/ML. 

JM: That's my favorite quote. I'm glad you liked it, as well.

JB: However, later in the book when talking about exponential advancements, much of the focus is on the singularity. You talk about Vernor Vinge’s prediction of the singularity ending the human era and Ray Kurzweil’s predictions about machines outpacing humans, all of which make people very afraid of AI/ML.

JM: Many technologists don't believe in the singularity. I hesitated to include it in How To Speak Machine. But because the word gets used so often, I wanted to talk about it in the context of how the word started to become used.

JB: Do you think people are right to be afraid of AI/ML? 

JM: I think to be afraid of what comes in the future isn't a great way to live. I would reframe it as people should understand what's coming -- which should make them less worried about not knowing what's happened. If you have a general idea of what's happening, you at least have a chance at living with it, or ideally, overcoming it.

JB: Is it a matter of realizing that what we have today is not scary, but it could get scary further down the line? 

JM: I think that being scared and more scared is not a good algorithm for life. But if it motivates you to change, and change for the better, then that's a good thing.

JB: You also suggest that when AI does bad things, we humans will be to blame. Does this mean we should be more afraid of humans than of AI? 

JM: The beauty of critical thinking is having the power to ask questions instead of just responding to everything that happens. Being concerned about the humans who are using AI at scale deserves our critical thinking.

JB: How can we help to educate people so they can essentially know the right time to freak out (if ever) and how freaked out they should be? 

JM: I am not sure. I thought that it would be good to at least start exposing more people to how computation works. How To Speak Machine is not my first attempt to do so. In 2005 I had an exhibition at the Cartier Foundation called Nature which was seven short animated films I made to depict the universe inside the computer. The fact that nobody knows about them today means that I failed. LOL. So I guess I'll keep trying.

JB: As an extension of the previous question, there is a lot of really interesting art being made using machine learning as a tool which is unfortunately being overshadowed by public discourse fixated around a false narrative of machines replacing artists and making art on their own. As a result, we also have some pretty silly “art” projects with paintbrush-wielding robots wearing wigs grabbing headlines about robots/machines replacing artists. 

JM: I enjoy seeing all the work being done out there. Even if it might seem minor, sometimes I think that a bigger outcome can only be understood when we look across all the pieces, no matter how small they might seem.

JB: The advice you give in your book is to “be what AI cannot do.” I believe AI/ML is good for artists because creativity and soft skills will be the hardest to automate, and this will put a premium on skills that artists excel at. What do you think of the art that is being created using machine learning today, for example, using GANs? 

JM: I'm excited by it all. I've learned so much by browsing what is being made out there.

JB: Do you think artists should be worried about being replaced by machines, and, if so, when? 

JM: I think artists should avoid being worried. Being insecure is fine. But not worried. Otherwise you cannot inspire others.

JB: Or alternatively, do you believe that AI/ML will highlight artists’ unique skills and increase their value?  

JM: My guess is that AI will remove a lot of the nonsense work that we think is important, and allow us to focus on what is truly interesting.

JB: Another aspect of the book that I appreciate is how you tie humanity back to computing. You do this in the book by pointing out that humans were the first computers, that concepts like recursion and fractals can be found in nature, and by pointing out that “computation has a shared ancestor: us.” As we live in an increasingly computationally driven world with inhabitants that do not yet “speak machine,” how can we help put people at ease and feel less disconnected? 

JM: I wish I knew the answer to that. I guess my answer is to continue to think critically about what we do. And to think critically about what I do. If we stay connected to each other, and we can stay connected to ourselves, then we are aware. And present. And then we are no longer disconnected.

JB: Obviously writing a book like yours is a helpful step, but are there other ways we can move in this direction? 

JB: I'm glad I wrote the book so I didn't have to think about these things anymore. And now every day I'm just trying to move things in a direction that I believe is important. We can all do that in whatever line of work we are in.

JB: What role, if any, do you think artists can play in helping society to become more comfortable in its own computational skin?

JM: Artists who like to say that computers do not matter can often do so because they are usually independently wealthy enough and do not need to be relevant. For those artists who have less privilege, there's no better time than now to take advantage of a medium that has such potential. So go learn computation deeply.

JB: Last question. In following the arc of your career, I see a theme of wanting to help people better understand computation so that it feels less alien and less intimidating. I see this with Design By Numbers, the programming tool you designed to teach less technical people how programming works, and now, with How to Speak Machine, where you try to explain the basics of programming and computation in words. Do you see this as being one of your life’s missions? 

JM: No. I don't think I ever set out consciously to do whatever I do. I try to be useful in the moment. That moment feels like right now with respect to computation.

JB: What did you hope to accomplish in book format that maybe you could not have achieved with DBN or through teaching and lecturing?   

JM: Like everyone, I work to be relevant. Being relevant means being fully open to how whatever you think is right could now be wrong. From a timing perspective, I think that How To Speak Machine is not too early and not too late, and by putting all this thinking in book form, I don't have to teach or lecture about anything. Instead, I can tell someone to simply read the book. And I can go back to work to try to stay relevant by bringing in new thoughts that I don't know yet. LOL.

JB: Thanks again for writing this book, I will be recommending it to folks for a long time!

JM: Wow! Thanks -- that means a lot to me coming from you.

Artists Hijack MoMA with AR Art Exhibition

The artist Damjanski, who we recently wrote about for his post-apocalyptic, sans-human iPhone app Bye Bye Camera, is at it again, this time with his MoMAR artist collective at the newly renovated MoMA, with a guerrilla AR art show in the Estée and Joseph Lauder Gallery (former Jackson Pollock Rooms).

Artnome fully supports Damjanski and the MoMAR collective in their rogue space claiming exhibitions of digital art. MoMA can spend $450 million on renovations, but no amount of money can keep savvy artists groups like MoMAR from subverting the museum from a place designed for the public consumption of art into to a space for the public display of art.

A recent study shows that over 75% of art in US museums is by white men. A second study shows that without having work in the MoMA and a handful of other institutions, artists have little to no chance of achieving great success. Until the system changes, the fastest route to diversifying museums and gaining recognition may be to beg forgiveness rather than seek permission and to follow the lead of MoMAR.

In support of the exhibition and in solidarity with our friends at MoMAR, we are sharing their full press release, as well as several GIFs from the exhibition below.

Erin Ko LA BARRERA

Manuel Rossner-REEF

exonemo STROKES

Akihiko Taniguchi AUGMENTED SELFIE

Official Release from MoMAR

NEW YORK, October 25, 2019— 

From October 25, MoMAR returns to MOMA with a smartphone based group show, using Augmented Reality technology. Launched in spring 2018, the non-profit gallery project hijacks the Estée and Joseph Lauder Gallery (former Jackson Pollock Rooms) for the third time. While art seems to be the ultimate place of freedom and individuation, art museums often resemble sacred devotional spaces in which perspectives and behavior know only one direction. MoMAR ties in with the artistic movement of 'Play Art’, which in the 1960s had a democratization of art in mind. Variable art objects made art itself tangible as a product of collective activities and value creation processes. The participants of 'Open to the Public' belong to the generation of the Internet. However, they have also started to dynamize the medium of art, its exhibition forms and ways of interpretation. MoMAR provides a platform for these approaches and therefore rethinks the mechanisms of art curation. Each artist has created new work specifically for the exhibition. Thus 'Open to the Public' takes MoMA's new exhibition concept at its word and provides for 'more than physical expansion'. It is less about the ownership or location of individual works of art than about the plurality and diversity of their experience. The VR-collage “La Barrera” by New York based artist Erin Ko is an augmented reality piece that addresses the signal-to-noise ratio in our everyday lives, at a time when the future seems especially uncertain. Her work plays with the idea that museumgoers are looking at art while the world is burning, melting, and falling apart. But is it simply a luxury, willful distraction, or a genuine appreciation for the present moment? The spatial work “Reef” by German artist Manuel Rosner creates an evolutionary spatial structure that is built by its own dynamics. In contrast to the white cube's neutral architecture it is a colorful mix-up which doesn't derive its aesthetics from expressive forms of art, but from software which uses color-coding for transmitting information. The work “Strokes” by the Japanese artist duo “exonemo” also questions the exhibition space in an abstract way. Each time the piece starts, your phone drips random strokes on the wall and beyond. This ephemeral painting only exists in a world-famous museum, made-up in your mind. Finally, the work “Augmented Selfie” addresses the partly libertarian, partly narcissistic trait of virtual self-representation. An avatar of Akihiko Taniguchi appears in augmented reality and a hand holding an iPhone appears in the first person view in the screen. Tap the screen to take a picture in the AR space and the picture is saved to your smartphone automatically. Akihiko's Avatar will mimic your player’s movements. The artist says: “This is an extended selfie where various things intersect in virtual space and real space, where they meet and gather.” After all, art takes place in the conversations of the exhibition visitors. Let me open Instagram. 

About the artists: 

The Japanese artist duo exonemo (by SEMBO Kensuke and AKAIWA Yae) was formed in 1996 on the Internet. Their experimental projects are typically humorous and innovative explorations of the paradoxes of digital and analog – computer networked or actual – environments in our lives. Their piece “The Road Movie” won the Golden Nica for Net Vision category at Prix Ars Electronica 2006. Exonemo have been organizing the IDPW gatherings and "Internet Yami-Ichi" since 2012. They live and work in New York since 2015. 

Erin Ko is a mixed media artist based in New York. Ko’s experiential art plays with Mediated Reality, Collective Consciousness and Layered Experiences. She combines traditional art making methods with new media tools to address our complicated love/hate relationship with technology. She has worked in video games and new media for 20 years. Some of those endeavors include time spent at Looking Glass Studios, Razorfish, and Blue Fang Games. 

Manuel Rossner creates spaces that merge playfulness, sobriety and abstract aesthetics. Often, but not necessarily, they intersect with reality. His work has been shown in international institutions and galleries. After founding Float Gallery in 2012, he designed a digital extension building for NRW-Forum Düsseldorf and a virtual building for Roehrs & Boetsch Gallery in Zürich. His solo exhibition for the Museum der bildendenden Künste Leipzig opens in autumn 2019. He lives and works in Berlin. 

https://okikata.org/ lives and works in Japan. His work includes installations, performances and video works using self-built devices as well as software. In recent years, he has focused on net art work – and sometimes VJing. Tanoguchi teaches at Tama Art University and Musashino Art University. Main exhibitions include "dangling media" ("emergencies! 004" at "Open Space 2007," ICC, Tokyo, 2007), "Space of Imperception" (Radiator Festival, UK, 2009), "redundant web" (Internet, 2010) "[Internet Art Future?]" (ICC, Tokyo, 2012). 

The gallery 

MoMAR is an unauthorized gallery concept aimed at democratizing physical exhibition spaces, museums, and the curation of art within them. MoMAR is non-profit, non-owned, and exists in the absence of any privatized structures. MoMAR uses Augmented Reality to overlay art onto existing artwork and frames housed in museums and gallery spaces around the world. Downloading the MoMAR app, visitors hold their phones over specified frames and spaces in order to view alternate artwork, and MoMAR exhibits. 

Download the MoMAR app at http://momar.gallery Alternatively, iOS 11 users, simply point your camera at the code for instant download. You cannot view the exhibit without the app. Unless you have psychic powers. LOL 

Press Contacts: press@momar.gallery For downloadable high-resolution images, register at http://momar.gallery

Augmenting Creativity - Decoding AI and Generative Art

I am thrilled and honored to announce that Artnome will be curating an exhibition titled Augmenting Creativity - Decoding AI and Generative Art at Nanjing University this November, 2019.

In preparing an exhibition in a country I have never been to and whose language I do not speak, I’ve been thinking a lot about translation and communication. Specifically, I have been thinking about the language of computing as an increasingly dominant and universal language which shapes all aspects of our lives despite only a small percentage of our population (developers and computer scientists) having any real fluency in it.

People with limited exposure to programming and training networks often assume creating generative art is a carefully planned, predictable, and linear process with a predetermined outcome, but nothing could be further from the truth. A common thread across the artists in this exhibition is a circuitous path of discovery, an addiction to surprise, and an ability to embrace and build on “accidents” or the unexpected.

There is, of course, no one “right” way to conceive of these surprises and the role that technology plays in the generative artmaking process. Some see code and networks as a just another tool. Others describe it as more of a creative collaborator. That each artist in this show has their own way of describing how technology augments their own creativity only serves to further individuate their work and brings a richness to the dialogue surrounding the exhibition.

Among artists, generative artists carry a unique burden of having to explain how programming and machine learning work without distracting their audience from their other artistic ambitions. Without at least some insight into computing and code, we as an audience cannot fully appreciate their artistry and craft.

The most engaging generative artists share the ability to seamlessly educate us on how their work is created and use that opportunity to reinforce their artistic objectives rather than to distract from them.

In his 2010 presentation at the Flash on the Beach conference, Jared Tarbell does a masterful job of using his artwork as an aid in explaining programming concepts. Tarbell starts by sharing his love of nature, something we can all relate to. He then explains how iterating or “looping” a simple command to create dots can empower us to recreate the same complex and beautiful patterns found in nature. He then uses the simple algorithm behind his famous 2003 work Substrate (which we are excited to feature in our exhibition) as an example.

Jared Tarbell, Substrate - 2003

As Tarbell describes it:

Let’s imagine in this system that we create a line and we allow this line to draw. The line will continue to draw until it hits the edge of the screen or another line and then it stops and then two new lines form. New lines will also only form at right angles to existing lines. So those are the rules. But if we repeat this process over and over again, something amazing happens, so I am just going to let this go.

The construction of Tarbell’s Substrate mirrors that of his presentation in that it builds on very simple definitions and actions to create something complex, rewarding, and beautiful. The fact that Tarbell shows us how his work is made does not detract from it. Instead, understanding Tarbell’s process only increases our curiosity and engagement.

Tarbell originally created Substrate back in 2003 using a programming language specifically designed for creatives called Processing. Casey Reas and Ben Fry, co-creators of Processing, wanted to create a language and a community that could make it easier for creatives to explore programming as a tool for making art.

The Processing language has been a huge success and has empowered hundreds of thousands of creatives by reducing the learning curve to use programming as an artmaking tool.

We are honored to be featuring Reas’ 2004 MicroImage in the exhibition. As with Tarbell’s Substrate, Reas’ MicroImage is a complex system constructed from the repetition of relatively simple parts and commands. By creating thousands of dots and programming them each with a simple set of behaviors designed to react to their surroundings, Reas explores the concept of emergence within a software environment.

MicroImage (Software 1), 2002/2014. Software, 1920 x 1080 pixels.

---

The dots follow simple rules, and there is no central system instructing how each individual dot should behave. Local interactions between the dots leads to the emergence of a type of swarm intelligence, a global behavior, which the individual dots could not exhibit on their own.

The results as seen in Reas MicroImage are breathtaking and feel like like a living, breathing, pen-and-ink drawing undulating with rythyms reminiscent of schools of fish or flocks of birds.

Reas shares his process:

The core of the “MicroImage” software was written in one day. The current version of the software has developed through a gradual evolution. While the base algorithm controlling the movement was constructed in a rational way, subsequent developments were the result of aesthetic judgments constructed through many months of interacting with the software. Through directly manipulating the code, hundreds of quick iterations were created, and changes were implemented based on analyzing the responsive structures. This process was more similar to intuitive sketching than rational calculation.

Substrate and MicroImage feel natural because they harness concepts and algorithms found in nature, but in the hands of generative artists like Tarbell and Reas, these algorithms create new worlds rather than replicate the one we live in. Part of what is new and powerful about generative art is the ability to go beyond painting an illusionistic image of nature and to actually create systems that emulate the creative forces of nature itself.

If programming can better help us understand the complexities of nature by harnessing hidden algorithms for the creation of art, could it also help us understand the complexities of our own minds and the minds of machines?

Artist David Young explores the similarities and differences between human and machine learning by training models with a severely limited data set. While most still consider art made with AI and ML to be “generative art,” the process is a significant departure from traditional methods used with tools like Processing. Young explains in his essay Tabla Rasa: Rethinking the Intelligence of Machine Minds :

Like AI programmers, artists working with AI don’t encode rules, but instead train networks. The resulting artworks look and feel different from previous forms of computer-based art for they reflect the organic messiness of their vast and inscrutable neural networks.

In an attempt to better understand machine intelligence, Young takes an empirical approach to training GANs (generative adversarial networks), observing closely as he sparingly adds new training data. By severely limiting the quantity of training data used for the model, Young hopes to better understand how AI builds an understanding of the natural world.

David Young, Still from Tabula Rasa (b67o,2600,2) animation - 2019

Our exhibition, Augmenting Human Creativity, features Young’s documentary animation Tabula Rasa (b67o,2600,2) which chronicles the learning process of his GAN. Watching Young’s model learn in compressed time is fascinating, but it raises more questions than it answers. Among the key questions prompted for Young were:

Do we need to develop a new concept of beauty to evaluate what emerges from working with these systems? And is it possible that a new aesthetic can give us a better, more empowered understanding of AI?

Young argues that all training data includes the emotional bias and irrationality of the human who selected it. This leads him to ask:

Given that we can’t help but use human terms to describe the behavior of AI systems — we say that the machine is “learning,” it “knows” something, it has “skills,” the word “intelligence” is inescapable — perhaps we should look for other human traits. Might machine “emotions” be a new way for us to approach and understand AI?

Whether we think of AI as an emotional entity or simply accept that the data it is trained on is imbued with the emotions and bias of the human who curated it, it is interesting to explore how it can interpret or even contribute to complex cultural concepts like fashion and costume.

Harshit Agrawal, Masked Reality - 2018

Artist Harshit Agrawal creates artificial intelligence/machine learning-generated faces, drawing inspiration from traditional mask cultures of different regions of India. Agrawal explains:

Throughout our years of existence as a culture, we’ve crafted and performed several kinds of rituals and ceremonies, both collective and individualistic as acts of transformation and transcendence. Masks and face-transformative decorations have been fundamental across the Indian culture in our journeys into unknown realms, in our celebrations of the malleability of human representation, or as a tool for practical disguise and entertainment.

It helps us engage with our world from a completely new vantage point, augmenting our sense of self, very similar to what technology, especially AI, enables today. What happens when these media of transcendence collide? Can we teach machines about our cultural heritage, and as a result, make them an instrument for our own exploration and engagement with our heritage?

Harshit Agrawal, Masked Reality - 2018

Agrawal’s project allows for users to explore both machine learning and the language of India’s mask culture directly through an interactive installation which we are excited to include in the Augmenting Human Creativity art exhibition.

Artist Robbie Barrat trained a GAN on Balenciaga’s fashion catalogs to produce radical new fashions and styles unlike anything a traditionally trained human fashion designer would ever come up with. Barrat particularly likes the absence of symmetry, the random placement of pockets, and the addition of non-functional adornments like handheld tassels.

AI Fashion - Robbie Barrat, 2018

AI Fashion - Robbie Barrat, 2018

Barrat uses Pix2Pix technology in combination with DensePose to map the new AI outfits to the models. DensePose tries to estimate human poses and "...aims at mapping all human pixels of an RGB image to the 3D surface of the human body." A simpler explanation of this is that Barrat is training the AI to not only recognize the clothing in the Balenciaga catalog of clothing, but also the poses of the fashion models, and is then mapping the new fashions on to AI-generated fashion models and postures.

Example showing use of DensePose and Pix2Pix

Barrat went a step further with his creative collaborator Mushbuh and produced physical garments which were recently included in a show at the Fashion Museum of Hasselt. We are excited to be showing Barrat’s original runway video in our exhibition.

AI inspired garments by Robbie Barrat and Mushbuh

Barrat has always been careful to describe machine learning as a creative tool and is hesitant to anthropomorphize the machine as an artistic partner. Barrat points out that with generative art made using machine learning, and specifically GANs (generative adversarial networks):

• A human chose the data set

• A human designed the network

• A human trained the network

• A human curated the resulting outputs

Barrat’s framing of GANs as a tool and his emphasis on the human contribution helps to explain why art made with machine learning varies so much from artist to artist despite using similar models. It is predominantly human creativity that is driving the creation of the work and human creativity that supplies the work’s purpose and meaning.

Artist Sougwen Chung acknowledges the importance of the human artist in generative art, however, she tends to think of machines more as collaborators than just tools. Chung is best known for her project Drawing Operations in which she engages in a drawing performance with a robotic arm. Chung explains, “The interface is not simply a mediating apparatus for creation, but a speculative agent of co-creation.”

Drawing Operations (Duet) is a performance centered on a drawing collaboration between human and machine. In this duet, Sougwen and Drawing Operations Units: Generation 1&2 create an improvised drawing corresponding to 3 themes, tracing a narrative of human and machine; Mimicry, Memory and Future Speculations. Each thematic chapter showcases an evolving robotic behavior linked to the artists explorations into art & Ai. The 25 minute performance is accompanied by a visual projection and an original score by Aquarian. The work is part of an ongoing series of Drawing Operations projects by Sougwen Chung.

Like Agrawal, Chung is comfortable mixing technology with tradition. In a recent interview, Chung explains:

I don’t see tradition and technology as oppositional or mutually exclusive. I utilize the technology of tradition, and are we not growing into a tradition of technology? Insofar as we can frame a narrative around tradition as that which is made by hand and technology as that which is made by machine, the work I’m doing is certainly an exploration of the possibilities in the intersection of both… one that is gaining clarity the further along I go.

In addition to robotics and neural networks, Chung’s work uses a variety of traditional art materials as well as digital tools including, but not limited to, Photoshop, Illustrator, Cinema4d, Processing, nano-controllers, electric violin, Wacom tablet, Arduino, projectors, and Madmapper.

Sougwen Chung, The Limitless, The Absolute -2017

Augmenting Creativity will feature Chung’s work The Limitless, The Absolute which she released alongside the Lunar Calendar for 2017.

According to Chung:

“The Limitless, The Absolute” is an artistic interpretation of the Bagua฀八卦; the eight Taoist trigrams that represent the principles of reality as organic, emotive, and experiential environments. The correspondences between the eight interlocking concepts of the Bagua are used in Taoist cosmology to interpret and contemplate reality and existence. The patterns formed as the trigrams interrelate provide the conceptual logics used by the artist to create delicate, complex, shifting environments resembling ecosystems.

Like Chung, artist and roboticist Alexander Reben is fascinated with the relationship between humans and machines. Reben’s work deals with synthetic psychology, artificial philosophy, and robot ethics to “probe the inherently human nature of the artificial.”

New technologies can often feel foreign and distant from our human nature, frequently leading to irrational fear and anxiety. Reben’s work explores and often collapses this distance by exploring the relationships between humans and machines on a more visceral and intimate scale.

Issac Asimov’s first rule of robotics is “do no harm.” Breaking this rule, Reben developed a robotic arm with a lancet that was capable of detecting a human presence administering a needle prick to the intruder.

The first robot to autonomously and intentionally break Asimov's first law, which states: A robot may not injure a human being or, through inaction, allow a human being to come to harm. The robot makes a decision to injure a person or not in a way the creator can not predict (in this video it decided for injury)

Reben explains:

It was kept simple intentionally, as one of the ideas with the work is that you don’t need an advanced AI or complicated system to start encountering interesting philosophical and ethical issues.

With just a few small adjustments, Reben turned his fear-inducing robotic arm into a soothing head massager, exploring the flip-side potential of administering pleasure between human and machine instead of pain.

Reben’s most recent project, Latent Faces, debuted last month at Ars Electronica in Linz, Austria. For the project, Reben built a photo booth and created an average portrait of over 700 of the attendees to the fair using machine learning and GANs.

Combinations of real faces in the latent space from over 700 people who visited the art installation at Ars Electrionica. (real people as represented by an AI, styleGAN)

Reben notes when people become active participants and see themselves as part of the project it, personalizes and transforms their relationship to the technology in was that simply seeing a tech demo of GANs does not.

We are excited to be including a variation of Latent Faces as part of the Augmenting Creativity show. The photos of visitors to the show in Nanjing will be processed by Reben in batches locally in San Fransisco to create a face of the average attendee in Nanjing.

It is an honor to continue to curate exhibitions of generative art around the world. I am extremely grateful to both Nanjing University for hosting the exhibition and to the many artists for their generosity in working with me to put together the show. The exhibition is open to the public, so if you find yourself near Nanjing, please do stop in.

Epilogue - Professor CJ Chen

“What’s beautiful is difficult,” as Plato said. Understanding art is also difficult  now -- especially digital art. We all kind of agree that everyone is an artist in the digital era due to the convenience of digital tools. However, we also know that not everyone can understand the complexities of writing computer code, which is an important part of understanding digital art. 

Accordingly, it causes difficulties for the audience to appreciate the artworks made by/with digital technology. It may not bother some artists, who feel the audience does not need to understand the code and programming behind the interface before they see the works, but it does bother researchers like me. 

We study art as well as digital technology. We are creating/imagining an ecological environment where artists, audiences, and machines -- including hardware and software -- can understand and communicate with each other. Because some of us believe that the future will be built by humans and machines together, we want to know and better understand the technology better from the beginning as part of the process of appreciating the art.

Making art could be the most creative and innovative way to help human beings talk about and understand artificial intelligence and machine learning. Understanding these technologies is important because it impacts the future of all humans, no matter where you are. That’s a reason why I, a researcher on digital media and humanities at Nanjing University, China, invited Jason Bailey, a critic and curator on digital art in the US, to hold this exhibition in China. 

Jason and I share the common belief that artists (and curators) can play a critical role in helping audiences to understand the new technologies shaping our world. As Jason says: 

Programming is an increasingly dominant and universal language which shapes all aspects of our lives despite only a small percentage of our population (developers and computer scientists) having any real fluency in it.

Fear comes from ignorance. We hope everyone will enjoy and appreciate the works as well as the beauty of the collaboration between artists and technology, human, and machine.

- CJ Chen

An Interview With Art Forger Robert J.C. Driessen

Foreword by Jason Bailey

I probably have more in common with notorious art forger Robert J.C. Driessen than I do with most blue chip art collectors: We both grew up dreaming of being artists, and we were both deeply disappointed to discover the art world is not designed to support most artists and often feels rigged (data suggests this is not just a “feeling”). 

Anecdotally, I spent much of my life in art school and in art classes surrounded by artists. Despite this, I have met only one person who makes a full-time living off of their art, and he is a political cartoonist.

So I can see how Driessen could easily have become disillusioned and desperate as a young artist and started justifying forgery as an act of revenge against a system that failed to recognize his own talents. It’s likely Driessen thought forgery was a crime that only impacted a tiny percent of very wealthy collectors and institutions. The problem is, this isn’t true. Forgery impacts us all.

Cache of Giacometti forgeries

In an open letter to art forgers I wrote back in October of 2017, which mentioned Driessen by name, I pleaded with forgers to stop thinking of their actions as victimless crimes. I reminded them that it is the would-be artists, the modern versions of their younger selves, who are the ones who miss out as a result of their forgeries. The damage is not merely mistaking their works for those of other artists, but the cumulative erosion of a credible artistic record of our most important cultural objects: our art.

To his credit, Driessen responded to my open letter to forgers to open up a dialog. From Driessen:

By coincidence I read your “OPEN LETTER” to Art Forgers, now one year ago.

My name is Robert JC Driessen and I forged “Giacometti” sculptures and many other artists I admired.

The fact I forged was that I was not bright minded enough to create something myself.

That is why I always tell people, “I am not an artist but I make art”…

…I did try, (after my prison sentence) to make my own art but the public does not seem to like it very much. 

But please do not worry, I will not forge “ART” any more.

I am very happy that Driessen no longer creates forgeries, and I hope other forgers follow his good example. But we need to stop celebrating forgery in popular culture as “a few rascals getting one over on the rich and powerful” and call it out for the irreparable damage it does to our shared cultural history. That said, I don’t believe anyone, even forgers, are all good or all bad. I believe everyone can change, and I believe that people deserve second chances. I also believe that if you see something as a problem (forgers and forgery in this case), it is better to engage with the problem and try to understand it than it is to ignore it and hope it goes away on its own. 

In the spirit of better understanding forgers, I’d meant to interview Robert Driessen since he reached out to me almost a year ago. I had not found the time and did not trust anyone else to do the interview. Then journalist and art historian Christina Cacouris reached out to volunteer to help with Artnome. Cacouris was willing to help with the less-exciting aspects of Artnome, including cleaning data. But when I read her writing and, particularly, her interviews, I thought data cleaning would be a waste of her talents. So I asked if she’d be interested in interviewing Driessen for Artnome. 

Cacouris obliged, and she did an excellent job with a tricky interview. She avoids the trap of lionizing or villainizing Driessen, and instead tries to understand him and his motivations. Much of what Driessen chooses to share with Cacouris has more to do with attacking the art world rather than with his own history as a forger. I must say this blame feels misplaced for me. If Driessen and other forgers had not produced the forgeries in the first place, there would be no concern with the work entering into the marketplace (and later, into our museums).

I also don’t believe any auction house would make it a regular practice to undermine public trust in their own brand, as trust — not art — is their stock and trade. With hundreds of years in operation as trusted brands, Christie’s and Sotheby’s have outlasted most brands from most other industries. Businesses do not achieve that type of prolonged success by trading long-term trust for a quick buck in the short run.

While I disagree with Driessen’s assessment, I appreciate him sharing his perspective with Cacouris so that we might better understand some of the motivations behind forgers and forgery moving forward. I am also delighted to welcome Cacouris as a contributor to Artnome, and I encourage you to read her other work, as it is excellent.

An Interview With Art Forger Robert J.C. Driessen by Christina Cacouris

Forgery of a Giacometti drawing by Robert J.C. Driessen

People have a fascination with forgery. Perhaps it’s because the art world has been so overinflated that when we hear about the Leonardo da Vinci selling to a Saudi prince for $450 million and then have it turn out to be most likely fake, we respond with glee. There’s something of a Robin Hood complex about it all: Only the ultra-rich are being swindled, so what’s really the harm?

“It’s absolutely not da Vinci,” says Robert Driessen of that spectacular auction. A former art forger, he says he can spot fakes a mile away. “Have you seen the before restoration and after restoration? It’s a totally different painting now. It’s unbelievable how people can fall for that.”

By his own estimate, Driessen created thousands of forgeries during his run as a forger in Europe, starting with oil paintings, then moving to sculpture. He sold these works to collectors, who, knowing they were fake, would then take the pieces to auction. At the apex of his forging career, he focused his production on Giacometti sculptures, before his arrest in July 2014. He spent a few years in prison, then relocated to Thailand, where he still makes Giacometti reproductions—with the disclaimer that they are not originals.

Driessen called me from Thailand to discuss his life’s work and perspective on forging. From our interview and from his self-published memoir, it’s clear that Driessen intended to be an artist himself, but fell into forging when he desperately needed money and received a commission to paint 19th century Romantic-style Dutch and German landscapes that weren’t exactly meant to be forgeries, per se, but also weren’t exactly meant to be anything else. 

He wavers between remorse for his wrongdoing and denouncing the art world as essentially being no better, therefore justifying his work. “I really mean it from the bottom of my heart: I love art, I dream art, I eat art,” he says. “But the art world is so loathsome you wouldn’t believe.” 

There’s an obvious incentive for auction houses to inflate the value of a work, which can be done in a variety of ways, from accepting something widely believed to be fake (see: Salvador Mundi), to price fixing (see: Christie’s and Sotheby’s ). The house always wins, and even the littlest white lies can begin to stack up. 

“They take a glance at where it comes from, and as long as they can sell it, they need art to sell,” he says. “As long as it looks good and they’ve got some reference, they don’t really care where it’s from.” 

“I think that 30-40% of all the auction houses is forged art,” he adds. 

The normal process for selling work from a private collection at auction involves having the work authenticated by a third party expert; the collector then takes their work and the authentication certificate to an auction house, like Christie’s or Sotheby’s, who then sets a starting price and brings it to auction. Intentionally or not, there’s a lot of room for error in the authentication process. 

Not every work by an artist has been documented, and artists often change styles throughout their career. There is rarely one singular marker that will clearly separate the fakes from the originals. It’s almost impossible to be certain, so, claims Driessen, most authenticators give objects the benefit of the doubt. 

Would a database like Artnome’s compiling the top 40 artists’ known works help combat incorrectly authenticated forgeries? “Yeah, but I don’t think it’s possible,” he says. “Artists, if you’re talking about the top 40, you’re talking about Picasso, Giacometti, Cezanne. These people hardly ever held records, and you still find in attics or basements some piece which is not registered. Of course most of them are fakes, but sometimes a good piece turns up. The problem also is, if it is an original piece and you go to an expert, the expert will not want to burn his hand and say, ‘No, I don’t think it’s original.’ That also happens.”

But, authenticators can be persuaded to give something the benefit of the doubt: “If a piece is good, if a piece is almost original, I do know from experience that if you say to an expert, ‘If you say it’s original, and I put it in the auction, and for the selling price, I’ll give you 20%, 30%, 40%’ — if it’s a piece that brings $10 million, come on! It’s a million [dollars] just for the expert to say ‘yes’.”

There’s also a built-in protection for the expert, should things go awry and the piece is discovered to be a fake: “They say, ‘Well, I can’t know everything. To the best of my ability, I thought it was genuine.’ That also happens.” 

The line between real and fake, authentic and inauthentic, can also be tenuous when looking at sculptures that are made from casts. Take Rodin, who willed his estate to the Nation of France in 1916, which authorized the government to make posthumous casts. There are some provisions; the date of the cast has to be inscribed on each sculpture, to delineate between those that were made by Rodin’s hand (or, if not by his hand precisely, at least made during his lifetime) and those that are being cast posthumously.

Every cast that the government makes of Rodin’s sculptures is, technically, a Rodin sculpture. What, then, separates that from the works by Driessen or other forgers who used casts they took from the original sources — are they any less authentic than a government-cast Rodin? 

In his memoir, Driessen talks about the prevalence of family members creating posthumous casts to make more money after an artist’s death, citing the German sculpture Wilhelm Lehmbruck as one example, with his widow casting many more pieces from his original molds. “I made Lehmbruck, as well,” he says, “but his wife did the aftercasts, and she wouldn’t let anybody be involved because she wanted all the money. Everything that she knew wasn’t made by her, she called fake.” 

“It is of course an extension of the original,” he adds. “But people don’t call it that way. They say no, there’s only five works, and now suddenly there’s 15 or 150; it’s still the same thing, but you can’t call it original anymore. But it’s still the same sculpture.”

I mention how this inflation ultimately lessens the value, which affects both the artists and their families and all of their patrons; I ask if he’s ever had ethical qualms about his practice as a forger and how it can affect others in this way. “Not whatsoever, actually. I never went to an auction house myself,” he says. “I sold it to people who were interested, and they knew what I was doing, but I never made the big money. I sold it for little money, and that’s how I lived my life. I was just making art.”

He catches himself. “Not making art. I was copying art. I’m not an artist.”

Tabula Rasa - Rethinking the intelligence of machine minds

Foreword By Jason Bailey

“While rationalism would say that the machine learns emotions from the training data, romanticism instead poses that they are innate.” - David Young

“In addition to memories from a long-distant conscious past, completely new thoughts and creative ideas can… present themselves from the unconscious – thoughts and ideas that have never been conscious before. ” - Carl Jung

David Young is one of the most articulate artists working in the AI art space today. Not only can he explain the technical workings of machine learning models like GANs (generative adversarial networks), but he is also expert in sharing his philosophical views on AI and how these views impact his artistic practice.

In his important essay Tabula Rasa - Rethinking the Intelligence of Machine Minds, Young compares the untrained machine learning model to John Locke’s conception of the human mind as a “tabla rasa,” or “blank tablet.” As Young explains, Locke believed we develop rational knowledge from our experiences in the natural world. As with the human brain, it is thought that machine learning models are also blank slates until they are trained on data. But as Young points out, machine learning models are trained on biased data curated by humans, and not (as with the human brain) on the natural world:

Given that the machine is reflecting on the sensation of biased data, what makes us think that the machine can possibly be rational? We must consider that the rationality offered by these systems may be an illusion.

How are we to think of the machine learning model if, as Young proposes, it may not, in fact, be rational? Young reminds us that the Romantic movement served as an antidote to the rationalism of the enlightenment and wonders aloud if we may need a similar antidote for our understanding of machine learning.

During the Romantic period of the arts, we placed “the artist’s emotions as the most authentic source of aesthetic experience,” according to Young. Young then asks if human emotion might be a better analogy than human rationalism when comparing AI to humans.

I’d personally like to offer up a Jungian alternative to the Lockean interpretation of the mind as a tabula rasa. Rather than starting with a “blank slate,” Swiss psychiatrist Carl Jung believed that we are all born with a set of shared ideas, a “collective unconscious.” According to Jung, this collective unconscious is filled with archetypes and visual symbols that resonate universally within humans regardless of their individual experience, geography, or era.

The artistic genius, according to Jung, was one who could pluck these universal symbols from the collective unconscious and manifest them as physical and tangible works of art. According to Jung, one way to do this was to tap into dreams. Jung posits that in dreams, the collective unconscious become more accessible. But what does this have to do with art?

Jung’s ideas on archetypes, dreams, and the collective unconscious were hugely influential on twentieth-century art, in particular with the Surrealists and the Abstract Expressionists. Artists including Jackson Pollock and Mark Rothko were heavily influenced by Jung’s writing and ideas, and many argue they saw themselves as tapping into the collective unconscious to unlock these archetypes and Jungian symbols through their work.

Whether we buy into Jung’s ideas of the collective unconscious or not, I think that we can agree that the best known abstract painters tapped into something primal which continues to resonate across boundaries, be they cultural, geographic, or generational.

When David Young shared a contact sheet of work from his latest machine learning model which was trained on minimal data, I was shocked at how similar the images were to twentieth-century abstract paintings.

Take a look at the visual comparisons below with known twentieth-century abstract painters on the top rows and David’s machine learning work trained on minimal data in the bottom rows.

Kenneth Noland

Three works from Ken Noland’s Target Series

Works From David Young’s Tabla Rasa

Mark Rothko

Examples of abstract paintings by Mark Rothko

Works From David Young’s Tabla Rasa

Agnes Martin

Examples of abstract paintings by Agnes Martin

Works From David Young’s Tabla Rasa

Alberto Burri

Examples of abstract paintings by Alberto Burri

Works From David Young’s Tabla Rasa

Sure, the easy explanation here would be that Young trained his model to produce output that look like abstract paintings from the twentieth century. We have seen many other AI artists create models designed to generate convincing new imagery in the style of past artists and artistic movements. But in Young’s case, this could not be further from his goals and objectives. In fact, when I came back to Young with this connection to abstract painting, he was actually a little put off at first as his intention had been to unlock a new aesthetic through AI (as he explains in his essay), not to revisit Abstract Expressionism.

This led me to wonder, could it be possible that machine learning models also have a collective unconscious? If so, could that collective unconscious include some of the same archetypes and symbols we have as humans? Is it possible that a machine learning model, when given little to no training material (as with David’s Tabla Rasa model), produces symbols that may come largely from its own collective unconscious rather than the sparse data it has been trained on?

I’m a pretty rational guy, and I am not sure if I fully believe Jung’s theories of the collective unconscious or that machines could potentially share that unconscious with humans. But I will say that the similarities between Young’s Tabula Rasa images and the abstract paintings of the twentieth century are uncanny. The similarities go beyond a surface-level visual comparison, extending to how they are conceived and appreciated.

Both David’s Tabula Rasa model and the Abstract Expressionist painters are trying to create artwork that taps into an unconscious knowledge, a knowledge that is developed before exposure to large volumes of external data. From an appreciation standpoint, both David’s images and the art of the Abstract Expressionists resonate on the same frequency for me. I believe Young has tapped into the same well that all great abstract art springs from - even if none of us could pin down that well’s location or even fully define it.

I’d like to thank David Young for giving me the opportunity to share my alternative interpretation of his work. In my experience the best artwork prompts multiple interpretations, and the greatest artists welcome those interpretations from others.

With that, I’m honored to share David’s thoughtful essay in its entirety below.

---

Tabula Rasa

Rethinking the intelligence of machine minds

Although AI has been around since the 1950s, advances over the past decade, particularly in machine learning, have brought about a radical change in what we call AI.

With this “new AI” programmers no longer encode rules but instead build network frameworks. Then, using large quantities of data, the networks are trained and their components adjusted — or tweaked — until the system starts performing as desired.

Part of the magic, and the enigma, of AI is that we can’t open up a trained network to see what “rules” it has learned, because it doesn’t have any. Its understanding has developed organically, from the bottom up. And, as a result, it is not possible to fully know why the machine behaves as it does.

Despite this mysterious opacity, the future is being built on AI. It is poised to be the latest in the inevitable progression of technologies that brings greater efficiency and optimization of all aspects of our lives — with little regard for any unintended consequences. And so we are resigned to the inevitable disruption that will follow in AI’s wake, cynically aware of how it may be used for the benefit of those in power, and that most people will have very little say in what that future will be.

It is in this context that artists have been exploring AI — as subject matter, as collaborator, and as a medium.

Like AI programmers, artists working with AI don’t encode rules, but instead train networks. The resulting artworks look and feel different from previous forms of computer-based art for they reflect the organic messiness of their vast and inscrutable neural networks.

My own work with AI, created with GANs (generative adversarial networks), exists in contrast to the vast scale of commercial AI. Using small data sets I am exploring how AI builds an understanding the natural world, and how the images I create relate to earlier art movements and technological innovations. (More information is in my previous essay Little AI.)

In this essay, I step back from that broader project in order to examine the very basics of AI. I want to learn what is fundamentally different about art made with AI. Can I then use that difference to create unique images?

Through all of my work I ask: Do we need to develop a new concept of beauty to evaluate what emerges from working with these systems? And is it possible that a new aesthetic can give us a better, more empowered, understanding of AI?

Romanticism

Fundamental to AI and machine learning is the concept of training — teaching the machine to understand its subject. But what does it mean for the machine to “know” something?

In philosophy, John Locke (1632–1704) proposed that the only knowledge humans can have is through experience. He believed the mind to be a tabula rasa  —  a “white paper” or “blank tablet”  —  “on which the experiences derived from sense impressions as a person’s life proceeds are written. There are thus two sources of our knowledge and ideas: sensation and reflection.”¹

This sounds remarkably similar to a contemporary AI system and how it learns. Just as with Locke’s conception of the mind, the machine begins as a blank tablet, knowing nothing. It is then trained with data  —  the “sensation” upon which it “reflects”  —  in order to learn.

In Locke’s model, one develops rational knowledge from experiences in the natural world. But unfortunately, the data we provide to the machine is not neutral, for embedded within all data are human biases — the preconceptions, irrationalities, and emotions of those who collected it. Even the tools we use to collect that data — from their interfaces that control what we can see, to their underlying APIs, code libraries, and sensor technologies — each adds its own particular bias. The technology stack is, essentially, a stack of bias.

Art created with AI is not exempt from this bias problem. The images that the artist selects to train the machine — whether created by the artist, or collected from other sources — can have numerous layers of human bias.

There’s more bias here than may be immediately apparent. “Learning Nature (b38,2519,3

Given that the machine is reflecting on the sensation of biased data, what makes us think that the machine can possibly be rational? We must consider that the rationality offered by these systems may be an illusion.

In the arts, romanticism developed as a reaction against the scientific rationalism of the enlightenment. It was based on the belief that reason alone can not explain everything, and instead emphasized emotion and individualism, placing the free expression of the artist’s emotions as the most authentic source of aesthetic experience.

Given that we can’t help but use human terms to describe the behavior of AI systems — we say that the machine is “learning,” it “knows” something, it has “skills,” the word “intelligence” is inescapable — perhaps we should look for other human traits. Might machine “emotions” be a new way for us to approach and understand AI? Could romanticism be the foundation for a new AI aesthetic?

Searching for Emotions

Emotions? Really?? This is just code! But… because the technology is inscrutable we haven’t been able to develop alternative metaphors for what it is doing. We remain constrained by language that leaves us with no choice other than to describe the process as if an entity is doing it. So emotions seem to be as good as any other method to build a new understanding about AI. A linguistic placeholder, perhaps, but a place to start.

While rationalism would say that the machine learns emotions from the training data, romanticism instead poses that they are innate. What if, in order to better understand the source of the machine’s emotions, we could train it using data that doesn’t have any embedded emotional bias? Doing so might then allow us to see if the machine exhibits any emotions, and if so, reveal something about their development.

Starting with solid colors, material which is theoretically bias-free, I started training with just a single black image. The machine, seeing nothing but black, learns nothing but black. And while we might imagine it cruel to deprive it of any stimulus, I didn’t really think that I was destroying an innate intelligence.

Not much to see when training on a single black image.

When I add a second color — a second sensation — we see more activity in the machine’s mind. The images generated reveal a kind-of stress in its reflection. New textures, organic shapes and multiple overlapping grids appear, seemingly competing for dominance. There is a tension in the images as the machine tries to resolve its understanding back to the original solid color sensations.

I continue to add training colors in order to see how the machine handles the additional sensations. Through this process we see increasingly complex patterns and dynamics. It’s interesting to note that the machine occasionally invents colors that it has never seen.

The green was invented by the machine.

As the learning progresses, we begin seeing emotions. There are periods of relative calm; the gentle tranquility of a machine mind incrementally building its understanding. But then there are also moments where it seems to give up, abandoning its progress in favor of a noisy chaos. A tantrum of static. But this chaos is different from when it first started learning — informed by its previous reflections.

Were these emotions present within the machine as it was creating? Are these images the expressions of an emotional mind? I would say that is highly unlikely. But what we can be certain of is that it has a way of sensing and reflecting which is truly different from us — it has developed its own understanding of the subject.

The Materiality of AI

The work exhibits specific visual characteristics, which reflect the unique nature of the machine. This decidedly non-human expression, as well as the machine’s active role in the artistic process, is the materiality of AI. And it is from this starting point that we can explore the development of a new AI aesthetic.

We see how the neural network’s organic complexity is revealed through a fluid form of understanding. The images are made up of cloud-like shapes competing for dominance. There are forms that evolve, and edges and boundaries that never solidify. It’s as though there’s a constant tension on each side — energy waiting to burst through.

The machine’s digital nature is, likewise, inescapably present. We see multiple underlying grids that echo one another at varying scales. Sometimes they seem to hold the organic shapes within their mesh. At other times their form takes over, revealing a digital complexity. And while it may border on visual chaos, the learned knowledge of the machine is almost always apparent.

The organic and the digital elements exist in a strange harmony as the machine works within its particular capabilities. Watching the machine create, it feels as though there isn’t a single entity orchestrating the work, but instead we’re watching a negotiation among all the pixels, grids, and shapes of the image.

As we consider the materiality of AI it is worth noting that the technology, too, is constantly changing — a moving target that never stops. New methods continue to be developed which allow greater resolution and “understanding.” We could be concerned that work is obsolete soon after it is created. But, like all aspects of this AI materiality, we have no choice other than to embrace this ongoing advancement.

And so, in order to create work which embraces a new AI aesthetic, I harness this materiality. If there remains a mystery to how AI works, then I choose to create work that evokes this mysticism. And if we acknowledge the bias present in almost all training material, then I embrace that, too — training it with material which matches my own research and aesthetic goals.

It’s important to note that the work I’m showing here is not in the artistic style of romanticism, but is more akin to abstraction. I’m doing this —  working with the characteristics of abstract art — so that I can better focus on the fundamental materiality of AI. (Perhaps this might be considered a form of abstract expressionism, where I am harnessing the (romantic) emotional intensity of the machine. But I leave that for a future conversation.)

The Sublime

If we want to measure the scale of the machine’s understanding — to get a sense of how ‘intelligent’ it is — we can consider the number of possible images it can create.

In the work shown here, each image exists at a point in a 128 dimensional space. (Technically this is referred to as the “latent space.”) The number of unique images that the machine can create is the same as the number of points that are in this space.

How many is this? A lot. The number is larger than the number of atoms in the universe. If you wanted to look at every single image, even at thousands per second, it would take longer than the age of the universe.

The scale of this borders on the unfathomable. In aesthetics this is called the sublime, a term which “refers to a greatness beyond all possibility of calculation, measurement, or imitation.”²

Can we experience the sublime as we look at the images made by the machine? Are these images, though merely pixels, imbued with the sublime?

As we look at an image we know that it exists in a vast space of possible images. Each of its neighbors, corresponding to the 128 dimensions, are different in its own way. Some of these differences may be recognizable to us — the size of a grid, or the color of a shape. But others are based on perceptions knowable only by the machine.

Tabula Rasa (b67j,2149,1

There is an awe to this scale for it is impossible for us to truly understand it. To imagine ourselves in that space is to evoke a sense of terror. We would be absolutely lost, never able to comprehend where we are, or where any movement would lead.

The sublime emerges as we try to understand the mind that lies behind the work. We know there is an non-human logic to its method, but its inscrutableness leads to a feeling of magic. We can watch as the machine learns — building its understanding, always with the possibility of collapse into chaos — but there is no “why” to what it knows. Its vast mind, so different from our own, is a mystery.

Tabula Rasa (b67j,1354,1

And throughout this work there also exists a melancholy sadness. For by training the machine on such small amounts of data, I have severely limited the diversity to this vast space. A universe of similarity, the potential of our AI mind denied.

This perspective can be a foundation for viewing more complex work. For example, in the flowers and landscapes of my Learning Natureproject we can now consider how AI’s materiality is manifested. Each image also gives us a window into the near-infinite, yet very limited, mind of the machine.

An Antidote 

We see so many examples of AI work that aim to mimic human talents. Machines producing results that give the appearance of understanding and knowledge. Novelties that flatter our own creative prejudices. Clever technological vanities.

And yet we know that AI is not as it is presented. It is built upon frameworks we only marginally understand, to learn it requires vast quantities of data, its energy consumption is wasteful, and its knowledge is incomplete and unexplainable. AI’s inelegance reflects many of the worst aspects of our modern era — our obsessions with speed, consumption, and scale; traits which not only polarize our society, but threaten to destroy our planet.

But looking at AI through the lens of romanticism liberates us from trying to enforce rationality on AI. We can embrace the “emotional” qualities of how it learns and understands — and how it creates. For it is by stepping away from mainstream AI, training the machine on reduced material and making a system of limited effectiveness, that we are revealing what is unique about it.

The strangeness of the works, with their evocation of the sublime, gives us pause to reconsider the inevitability of AI. And their beauty enable us to embrace AI, for beauty is an antidote. It excites the imagination. It gives us something to hold in our minds and reflect upon. It lets us contemplate what is special about this technology.

Art gives us the space to develop new intuitions for AI. It empowers us to consider alternatives for what AI can be, and for the world we want in the future.

---

Those interested in learning more about David and his work should read the excellent writing on his personal website and visit the Kate Vass Galerie to see additional work.

Can AI Art Authentication Put An End To Art Forgery?

In June of 2006, Christie’s auction house put the 1927 painting La Horde by artist Max Ernst on the cover of the catalog for their evening sale of Impressionist and Modern art. Typically the catalog cover is reserved for the most exciting work in the sale, and Christie’s was certain they had a clear winner with this Surrealist masterpiece. As they wrote in the May 2006 press release leading up to the sale:

Leading the Surrealist section of the sale is La Horde, 1927 (estimate £2,500,000-3,500,000), one of the finest examples from a small series of strange and powerful paintings by Max Ernst, made with his then newly discovered grattage technique. “This exciting discovery is the most important work by Max Ernst to appear at auction in at least a decade,” said Olivier Camu, co-head of the evening sale.

There was only one problem… the “most important work by Max Ernst to appear at auction in at least a decade” was actually created by notorious art forger Wolfgang Beltracchi.

Wolfgang Beltracchi in studio

Someone did a pretty good job of burying Christie’s information on this embarrassing sale, but after a fair amount of detective work, I was able to track down a link to the press release and a low-res image of the catalog on Pinterest.

To be clear, I don’t think Christie’s was being negligent. I am sure they spent top dollar on due diligence to authenticate La Horde before bringing it to auction in such a bold and public way. The problem is human connoisseurship is more fallible than most in the art world want to believe or admit.

Other industries relying on human expertise and visual acuity for high-stakes decision making are realizing that AI can improve outcomes. For example, new research from Google published in the journal Nature Medicine last May shows that artificial intelligence is “as good or better than doctors at identifying lung cancer in CT scans”.

Credit: Nature Medicine Overall modeling framework.

The question is, can artificial intelligence be “as good or better” than art authentication experts in detecting that paintings like La Horde are forgeries? The answer is yes.

Two women in Switzerland, Dr. Carina Popovici and her co-founder Christiane Hoppe-Oehl, have developed an algorithm that correctly detected La Horde and several other known fakes using just a single photograph of the artworks in question. Their solution, the Art Recognition algorithm, looks at brushstrokes and produces an easy-to-read heat map that pinpoints which areas of the painting are most suspect. Their neural network is trained using machine learning and a comprehensive set of images of the artist’s works.

Art Recognition cofounders Christiane Hoppe-Oehl and Dr. Carina Popovici

Fairly skeptical and armed with a long list of technical questions (from myself and others), I set up a call with Dr. Carina Popovici to get a better understanding of how the Art Recognition algorithm works. Dr. Popovici was gracious, patient, and actually rather excited to have some challenging technical questions to address. By the end of the interview, I had nothing but admiration and awe for the impressive work the Art Recognition team has done in developing a scalable solution for attacking the world’s art forgery crisis.

Convinced others will feel the same, I am sharing my full interview with Dr. Popovici below.

Fighting Art Forgery With Machine Learning: An Interview With Dr. Carina Popovici

Art Recognition cofounder Dr. Carina Popovici

Jason Bailey: I love interviewing people working at the intersection of art and tech because they always have really interesting backgrounds. I’m always curious to learn if they started with a strong interest in art and migrated towards tech or the other way around. Please share a bit about your background with us. When did you start getting interested in art? Which artists are your favorite? Have you always been technically inclined?

Carina Popovici: Indeed, I have always been technically inclined. Back in my home country of Romania, I went to a high school with strong focus on math and physics. I was fascinated by the physics of elementary particles, which later also become my main topic of research. At the same time, I always had a strong interest in arts, not just visual, but also literature and theater. I mostly like Surrealists – Dali is my favorite, but in recent years I discovered Max Ernst, whom I find quite amazing.

Another artist I admire is Modigliani, also because of his turbulent life and tragic death. Although strictly professionally, Modigliani is probably not the best choice, being as he is the most forged artist of modern times.

Portrait of a Woman, by the Modigliani forger Elmyr de Hory, circa 1974

JB: How did you and your co-founder Christiane meet and decide to start a company together to detect art forgery?

CP: We met at Switzerland’s largest bank where we worked together for a while. Our career paths have split, but we stayed friends. And when the idea of an AI program to detect art forgeries eventually emerged, we discussed it thoroughly together. In 2017, I started working on the algorithm full time, while Christiane followed one year after. Meanwhile, it became clear that the program has a great potential and there is a huge interest on the market, so at the beginning of 2019 we decided to take the plunge and start a business here in Zurich.

JB: Your academic background is in particle physics. How does this translate to using machine learning to identify forgery in art?

CP: We have a lot of practical experience in applying mathematical/machine learning (ML) models on large data sets. The problems that we’ve worked on across the years are of very different natures, yet they all require the capacity to translate a specific problem (which can be in physics, finance, and now art) into a mathematical model and then find the right method to solve it. What is also common to all applications is the ability to find ways to handle data which are not as good as you would wish them to be. You need to continuously improve the models and use your creativity to make them solve the issue at hand. Also, neural networks were already a hot topic 15 years ago, but only in recent years new algorithms and computing power which can handle the large data at a sufficient pace became available.

JB: How does the actual Art Recognition algorithm work? What is the model actually looking at in the artwork? Are there aspects of it that are essentially a black box, or are you able to understand all aspects of the analysis?

CP: The algorithm is based on a deep convolutional neuronal network which we train to “learn” the characteristics of an artist from a set of original artworks by that artist. Once the training has been completed, the algorithm has learned with extremely high precision the characteristics of that artist. 

When a new, previously unseen artwork is being analyzed, the same type of features are collected and compared to the already stored one. If they match, the new image is labeled as original; otherwise, it is a fake. Probabilities to distinguish original from fake can be higher than 90% depending on the style and artist, and we are continuously working on improving accuracy. 

The black box character of AI algorithms is, of course, very well known. We as developers do not wish to bias the training. Therefore, we allow the network to choose itself the relevant features; the only feature which we give as “input” is the brushstroke. This comes at a cost, as it is not easy to find out what features have actually been learned. So we came up with the idea of producing a heat map which is meant to provide a visual interpretation of the decision process.

Beltracchi fake in the style of Max Pechstein

Art Recognition heat map highlighting areas of concern

JB: What advantages does Art Recognition’s approach have over traditional art authentication and forgery detection tools/approaches? 

CP: One advantage is that the analysis can be run based on images alone - no transport of the original artwork is necessary. Another advantage is the speed, as we deliver our result in just a few days or even hours, whereas traditional committees sometimes need months. The method is not invasive; one doesn’t need to remove samples from the artwork, as it is done for chemical analyses. Finally, we believe that the objectivity provided through the computer is also an advantage, especially in cases where experts disagree and have long-lasting disputes.

JB: A combination of cloud computing and open-source machine learning platforms like Tensor Flow have made it easier than ever for anyone to use machine learning. What does Art Recognition do that other people could not do on their own with these freely available ML models? What is your secret sauce, your competitive edge?

CP: We have been working on the algorithm for roughly three years now, continuously improving it and taking our experience from other fields into account. The algorithm is specifically streamlined for art authentication, and it includes a number of technical tweaks for that purpose. It works flawlessly and delivers very high accuracy. By now we have tested it on many known forgeries, we ran blind tests with art experts, and we are now working closely with art experts to fine tune the quality of the algorithm even more. 

From a non-technical side, we have invested a lot of time into building up an image database of high-quality images of original artworks, as well as a trusted network of art experts and advisors. 

Dr. Carina Popovici at work refining the Art Recognition model

JB: What materials are you using to train the model? Are you using transfer learning or have you trained your models from scratch?

CP: The input data are good quality photographic reproductions of artworks by that artist. We only use images from trusted and verified sources such as catalogue raisonnés or museum databases. Ideally, the images should be of high resolution in order to make finer details available to the algorithm. If the resolution allows it, we generate several patches per image, thus augmenting the data set and making sure that no information is lost. 

We use a convolutional autoencoder as a feature extractor. The autoencoder is trained with mean pixel-wise reconstruction loss, but we also add some terms to the loss function adapted from style transfer. The piece coming from transfer learning has the explicit purpose of capturing the main brushstroke characteristics. All other features are trained from the scratch.

JB: What type of information do you provide your clients with after the analysis is run?

CP: We provide a technical report showing the result (original/fake) together with the probability, and a heat map showing the main features which led to the decision. We also include a detailed explanation of our method for their reference.

Close up of an Art recognition heat map

JB: How accurate is the analysis you are running? What kind of testing have you conducted to validate your models? Have you been able to test known forgeries against your model? If so, how did they perform in those tests?

CP: Before launching the actual training, we split our original data set into a training and validation set. The paintings are split randomly, with 80% of images kept for training, and 20% for validation. The validation data set has the important role of providing an unbiased evaluation of the trained model, as no painting in the training set appears in the testing set, and vice versa. The validation accuracy is in most cases above 80%.

By now we successfully tested the model on several known forgeries by Wolfgang Beltracchi and Otto Wacker, and on the Lost Ales Sketchbook by Van Gogh (generally held to be a forgery). We also ran blind tests on several artists, for example, Cezanne or Daumier. In all cases, the results were as expected.

'Still life' in the style of Cézanne by forger Tom Keating

Art Recognition heat map highlighting areas of concern

JB: Some folks have suggested to me that subdividing images is good for capturing low-level features like strokes, edges, shadows, shapes, etc. But the downside to this approach might be that you lose the overall structure of the painting when you split it up. How do you think about this balance?

CP: We agree with this. That is why we include in the training both the patches, which capture the low-level features as well as the images as a whole which contain important information about the composition. We have even noticed that for some artists such as Manet, this type of overall structure is at least as important as the details of the painting.

JB: Does your approach tend to work better for certain types of artworks (drawings vs. paintings) or certain artists (Realism vs. Abstract)?

CP: So far, we have concentrated on paintings and only analyzed a few drawings, also because we mostly got requests on painting. We are working on enhancing the model to better handle drawings.

In terms of artists, the algorithm tends to work better on Impressionists, the reason being that this style is very rich in structure and brushstrokes are mostly well defined. But we are also happy with the results on other styles (Post-Impressionism, Expressionism). We actually analyzed only one painter with elements of Abstract art (Auguste Herbin). It worked very well but it is probably not enough to make a general statement. 

Auguste Herbin, Composition sur Fond Vert - 1942

JB: I imagine that the ratio of size of the actual paintings to the digital images varies across your data set. This means some images may have a high level of detail while others may not. Do you have any concerns that this could inadvertently introduce bias?

CP: We do train on paintings of very different sizes, some larger and some very small. Our approach is to scale and crop all the images (both patches and full paintings) to equal size before feeding them into the neural network. While this doesn’t solve directly the problem of actual vs. digital size, it does make the analysis thoroughly consistent. The different levels of detail are blended together in the learning process, and that is perfectly okay if we are satisfied with the validation accuracy. 

JB: I am fond of saying that data and analysis are only as good as your ability to communicate the results. I love your heat maps because they are so intuitive and easy to understand. Can you share a bit more about how you are generating them? 

CP: Thank you. The heat maps are generated by evaluating the “importance” of each pixel in the decision process – the technical word for it is “weighted evidence.” In practice, this is done by removing a small number of pixels, replacing it by the average of the environment, and recalculating the overall class probability with this modified input. By comparing this new probability with the original class probability, one can check what is the influence of this group of pixels on the probability value. The process is repeated by going through the whole image and storing the modified probabilities. These variations result in different patterns and are being assigned different colors.

The patterns emerging as explained above basically show what are the features that had the biggest weight in the decision process, the idea being to make the decision of the network more understandable for the human viewer. In general, heat spots occur in the regions which comprise more structure (corners, edges, shapes, etc.), but they can also appear on flat pieces where the brushstroke changes direction.

JB: Do you see Art Recognition as being robust enough to detect forgeries on its own or is this just the first step in the journey for collectors who want to authenticate work from their collections? If just the first step, what other steps do collectors take after you have analyzed a photo of the work in question?

CP: All tests have shown that the Art Recognition program and results are very robust. However, we think that a rigorous investigation, especially when it comes to very valuable pieces, should also include other aspects, such as provenance research. All the elements can then be put together to reach the final conclusion. 

The Art Recognition results could be, for example, used by the collectors as a precheck before approaching an art expert, but also by the art connoisseurs themselves as an additional piece of evidence when delivering their expertise.

JB: What other scientific approaches and tools do you think best compliment Art Recognition’s machine learning-based detection?

CP: Chemical analysis of pigments or carbon detection can help to find out if the materials used (canvas, paint) are from a certain time period, but that is, of course, no guarantee that the artwork is actually original. In fact, it is known that skilled forgers have prepared pigments contemporary to the artist they forged and also used the canvas of ‘worthless’ paintings from which they stripped off the color.

One aspect which is very important for us is the quality of the images. The company Artmyn, based also here in Switzerland, is using high-tech scanners and cutting-edge technology to produce very high-resolution images. This kind of input would certainly help Art Recognition to deliver even better results. 

JB: Do you see AI and machine learning eventually replacing human authenticators, or do you see AI/ML as a tool that will augment human connoisseurship?  

CP: We see AI/ML as a helping tool to increase human connoisseurship. It can be especially helpful if experts disagree or they decided not to opine on authenticity anymore. Other situations where we could help are the cases where the authentication committees have dissolved or where simply no expert exists.

JB: The art world can be notoriously conservative and slow to adopt new technologies. How has your technology been received so far? What objections have surfaced and how are you dealing with them?

CP: More or less all the art specialist we have worked and spoken with were very enthusiastic about our service and gave us a lot of honest feedback. Every encounter which happened over the past three years has helped with improving our service. 

There have been, of course, also a few situations where people showed doubt and were straightforwardly reluctant to the idea of bringing AI methods into art. As you say, these are the very conservative and rather old-school personalities. Our strategy in such cases is to simply demonstrate that it works. For example, on a painting which we analyzed, the expert was quite amazed to see not only that the result was correct, but also that the features highlighted on the heat map were matching exactly on his expectations.     

We are currently working closely with art experts not only to ensure the product we provide is really useful for them, but also to gain credibility in the market. 

JB: The world of authentication is very high stakes. A work being deemed authentic or forged can mean the difference between a painting being worth tens of millions of dollars or essentially nothing. Because of this, several authentication boards have been tangled up in lawsuits with angry collectors who disagree with their opinions. The Warhol authentication board famously ran out of money for this reason and had to disband. As a fledgling two-person startup just getting off the ground, how are you thinking about the potential for these issues? 

Self Portrait that was central to the law suit leading to the shut down of the Warhol authentication board

CP: We are indeed aware of all of the legal issues which can potentially emerge. We also know that some art experts seem to move away from “authentication” for the reasons you mention. Yet because of the (occasionally) hostile view towards authentication, I think it is even more important to have at hand a tool which is completely objective, emotionless, and has no fear of lawsuits. 

While we are continuously working on improving the accuracy, it is well known that, by its nature, no AI algorithm can be 100% certain. We ensure our clients that the analysis is done most carefully and with due diligence, but we also explain to them that we cannot provide 100% accuracy. The clients have, of course, the freedom to use the authenticity evaluation resulting from an algorithm in any way they find meaningful for their purpose, but we exclude all warranty on any further use of the technical report we provide to them.

JB: Many technical folks I have spoken to are suspicious that machine learning can correctly detect forgery based on images alone. You yourself have a Ph.D. in particle physics and your co-founder Christiane Hoppe has an advanced degree in mathematics. What are these other folks missing that the two of you see in the potential for this solution/approach? 

CP: It would be interesting to know why are those folks suspicious. In any case, teaching computers how to model large amounts of data in a meaningful way is nowadays a global trend reaching out to all domains, and I don’t see why art — or art authentication, for that matter — should be an exception. When developing an algorithm for a problem which is a rather atypical, one should, of course, take into consideration the particularities of that problem. To give you an example, when preprocessing the images, we are careful to not cut them into patches smaller than the brushstroke and we make sure to remove effects such as light spots, shadows, etc. Our network architecture is also specifically designed for this type of image recognition. So I wouldn’t say that there is a big secret that people out there are missing, but rather many small details which should be embedded at every step of analysis in order to have it working properly.

Finally, the results alone should speak for themselves, as in all tests, the algorithm worked perfectly well.

JB: What is the long-term vision for Art Recognition?

CP: Our vision is that Art Recognition will become a trusted label and that every artwork coming up on sale has been checked by our algorithm. 

JB: Why have you decided to partner up with Artnome?

CP: We are really thrilled to see that you are so enthusiastic about the same topics as we are, and it is great fun to team up and move the case forward together. Your network and images are a tremendous help for us. At the same time, we hope we can help you fulfill one of your dreams – bringing down forgery in the art market!

JB: How do people engage with you if they want to know if they have a forgery or an authentic work?

CP: For the time being, they can send us an e-mail with a photo of the artwork in question and their assumption on the artist. Our algorithm then analyzes the image, and within few days, we get back to them with a full report. We are planning to soon give the possibility to our clients to upload the images directly via our website.

Conclusion

We need to stop celebrating forgers as lovable rascals getting one over on rich collectors and start celebrating real heroes like the Art Recognition founders, Popovici and Hoppe-Oehl. The reality is that art forgers undercut our shared humanity by compromising the historical record of our most important cultural objects: Our art. I’m proud and excited to be both a partner and an advisor to Art Authentication and look forward to working with them to put an end to forgery in our museums on the art market in my lifetime.

Collecting in the Age of Digital Reproduction

Foreword by Jason Bailey

I have a handful of people I consider personal heroes in the world of generative art. Casey Reas (along with Ben Fry) is at the top of that list for inventing the Processing programming language. Through Processing, Reas and Fry put the ability to create code-based art within reach for hundreds of thousands of artists who otherwise would never have learned how to code. It’s hard to overstate the significance of their contributions. Likewise, Reas has consistently elevated digital art — and specifically, generative art — in his personal artistic practice, having shown work in prestigious museums and galleries around the world.

Now, with his important essay Collecting in the Age of Digital Reproduction, Reas again helps to lead the way, this time in how we think about collecting digital art. After reading a draft, I immediately asked Reas if I could share it as a guest post on Artnome, as this is exactly the kind of thought leadership this space needs and the kind of thinking I created the Artnome site to promote. With that, I am honored to share with you this insightful essay by Casey Reas.

---

Collecting in the Age of Digital Reproduction

In the art market, almost everything sold is an object such as a drawing, a painting, a sculpture, an installation, or a photograph, but there are some exceptions. These deviations may include a contract, a set of instructions, a digital video file, or a software file. These are all examples of art as information rather than material. For example, a purely digital photograph is different from a printed photograph or negative. A digital photo stores an image as a sequence of colors defined as numbers that can be exhibited on a screen or projected. In contrast, a printed photograph is continuous tones of gray or color values that are embedded in the structure of a unique piece of paper.

While more and more digital media works are created by artists, the art market has been hesitant to collect works that are not tangible and tactile. In contrast, other kinds of collecting not related to the art market have shifted toward digital, information archives. The most iconic shift from material to digital collecting has happened with music. Traditional art collecting and traditional music collecting have little in common. Art artifacts are typically one-of-a-kind or in limited editions in quantities much smaller than pressings of a vinyl record or compact disc. However, contemporary music collecting and the future of collecting contemporary art might eventually be more similar than different — this is the speculation of this text.

Collecting Files

The shift in collecting music was the transition where the information version of the media was in more demand than the material form — when the desire for the MP3 files of an album or track eclipsed desire for the vinyl or compact disc. The widespread catalyst for this change was a piece of software called Napster and how this software engaged people with peer-to-peer (p2p) file sharing. This was enabled by the internet and through a cultural shift in ideas about sharing media.

When Napster was released in 1999, it started a new way of collecting, sharing, and trading. Before this, people could copy tapes and compact discs for friends, but the scale, speed, and ease of sharing through p2p systems made this new way fundamentally different from what came before. The distribution system of brick-and-mortar stores vs. direct download have little comparison. Behaviors and ideas about property shifted quickly. In 1999, a search for Aphex Twin made it easy to download the Richard D. James Album and Come to Daddy. No payment was needed and none was expected by people using the software. These files were being shared by many computers around the world and they were quickly transferred to any connected computer. This same pattern, followed again and again, could lead to downloading hundreds of hours of music without any payment. This felt normal at the time, but it wasn’t legal. The response from the gatekeepers and copyright holders was controversial and punitive — people were prosecuted.

As music distribution transitioned from records to tapes to compact discs, the primary model was the same. The collector would buy a thing: a vinyl disk with grooves, a magnetic tape wound around spools, a thin polycarbonate circle encased in plastic. Tapes and compact discs were different than records because of the ease of making copies. In the case of tapes, they were noisy and copies degraded, and with compact discs, they were perfect copies. However, from vinyl to compact disks, the model of buying an object that became property was continuous. A collector could visit the used record store on Canal Street and sell her copy of Remain in Light because she owned that object. That same collector could also buy a used copy of Dark Side of the Moon that someone had earlier sold to the store. The full story of collecting records, tapes, and compact discs has another layer to it. Because these objects became property when they were bought, the owner could loan them to other people for as long as she wanted. The Queen is Dead could be loaned to a friend for a few weeks and if she liked it, that tape could be traded for Three Imaginary Boys. This whole system was flexible and legal and it worked well.

Today, people still buy compact discs, tapes, and records, but new options are more dominant and maybe more importantly, they are more convenient. Streaming services provide wide, but still limited, libraries of music. When music is purchased, it’s often digital files, either one song at a time or a full album of tracks. However, if a song is bought as a digital file for ninety-nine cents, this is different from buying a used cassette single for the same price. The cassette is property and the digital file is licensed. The license states that the music isn’t owned by the person who buys it. That person doesn’t have the right to loan it to someone or to resell it. In fact, that person doesn’t even have the right to give it to someone. This is legal, but it doesn’t feel right. This is the era of the arcane end-user-license agreement (EULA) that defines what we can and can’t do with the media and software we spend money on, but do not own as property.

The focus on music clarifies the differences between ownership versus a license to use something. Also, music was the first powerful media industry to be deeply threatened by downloadable digital media, followed by film/video, and then publishing. The last twenty years have yielded interesting shifts and now things feel roughly stabilized for the moment.

New Collections, New Media?

So far, this discussion yields a few difficult-to-answer questions: Can accessibility be a part of collecting art or is it in opposition to what attracts people to collecting? Will people still be interested in art if it’s not exclusive or a marker of status? Will the existing audience for collecting art shift or does a new kind of collection require building a new audience? Can or should the audience for collecting art scale to millions of people?

The kinds of collecting that people broadly engage with like music, cards, and comics are only possible when the work is mass media, meaning the items that people are collecting are in unlimited or large distributions. This kind of collecting might only work with things that are property and not licensed, as outlined above. The people who collect these things need to have the rights to sell them, to trade them, and to give them to others.

Some visual artists who work with media such as video and software look to this type of collecting with envy, but with conflicted feelings. These artists want anyone who would like to collect the work to be able to have it, but they also realize the advantages of the gallery system that is in direct opposition to this goal. On one hand, artists don’t want barriers of cost to prevent people from collecting the work, but artists are often specific about how to control the presentation of the work. For example, some works need to be viewed in precise lighting conditions and at a specific scale. A work with these requirements isn’t mass media, but is created for a unique physical space that can only be created within a controlled environment, as in a gallery or museum. The traditional gallery system also supports its artists through administration — taking care of sales, paperwork, promotion, shipping logistics and press communication. These services can only exist with a regular flow of sales of unique or editioned works with a high price point.

In theory, an artist might decide to make one work for a gallery and another work for mass distribution. This decision could be different for each work created, but this decision isn’t possible with traditional visual art that has an “original.” This is the essential difference with creating works that are digital media. With digital media, each so-called copy is identical to the so-called original. The very idea of an original with digital work isn’t applicable. Each copy is simultaneously as original and not-original as all the others. Therefore, an artist who works with digital media could have the option to make work for the gallery or to make mass media works that can be collected in an accessible way, but in practice this choice doesn’t exist.

At the present moment, an artist who creates works in mass media has the option to either make work available through a gallery or to publish the work for free and open online access. It’s true, there have been many initiatives to provide a third option over the last two decades, but they have either failed in succession or have not had a broad impact. I think these disappointments are the result of the inability to develop a base of collectors and this is partially a platform issue where existing technologies don’t support what people want. If the platforms do evolve and a collector base is nurtured, is it even possible for an artist to work in both ways? Can an artist succeed at playing the gallery game and making work that is financially accessible for a wider audience? Can an artist avoid galleries entirely, make only digital mass-media work, and still support themselves?

The social and emotional shift required for people to start collecting digital art, as they do music, feels like the difficult part, while the technical aspect is now possible. One way to manage the logistics is a blockchain, a frequently hyped and misunderstood set of technologies. At its most simple, a blockchain is a list. The blockchain innovation is the way the transactions are verified so they are confirmed to be accurate. A blockchain is open for anyone to read and it is distributed so everyone can access the data. The blockchain is the key to moving away from licenses toward property for digital media because the information is accurate and it isn’t centralized. Further, a blockchain can record loans and a history of sales transactions for a work. The provenance of the work is public and it’s guaranteed to be accurate; it can’t be forged.

As an example, an artist might decide to release a digital photograph to be collected. This photo can be a unique work or an edition of any number. The process starts by entering data about the work: title, medium, size, year, edition size, etc. This data, as well as a unique distillation of the photograph file is recorded into the blockchain. This “unique distillation” is called a hash or more colloquially a digital fingerprint. At this point, a record of the photo exists as a piece of property that can be transferred. If the decision is made to release an edition of ten, there are ten unique pieces of property. If a decision is made to release an edition of 2000, there are 2000 unique pieces of property. As the creator, the artist can then transfer the ownership to someone as a gift or can transfer the ownership to someone who buys it. This transaction need not be done personally or “by hand.” Software can automate this through a website or application, or an authorized representative can take care of the transactions. Once the change in ownership is recorded into the blockchain, the new owner has all of the rights to sell it, loan it, etc.

It’s important to clarify that the file itself, the artwork, can be copied and shared with others outside of the blockchain. A blockchain is unrelated to copy protection or digital rights management (DRM.) The blockchain keeps track of the transactions, but it doesn’t constrain what happens to the file; this is up to the owner. For example, if someone sells a digital photograph and that transfer of ownership is recorded into a blockchain, the original owner loses her rights to ownership, but she can keep a copy of the file. (There are ways to change this, but that is separate from the blockchain.) The new owner can do whatever she wants with the file, including the decision to copy it as many times as she wants and to distribute those copies. The people who have these copies are not the owners of the work.

This is a new kind of possession that requires a shift in ideas. One way to think about it is similar to how copyright works. Imagine a poem that is under copyright. Anyone might gain access to the poem in a book at the library or through the internet, but only the owner has the right to grant distribution, among other things. With the photograph example, other people might have a copy of the photo, but they don’t hold the rights of ownership for that image.

A Forking Path

There are two broad ways that collecting digital media might develop. One direction mirrors the existing art world through selling unique or exclusive things. This is the traditional, conservative model where a digital media file is sold as a unique artwork or as a small edition. This could be selling unique digital artifacts for a high price or an edition of a few hundred. This thinking is preceded by markets for printed work on the high end including lithographs and etchings in editions that make them more affordable than unique paintings and drawings. Numerous examples of this model for selling digital media are under development by individual artists and by venture-capital-funded initiatives. This might be a good business model if it works, but it’s not transformative for artists, the arts, or for society. It might broaden the possibility for more people to collect art, specifically digital art, but it remains exclusive. While this model will hopefully succeed and it will support artists, there’s an alternate opportunity.

The unique material of digital media presents a new opportunity for collecting that wasn’t possible with prior forms of visual art. This is only possible with digital media because of the absence of an original and the condition that all copies appear to be identical. As a hybrid between unique objects and the unique qualities of digital objects, each “identical” digital copy can have a unique and verifiable digital signature. Therefore, each version can look and perform interchangeably, but each is singular. Each version can be registered, bought, sold, loaned, and traded as a unique object, and importantly, each can be accurately verified. This supports a new model of collecting digital media where an edition can be sold in large numbers for lower cost or a large number of unique works can be managed, but each still includes all of the traditional rights of ownership. For example, edition 132/500 or edition 5243/20000 can be bought, recorded, and verified. In a different direction, an artist can produce any number of unique works in a series. To use an older idea from photography, imagine each image on a contact sheet can be sold as a unique work. Or, more likely, a set of images edited from a few rolls of film. The logistics of doing this were prohibitive with analog media, but digital media can fully support it. As another example, imagine a piece of software that generates images. Each of these images could be offered as a unique work within a series, rather the more traditional model of editioning only one of those files.

There are other unique elements to collecting digital media that weren’t possible until recently. To recede back to music for one final comparison, the thing that made collecting music on records and tapes possible was the accessibility of playback hardware. For that media to be collected, there needed to be a way to experience it. For example, record players were available in a range from inexpensive models to exotic devices for audiophiles, but access to listening to vinyl was possible on an individual, family, or community scale. At the time collecting records was emerging, digital media did not yet exist. It has only been in the last few decades that digital media playback hardware has become accessible and high-enough quality to engage large groups of artists to create work for it. Hardware that plays digital media files recently shifted in a positive direction with the development of slim, high-resolution flat screen technologies and more powerful home media players, including video game systems and hardware embedded directly into televisions. Now, millions of people have easy access to digital media files through the internet that can be experienced through mobile phones, tablets, and new kinds of televisions. In addition to the devices that people already have, there’s a growing number of hardware manufacturers who are focusing specifically on designing screens for the display of digital art.

To have a broad impact, a new model of collecting media art will need to be extremely accessible. It will need to work broadly across cultures, incomes, and age; it needs to be approachable for millions of people. This new model will need to be decentralized. It must support diverse groups curating and selling work without centralized control. The proprietary “store” systems created by contemporary tech giants are the opposite of what is needed. Individuals and small groups should be able to create their own “galleries” or “labels” and individual artists should have the freedom to release and control their own work directly.

As long as artists continue to live within capitalist economies, a new form of collecting digital media will need to provide financial support for the artists that participate. It will still be an option for artists to post work online without compensation, but an alternative should exist for people who want to pursue it. The entrenched gallery system doesn’t support this kind of work and the model of finding grants, fellowships and commissions for digital work supports too few. Similarly, some artists have been successful on different crowdsourcing platforms, but those systems and audiences favor a narrow range of practices and sensibilities. Artists who create digital media do not have enough opportunities to support themselves with their work.

The Dematerialization of the Art Object

Today, even though art is still associated with wealth and traditional media, the need for either to be required for art was dismantled deep within the twentieth century. With certainty, it can be asserted that the status of an object as art is unrelated to scarcity, cost, or materials. It’s time to proceed with what was started many decades ago. Artists have long used mass media and have explored new materials. For example, it’s been over fifty years since On Kawara started the I Got Up series by sending postcards stating the time he woke up each day and twenty-seven years since Felix Gonzalez-Torres placed a pile of candy in the corner for the museum audience to take for his Untitled (Portrait of Ross in L.A.) The spirit and legacy of Dada, Fluxus, and net.art remain relevant today.

It’s time to build on those ideas and to move forward, rather than to regress or maintain nostalgia for the power relationships and dependencies of patronage and exclusion. In the future, a native digital media collecting ecology might be radically different from the current art market, but this new kind of collecting probably won’t affect the existing market. The current art market is a niche that will continue to evolve, but not in this direction. If a new model succeeds, it will emerge as new territory.

Acknowledgements

Thank you to Sean Moss-Pultz and Casey Alt for an exchange of ideas on this topic over a number of years. Thank you to Addie Wagenknecht and Kerry Doran for reading drafts and contributing excellent comments and questions.

Breeding Paintings With Machine Learning

This week at the Ars Electronica festival for art and technology in Linz, Austria, Artnome friend and artist Alexander Reben will be debuting a photo booth that mixes faces together in latent space. Reben calls the project Latent Face, and according to him, “It looks into the stylegan flicker model for the latent vector closest to the input (which is two or more images of faces), then the vectors are operated on to find the midpoint vector and produce the output.” That’s a fancy way of saying it creates something of an average from the two or more faces provided as inputs to produce a convincing-looking digital offspring.

Grant Wood, American Gothic - 1930

As an art history nerd, I of course had to ask Alex if it was possible to cross-breed painted portraits. He obliged, and his results are pretty great.

First up, Grant Wood’s American Gothic, maybe the most iconic double portrait of all time. What many people may not know about American Gothic is that the woman in the painting is the artist’s sister, and the man was actually not Wood’s brother-in-law, but his dentist. So this iconic couple were never actually a couple at all, but that did not stop us from wondering what it would look like if they had a child that grew into an adult. You can see the results below and the animation at the top of this post.

Grant Wood, detail from American Gothic - 1930

Grant Wood, detail from American Gothic - 1930

Alex Reben, American Gothic Hybrid

One of my favorite real-life artist couples is Frida Kahlo and Diego Rivera. It is well-documented in her work and her journals that Frida had always wanted to have a child with her husband Diego. Tragically, Kahlo suffered horrific injuries in a bus accident when she was just 18 that made that difficult.

I asked Alex if we could find the latent vector between the striking self-portraits of Kahlo and Rivera below.

Left: Frida Kahlo, Self-Portrait with Monkey - 1938; Right; Diego Rivera, Self-Portrait - 1949

Alexander Reben, Midpoint between Kahlo and Rivera self-portraits

You can see the animation moving through latent space between the two self-portraits below.

Regular Artnome readers may recall that we averaged every painting by Van Gogh into a single image. For that image, Artnome data scientist Kyle Waters set the aspect ratio for all the portraits to the same dimensions and averaged out the color for each pixel location. It produced the ghostly-looking portrait below.

Kyle Waters, Average van Gogh Portrait

Unlike Water’s pixel average portrait, Reben’s averages (or more precisely, midpoint images) from StyleGAN are creating entirely new facial structures, not just averages of pixel color values. In this way, it feels more like breeding paintings than creating a visual average. Reben pointed out that, “You can often see this in the hair and facial hair, which many times is a different style than either of the two input images.”

Vincent van Gogh,

Vincent van Gogh,

Alexander Reben,

So how does this all work? Okay - let’s start here. Right now you have a digital twin for every photograph ever taken of you in the past, or that will ever be taken of you in the future. These twins exists in a near-infinite matrix of possible faces (called latent space) produced by a deep-learning algorithm developed by Nvidia called StyleGAN.

Like all GANs (generative adversarial networks), StyleGAN is comprised of two neural networks: a generator and a discriminator. In this case, Nvidia trained the generator on 70,000 high-quality images of human faces at 1024×1024 resolution taken from Flickr until it was able to fool the discriminator into believing the images it produced were real human faces. You have probably already seen projects that use this model like This Person Does Not Exist, which became very popular earlier this year.

Reben’s project is interesting to me for two reasons. First, it focuses more on the ability to perform mathematical functions on the faces, and second, it personalizes this technology by finding and cross-breeding images that look like the participants in his photo booth. 

In other words, seeing hyper-realistic faces of people that don’t really exist is cool, but discovering I have infinite digital twins that can breed to make offspring with everyone else’s digital twins… well, that is much cooler in my book! 

As an example, here is one of my digital twins:

The image on the left is a photo I took with my phone and sent to Alex; the image on the right existed in latent space long before I ever took the photo of myself (in the same way my date of birth existed as a series of numbers before I was born).

To help describe how the process works a bit more, Alex and I took one for the team and made our own handsome man-baby together (whom we call “Jalex”).

Let’s walk through the steps behind the creation of our little bundle of joy.

Step One: Take Two Photographic Portraits

Step Two: Find Our Twins In Latent Space

Reben is using code called StyleGAN Encoder that that identifies and locates the latent vector (the digital twin) within latent space that most resembles the input image. You can actually see it honing in on the right image in latent space in the gifs below.

Step Three: Find the Midpoint

Once Alex located the location of our digital twins in latent space, he could then travel to the arithmetic midpoint between the two and find our “average,” or genetic offspring. I have to say, we produce a pretty handsome looking man-baby.

What Other Math Can We Do?

Again, because our digital twins are described numerically, there are several other mathematical operations that can be performed. If you subtract Alexander’s face from mine, you get this happy little creeper.

And likewise, if you subtract me from Alexander, you get this teen heartthrob.

Kind of weird, right? Whatever features of ours that have been subtracted from the other were apparently pretty important to us looking like our handsome selves.

Reben finds that people respond more to images of people that they know, like the interpolation gif between our two most recent presidents. I find it funny that latent space has no real answer for Trump’s hair.

However, even more engaging than combing pictures of famous people is having your own image combined with someone else. Reben is aware of this, and it is the core idea driving his photo booth at Ars Electronica. For example, I had Reben create a digital offspring for me and my wife. The results were kind of astonishing and bizarre-feeling in a way that you could only feel if it was your image.

I’m sure Reben’s photo booth will be a hit at Ars Electronica. If you happen to be there, be sure to swing by and say hello to Alex!

Generative Photography - An Interview With Gottfried Jäger

Foreword by Jason Bailey

It may be hard to believe, but there was a time in the not too distant past when photography was viewed with as much prejudice and suspicion by the the art world as digital art is today. Many believed that because photography involved chemicals and machinery instead of the “human hand and spirit,” it could not be considered on par with drawing and painting. Instead, photography was seen as being closer to the fabrics being mass produced by machines in the mills than it was to the fine arts. The influential French poet and art critic Charles Baudelaire even worried that photography would erode the very foundation of the fine arts:

As the photographic industry was the refuge of every would-be painter, every painter too ill-endowed or too lazy to complete his studies, this universal infatuation bore not only the mark of a blindness, an imbecility, but had also the air of a vengeance. I do not believe, or at least I do not wish to believe, in the absolute success of such a brutish conspiracy, in which, as in all others, one finds both fools and knaves; but I am convinced that the ill-applied developments of photography, like all other purely material developments of progress, have contrib­uted much to the impoverishment of the French artistic genius, which is already so scarce.

Just a few years after making this bold anti-photography statement Baudelaire’s stance on photography must have softened a bit as he chose to sit for the photographer Étienne Carjat for the portrait shown below. It would, however, take more than one hundred years before a popular market for photography as fine art would develop.

Étienne Carjat, Charles Baudelaire - 1863

In the early 1950s, photographer Henri Cartier-Bresson, often referred to with reverence as “the father of photojournalism,” published his book The Decisive Moment. In it, he differentiates photography from painting by pointing out that painting can be slowly worked over time in contrast to the photographer, who is capturing a precise moment which will never be reproduced or occur in the same way again. As an extension of this ethos, Cartier-Bresson was against manipulation and cropping of photographs in the darkroom. The art of photography was in the framing of unique events in real time through the camera’s view finder, not in the manipulation of chemicals and machinery.  

Henri Cartier-Bresson, Behind the Gare Saint-Lazare - 1932

As Cartier-Bresson shared in a 1957 interview in The Washington Post:

Photography is not like painting. There is a creative fraction of a second when you are taking a picture. Your eye must see a composition or an expression that life itself offers you, and you must know with intuition when to click the camera.

That is the moment the photographer is creative. Oop! The Moment! Once you miss it, it is gone forever.

Against this popular conception of photography as means of faithfully capturing fleeting moments of reality, and decades before a popular market for photography as fine art would emerge, a group of radical photographers posed a direct challenge to Cartier-Bresson’s notion of the “decisive moment.” They created non-representational photographs that could be mechanically reproduced through a series of algorithmic actions. Rather than capture moments of reality, these photographers were producing their own realities or “aesthetic states” through mechanical and chemical manipulation. Though they owed a debt to concrete art with its strong emphasis on geometrical abstraction, these photographers were more influenced by media theory, cybernetics, and semiotics than the traditional art history and art theory of their time. They borrowed the term “generative” from German semiotician Max Bense’s book, Generative Aesthetics, to describe their approach, eventually settling in on “generative photography”. The generative photographers developed repeatable programs for photographic image making which predated and then evolved side by side with early generative computer-based art.

While generative photographers like Gottfried Jäger, Herbert W. Franke, and Hein Gravenhorst may not be household names yet, they should be. I believe generative art is the most important art of our generation as it makes the digital revolution visible and best reflects all that separates us from those who came before us. First through photography and then through computers, the ideas and work of these artists form the foundations for all of generative art.

Porträt Gottfried Jäger, Paris photo - 2017 - photographer unknown

One reason you may not know of these great pioneers is that although they were prolific and have written hefty volumes of theory explaining their approach, few of these books have been translated into English. We are seeing renewed interest in this work, with several generative photographers having been shown in the recent exhibition Shape of Light at the Tate Gallery London and in the current show Automat und Mensch at the Kate Vass Galerie in Zurich. I was lucky enough to co-curate the latter show with generative photography expert Georg Bak. It was through Bak that I learned of these pioneers and their important contributions. Bak has had the foresight to interview these important artists while they are still available to shed light on their important work and contributions to the history of generative art. 

What follows is the first of several important interviews between Georg Bak and the early pioneers of generative photography. In this interview, Bak speaks with Gottfried Jäger, who is credited with naming the movement “generative photography” and is one of its most important practitioners. We at Artnome are thrilled to feature Bak’s interview with Jäger in English for an audience who may not otherwise learn of his important work and the ideas behind the origins of generative photography and the generative art movement.

Interview with Gottfried Jäger by Georg Bak

Gottfried Jäger presents his pinhole structure 3.8.14 F 2.6, 1967, b/w camera - photograph by Ursel Jäger

Georg Bak (GB): In the exhibition Automat und Mensch, we have curated a show around artificial intelligence within its art historical context. The title of this exhibition refers to a book by Karl Steinbuch from 1961, which you have mentioned to me several times in our numerous conversations. Can you describe to us how you discovered this book in the early 1960s and to what extent it was influential for your artistic practices?

Gottfried Jäger (GJ): This book was recommended to me by Hein Gravenhorst, a longtime friend of mine and fellow artist in generative photography, who brought it to my attention in the mid-1960s. I met Hein for the first time through Manfred Kage, whom I was visiting for an interview in Winnenden near Stuttgart. Gravenhorst and Kage were successfully collaborating in the making of "polychromatic variations." Our first meeting was very fruitful and the beginning of a long-term collaboration which lasted up until now.

Karl Steinbuch: Automat und Mensch. Kybernetische Tatsachen und Hypothesen. Springer Verlag, Berlin-Heidelberg-New York, 3. ed., 1965. First ed. in 1961.

Herbert W. Franke: Kunst und Konstruktion. Physik und Mathematik als fotografisches Experiment. F. Bruckmann Verlag, Munich, 1957

The book Automat und Mensch - Kybernetische Tatsachen und Hypothesen was seminal for my way of thinking. It was written by an engineer, and his language was congruent to mine, both being knowledgeable about new technology in those days. This book got me familiar with the term "information," which helped me to get away from the myth of the "Geistigen" (spiritual) and metaphysics in art. Steinbuch stated: 

What we are observing regarding intellectual functions is reception, processing, and transfer of information... It isn't evident or likely at all that you need further requirements than physics in order to explain intellectual functions.

According to this statement, a human brain and a hand weren't necessary anymore for an artistic expression. It could be also a technical apparatus. A camera or a computer were capable of achieving intellectual results. Nowadays these kind of questions have become obsolete and photography is undoubtedly acknowledged as a form of art. But in those days it was not the case, and it had to be legitimized and justified.

GB: Another book that you often talked about is Kunst und Konstruktion by one of the earliest pioneers of computer art, Herbert W. Franke, with whom you are still befriended. How did you meet him and how was your exchange?

GJ: I came across that book at the Bibliothek der Staatlichen Höheren Fachschule für Photographie (State College Library for Photography) in Cologne, where I had studied photography from 1958 to 1960. That little book from 1957 published at F. Bruckmann in Munich instantly woke my interest. It was not only the title and a nice cover of the book, but especially its subtitle, Physik und Mathematik als fotografisches Experiment (Physics and Mathematics as a Photographic Experiment). That was a credo, a program. And nowadays I realize: This approach has led my way throughout my whole career as an artist.

Double page 38/39 from Kunst und Konstruktion: (l.): Herbert W. Franke and Andres Hübner, Pendulum Oscillogram, Contaflex, photographed from a screen; (r.): idem, Verdrillter Gummiring, camera photograph; both photographs not dated (c. 1957)

I began my approach to this completely new and fascinating world through imitation. I was reproducing some of the images that I had discovered in the book - for example, the photograph Verdrillter Gummiring (Twisted Rubber Ring) by Herbert W. Franke and Andreas Hübner. But a few years later in 1966 during my summer vacation at Bodensee, I decided to contact Herbert W. Franke and asked for an appointment ... granted! This was followed by an exciting first meeting in Wolfratshausen near Munich. We became friends and have been cooperating on several occasions. The first results of our cooperation were the exhibition Generative Fotografie in 1968 and the book Apparative Kunst. Vom Kaleidoskop zum Computer, which we had published together in 1973 with DuMont in Cologne and had gained wide international recognition.

Gottfried Jäger: Untitled, Hommage à H. W. F., camera photograph, silver gelatine print, 16 x 10.5 cm, 1958

GB: In 1968 you organized the exhibition Generative Fotografie at Kunsthaus Bielefeld which can be regarded retrospectively as the manifesto of this important art movement. Can you share with us how you came up with this exhibition and what the reaction was within the art scene at that time? 

GJ: Let me tell you the whole story. In 1965 I was invited to show my "Lichtgrafiken" (light graphics) at the group show Fotografie '65 in Bruges. I was visiting the show alongside some students of my class at Werkkunstschule Bielefeld. Although the succinct title of the show wouldn't suggest any specific expectations, this exhibition was a counter-reaction to another exhibition taking place at the same time, also in Bruges, an exhibition organized by Karl Pawek in conjunction with the German magazine by Stern,Weltausstellung der Photographie: Was ist der Mensch. While the latter was the "grand opera," our show Fotografie '65 was rather a chamber concert. A sensational realism was countered by a radical formalism. This is how we experienced it on spot and how I later featured it in the article Signale eines neuen Programms at the art magazine Foto-Prisma. Some works that captured my attention were landscape studies by the Belgian artists Yves Auquie and Robert Besard, close-ups by Anton Dries, nudes by the Swiss artist René Mächler, and especially the filigrane "chemigrams" by Pierre Cordier and the elegant "photograms" by Kilian Breier, as well as the "Lichtstrukturen" (light structures) by Roger Humbert. The latter inspired me to organize my own group exhibition with a special focus on the formative potential - concrete as well as constructive - of photography. This was the moment when the idea was born to organize an exhibition titled Generative Fotografie which would take place three years later at Kunsthaus Bielefeld.

Max Bense: Aesthetica. Einführung in die neue Ästhetik. Agis Verlag, Baden-Baden, 1965

It took me a few travels to visit this small circle of artists until I finally spoke about the idea to the former director at Kunsthaus Bielefeld, Joachim von Moltke, in 1967. I presented him my "Lochblendenstrukturen" (pinhole structures) alongside some photographic works by Kilian Breier, Pierre Cordier, and Hein Gravenhorst. Surprisingly, he instantly agreed to make this exhibition happen in January, 1968, and it was to become the last show at the beautiful Kunsthaus Bielefeld. Soon after, the building was torn down and instead the Bielefeld Kunsthalle was built at another place.

For the title of the show, I was inspired by the last chapter of the book Aesthetica by Max Bense. His theories were summarized under the title Projekte generativer Ästhetik. The following sentences were seminal for me: 

Generative aesthetics is an aesthetic by creation. It allows a methodical creation of aesthetic states by dissecting the creation in a finite number of specifiable and describable single steps. 

So basically it was not about creating "high art," but about generating "aesthetic states." This was a key element for a new discipline in the art. I called up Herbert W. Franke and asked him for his opinion regarding the exhibition title Generative Fotografie, and spontaneously he thought, "It sounds good.”

Generative Fotografie, invitation card for the exhibition at Kunsthaus Bielefeld, 1968, designed by Heinz Baier

The exhibition was quite successful. Local press reacted in a rather positive way. Even such prominent artist colleagues such as Otto Steinert affirmed during a board meeting of the Deutsche Gesellschaft für Photographie (German Society for Photography) which took place in the exhibition space of Generative Fotografie. Follow-up exhibitions around this topic took place at Galerie Spektrum in Hannover, in Antwerp, and at other places. The studies class of Kilian Breier from HFBK Hamburg came to visit the show. Among the students was Karl Martin Holzhäuser, a later colleague and friend of mine at Fachhochschule Bielefeld (Bielefeld University of Applied Sciences). But it was only after the manifesto-like book Generative Fotografie from 1975 which I co-published with Holzhäuser and Franke when theory of this art movement became more widespread in the broad public.

Exhibition view of Generative Fotografie at Kunsthaus Bielefeld, 1968; included works by Kilian Breier (l.), Pierre Cordier, Hein Gravenhorst (l.) and Gottfried Jäger (r.; curator); catalogue text by Herbert W. Franke

GB: Could you define what generative photography means and how this term evolved? Looking back in art history from our perspective, wouldn't it be almost evident that generative photography was the origin or source of digital photography? 

GJ: I have already mentioned the definition of generative aesthetics by Max Bense. We discovered therein a new artistic approach in the handling of photographic techniques. Nowadays the term is widely acknowledged and can be retrospectively seen as an early artistic form from the mid-1960s, which was exploring for the first time the synthesis between light images and data images, an interplay between cameras and computers. The imagery was on an abstract level, without any reference to objects or symbolic signs. It was all about generating aesthetic states, where natural analog media (light, camera, light-sensitive support) and mathematical instruments (number, computer, program) were interacting.

But I have to clarify that in the early days, generative photography was not made with computers. They were primarily light images made through experimental photography. But we have connected the techniques with the methods of computers, meaning numeric programs. Our method was also a rejection of the single image and the "decisive moment" (Cartier-Bresson) towards a serial and logically reproducible work series.

Ursel and Gottfried Jäger with “pinhole structures” (Lochblendenstrukturen), 1967, from the exhibition Generative Fotografie; photograph by Günter Rudolf, 1968

My first definition of generative photography reads as follows: 

"Generating aesthetic structures on the basis of predefined programs which are processed through photo-chemical, photo-optical, or photo-technical operations in order to achieve an ideal and functional connection within all involved elements that form the composition of the aesthetic structure."

This was the opposite to "subjective photography" (Steinert) or "totale photographie" (Pawek). It was an alternative to these very dominant positions in photography at the time.

Insofar I wouldn't dare to say that "generative photography" was the origin of digital photography. Digital photography is a general expression for a chip-based optical image technique of our time. The origin goes back to Steven Sasson and his patent at Kodak in Rochester in 1975.

Herbert W. Franke (l.) and Gottfried Jäger (r.) at the exhibition Wege zur Computerkunst, Kunsthalle Bielefeld, 1971; photograph by Ursel Jäger

GB: In the 1960s there was an international computer art movement with the New Tendencies in Zagreb and the extensive exhibition Cybernetic Serendipity at the ICA in London in 1968. Was there any awareness for this rather marginal movement at that time? Were there any connections to, for example, the ZERO or the Op-Art movement? 

GJ: We were active participants of these movements. We contributed with lectures and publications as well as presentation of our artworks in exhibitions. So for example, Gravenhorst and myself were exhibiting in Zagreb, and we also took part at the symposium. Kage and Breier were both part of the ZERO group. Nevertheless, the art establishment didn't truly take notice. 

Regarding the acknowledgement of our work (generative photography) as a legitimate form of art, we had to fight on two levels against the perception of "photography as illegitimate art" (Bourdieu, 1983). It was only in 1984 when photographs were being accepted legally as works of visual art according to German copyright law. On the other hand there was a general skepticism whether computers could be regarded as an artistic medium. The rather cold outputs of computer art couldn't withhold the traditional conception of high art. So as photographers, we were basically not regarded as real artists, and as generative photographers, we were not regarded as real photographers.

GB: Was there a market for computer art? Who were the collectors and gallerists dealing with that type of art? 

GJ: In the 1960s, there didn't exist a significant market for generative art. For photography it was Käthe Schroeder who opened Galerie Clarissa, the first gallery for photography in Germany. This gallery also specialized in experimental and generative art. When she donated her collection to the Kestner Museum in Hannover in 1968, the list of included artists was almost an encyclopedia of the avantgarde movement in photography in the 1960s. Among the artists were Théodore Bally, Monika von Boch, Hein Gravenhorst, Heinz Hajek-Halke, Roger Humbert, Manfred Kage, Gottfried and Ursel Jäger, René Mächler, Floris Neusüss, Heinrich Riebesehl - just to name a few. 

GB: In the exhibition "Automat und Mensch," we are presenting a series of your pinhole structures from 1967. Each artwork is titled with a code such as, for example, 3.8.14 D 3.1. Can you describe the underlying method and idea behind this series and how you developed your concept of pinhole structures further at the dawn of the digital age?

GJ: The first so-called "Lochblendenstruktur" (pinhole structure) was conceived on the light table where I was experimenting with transparent superimposed slides containing patterns and dot matrices. The results were peculiar geometric patterns and shapes which I have further developed by using optical devices.

Multiple pinhole camera which can be rotated

The program and steps in order to make pinhole structures I have described later in a diagram, Entscheidungsstufen beim Aufbau modifizierter Lochblendenstrukturen, in 1976. This diagram was recently published in the exhibition catalogue Shape of Light by Tate Modern in 2018, and it illustrates the steps which lead to a certain pinhole structure. The code is equivalent to the steps as, for example, in 3.8.14 F 2.5, 1967, depicted in the diagram.

Although the first numeric character stands for the aparative system (multiple pinhole camera), the second one for the geometry of the multiple pinhole aperture, and so forth, I wouldn't want to describe all the parameters that lead finally to an "aesthetic state." But maybe you can comprehend now that I was basically building step by step a rational "history" of a photo- and data-based process.

Gottfried Jäger: Entwicklungsstufen beim Aufbau modifizierter Lochblendenstrukturen der Serie 3.8.14., 1967, diagram, drawing on paper, 70 x 100 cm, 1976

In the 1990s I started to use computers in my work. My friend and IT engineer Peter Serocka, at that time director at the visual laboratory of the mathematical faculty at University of Bielefeld, has developed a computer program based upon my optical system of two superposed pinhole structures - which he called matrices - which enabled him to imitate my pinhole structures. At the same time I shockingly became aware by reading a magazine that, independently from us, a Japanese group of mathematicians came up with the same results. So we got in touch with each other and we started a fruitful technological exchange with them, which resulted in reciprocal publications. This was basically my first access into my artistic practices with computers.

It was high time to deal with the new system also practically and not only theoretically. And as it is often the case when a historical shift of a system takes place, the shift started again through imitation and simulation of the pre-existing. I tied in with the pinhole structures and edited them with inherent instruments of computer technology and developed new image orders. Nowadays I am working in a hybrid way, a mix between conventional photographic techniques as well as cybernetic, computer-based instruments. 

GB: In the beginning of generative photography, it was difficult to categorize this new form of photography while in Europe, especially, two main directions of photography were dominating the discourse on photography. On the one side there was subjective photography and the Otto Steinert school; on the other side, Karl Pawek's conception of documentary photography. You mentioned to me several times that leading art theorists were rather suspicious about generative photography. Retrospectively, one can say that generative aesthetics is widely used as a term among contemporary artists. Can we say that generative photography has finally entered into the art historical canon? 

GJ: I have already referred to these two names in a previous question. Both were antipodes at the time when I developed the idea of generative photography. On the other hand, they also enhanced a disruptive evolution. Nowadays it is almost unimaginable how intense these grave battles between the two sides of realism and formalism, between document and experiment, were waged. It was a power struggle for interpretive sovereignty of what photography fundamentally was - its ontology - assuming that it would be possible to define this ambiguous notion exclusively.

An important step in the evolution on the subject was the exhibition Ungegenständliche Fotografie in Basel in 1960, which led to the exhibition Konkrete Fotografie in Bern in 1967, organized by an independent group of four Swiss artists referring to their idea of a self-referential form of photography to "concrete painting" (van Doesburg, 1930) and the Swiss concrete artists (Max Bill, etc.). One year later this was followed by the exhibition Generative Fotografie in Bielefeld. This is a short review of the history.

Nowadays the term "generative photography" can be found in almost every encyclopedia on photography. Currently, it will be used in the context of "pre-digital art," mentioned as such in the exhibition catalogue Shapes of Light - 100 Years of Photography and Abstract Art at the Tate Modern in London in 2018.

Exhibition view Shape of Light. 100 Years of Photography and Abstract Art at Tate Modern, London 2018. Gottfried Jäger, Lochblendenstrukturen (pinhole structures), 1967

According to Wikipedia, it is described as "attempt for photography by using the tools of emerging computer technology and its graphic to enable a new form of artistic imagery". Fifty years after its introduction, it is an established category of photography and has entered in the art historical canon. 

GB: Looking back 50 years, one can say that you are the father of generative photography, and you have published so many important books and essays. A selection of your most important writings have been assembled by Bernd Stiegler in Gottfried Jäger. Abstrakte, konkrete und generative Fotografie. Gesammelte Schriften, which was published in 2016. What would you consider as the theoretical basis of generative photography? Which essays (theory) are essential?

Vilém Flusser and Gottfried Jäger at 5th. Bielefeld Symposium on Fotografie at Fachhochschule Bielefeld, 1984, photography by Ralph Hinterkeuser

GJ: Partly I have already responded to this question through my previous statements. I could add to it a few names and thoughts. Based on Noam Chomsky's "generative grammar" as well as the thoughts on "generative aesthetics" by Max Bense and "cybernetic aesthetics" by Herbert W. Franke, the sources of generative photography derive not primarily - as opposed to "concrete photography" - from art historical sources, but rather from linguistics and philology, mostly from semiotics. My colleague from Bielefeld, Kirsten Wagner, has published an important essay in the exhibition catalogue Die Bielefelder Schule. Foto Kunst im Kontext in 2014: Generative Ästhetik im Kontext der Bielefelder Schule. This text contains a much more comprehensive answer to this question than I am capable to give you at this point. From my point of view, I would like to mention Herbert W. Franke as my first spiritus mentor and Vilém Flusser as my sharpest critic. Both have fundamentally influenced my career as an artist. I learned from Franke how to play with the machine - and from Flusser how to play against the machine. Today I would count the philosopher Lambert Wiesing and the literary theorist Bernd Stiegler among the theorists who help me on my way.

This interview took place on 29th July 2019 in German and was translated into English by Georg Bak

The Generative Portraiture of Espen Kluge

“The force of art lies in its immediate influence on human psychology and in its active contagiousness.” - Naum Gabo

Espen Kluge, ash when he arrives, every time - 2019

I have a deep and well-documented love for generative art. I’ve gone as far as saying that generative art is the most important art of our generation, as it best reflects the digital transformation that so deeply separates us from all that came before us.

However, the nature of generating art with code typically leads to results that are highly geometric, abstract, and non-representational. And my first true love in art will always be portraiture.

So when I discovered the arresting generative portraiture of Norwegian artist Espen Kluge several weeks ago, I was immediately taken in.

Espen Kluge, makes great choices while angry - 2019

I love portraiture because evolution has trained us to read the nuance of human facial expression with a higher degree of sensitivity than any other visual input. From finding a mate to sensing a foe, our ability to read the subtle expressions of others is critical to navigating many of the most important and meaningful aspects of life.

Our supercharged sensitivity to these facial signals creates a playground for portrait artists who can combine, bend, exaggerate, and break the signals to manipulate us and make us feel with heightened intensity.

Kluge does this masterfully in his new series of portraits titled Alternatives which source and alter photographic portraits to tell stories alternative to their origins.

Espen Kluge, superclear when I say it’s risky business - 2019

Though they are composed of two-dimensional vectors, Kluge’s portraits feel structural, almost architectural. They immediately reminded me of a blend of the dynamic sculptures of Russian constructivist Naum Gabo mixed with a hint of mid-century modern symmography (aka ‘70s string art).

Linear Construction No 2 Naum Gabo, 1950

1970s symmography string art

With twenty years’ experience working in digital media, if I see something beautiful and I can’t figure out how it was made, I feel deeply compelled to reach out to the artist.

I set up a call to speak with Espen Kluge about his work and caught him during a visit to Sweden to help his fiancee prepare her apartment for sale.

I was honestly a bit surprised to hear Kluge speak perfect English because I had assumed his deadpan titles like “makes great choices while angry“ and “cruise ships and police cars“ were in part a function of English not being his first language. He explained that he gets his titles by pulling snippets from the text associated with the source images he uses to create his portraits. He was a bit apologetic about this, and wondered aloud if he should spend more time on the titles. I assured him the titles added just the right amount of mystery and fun while avoiding the trap of becoming overly pretentious and serious.

Kluge shared that he grew up in Norway, where he still lives and works today. He earned a bachelor’s degree in fine arts but then pivoted and studied composition in music for TV and film for his master’s degree.

Kluge’s music, like his portraits, is psychologically nuanced and exploratory. You can hear his soundtrack for the 2018 television show Who Killed Birgitte? on Spotify. I listened to it while writing this post and highly recommend it. It’s my new go-to music for writing.

I wanted to know how someone who went to school for visual art and then film scoring got into programming and code-based art.

Kluge explained that while studying for his master’s degree, he had developed an obsession for an idea that he could not shake:

What if there was a keyboard that had keys that lit up not based on anything that was pre-composed, but in real time, based on your own improvisation?

Kluge became so obsessed with this idea that he taught himself how to code from scratch at the age of 27 so he could start his own company and build his invention.

Drawing from the patent awarded to Kluge for his improvisational keyboard

Though his improvisational keyboard was programmed in Objective C, Kluge often took breaks to create more visual projects in Processing and eventually JavaScript. One such project was based on the idea of making an interactive logo for his website. The logo was a portrait, and Kluge wanted it to take on the colors of the pages it appeared on and to “do something interactive,” as he described it.

Though he failed at his initial objective, Kluge ended up creating a really cool piece of code that took in photographs as input and then output what would become the initial foundations for his colorful vector-based portraiture series.

Original logo project image that spawned Espen Kluge’s current portrait series

At the time, busy getting his master’s degree and building his company, Kluge decided to put the logo project on the back burner, where it remained for many years. It was only this past January that he rediscovered the project and began refining the code again.

I asked Kluge how his code works. He explained that the code loops through all the pixels in a raster image and chooses some at semi-random, then engages those pixels by putting lines between them. He added that he is able to adjust five or six variables to do things like focusing in on one color more than another.

Espen Kluge, cruise ships and police cars - 2019

Kluge admits the code is not super clean, but through his manipulations, he has built his own tool for making art. The results speak for themselves.

Kluge would like to get to a spot with the project where he can do less and less work on the images so that it becomes more generative. But for now they require a lot of creative intervention to yield the results he is happiest with.

Espen Kluge, they don't have this for humans - 2019

For now, each image can take Kluge from ten minutes to an hour to create. The process requires that he source several input images, as some just come out “flat and boring” and he is not always sure why. But over time, his intuition has improved as to what makes for better input images. For example, photos with distinct facial expressions, open mouths, good lighting, rich skin tones, etc., seem to work best. He really likes photos of dancers with makeup, as they tend to have strong or exaggerated features.

Kluge often takes the base photo into Photoshop and changes the colors and gradients as a middle stage based on what he knows will work well with the algorithm. You can see from the before and after images below that the output image often bears very little resemblance to the source image he starts with.

starting point (thinking outside the ocasahedron)

thinking outside the icosahedron

Kluge has built an assembly line of code variations he likes into the software tool CodePen so that he can rapidly try out different configurations on his source images. He claims to have “about 100 saved combinations.” He will typically try an image across 20 or so of his CodePen variations, searching for the best possible output and making further changes to the code when necessary.

I love Kluge’s description of his organic coding process for creating art. It flies in the face of the popular misconception that programming art is somehow a linear process, when in actuality, it is almost always circuitous.

When I asked Kluge if he is still surprised by the outputs he is getting, he replied:

Oh, yeah, I wouldn’t have the motivation to do this at all if I wasn’t surprised every time. It’s a pleasure every time I get close to something I like. I don’t have a good drawing hand. This is something I use to be my drawing hand. Sometimes the lines and shapes can be really beautiful, and I don’t think I could calculate that. It’s impossible for me to have these things in my head before I start. I would like to think this is true for all generative artists. It is a very playful process.

Espen Kludge, any individual - 2019

Because Kluge’s portraits feel so three dimensional to me, I asked him if he had played with adding a Z dimension to the pixels. He shared that he had considered importing them into Houdini (a 3D software package), but wanted to keep them as pure code since that is how they were born. He added that he may explore doing some work with them in 3D using Python.

I was also curious if he had explored making them interactive, animated, or motion based. Kluge explained that he fears that adding animation to the portraits would demystify them for him and could make it something he feels it shouldn’t be.

Espen Kluge, ladyfriend - 2019

The look of the portraits varies broadly across the series based largely on the quantity and density of the lines. Some of the more dense portraits look almost like photographs while other more sparse portraits look more geometric and abstract.

Not that it drives his decisions or his process, but Kluge did share that the ones that get the most likes on social media tend to be somewhere in the middle.

Espen Kluge, more extroverted than you - 2019

Personally I think Kluge’s choice to share works that range in line density is itself important. The sparse ones help me appreciate the dense ones more and vice versa. It is an exercise in contrast, and the work benefits from the display of the full spectrum.

In fact, I think the shift toward and away from abstraction through variation in line density is a big part of what makes Kluge’s portraits feel so fresh and emotive, especially when compared to traditional generative art which can often feel cold, geometric, and repetitive.

Espen Kluge, 100 atoms in my favor - 2019

An artist’s artist, Kluge is candid and open about his process, one which is beautifully honest and imperfect. When I asked him about the similarities between programming with Java and film scoring, he shared:

Both are programming languages, Java script and musical notation. But I have never been very good or very intuitive with code or notation - I have always been very chaotic with both programming and notation. I took longer than most students to get there with music notation. But sometimes you take the longer road and you get your own very unique experience with things, which I think results in something very cool and creative.

As someone who also spent their life breaking things and doing things the “wrong” way, I couldn’t agree with Kluge more. I also agree that the uniqueness of his work is likely a direct result of his struggles and unique journey.

Kluge, his music, and his art are truly fantastic treasures. It is artists like Espen Kluge who keep me excited about discovering new work and sharing it through my writing on Artnome. Additionally, in my new capacity as senior curator for Kate Vass Galerie, I have the added pleasure of making introductions for artists like Kluge to help further raise his profile and identify a network of collectors and appreciators for his work. Thanks to Espen and Kate, the work I have curated for this article is also now available to Artnome readers and collectors of digital art. You can find both digital and printed works at the Kate Vass website.

Collecting Art by Women: Wise Investment or Last Resort?

The team behind Sotheby’s Mei Moses Index recently published a report showing that over the past six years, works by female artists that have returned to auction have seen price gains of 72.9% versus 8.3% for works by men.

Source: Art Agency, Partners Where Women Outpace Men in the Market

We at Artnome were not surprised by this. Our core thesis for finding value in the art market has long been that the market will see a dramatic correction for the massive institutionalized gender and racial bias that has been baked into it from its inception. We believe this correction will be largely driven by socially progressive Millennials and Gen Z’ers who are the first to widely recognize that gender and race have no bearing on artistic skill and ability.

From the 2018 Pew Research Center Report: “Generation Z looks a lot like Millennials on Key Social and Political Views

What we did find surprising and disappointing was the explanation for the trend in rising value for art by women given by the report put out by Art Agency Partners. The report suggests that the reason more collectors were buying art by women was that all the good work by male artists was taken. In explaining “What Happened and Why,” the report specifically offers up this theory:

When the market runs out of art produced by its superstars, it tends to move onto related areas for supply.

Jenny Saville, Propped (1992) Sold for $12.4 million

At first I thought I was being a bit too sensitive here. It is not like the report was literally suggesting the main reason collectors would buy a painting by an amazing female artist like Jenny Saville is because there were simply not enough paintings by male “superstars” like Francis Bacon and Lucian Freud, right?

Oh, but then I read on, and it couldn’t be more clear that that is exactly what the report is trying to suggest.

Meanwhile, as supply of their (Francis Bacon and Lucian Freud) major paintings has become restricted, buyers who feel priced out might be tempted by the psychologically charged, fleshy paintings produced by the younger, female British artist Jenny Saville.

There’s lots wrong with this. First off, positioning Jenny Saville as a consolation prize or last resort for those who can’t acquire work by Francis Bacon and Lucian Freud implies her work is inferior. We would not accept this type of comparison in other industries. For example, when Marry T. Barra was named CEO of GM, it would have been absurd and insulting to suggest it must have been due to scarcity of male CEO candidates. So why should this same logic be acceptable in the art world?

Worse, it turns out the number of works at auction by Bacon and Freud has actually increased over the last three years, and during that same time period, paintings by Saville have been as or more scarce at auction.

Supply of works at auction priced at at $100K or more.

So the idea that Saville’s work is somehow abundant and can fill in the gaps for the Bacon and Freud markets is off base. In fact, it looks to me like the report has it backwards. If you can’t collect one of the few Saville paintings available at auction, you may be stuck bidding on a Bacon or a Freud instead as a consolation.

Helen Frankenthaler, Blue Reach , 1978. Sold for $3 million

Lee Krasner, Joan Mitchell, and Helen Frankenthaler have all set new records at auction in recent years. At Artnome we would suggest this is a market correction for these amazingly talented artists who have been dramatically undervalued by an antiquated art world built on severe gender bias. The report, however, suggests it may be because of a tiny dip in the supply of Jackson Pollock and Willem de Kooning paintings:

Take Abstract Expressionism. Supply is dwindling for premium examples of the most sought-after works, such as those by Jackson Pollock and Willem de Kooning. Between 2016 and 2018, seven works by Pollock were sold and 26 by De Kooning, a dip on the previous two years, in which 11 and 35 works were sold, respectively, according to data from artnet.

Subsequently, the market (dealers and auction houses alike) have begun to focus buyers’ attentions on the work of their more historically overlooked or undervalued contemporaries, notably Joan Mitchell and Helen Frankenthaler.

Right, so you’re telling me that Krasner, Mitchell, and Frankenthaler all recently broke their personal records for sales at auction not because they are amazing and because society is becoming more progressive, but instead because there were four fewer Pollock paintings available in a two year period?

Joan Mitchell, Blueberry (1969-70) set a $16.6m record for the artist at Christie’s New York last year.

There are 499 female artists in Mei Moses All Art-Female (AAF) index which includes 2,472 repeat sales. Are we to write off the dramatic increase in value seen across the entire All Art-Female index as somehow being the result of a supply shortage in art by men? I am not buying it.

Lee Krasner’s The Eye Is the First Circle, 1960, sold for $11.7 million.

In fact, if you just cherry pick a handful of male Abstract Expressionist artists and the time frame you are looking at, you can tell pretty much any story you want.

I used MutualArt to look at the supply of paintings at auction by Pollock, de Kooning, and six other male Abstract Expressionist artists over the last four years. Remember, this coincides with the precise time period in the report where prices for art by women really started to outshine their male counterparts.

Supply of works at auction priced at at $100K or more.

I learned that while the supply of work at auction by de Kooning was down slightly (but still relatively abundant), work by other male Abstract Expressionist painters, including Pollock, has actually risen in the last two years. So there is actually no reason to believe a “shortage” of work by male Abstract Expressionists drove the record-breaking sales and rising value of art by female Abstract Expressionist painters at all.

Unfortunately, the way the mainstream art world media works, most major art news outlets just passed on the explanation directly from the report without questioning or challenging it. According to coverage of the report in Penta, a subsidiary of Barron’s:

One reason works by female contemporary artists are doing so well is those by their male peers are getting scarcer, the firm said.

The Art Newspaper passes on the example of the Abstract Expressionists, stating:

…for instance, in the Abstract Expressionist market, a dearth of works by male artists such as Jackson Pollock and Willem de Kooning has coincided with a new focus on Joan Mitchell and Helen Frankenthaler.

Likewise, articles from Artnews and Artnet fail to challenge the report’s assumptions that the success of female artists must be driven by a lack of work by men.

Yes, men still dominate the art auction market by volume and price. But the real reason art by women accounts for just 10% of work sold at auction and just 7% of the value created at auction is that the art world has long harbored misogynistic views that have held women back from having their work valued on an equal playing field.

And the real reason work by female artists is rising faster than the work by men is because it is just as good but has been greatly undervalued as a result of systemic gender bias.

Georgia O’Keeffe, Jimson Weed/White Flower No. 1 (1932) sold for $44.4 million

What is especially sad in this instance is that some rare positive news on the growth of the market for art by women was instantly attempted to be explained away. Leading with the explanation that there must be a lack of work by men instead of recognizing the overlooked merit of female artists and the progress we are making as a society only serves to further perpetuate gender bias in the art market.

To be clear, I believe the authors of the report mean well and have good intentions. But I was not comfortable letting this explanation proliferate unchallenged as I believe it is both unfair and incorrect.

It is great to see data showing rising prices of art by women - but sad to see a biased interpretation of the data robbing female artists of the credit for this long overdue market correction.

Artnome Turns 2, Announces Partnerships and Advisory Board

Artnome turns two this month and we are celebrating by announcing two exciting new partnerships and a brilliant new board of advisors to help us navigate the next two years! Both our partnerships and our advisory board align perfectly with our two primary goals of improving the art historical record and promoting artists working at the intersection of art and tech through interviews and articles. 

As many of you know, Artnome co-curated a show on the history of generative and AI art with Georg Bak called Automat und Mensch, which had a successful opening last month in Zurich.

From Left: Robbie Barrat, Herbert W. Franke, Mario Klingemann, Kate Vass, Jason Bailey, Georg Bak

Working with Kate Vass and her team was a fantastic experience, and I am excited to announce our partnership, making Kate Vass Galerie the official gallery of Artnome! Kate’s most recent shows on blockchain art and generative art, respectively, line up perfectly with the topics and artists we cover and love. 

Under this new partnership I will continue to curate for the Kate Vass Galerie, make introductions for more up-and-coming artists, and will be linking to and advertising the Kate Vass Galerie as the official Artnome gallery. In keeping with both of our missions and goals, we are looking at adding a blockchain-based gallery to make collecting digital art accessible to the next generation of collectors.  

Jared Tarbell, Substrate.GZB, 2019

Artnome’s second partnership is with Art Recognition, another amazing Swiss company with strong female founders. Particle physics Ph.D. Dr. Carina Popovici and her co-founder Christiane Hoppe-Oehl have developed AI to assign the probability of forgery based on photographs of an artwork. Specifically designed to augment, not replace, human connoisseurship, Art Recognition produces a heat map alerting human experts to areas of potential concern. I could not be more thrilled to team up with Art Recognition to help improve authentication and fight forgery in the art market. 

Example heat map from Art-Recognition. The regions highlighted in blue represent the salient features which support the prediction that the artwork is fake, whereas areas highlighted in red show evidence against the classifier's decision.

As Artnome’s primary partners, Kate Vass Galerie and Art Recognition provide funding to help offset the cost and time I put into writing articles for Artnome, gathering data, and creating analysis in order to make it freely available to the public. Put more simply, when you support Kate Vass Galerie and Art-Recognition, you directly support Artnome in our mission to fight art forgery and highlight underrepresented artists whom we feel deserve more attention! 

Last but not least, I have asked an amazing list of industry experts, all of whom have already played important roles in the growth of Artnome over the last two years, to become formal advisors as Artnome moves forward and continues its growth. These are all people for whom I have the deepest respect and who have already provided critical guidance and opened up opportunities for Artnome. 

Anne Bracegirdle, Former Associate Vice President, Art+Tech at Christie’s

Anne reached out to me in 2018 to invite me to moderate two panels at Christie’s inaugural Art + Tech event in London. That was a huge leap of faith, given I had zero experience presenting or moderating. Having “moderator at Christie’s” on my resume has opened countless doors and created new opportunities and friendships ever since. I was again invited by Anne to speak, this time on “AI and art” in 2019 which has further raised the profile of Artnome and helped to solidify our brand as experts in art and tech. Anne has great passion and commitment for the digital art space and will play a key role in raising its profile as “the” important genre for our generation.

Bernadine Bröcker Weider, Co-founder & CEO, Vastari

I’ve had the pleasure of attending several conferences around the world with Bernadine, and she brings amazing energy and leadership everywhere she goes. Bernadine is the CEO and founder of Vastari, the largest online facilitator of international touring exhibitions and loans between private collectors, museums, and exhibition producers. Vastari facilitated over 456 matches of content to venues in 2018 and works with thousands of museums around the world, including eight of the top ten most visited.

Nanne Dekking, CEO & Founder at Artory and Chairman of the Board of TEFAF

I’ve had the pleasure of sharing the stage with Nanne at several conferences, and we immediately hit it off based on our common mission and principles. Nanne is an eloquent advocate for change in the international art market and is using his long art career to improve the art historical record and to bring transparency to the art world. As Founder and CEO of Artory, Nanne has developed the first standardized data collection solution by the art world, for the art world.

Nanne was asked two years ago to become the chairman of the board of trustees of TEFAF. In this role, he has greatly contributed to make vetting rules for authentication and selection criteria more transparent and to avoid any commercial interest. Prior to Artory, Nanne held senior roles with Sotheby’s and Wildenstein & Co.  

Jessica Fjeld, Assistant Director, Cyberlaw Clinic, Harvard Law School

Though I did not anticipate it, I have met many talented art lawyers since starting Artnome. None of them came across as understanding art, law, and tech as well as Jessica has. I interviewed Jessica for a recent article I wrote exploring why copyright for AI art is so complicated. She navigated the intersection of art, tech, and law with ease while putting even the most complex aspects in terms that anyone could understand. I am thrilled to have her unique perspective and invaluable advice moving forward.

Ahmed Hosny, Machine Learning Research Scientist, Dana-Farber Cancer Institute

Ahmed’s “Green Canvas” project was the inspiration for my interests in machine learning and art, and he is solely responsible for my initial explorations of blockchain and art. Ahmed is a tech visionary and is the first person I go to when I run out of ideas or have questions about new technologies and how they work. You can learn about his amazing work using machine learning to identify and treat cancer and other important projects on his website or on his LinkedIn profile.

Marion Maneker, Editorial Director at Art Media Holdings (Penske Media Corporation)

I cold called/emailed Marion several years ago after stalking him on his excellent site Art Market Monitor to tell him I had the world’s largest database of complete works. Rather than hang up on me, he agreed to meet with me in person and educated me on the art market. Several years later, he is still educating me, and it has shaped the analytics I am developing from my data to make them far more relevant than they otherwise would be. If you are interested in the art market, Marion’s site is the best place to learn about it and to follow important news and developments.

Thanks for your support over the last two years! If you have questions, suggestions, or concerns you can always reach me at jason@artnome.com.

Robbie Barrat Interview At Christie's With Artnome

Artnome has two missions:

• Improve the art historical record through better data to help fight against forgery.

• Highlight work by artists who deserve more attention, including artists working with tech and artists from underrepresented communities.

Thanks to my good friends at Christie’s, Anne Bracegirdle and Marisa Kayyem, I was able to share both of Artnome’s missions on stage to a live audience at Christie’s NYC last month.

First I presented on machine learning and analytics in relation to the art market, second by interviewing my favorite AI artist, Robbie Barrat, on stage. Christie’s did a phenomenal job recording the presentations and I am excited to share them with you below.

Machine Learning and Analytics: Predictions for the Art Market

An Interview with Robbie Barrat and Jason Bailey

Again I am super grateful to Christie’s for giving me this platform to shine a light on art and tech. I encourage you to watch all the presentations from the summit here.

If you found the topics in the videos interesting I would also encourage you to sign up for our newsletter below. And as a teaser, be on the look out for some big announcements from Artnome in the next few weeks to celebrate our second birthday!

Machine Learning for Art - Deep Kitsch or Creative Augmentation?

Machine learning is at its best when used as a tool for augmenting human capabilities, not for replacing them. And while we may not all be able to build our own machine learning models from scratch, new tools like Runway ML and Joel Simon’s forthcoming Artbreeder are opening up access to these machine learning models for everyone. Will this flood our screens with infinite images of deep kitsch? Or can machine learning augment human creativity on a larger scale and point towards a new direction for art?

Animation by Alexander Reben using real + drawn faces model by Joel Simon in Artbreeder.

My most important mentor, my high school art teacher Marco Marchi, taught me that creativity can start with new tools, but it should never stop there. This is especially true with machine learning models that can apply eye candy like filters at the push of a button, only then to devolve into derivative and contrived visual effects.

Marco also taught me that the beauty in making art is in the discovery process. Likewise, art appreciation is about unpacking the artist’s discovery process and finding your own discoveries along the way.

The future of work according to The Jetsons

There is not much discovery in pushing a button to apply a single filter or machine learning model, nor is their much to unpack, but I believe it can be a starting point for a more interesting artistic journey if used as a tool for augmentation.

Marco passed away recently, and I have been thinking about him a lot and revisiting his work. Over the weekend I played around with style2paints, a machine learning model for colorizing sketches, and tried it on several of Marco’s sketches.

Earth Gift, pen and ink drawing by Marco Marchi

Earth Gift, pen and ink drawing by Marco Marchi, colorized in style2paints

I wondered what Marco would have thought of these new machine learning tools. I decided that he would approach them with with openness and curiosity rather than judgment and suspicion. With that in mind, I decided to do the same. In the rest of this article, I take style2paints for a spin documenting and sharing my creative process along the way while trying to discover if machine learning for the masses will augment creativity, produce more kitsch, or both.

AI as a tool for augmenting creativity and introducing unpredictability

After playing with Marco’s ink drawings, I decided to run other images with simpler compositions through style2paints to try and learn its quirks and breaking points. Etchings and drawings from online databases like the MET’s open access collection provided me with plenty of interesting fuel.

Portrait of Charles Meryon, 1853, Félix Henri Bracquemond,

Portrait of Charles Meryon,, updated in style2paints

I pretty quickly figured out that recognizable images with fewer details and less shading actually produced the most realistic outcomes. But the most realistic results are rarely the most interesting.

I find breaking tools is the fastest way to get to their creative potential. By playing with a combination of filters for hue, brightness, and grayscale in Runway ML, I was able to break the style2paints model a bit.

Portrait of Guido Arnot 1918 by Egon Schiele

Portrait of Guido Arnot, updated in style2paints

There are known problems with machine learning models showing skin tone bias in favor of lighter-skinned people sometimes called “whitewashing” (this is due to disproportionate or biased training data skewed towards caucasians). With certain filter combinations in Runway ML you can actually approximate the reverse effect and produce a wider variety of skin tones as seen in these Egon Schiele drawings.

Standing Nude Facing Right, 1918, Egon Shiele

Standing Nude Facing Right, updated in style2paints

I then tried some more unusual subject matter as an input to see if I could break the model that way - for example, Thomas Rowlandson’s Comparative Anatomy, which comically compares the human head to that of an elephant and a bull.

Comparative Antomy, 1800-’25, Thomas Rowlandson

Comparative Antomy, updated in style2paints

I loved the results from these cartoonish images. It seemed the less details I provided the model and the less realistic the characters, the more interesting the results were.

I thought about other sources for simplified minimalistic inputs like cartoons and coloring books. Pretty quickly I wondered what I could get by grafting coloring book images onto historic etchings?

I started in Photoshop, but the results were a bit disjointed. I then I printed the images out and drew on them directly using pen and marker to unify the images and to make them my own, and then re-ran the model.

Drawing directly on the printouts by hand was producing the most interesting results. Eager to explore, I made a quick dumb sketch of a monster staring at Homer Simpson to see how the model would handle it.

To my surprise, it produced a pretty compelling ghost from my crappy doodle.

I now had an image that felt really satisfying and unique. Sure, it is not going to the Louvre anytime soon, but I was developing a visual language and an understanding of the tool that gave me some control while leaving a delightful amount of unpredictability.

I also really liked the idea of blending of highbrow historical etchings from the MET database with lowbrow cartoons from children’s coloring books. I tried a few others, including a mash up of a Picasso drawing and Garfield I called Picarfield.

Picarfield Glitched, Jason Bailey, 2019

The clean version of Picarfield was kind of boring. But once I started breaking the model a bit more, it produced some glitchy versions full of artifacts, which I actually like in this case.

I often find running towards things that repel you and exploring them more deeply is a great trick for prompting creativity. One of my favorite painters, Albert Oehlen, is a master at this. In a recent interview Oehlen shared:

If someone stands in front of one of my paintings and says, 'This is just a mess', the word 'just' is not so good, but 'mess' might be right. Why not a mess? If it makes you say, 'Wow, I've never seen anything like that,' that's beautiful.

Ohlen’s Computer Paintings transform the Mac paint/inkjet aesthetic into high-art.

Albert Oehlen’s Prix Ars Electronica (1991)

I ultimately decided creating analog work to feed into the model was the more interesting direction. Introducing handmade images feels like a good way to fight back against the often cookie-cutter output of machine learning models. I made a super creepy contour drawing of my wife (sorry, sweets, you don’t really look like this) and ran that through style2paints.

And then “put a bird on it”… just because.

To redeem myself, I did a few more accurate (and flattering) life drawings of Erin. I found it more than a little ironic that playing with AI models on my computer had lead me to do some life drawing for the first time in years.

In the end I didn’t create any masterpieces (there is a reason I spend more time writing than making art these days), but the process was an adventure, and the more I experimented, the closer I got to making interesting work. I think process and exploration is the secret sauce that makes any art interesting.

The reason I admire and write about artists like Helena Sarin, David Young, and Robbie Barrat so regularly is that they are going beyond simply applying models: they are deeply exploring the creative process, and it shows in their work.

Pocos Frijoles, an artisan coffeeshop, Helena Sarin

Helena is prolific and generous in sharing her process on Twitter and will soon be launching a new site. It is fun to watch her evolve as she explores different directions and influences continuing to blend her handmade paintings and drawings with machine learning tools.

Nude Study, Robbie Barrat, 2019

Robbie recently looked to traditional methods of artistic training as part of his process, sharing “I’m trying to get better at using neural networks for making artwork by engaging in more traditional exercises (figure drawings this time).”

(b62a,unknown01,1) from the Tabula Rasa series, David Young, 2019

David Young breaks down machine learning models to the fewest possible components as a way of “revealing the materiality of AI.” As he describes them on his site, “These images are an exploration of how a machine learns. They were generated from no more than a handful of training images.”

All three artists use machine learning models to make art, but they could not be more different. That is because the tools are only a starting point - it is their process which makes their work rich and unique.

On the opposite side of the spectrum we have “artists” giving their machine learning models human names and putting wigs on robots to reinforce disingenuous claims that AI is replacing human artists. This is sad, as art has great potential to educate an anxious and confused public on the actual capabilities of machine learning. When the AI hype dies down and we head into the next AI winter, we should see fewer uninformed dystopian AI art projects. These projects will thankfully be lost to history while those using AI and ML as a tool instead of subject matter should continue to grow and break new ground.

I am also optimistic that with Runway ML and Artbreeder, we will see some more traditional (analog) artists bringing deeply established creative process to these new (digital) tools. Those more traditional artists may not have the technical chops to train their own machine learning models, but I am hoping that a rich sense of artistic process will expand the work that we see being created today into new and exciting directions.

This article was written in loving memory of my high school art teacher Marco Marchi. Marco gave purpose to the lives of thousands of students by being a friend and mentor and teaching us to think creatively, independently, and to challenge the status quo. The idea that I could someday be like him when I grew up helped me make it through high school and into college. Rest in peace, Marco.

Bye Bye Camera - an App for the Post-human Era

In a climate where endless hype has people paranoid about AI stealing humans’ jobs, a new app called Bye Bye Camera uses neural networks to eradicate humans from the world altogether. Bye Bye Camera simply removes people from photos and fills in the background. The app is being launched today by an artist who goes by “Damjanski” and his art collective Do Something Good.

According to Damjanski:

I’ve created this project together with two of my longtime collaborators, Andrej and Pavel, from Russia. A couple of years ago I created a collective called Do Something Good where I connected all the people I’ve collaborated with online. By now we’re 16 people around the world from different fields and collaborate on different projects.

The app takes out the vanity of any selfie and also the person. I consider Bye Bye Camera an app for the post-human era. It’s a gentle nod to a future where complex programs replace human labor and some would argue the human race. It’s interesting to ask what is a human from an Ai (yes, the small “i” is intended) perspective? In this case, a collection of pixels that identify a person based on previously labeled data. But who labels this data that defines a person immaterially? So many questions for such an innocent little camera app.

At a high level, Bye Bye Camera works by combining functionality from an image recognition app called Yolo with a neural network that analyzes the background and tries to repaint it.

Artnome editor Erin Bailey disappears into unruly hedges

Sure, this functionality has existed in other photo apps and tools, but in Damjanski’s hands, this feels more like an artwork than a business venture. For one thing, other tools are used to make corrections - Bye Bye Camera just wipes everyone out!

Damjanski reinforced my perception of his app, sharing with me:

A lot of friends asked us if we can implement the feature to choose which person to take out. But for us, this app is not an utility app in a classical sense that solves a problem. It’s an artistic tool and ultimately a piece of software art.

The idea of “app as artwork” may seem strange at first. It makes more sense when you remember that most of the serious AI artists feel strongly that it is the models they are training and not the output that should be considered the artwork. Once you arrive at this conclusion, highjacking the Apple App Store and Google Play Store as your own personal art gallery makes a lot of sense. Again, from Damjanski:

First and foremost I see this as an artistic tool for me to create more art. It’s almost like a new ‘brush’ I am using. A year ago we created another tool called “No Shutter App” that only takes pictures while the phone’s shutter loads. In both cases it’s a way for me to enhance my artistic practice. But we also like opening it up to the public and see what other people create with it. A commercial success is not the primary goal here.

Demo photo provided by Bye Bye Camera

The results from Bye Bye Camera are often eerie and remind me of work by one of my favorite photographers, Lewis Baltz. Baltz created gorgeous photographs of generic office parks completely void of humanity.

Lewis Baltz, South Corner, Parking Area, 23831 El Toro Road, El Toro (1973)

While Baltz photos are sans-human from conception, there is, of course, also a long and rich history of manual and mechanical removal of humans from photographs for a variety of reasons.

In Soviet Russia it was common practice for censors to remove people from photographs as part of the propaganda efforts. The image below was taken near the Moscow Canal while Nikolai Yezhov was serving as the water commissar. After Yezhov fell from power he was killed and his image was systematically removed from the photograph.

It is important to remember that in the early days of Soviet censorship, the photograph still had an air of truthfulness to it. So carefully removing a person from a photograph had the haunting effect of visually rewriting them out of history.

In today’s post-truth society where photography as a reliable record of reality has long since dissolved, erasure of humans from photographs takes on a different meaning. In 2007, artist Michael Somoroff erased the figures from photographer August Sander’s famous photographic series People of the 20th Century, creating his own series titled Absence of Subject.

August Sander, Pastrycook (1923)

Michael Somoroff, Absence of Subject (2007)

Unlike the Russian censors, Somoroff’s goal was not to rewrite history, but rather to reimagine it through a postmodern lens while paying homage to Sander. By removing the human figures intended as the primary point of interest, he is pushing the background into the the foreground, creating an entirely new image in the process.

Regular Artnome readers will recall that I used a similar process in my previous article in an attempt to create a neutered version of David Hockney’s Portrait of an Artist (Pool with Two Figures) by using algorithms to remove and replace the human figures.

Jason Bailey, Olive Synchronism (After Hockney), version 2 (2019)

My goal was to make a copyright-free version of famous paintings by systematically identifying and removing the “visually important” parts and replacing them using Photoshop’s “context aware fill.” Though the goals of Somoroff, Damjanski, and myself are different, the approaches are quite similar. This is not all that surprising as these new AI/ML-driven tools have built in affordances that encourage appropriation and radical remixing. We can expect this to be a trend that only grows more extreme over the next few years, manifesting in deep fakes and other forms of extreme photographic and cinematic modification.

If Damjanski’s name sounds familiar to you, it may be because you remember him from his guerrilla art exhibition in the Jackson Pollock gallery at the MOMA titled MoMAR. The project used AR (augmented reality) to replace Jackson Pollock paintings with work by generative artist David Kraftsow from his YouTube glitch series.

According to Damjanski, MoMAR is “an unauthorized gallery concept aimed at democratizing physical exhibition spaces, museums, and the curation of art within them.”

Since we think of Artnome mascot Frida as being a human, we were delighted to see that Bye Bye Camera feels the same way!

While Bye Bye Camera does not co-opt a physical space as with MoMAR, Damjanski is introducing his art into a public distribution channel, the app store, which we do not typically associate as being a “gallery.” With more than 93 million selfies posted daily, I, for one, welcome Bye Bye Camera as perhaps our last best hope of stemming the avalanche of endless selfies in favor of some more Baltz-ian subject matter in my Instagram feed.

Mass Appropriation, Radical Remixing, and the Democratization of AI Art

The tools for using artificial intelligence and machine learning to create art are currently complex enough that only a small number of highly technical people have mastered them. This came up on a recent panel I moderated on AI art at CADAF (Contemporary And Digital Art Fair) in NYC.

Specifically, I asked the panel if you “need a degree from MIT and to be a white guy over 40” to create art with artificial intelligence and machine learning, as many of the panels I attend on AI art have seemed particularly homogenous. The question was, of course, somewhat rhetorical, as I am well aware of great work being done with AI by non-white and non-male artists. But the balance for panels so far does seem to skew in that direction.

One way to increase the diversity in art created with artificial intelligence is to continue to invest in making code open source. However, not everyone can or will learn to code, but with the development of new tools like Runway ML, you may not have to.

I have been following Runway ML since May, 2018, when I read cofounder Cristóbal Valenzuela’s excellent essay “Machine Learning En Plein Air: Building Accessible Tools for Artists.” In the essay, Valenzuela compares the current barriers for artists in using AI to the barriers that would-be artists faced in creating their own pigments and paints prior to the invention of the collapsable paint tube.

Valenzuela points out that many credit the portability of the paint tube with making it practical for artists to paint plein air (outdoors) in oils. The great painter Pierre-Auguste Renoir even credited the invention of the paint tube with triggering “Impressionism” and assisting to usher in modern art at large.

“Without colors in tubes, there would be no Cézanne, no Monet, no Pissarro, and no Impressionism.” - Renoir

Like Valenzuela, I believe the democratization of artificial intelligence for artists and designers will drive a revolution in aesthetics. Specifically, I believe we will enter an era of mass appropriation and radical remixing of visual materials like we have never seen before.

In this blog post, I argue that it is the nature of AI and ML as art making tools that they will require an enormous amount of visual material as fuel to train their models. It is also the nature of AI and ML that the resulting artworks are essentially radical remixes of the work that is used to fuel them.

We can already see this trend in “appropriation” and “remixing” of visual material in the work of accomplished AI artists. Mario Klingemann’s Lumen-award-winning piece The Butcher’s Son for example, was trained on a large number of pornographic images.

The Butcher’s Son, Mario Klingemann, 2018

Klingemann explained in an interview with Fast Company in 2018 that he chose to train on pornography because it is “one reliable and abundant source of data that shows people with their entire body.” He then added that sports would have been another source, but he is not that into sports.

Similarly, AI artist Robbie Barrat has famously trained models on famous portrait paintings of nudes and even trained a Pix2Pix model on the complete Balenciaga online fashion catalog.

AI Fashion, Robbie Barrat, 2018

AI Fashion, Robbie Barrat, 2018

Barrat’s most recent work remixes appropriated imagery in a slightly different way. For his series of artworks Corrections, Barrat starts with images of classical paintings by old masters and applies a custom “flesh finding” algorithm.

Saturn Devouring His Son (After Peter Paul Reubens), Robbie Barrat, 2019

Barrat shared the genesis behind his Corrections in a recent interview I did with him for the catalog for the Automat und Mensch exhibition.

When I was in France in Ronan’s studio [French painter Ronan Barrot], I saw that he had a scene that he was not satisfied with, and he covered over the parts that he didn’t like with bright orange paint. And then he filled those parts back in. I thought that was really interesting, and he called those “corrections.” And during our confrontation, he also corrected a lot of my AI skulls. I was really fascinated with the painting over the parts you don’t like and redoing it. I thought that was a really cool way of doing it.

So I basically am teaching the neural networks to do pretty much the same thing. I’m using Pix2Pix, and it is basically trying to learn the transformation from a patch of a painting with the center part missing back to the full patch of the painting. If you look up “neural inpainting,” you’ll find this.

While Barrat and Klingemann both typically appropriate and remix from large public data sets of images, other AI artists like Helena Sarin, David Young, and Anna Ridler choose to train their models based on their own photography and hand-drawn/painted artwork. Anna Ridler, for example, shot a large corpus of tulip photos to train her Mosaic Virus video piece which draws “historical parallels from ‘tulip-mania’ that swept across Netherlands/Europe in the 1630s to the speculation currently ongoing around crypto-currencies.”

Anna Ridler, tulip photos used to train Mosaic Virus

Even when training models using their own materials and avoiding appropriation, the process of using neural networks to create art is ultimately still one of remixing.

Some artists might argue with me here and point out that neural nets are creating entirely new works, not remixing old ones. But there is no question that the visual materials that artists curate and train the models with have a huge impact on the end results. It is this process, in particular, that I am referring to as “radical remixing” - radical in that I agree it is far different from traditional visual remixing through techniques like collage or even sampling in music.

Once you see the pattern of appropriation and remixing present in the work of AI art pioneers, it becomes easier to imagine what this might look like once the tools become more accessible to artists and designers at large. I believe it will start a new aesthetic revolution that will change the world through art, advertising, and political propaganda - all heavily based on mass appropriation and radical remixing of visual imagery.

How close are we to the democratization of AI for artists? I decided to be the guinea pig and jump in to Runway ML and see how challenging it would be for me to play with these tools to create new images.

Runway ML - Artificial Intelligence for Augmented Creativity

I have wanted to explore Runway ML and write an article about it for many months now, but I though it would take a lot of time to learn how to use it. I was wrong. In less than an hour I was up and running, creating multiple projects. Pretty quickly I became convinced Runway ML is onto something huge here and will be the Photoshop for the next generation of artists and designers.

Before I share more about the projects that I built using Runway ML, I should explain a bit about how the tool is constructed at a high level. The team at Runway ML does not necessarily write the actual algorithms for artists and designers to use. Instead, they have designed a framework to make it possible to integrate the latest algorithms from academic research into an intuitive interface that does not require programming skills on the part of the artist or designer.

For example, the first thing I tried in Runway ML was to colorize an early black-and-white scene from The Wizard of Oz. Runway ML has added Jason Antic’s excellent DeOldify model into their interface for color correction. It was super easy: I just loaded the clip in Runway ML and applied the filter. I didn’t even need to read the instructions or documentation.

The whole process took less than 15 minutes, and most of that was processing time. The interface was intuitive enough that I did could just use it.

With some newfound confidence that I could use the tools in Runway ML, I began to explore and run other models. I started thinking more about what visual materials I would want to play with. This is just the nature of Runway ML - you think, ‘I’ve got access to these cool models - what visual material can I apply them to?’ The tools in Runway ML simply do not do much until you feed them visual material; hence, my belief that we are hurtling towards an era of mass appropriation and radical remixing spurred by the inevitable democratization and adoption of these new tools.

My favorite visual materials are the artworks from 20th century art. Unfortunately, all the work created since 1923 (the work I love most) is still under copyright. With Robbie Barrat’s Corrections in mind, I wondered if there was a way to identify the visually important parts of these artworks, not to correct them, as with Barrat’s work, but to surgically remove their “hearts,” leaving only the parts that are unimportant enough to be legally shared with the masses as new works of parody.

Runway ML had exactly the model I was looking for. A model called “Visual-Importance” trained “neural networks on human clicks and importance annotations on hundreds of designs” to identify the “visually important” and “relevant” areas of images based on human perception. So I decided to run some copyrighted images from 20th century art through the model to identify the important parts of the images and then isolate and remove them.

I’m not a lawyer, but I remembered from my interview with Jessica Fjeld, assistant director at Harvard Cyber Law Clinic, that fair use of images is largely based on the “amount and substantiality” of the portion of the work you are pulling from. You are most likely to run into problems if you take the most memorable aspect, often referred to as the “heart” of the work. My entire process is based on using the most advanced technology we have at our disposal to surgically remove the “heart” and to remix the “visually unimportant” scraps into a new work of parody.

I started by running David Hockney’s wonderful Portrait of an Artist (Pool with Two Figures), which recently sold for $90M at Christie’s last fall. The heat map below produced in Runway ML shows the most “important” and “relevant” areas of the painting highlighted in white.

I then took this image into Photoshop and ramped up the contrast to isolate the important areas from the less important areas in clear blocks of white on black.

I erased the “visually important” areas of the artwork, leaving only the irrelevant portions. I was actually really happy with these results and almost stopped there.

To go a step further in transforming the work to make it my own, I used Photoshop’s content-aware-fill plugin. The plugin uses an algorithm to fill in all the white spaces by using the remaining areas of the work, the areas that were deemed “visually unimportant” by the neural net in Runway ML.

Olive Synchronism (After Hockney), version 2, Jason Bailey, 2019

Though I love Hockney and the original painting, I believe my version is a new work which has an entirely different feel. I am quite fond of it. It has a vibe similar to Lewis Baltz, one of my favorite photographers who is known for finding beauty in desolation. The new version is most notably absent of the emotion, narrative, and price tag present in Hockney’s original painting.

I am, of course, not the first artist to create art from art. There is a long tradition of artists painting other artists’ paintings within their own painting. These paintings about paintings were especially popular in the 17th and 18th centuries.

The Art Collection of Archduke Leopold Wilhelm in Brussels, David Teniers the Younger, 1650

Marcel Duchamp famously painted a mustache onto Leonardo Da Vinci’s Mona Lisa in 1919 and named the work of parody L.H.O.O.Q., which spoken in French translates roughly to “she has a hot ass” in English. Duchamp considered the work a “rectified ready-made,” a new work transformed by his addition of facial hair and the new title.

L.H.O.O.Q., Marcel Duchamp, 1919

Artists have long used direct appropriation of visual materials as a core part of their work - AI is just a new tool with deep potential to take it to a another level. Collage was raised to an art form by Picasso and Braque in the 1920s, and other artists like Kurt Schwitters, Hanna Hoch, John Heartfield, Jess Collins, and Richard Hamilton, spent their careers mastering it.

Roy Lichtenstein’s entire body of work was based on appropriating images from comic strips. Lichtenstein would enlarge, isolate, and crop elements directly appropriated from pop culture to create new works of art. His first appropriated work was directly lifted from a comic strip of Mickey Mouse and Donald Duck fishing. He titled the 1961 work Look Mickey.

I decided to extend the chain of appropriation and identify and remove the important elements from Lichtenstein’s appropriated work below.

Sketch for Nut Aestheticism (after Lichtenstein, after Disney), Jason Bailey - 2019

Nut Aestheticism (after Lichtenstein, after Disney), Jason Bailey - 2019

In this case I think I like my erased work better than my final version with the missing areas automatically filled in. The version with the white splotches would make a really cool graphic on a t-shirt.

Notably, I am also not the first artist to erase portions of another artist’s work and to claim that the results are a new work of my own authorship.

In 1953, artist Robert Rauschenberg asked himself if it was possible to create a new artwork completely from erasure (as I have). He started exploring this idea by erasing his own drawings. He felt that the result was not sufficiently creative, so he decided to ask William de Kooning, whom he looked up to as an artist, if he could erase one of his drawings, instead. After some hesitation, de Kooning gave Rauschenberg a drawing created with mixed media that he was fond of, but also knew would be a real challenge for Rauschenberg to erase.

Traces of ink and crayon on paper, with mat and hand-lettered label in ink, in gold-leafed frame

25 1/4 x 21 3/4 x 1/2 inches
(64.1 x 55.2 x 1.3 cm)

de Kooning chose well, and it took Rauschenberg two months to erase the piece. Rauschenberg finished the piece by putting it in a gilded frame and having his friend artist Jasper Johns add an inscription reading “drawing [with] traces of drawing media on paper with label and gilded frame.” The work now hangs in the SFMOMA.

Rauschenberg was also no stranger to appropriation. In 1979, he was sued by photographer Morton Beebe for including a photo Beebe had taken in 1971 of a cliff diver, which Rauschenberg used in his own piece titled Pull from 1974. In a letter responding to Beebe, Rauschenberg argued:

Dear Mr. Beebe,
I was surprised to read your reaction to the imagery I used in Pull, 1974. Having used collage in my work since 1949, I have never felt that I was infringing on anyone’s rights as I have consistently transformed these images sympathetically with the use of solvent transfer, collage and reversal as ingredients in the compositions which are dependent on reportage of current events and elements in our current environment hopefully to give the work the possibility of being reconsidered and viewed in a totally new concept.

I have received many letters from people expressing their happiness and pride in seeing their images incorporated and transformed in my work. In the past, mutual admiration has led to lasting friendships and, in some cases, have led directly to collaboration, as was the case with Cartier Bresson. I welcome the opportunity to meet you when you are next in New York City. I am traveling a great deal now and, if you would contact Charles Yoder, my curator, he will be able to tell you when a meeting can be arranged.
Wishing you continued success,
Sincerely
Robert Rauschenberg

Perhaps not surprisingly, Rauschenberg’s estate has been one of the more liberal estates when it comes to opening works under copyright for scholarly use and public consumption (partly why I felt it was okay to use the erased de Kooning drawing above in this post).

In homage to Rauschenberg as a great appropriator and master remixer of visual imagery, I ran his 1954 work Buffalo II, which recently sold for $89M at auction, through my visual neutering process.

Huckleberry Objectivity (after Rauschenberg), Jason Bailey - 2019

I love Rauschenberg, but I think I actually like my “visually unimportant” version even more than the original.

Rauschenberg passed away in 2008, but from what I know of him, I believe he would appreciate my posthumous collaboration in the spirit of his own visual remixing and appropriation.

Enthused by the results from processing Rauschenberg’s largely abstract Buffalo II, I was curious about how my process would handle a completely abstract work. Since de Kooning seemed okay with Rauschenberg’s erasure of his drawing, I figured he would not have minded if I ran his painting Interchange through my process, as well.

Persimmon Precisionism (after De Kooning), Jason Bailey - 2019

De Kooning’s Interchange is best known for reportedly selling for $300M in 2015. Mine is quite nice, as well - I don’t miss the heavier black lines. I feel like my version almost has a Diebenkorn feel to it.

Conclusion

Historically, when new tools for copying, manipulating, and multiplying existing images become available, we see an upsurge in appropriation and remix-based art. We saw this with Andy Warhol and Robert Rauschenberg, who co-opted screen printing for fine art and in photography with artists like Sherrie Levine and Richard Prince.

Series of people who do not actually exist created using GANs - https://www.thispersondoesnotexist.com/

With AI, we have tools that are sophisticated enough to generate endless convincing images of people who do not exist and to map our faces and our own physical movements using motion transfer onto people who do exist, as if they were our own personal puppets. Add to this the fact that we are making these powerful tools available to the most tech savvy generation ever, one that “gets” artistic appropriation as seen in their love KAWs, and you have the makings of an artistic revolution.

Think of Photoshop’s impact on visual culture for everything from politics to pop culture over the last 20 years. If you multiply that by ten, you will get my approximation of the impact accessible tools for using AI could have on art and design in the next 20 years.