Generative Art Based on the Input of Emotions

Abraham Schaecher Author

07/31/2021 Added

19 Plays

Description

Generative Art Based on the Input of Emotions UCARE SUMMER 2021

Searchable Transcript

Search:

[00:00:02.206]Hello everyone, my name is Abraham Schaecher and today I'll be
[00:00:05.222]presenting on my project which is generative art based on the input of
[00:00:08.573]emotions. I worked and collaborated with my advisor Dr. Robert Twomey
[00:00:12.580]on generative artwork and visual research at the Hixson-Lied College
[00:00:16.603]of Fine and Performing Arts at the University of Nebraska Lincoln.
[00:00:26.485]I think it is important to have some background information about this
[00:00:29.778]project. So the idea of artificial intelligence art and generative art
[00:00:33.186]has been explored and researched upon for decades. In fact, numerous
[00:00:37.193]examples of recent computational developments include the GauGan
[00:00:40.673]the Dall-E, BIG GAN, Style GAN, GAMEGAN, Gan-LSTM and many
[00:00:45.476]more examples. In fact, various applications of machine learning has
[00:00:49.495]been used in creating landscapes, generating images from text and
[00:00:53.519]creating fantastical creatures as a few examples.
[00:00:58.555]Now I also think that the purpose of the project is not only to explore
[00:01:02.179]creativity and originality but also to use various works of human made
[00:01:06.179]art to explore emotion based generative art. In fact, some areas of
[00:01:09.644]this project includes the creativity of a GAN, the artistic representation of
[00:01:13.324]a GAN, and also answering the question on whether or not a
[00:01:16.220]generated artwork could be near or at the same level as an artist. This
[00:01:20.624]project isn't necessarily siding with whether or not a GAN could be
[00:01:24.649]creative but a new perspective. Could a GAN be used as a tool to
[00:01:29.047]help understand or interpret emotion in the artworks?
[00:01:34.504]So, I think it is important to divide up the section into five key phases.
[00:01:38.946]The first is to be familiar with the San Diego Museum of Art study, as
[00:01:42.552]in examining the data, the categorization of art, and looking at
[00:01:46.855]the original museum artworks as well. The next step is to explore and
[00:01:50.806]decide on which GAN to implement for the project which will be
[00:01:53.890]between the BigSleep, Aleph2Image, and BIGGAN and CLIP. Then, I would
[00:01:58.823]generate results based on the GAN and use textual descriptions to
[00:02:02.489]create data as in generating artworks. I would then take those
[00:02:06.406]samples and conduct a survey to rate these samples compared to the
[00:02:09.821]museum original artworks. Finally, I will take the survey ratings and
[00:02:14.704]analyze them for observations and conclusion.
[00:02:18.604]To start off, I recently got a hold of the Training Data and the museum
[00:02:22.037]original artwork that Professor Twomey has for his project in
[00:02:25.493]association with the San Diego Museum of Art. Now, of the 20
[00:02:28.420]museum original artworks, I chose three artworks to be used. Those are
[00:02:33.134]Kilauea Calera, Sandwich Islands by Jules Tavernier, After Many Days by
[00:02:37.540]Thomas Hart Benton, and Red Blossom by Alexej von Jawlensky.
[00:02:41.365]These were chosen because they were diverse in the aesthetic such as
[00:02:44.647]a landscape, still life, and portrait.
[00:02:48.474]I then got a hold of a Generative Adversarial Network and compared
[00:02:51.423]samples using the BigSleep GAN, BIGGAN and CLIP, and
[00:02:54.740]ALEPH2Image. So after comparing the samples...
[00:02:59.323]I decided to choose the BigSleep GAN for this project. I chose this
[00:03:02.474]because the aesthetic the BIGSLEEP GAN had was identical to the
[00:03:06.240]samples I chose for this project and that the other two GANs had a
[00:03:09.923]unique aesthetic that would be different to the museum original artwork.
[00:03:19.861]So once I chose the GAN I was going to use, I used the textual prompts
[00:03:25.918]from the official website of the San Diego Museum of Art and had to
[00:03:29.313]paraphrase the description to avoid unnecessary descriptions and words
[00:03:33.346]for a good sample size. These descriptions will allow me to create
[00:03:36.879]8 generative artworks from that specific museum original artwork.
[00:03:41.479]As you can see in the next images, these are the generative results
[00:03:44.862]from each of the paintings.
[00:04:16.577]So, my method of research to compare the ratings was to create a
[00:04:20.110]Google Form survey that rates the images on a scale of 1 to 7 based on
[00:04:23.893]various emotions. These emotions were based on the Aesthmos Survey
[00:04:27.526]which captures aesthetic experiences. I opened this survey to
[00:04:32.192]the public for around a week and the people who took the survey
[00:04:35.475]were anonymous. Overall, 10 people took the survey and the results were
[00:04:39.459]gathered in a spreadsheet for future analysis.
[00:04:43.559]I also think it would be helpful to see what types of graphs we are about
[00:04:46.825]to see. The first is a difference plot which compares the average
[00:04:49.941]difference of the ratings by emotion from the eight synthesis artwork
[00:04:52.825]and the museum original rating. A positive difference indicates the
[00:04:57.491]emotion was more felt than the museum original while a negative
[00:05:00.858]difference means that the emotion was not as felt. The other type of
[00:05:04.158]graph we are about to see is a box and whisker plot which means all
[00:05:07.691]eight synthesis artworks are graphed in terms of minimum, maximum,
[00:05:10.974]mean and medians of the ratings. A box is shown per emotion and a dot
[00:05:15.407]indicates the painting was an outlier in the data sample.
[00:05:19.123]So as you can see here, some generative samples such as V5 and
[00:05:25.157]V1 indicates a unique outlier that is more unsettling and less beautiful
[00:05:30.356]than the museum original painting respectfully. And here are the eight
[00:05:35.906]samples to compare the results to by the dots. This also applies to the
[00:05:47.760]other samples for the next two paintings in terms of emotion and
[00:05:50.940]comparison.
[00:06:32.353]And here is the box and whisker plots for each of the three paintings.
[00:07:01.824]So we come to the conclusions based on the research. Most of the
[00:07:06.191]generated artworks varied widely in terms of the emotional response
[00:07:10.108]and reaction. In these cases, Kilauea had an interesting case of where the
[00:07:14.858]categories of calm and beautiful had a negative rating difference. After
[00:07:19.142]Many Days had indifference as a positive rating difference while
[00:07:22.808]unsettling had a negative ratting difference. Finally, Red Blossom had
[00:07:27.142]fascinating, funny, and beautiful as a high rating difference compared to
[00:07:31.189]the museum original. Also to keep in mind that there are a few
[00:07:38.622]relationships between the data such as indifference, beautiful and calm
[00:07:43.058]had inverse relationships and some emotions are similarly ranked to
[00:07:47.824]other emotions. Now some possible explanations for the change in
[00:07:51.575]ratings between the generated artworks and the original paintings
[00:07:54.359]may include the color aesthetic, the objects in the paintings, and the
[00:07:57.825]composition of the generated artworks that had a different response.
[00:08:03.973]There is definitely going to be further research in terms of creating
[00:08:07.007]artwork on a specific emotional adjective for individual paintings. I
[00:08:11.923]am exploring the idea of showing the synthesis artworks side by side
[00:08:15.589]in a exhibition. The use of animation is also another possibility and Image
[00:08:21.488]Classification is another area to explore in the future.
[00:08:32.105]Finally, I would like to thank a few individuals and groups for this
[00:08:35.021]project. First off UCARE for having the funding and resources to help
[00:08:38.705]this project. The Johnny Carson Center for Emerging Media Arts for
[00:08:42.438]allowing me to have access to the equipment and facilities for this
[00:08:46.021]project. Professor Twomey's Art and Empathy project and the San Diego
[00:08:49.870]Museum of Art for having me have the preliminary data for the project.
[00:08:53.837]The creators of the BIGSLEEP GAN for having access to the GAN for this
[00:08:57.471]project. I also would like to thank a few individuals as well. Professor
[00:09:01.020]Twomey for advising me and guiding me through the project in the last 10
[00:09:04.120]weeks, Sydney Kessler for the preliminary data analysis for the Art
[00:09:07.669]and Empathy Group. Dr. Ying Wu and the great individuals in the Art
[00:09:11.969]and Empathy Research Group for allowing me to join their meetings
[00:09:15.319]and gain valuable knowledge and insight into their research and
[00:09:18.820]suggestions for this project. I finally would like to thank the people who
[00:09:22.702]took the survey for the generated artworks as without them, this
[00:09:27.335]project would not exist.
[00:09:29.502]Thank you so much everyone for listening and I will see you soon.
[00:09:33.535]Thank you.

The screen size you are trying to search captions on is too small!

You can always jump over to MediaHub and check it out there.

Comments

0 Comments

Generative Art Based on the Input of Emotions

Description

Searchable Transcript

Comments icon comment

Related Channels

Comments