Steps to Solving the 'Picasso Problem' in Neural Networks

By Surbhi JainReviewed by Susha Cheriyedath, M.Sc.Apr 15 2022

In an article recently published in the journal Displays, researchers presented the first step in solving the Picasso problem by combining symbolic and neural approaches.

Study: Combining neural and symbolic approaches to solve the Picasso problem: A first step. Image Credit: ktsdesign/Shutterstock.com

Background

Deep neural networks have attained superhuman performance in a variety of disciplines during the last 10 years. Deep neural networks could learn to analyze a document by detecting grammatical patterns or translating it into virtually any other language in the domain of language processing. Despite these remarkable accomplishments, deep neural network models suffer from a well-known flaw, which is they perform poorly on tasks that are evident to humans.

In the natural language processing sector, for example, even the most powerful models could not recognize the grammaticality of phrases that any competent speaker would recognize. Models derived from convolutional deep neural networks can also fail to distinguish between an image and its corresponding meshed-up counterpart in the computer vision domain.

The well-known Picasso Problem, having a convolutional deep neural network, or any model built on it, is unable to discriminate between an image and its distorted counterpart. Even the most performant models are tricked by modified or distorted faces; even state-of-the-art picture recognizers are fooled. For humans, the challenge is trivial since they can take advantage of some prior knowledge of the application domain. Background knowledge could be communicated in this scenario by limiting the position of each face component in relation to the others.

About the Study

In this study, the authors provided a bridge between convolutional neural networks (CNNs) and machine learning. Answer Set Programming (ASP) was used to solve the well-known Picasso Problem. Images were detected automatically. The core idea was to first use the major benefits of the neural network approach for image identification and then use explicit knowledge defined by logical rules to solve the challenge of detecting well-formed images.

The researchers combined a neural-based technique with a purely symbolic approach to solving the Picasso Problem by using the well-known ASP paradigm to codify and reason about restrictions on the relative arrangement of the pieces of a face. The neural network used an image as input and recognized the various elements of the faces (nose, eyes, and so on), built bounding boxes, and labeled them. The data from the boxes in the neural network output was pre-processed by rotating and converting it into grids. Information on the produced arrays and Clingo constraints was then added to a string one at a time. To ensure that the string was solvable, it was submitted to an ASP solver. The outcome from the preceding stage was examined.

The team presented the Picasso problem by starting with convolutional deep neural networks to gain a more technical understanding of its causes. The entire architecture of the proposed system was illustrated, and its components, both neurological and logical, were discussed in detail. Based on the combination of CNNs and logical clauses of ASP, an alternate and simpler technique to solve the Picasso issue was suggested.

Observations

Using the Google application programming interface (API) to test 12 photographs of faces with the Picasso problem, it was discovered that they were all recognized as "person" and/or "face," with 60% to 96% accuracy. The logical phase consisting of pre-processing and constraint evaluation added about 38% to the time. A mean Average Precision (mAP) of 95.6% was estimated in terms of bounding box precision when only the boxes having an Intersect over Union (IoU) larger than 0.50 were considered. In terms of segmentation accuracy, an mAP of 95.0%, with an IoU of greater than 0.50 was determined.

More from AZoM: Industry 4.0 and Chemical Processes

If the list was empty, it signified that the constraints were not met, and so the face under examination was unsuitable or distorted. If the list was not empty, however, it indicated that the face was acceptable. Preliminary trials indicate that the proposed strategy was promising and could be viewed as a first step toward addressing the Picasso Problem, as well as a demonstration of the benefits that could be realized by combining a neural and a purely symbolic approach.

The proposed technique allowed them to evaluate whether the image was a well-formed face or not, which prevented neural network approaches from being harmed by errors.

Conclusions

In conclusion, this study elucidated that by constructing a bridge between CNNs and ASP, the first solution to the Picasso Problem could be offered, which is a critical problem in the automatic detection of images. The primary idea was to first use the major benefits of the neural network ap335 approach for image recognition and then use explicit knowledge defined by logical rules to address the problem of detecting properly shaped images.

Based on the basic and preliminary experiments, it was observed that the proposed approach was promising and can be viewed as a first step toward solving the Picasso Problem, on the one hand, and a witness to the potential benefits of combining a neural and logical approach on the other.

Disclaimer: The views expressed here are those of the author expressed in their private capacity and do not necessarily represent the views of AZoM.com Limited T/A AZoNetwork the owner and operator of this website. This disclaimer forms part of the Terms and conditions of use of this website.