Experimenting with a framework to check if AI can tell what User's might find attractive

Can AI help to predict what users find attractive or desireable?

User Feedback & Analysis with Machine Learning - an experimental study

The objective of this project was to create a method for analyzing the user's feedback of spatial characteristics of a space that they find most attractive and transferring these features to another project.

In order to narrow down the field of research we decided to focus on only one type of space, namely the public squares. We have selected a number of public squares in Spain (located in Madrid, Barcelona, Bilbao and Granada) and around the World in US, UK, and France for this study.

My role in the project was to work on the Natural Language Processing - data collection, data cleaning, creating a ML model & data visualizations. Some images in the project may be low quality or blurry due to output from ML model

Timeline

Mar-May 2020

Team

Sergey Kryuchkov

Pedro Ribeiro

Yara Gadah

Toolkit & techniques

Python

Pytorch

TensorFlow

Numpy

Deliverables

Code

Paper

Github Repo

TL;DR

Incorporating spatial analytics would allow us to gather key feedback & insights from big data. Can such a process be applicable for designing products?

New methodology - We ran the semantic analysis of the user reviews to extract the quality assessment of the urban spaces, then trained the neuron network to recognize the spatial features from the satellite photography and visual graphs based on the space syntax technology to derive conclusions about the relationship between spatial qualities and attractiveness expressed by the verbal assessment

food for thought

Can we adopt such a process to gather user data insights for digital product design projects?

Public square - Place Reial, Barcelona

SOLUTION

The attractiveness of a space is a measurable category

We identified correlations between spatial characteristics of the public space. Although the results of the relationships for some matrices were sparse, our experiment demonstrated that the attractiveness of a space is a measurable quantity

Results from the experimental study on a small dataset

METHODOLOGY

How can we assess and encompass a place's spatial features and the feedback of its occupants?

We created a methodology to assess the characteristics of a space by using semantic analysis from social media reviews, spatial features from satellite photography and visual graphs from space syntax application. Following that we added the features weight on a linear scale to get the KPI's as the output

Proposed Spatial design methodology

Process

We ran Polarity/Subjectivity Analysis onto the Dataset for each Public Square to get the overall popularity scores

I trained a LDA model to get coherence values for the number of topics in the data and identified the most popular words in each topic. The PCA analysis showed us which are the most distinctive topics from the dataset and we could extract 5 main categories from the data based on PCA. This was the first part of the analysis. In order to link this data to the features of the image segmentation , we identified the frequency of each popular word in each topic and assigned a value between 0 to 1 based on token values that we got from the PCA analysis. Here you can see the 14 categories of image segmentation that are relatable to the 5 categories of NLP.

Word cloud for the most common words

Sentiment Analysis on the social media reviews

Categorization of texts

METHODOLOGY

Semantic segmentation for Urban squares

Public spaces are different. Some are open and quiet while others are busy and compact, others wide and empty, while others are full of amenities and features. Having in mind the idea of creating a replicable method, we have chosen to work with semantic segmentation for satellite imagery which refers to the process of linking each pixel in an image to a class label – our urban features. By identifying these features – benches, trees, vegetation, pavements, fountains, etc. – we were capable of understanding the percentage of each one in a certain space, measuring the ones with more impact and understanding the proportions between them. By doing so, our perception from the spaces could be, in a better way, connected to the user experiences and their perception of the space.

Semantic segmentation

I created an experience map to identify the potential areas where CUI can add value to the service

EYE LEVEL VISIBILITY GRAPH

Visual Graph Analysis

Space Syntax has been investigated to incorporate a level of contextual urban data to enhance the comprehension of urban squares' characteristics. Urban square evaluations are generated for three space syntax analysis criteria using the DecCodingSpaces Toolbox plugin. Space Syntax values offer insights about eye-level visibility graphs originating from the public square's spatial arrangement in three aspects. The average visible area of a public square, the compactness which establishes the ratio of the square of the perimeter to the area, and ultimately the average value of circularity in a public square. Consequently, these space syntax values were combined with both sentiment analysis and image segmentation analysis for all public squares to better comprehend the correlation between these various analyses and deduce conclusions and understanding about the qualities that distinguish a good square from a less desirable one.

Density of Visibility

Visibility Graph Analysis

CONCLUSION

Next Steps

We created an analytical and design tool to facilitate the process of recognizing and transferring the successful urban scenarios. However, so far we have mostly concentrated on the analytical outcome of the A.I. application, while the design tool is still to be developed. There are a few directions for further experimenting: style GANs can be used to create an automated tool which will be adding certain formal qualities to the design. It can work in different ways: contextualize the design proposal or, on the contrary, translate it into the formal language of the prototype. Alternatively, a context reader can be created, which would generate surface textures for the urban interventions based on the context images interpolated in latent space. 2D-to-3D algorithms can be used to generate geometry from the raster patterns of the satellite or context photography etc.

Experimenting with a framework to check if AI can tell what User's might find attractive

Can AI help to predict what users find attractive or desireable?

User Feedback & Analysis with Machine Learning - an experimental study

My role in the project was to work on the Natural Language Processing - data collection, data cleaning, creating a ML model & data visualizations. Some images in the project may be low quality or blurry due to output from ML model

Sergey Kryuchkov

Pedro Ribeiro

Yara Gadah

TL;DR

Incorporating spatial analytics would allow us to gather key feedback & insights from big data. Can such a process be applicable for designing products?

food for thought

Can we adopt such a process to gather user data insights for digital product design projects?

SOLUTION

SOLUTION

The attractiveness of a space is a measurable category

We identified correlations between spatial characteristics of the public space. Although the results of the relationships for some matrices were sparse, our experiment demonstrated that the attractiveness of a space is a measurable quantity

Results from the experimental study on a small dataset

METHODOLOGY

How can we assess and encompass a place's spatial features and the feedback of its occupants?

Proposed Spatial design methodology

Process

We ran Polarity/Subjectivity Analysis onto the Dataset for each Public Square to get the overall popularity scores

METHODOLOGY

Semantic segmentation for Urban squares

Semantic segmentation

I created an experience map to identify the potential areas where CUI can add value to the service

I created an experience map to identify the potential areas where CUI can add value to the service

EYE LEVEL VISIBILITY GRAPH

Visual Graph Analysis

Density of Visibility

Visibility Graph Analysis

CONCLUSION

Next Steps

WORK

ai/ML explorations

social