1. Both humans and machines are needed to deliver the best result.
Pinterest’s SVP of Engineering, Li Fan, shared the challenges and opportunities of creating a visual discovery engine. Fan described the challenges with naming an image while also understanding what’s in it and the style behind the image. Correctly defining image attributes is crucial for delivering a successful user experience. For example, a living room has multiple items in it. The company uses computer vision technology to break down the image, understand the objects within the picture and recommend similar things for you to consider.
Since there are 100 billion pins in the database, Pinterest can’t rank every pin for every user in real time. To accomplish this herculean feat, Pinterest uses a graph-based recommendation engine that filters the candidate recommendations for every user. It uses machine models to predict the engagement level of a Pinterest user to a pin and the relevance of pin for a user. It sounds like a simple classification problem, but the system not only has to detect an item, such as a chair, within millions of images. Additionally, the AI system has to understand the style of the object, requiring Pinterest to create feature vectors to help recommend an image based on a user’s style. Where does the human element come in? Data specialists clean, validate and label the data.
Every night that data is fed into a model and retrained to improve the result. It’s a partnership between Pinterest employees and computer algorithms. Other sessions discussed how people are working with machines to classify data but also to review the results. Takeaway: There’s a role for human plus machine collaboration. In fact, Paul Daugherty, Accenture’s chief technology & innovation officer, has written a book on this called Human + Machine: Reimagining Work in the Age of AI.