“The highest education is that which does not merely give us information but makes our life in harmony with all its existence” – Tagore

Andrew Ng gave an interview to EE Times in early September 2023, where he talked about the next AI revolution coming to images and about a future with LVMs (Large Vision Models).

What is an LVM?

Large Language Models (LLMs) are now well-known thanks to ChatGPT’s phenomenal success. They can analyze and comprehend vast volumes of sophisticated data, including text, images, and other types of information, making them unique. These models analyze and learn from enormous volumes of data using deep learning techniques, which enables them to spot patterns, forecast the future, and produce high-quality results. Large language models’ capacity to build natural language material that closely mimics human writing is one of their main features. These models are helpful for applications like language translation, content generation, and chatbots since they can generate logical and persuasive written passages on various subjects. Similarly, LVMs can recognize and classify images with astounding precision. They can produce in-depth descriptions of what they see and recognize items, scenes, and even emotions that are depicted in photographs. These models’ distinctive abilities have several real-world applications in areas like artificial intelligence, computer vision, and natural language processing, and they have the potential to alter how we use technology and handle data fundamentally.

The applications of this are enormous. Imagine a computer looking into human tissues and counting the exact number of cancer cells. LVMs combined with the ever-evolving LLMs can count, classify, and predict the stage and rate of progression.

And like LLMs, LVMs can be trained through a technique called Visual Prompting. In this technique, a user prompts the model to produce the desired output by suggesting a pattern or image that the model has been trained to recognize and respond to in a certain way.

Here is an Example

Cell detection

On the left: We have the visual prompt showing the system a white space (long stroke) and on a cell by a dot.
On the Right: The system has detected cells on the petri dish, ignoring the empty space.

Some examples of LVMs are

CLIP: Developed by OpenAI, CLIP (Contrastive Language–Image Pretraining) is a vision-language model that’s trained to understand images in conjunction with natural language.

Google’s Vision Transformer: Also called ViT is a model for image classification that employs a Transformer-like architecture over patches of the image. An image is split into fixed-size patches, each of them is then linearly embedded, position embeddings are added, and the resulting sequence of vectors is fed to a standard Transformer encoder. This is now trained on 22 billion parameters. You can read about the latest update here.

LandingAi: Their flagship product, LandingLens™, is designed to make computer vision accessible to everyone. It provides an intuitive platform to create a custom computer vision project in minutes. You can upload images directly into LandingLens, label objects in your images, train your model, evaluate its performance, and deploy it to the cloud or edge devices.

What’s in it for Marketers?

Large Visual Models (LVMs) can be beneficial in several ways. Here are a few examples:

1. Data Analysis and Insights: LVMs can analyze large volumes of social media data, including text, images, and videos, to extract valuable insights. By applying natural language processing (NLP) and computer vision techniques, LVMs can identify trends, sentiments, topics, and key influencers in social media conversations.
2. Social Listening and Monitoring: LVMs can help monitor social media platforms in real time to track brand mentions, customer feedback, and emerging trends. Marketers can gain a deeper understanding of customer preferences, sentiment, and engagement levels by analyzing social media data.
3. Image and Video Analysis: LVMs excel at image and video analysis tasks, enabling marketers to analyze visual content. They can identify objects, scenes, and logos – even detecting inappropriate or harmful content.
4. Social Media Advertising: LVMs can enhance social media advertising campaigns by analyzing user behavior, interests, and demographics. Marketers can leverage LVM-generated insights to target specific audience segments effectively and optimize ad placements for better performance.
5. Social Media Influencer Identification: LVMs can assist in identifying relevant influencers for influencer marketing campaigns. Analyzing social media data and engagement metrics can help marketers find influencers who align with their brand values – having an authentic connection with their target audience.
6. Customer Segmentation: Large vision models can analyze customer data and segment customers based on their behavior and preferences. This allows businesses to target specific segments with personalized marketing messages, thereby enhancing the customer experience and increasing CLV.
7. Optimizing Marketing Spend: By identifying high-value customers, businesses can optimize their marketing spend to focus on retaining these customers and acquiring similar ones. This targeted approach can lead to a higher return on investment and increased CLV.

To realize some of these benefits above; the LVMs will need to be used in conjunction with LLMs.

LVMs are far from perfect and have a few issues related to hallucinations, labeling issues, biases, and privacy concerns but these current offerings will continue to evolve and find their place in the future of Marketing.