Mobile Visual Search Bridging The Gap Between The Real & Digital World

People are relying more and more on mobile devices to help them navigate their daily lives.

Whether it’s finding a place to eat, researching a product, or connecting via social channels for business and pleasure, mobile devices becoming are our constant companions.

And, as our reliance on smartphone and tablets to be conduits of relevant, real-time information grows, the lines are blurring between the time spent in the physical world and our experiences in, and expectations of, the digital world.

At the forefront of this narrowing gap between physical awareness and digital experience is Mobile Visual Search technology.

How Does MVS Work?

Mobile Visual Search, MVS for short, uses the images captured by a mobile device’s built in photo app to put users on the path to an immersive digital experience — including a route to purchase — linking the virtual and physical worlds.

Users start by simply taking a picture of an object. It could be flat like a movie poster, catalog or billboard; or, it could be a physical object like a wine bottle or pair of shoes.

After users store the photo in their smartphone app, a few things can happen. Users could be guided through a series of choices for additional information, or be given the ability to share content on social networks, or be provided with the option to find and purchase items online for delivery.

While, at first glance, MVS might appear to be a QR code on steroids — mobile’s first attempt at linking physical objects with online content — it’s different in a few important ways.

MVC technology uses image recognition to analyze and search a database based on the visual content captured by the phone’s native photo app. While MVS is often leveraged by a brand’s app, MVS technology doesn’t rely on a specific app, like a QR code does, to begin the search. It uses the more intuitive native photo app already in use on the mobile device.

Also, MVS doesn’t rely on a marketer to publish a bar code — any object can become the search criteria. This visual search content can be as large as a building or as small the print on the label of a wine bottle.

While Google Glass might be a sexy what-if case for MVS, early implementation of this technology by marketers tends to revolve around augmenting a print experience such as a billboard, catalog or poster. A good example of this technology is a recent campaign using Disney UTV Digital’s new smartphone app in India.

Product Placement

Users can snap a picture of a poster of a popular Bollywood movie and instantly be connected with more content such as movie trailers, behind the scene videos, and Tweets from the film’s actors.

The technology offers opportunities for new partnerships involving product placement, in which users can see a product, snap a picture and purchase it online via the mobile device at the moment of intent.

MVS Marketing

The opportunities apply to more than media companies. Retailers are exploring MVS technology as a way to collapse the distance between seeing and shopping; and these scenarios tend to use the 3-D capacity of MVS instead of relying on print.

A recent example is Macy’s campaign with NBC earlier this year, which let consumers watch clothes from different designers come down the runway. They could then shop instantaneously for these outfits from their mobile devices.

Macy’s is also using desirable gated content to encourage adoption of the technology needed to make MVS useful to consumers. For example, users are rewarded with special video content for downloading the app that utilizes MVS, ensuring that users are more likely to integrate Mobile Visual Search in the future.

Amazon.com has also incorporated Mobile Video Search into its Flow “augmented reality” app, which lets people take pictures of objects or UPCs or QR codes, and returns results — including product listings from its own store. The app is an ideal showrooming enabler, letting people see whether they can get a better deal by buying online.

Amazon Flow screenshot

In trying to better understand how MVS might be adopted in the future, we can look to Shazam, which takes a similar approach  — marrying audio and mobile together to give a user access to rich related content.

The Shazam app recognizes music and TV that’s playing nearby. In addition to identifying the audio content, Shazam allows a user to create playlists, watch previews, find lyrics, and more.

With more than 100 million mobile users accessing the Shazam app at least monthly (a 34% year-over-year increase), it’s clear that mobile users crave interconnected content delivered to their devices.

Boon For Users & Marketers

MVS engines may be a boon to both marketers and content-hungry consumers, but there are drawbacks. Searches must deliver fast results in order to provide a good user experience.

Shazam, for example, has a core team that focuses on improving their app’s speed. (Today, it takes less than 4 seconds for the Shazam app to recognize audio and deliver related content.) In addition, recognizing photographed objects can be challenging.

Each smartphone model will take a slightly different quality picture, and each object can be photographed in multiple ways, from varying distances, in a range of lighting conditions.

MVS accounts for these inconsistencies by analyzing the histograms, or distribution of data points, within the image, as well as the geometric consistency of the snapshot compared to the search result.

Mobile Image Search

Despite the challenges, interest in MVS is growing. In November of 2013, a Latin American telecommunications company run by billionaire Carlos Slim, América Móvil, invested $60 million in Mobli, a mobile image- and video-sharing tool.

And earlier this summer, Imperial Innovations invested £1.5 million in Cortexica, a visual search company whose main product, FindSimilar, delivers clothing inventory search results when a user snaps a photo of clothing or accessories.

MVS has some distinct promises for mobile marketers as it allows them to connect users with content that those users may not have found otherwise. MVS offers up the ability to collapse the time between seeing and purchasing a product.

When implemented properly for users, the realities of the physical world meld with the discovery and entertainment of the digital world. For marketers this integration of digital physical also means richer, more relevant data on users preferences and practices.

Opinions expressed in this article are those of the guest author and not necessarily Marketing Land. Staff authors are listed here.

About The Author

Lynn Baus

Lynn Baus is a Creative Director with Responsys — a technology company with a focus on cross-channel digital direct marketing solutions. Over the last seven years she has created campaign for leading brands, which have been recognized industry wide for their high creative standards and campaign performance.