power of data @powerofdata-blog - Tumblr Blog

so much to do in mobile - cross device tracking & measurement - part 5

How many tracking tags does your site have? On one site I found 70.

What do all these tags do? In my opinion, they fulfill mainly three functions.

1) Web analytics. Funnel, page flow, traffic source.

2) Cross-channel marketing attribution. Either single touch or multi touch.

3) Optimization. DMP, DSP.

Optimization, especially real-time optimization, is the main driver for multiple tagging. Optimizers prefer collecting the data directly from the site, rather than depending on another third party data provider.

The sheer number of tags and the lack of data control gave birth to a new type of data service - tag management system. Tag management tools are designed to satisfy all three functions. In doing so, they risk becoming too complicated themselves. Major players in this space all offer their own packages. Many smaller players provide niche solutions.

Same tracking functions are needed for mobile apps. However, mobile app has much smaller real estate, therefore presents some unique challenges.

- Mobile advertising is segmented. Some companies wall off competition by not recognizing third party tracking. This does not result in the intended monopoly of marketing dollars. Most companies still buy from multiple publishers and ad networks. And multiple sdks get added to the app. Everyone recognize not de-duping among traffic sources is an issue, which leads to duplicate payout. A number of companies arise to address this challenge by being the mobile attribution engine. The key to their success is the acceptance by dominant publishers, namely, Facebook and Google.

- There is yet a sdk management tool for apps like the tag management tool for website. Few sdks are truly server side codes. The ever changing mobile landscape forces companies to keep on updating sdks. All add to the burden of app developers. Since app updates never capture the whole user population, we will always have "residual" data from the older version.

- Cross-device tracking is evolving but yet a reliable operation tool. Google published studies on cross-device usage to demonstrate their mobile AdWords effectiveness. In these studies Google may use their own log-ins to link users across devices. However, when they offer cross-device analytics product, they rely on the client's user IDs. The difference is Google's coverage is substantially larger than the client's. Facebook recently announced a cross-device tracking and advertising product. I'll be waiting to see how they tackle the problem. Mostly likely Facebook will limit such insight within their network. Another cross-device marketing provider Drawbridge is brilliant in finger printing, but cannot share such insight at the user level, which makes the intelligence unusable to a company's other marketing programs. By consolidating data across devices we were able to make progress in building a cross-device user profile. Now we are investigating ways to push such insight out to the campaign level in real time.

so much to do in mobile - cross device tracking & measurement - part 4

Once we have all the key metrics tracked across devices, linked by user identifiers, the possibilities of analyses are endless! Here are just a few examples of interesting and actionable insights.

Device overlap. Increasing number of users engage with a brand on multiple devices. Counting the purchasing customers on different devices, a company may produce a device usage map such as the following. In this example, majority of users are still buying on the website. Tablet offers a sightly better purchasing experience than mobile phone. Users who purchase on all three devices are a small population. The data becomes more interesting when another metric, such as revenue, is overlaid on the customer counts. The revenue per customer in the small all-device group (g) could be three times higher than the computer group (a). When users use another device in addition to the computer, their values double that of a single device user. Of course, loyal customers are eager to adopt mobile therefore higher in value. However, if by engaging users on one more device can double their value, it will change the formula a company uses to calculate the cost per app download.

Funnel. Marketing dollars are spent to bring users through the "front door". Of all the traffic coming through the "front door", how many exit with a positive ROI? And where do users drop during the conversion path? The questions are best answered by the funnel analysis. The same concept applies to mobile. Companies need to compare funnels of different devices side by side in order to understand which provides the better user experience and ROI. The challenge is to produce comparable metrics using different methods. For example, most web analytic tools count a new session when the browser is closed, or idle for a certain amount of time (typical 30 minutes). App open is the comparable metric to web visit. However, when user exits an app, it still runs in the background. Most mobile tracking tools break the app sessions by a 30-minute interval. A lot of data consumers do not understand such subtle differences and can risk misinterpreting the numbers. When properly constructed, the funnels can be quite revealing. In the following example, the web and app are similar in persuading visitors to start the order process, but the app has an advantage in closing the deal.

User mobile profile. Most analytics are done at the aggregate level, while marketing is delivered and product is experienced by each individual user. There is great value to include mobile metrics in the traditional user profile. In the following example, the two users may look the same by the spend level, but they are unique. They have different preferences for marketing communication. User A cannot be reached through email or display campaigns, but is receptive to push notification. User A is active on web first then migrates to mobile, while User B is a mobile only customer.

Time spent. Time spent is a telltale for user engagement. I used to ask interviewees this question. If a website has twice longer average time spent on site than another website, what does it mean? Those interviewees who understood this can be both a good sign - that users are more engaged, and a bad sign - that users are confused, are those who got to the second round. With the competition for users' attention these days, users are less patient. Instead of trying to figure out how to use a product, they may simply uninstall the app. Getting them back requires paying the acquisition cost twice. In addition to the average, extremely short and extremely long time spent should be closely examined. If the website and app provide the same feature, it will also be interesting to compare the time spent on the two versions. Another time spent related question is the "cannibalization" between mobile and non-mobile. Fewer and fewer people are asking this question. Why? Maybe because businesses no loner look at mobile as a "stealer". Regardless users are spending more time in total on all devices, or shifting their time from one to the other, companies want to provide a fluid experience wherever users are.

so much to do in mobile - cross device tracking & measurement - part 3

With the user identifier mapping in hand, now it's time to join the data and see the full picture! But wait, where is the data?

The first data source comes to mind is the many mobile tracking and measurement providers. How many? Hundreds. Even with Facebook there are a dozen "trusted" mobile tracking providers including their own solution. The abundant choices make choosing difficult. Using kitchen as analogy again. By going with one solution you may feel you are getting the power of a gas stove, but losing the sleekness of an electric one. Not surprisingly, a lot of companies ends up installing multiple "stoves". More on the mobile tracking solutions later.

The often overlooked data source is actually the data currently in the database. Think about the key metrics. For an eCommerce company these may include registrations, visitors, sign ins, add to carts, and check outs. Most key metrics are collected together with a user identifier. However, the data don't immediately tell the story. Why?

There is 17 years gap between the introduction of first web site and and the first mobile app. Among the Nasdaq companies, ~30% IPOed in the past 17 years. When the other 70% companies started before mobile taking over the world, their databases were not built with the notion of "cross-device". A simple question like "how many users who registered on the mobile device purchased on the web in the past 30 days" requires both registration and order to be stamped with a device indicator.

Older databases, like older codes, are almost always more difficult to change than writing the new one. But change they must. When the key metrics are consistently tracked across devices with the device indicator, linked by the user identifier mapping, the analysis can really begin.

The definition of "device indicator" may differ by businesses. Some need only high level categorization such as "desktop", "phone", and "tablet". Some need go down to the granular level such as manufacturer and screen size. By recording the user agent, it allows the maximum flexibility in mapping out various level of granularity. Open source user agent mappings are available. They are not 100% accurate, but more efficient than maintaining such mapping in-house.

so much to do in mobile - cross device tracking & measurement - part 2

The best linkage of a single user across devices, obviously, is the User ID, Data with User ID can only be collected after user logs in. Logging in used to be a big deal. Sites worried about scaring users away by forcing them to log in too early. The proliferation of subscription based services (such as the flash sale) and apps have eased users' concern. Or maybe today's users have higher tolerance of less privacy.

If the log-in is not by default, such as in the case of app usage, the log-in prompt should be placed at critical points of the user engagement, a.k.a. "money pages". Once the data is collected with the User ID across devices, it can then be joined by the User ID and sequenced to paint a complete picture of user path. For example, a user may select an item on the desktop web and place it in the cart; and check out the item on app a day later. If we only look at the revenue coming from the app, we might miss the fact that the whole shopping experience started on the website.

The pre-login data also holds rich information, especially the data indicating traffic sources. When a user clicks through a Google keyword and lands on the homepage, she/he is unlikely signed in. So a secondary linkage between User ID and other user identifiers such as site cookies and device IDs is required. The key is to collect the pair of User ID and another identifier; and join the multiple pairs to create a user identifiers mapping. Such join usually needs to be ran on frequent basis, if not daily, to keep the IDs fresh.

Of course, a user can opt out and "go dark". It is safe to assume about 30% of the web users either reject or clean their cookies so often that their data is hard to interpret. The number of users who opt out mobile device ID is probably much smaller. In the analytical world we often choose not to worry about the data we cannot collect. Only when the lack of data is so significant that the collected sample is no longer representative, other technical alternatives can be explored to address the issue.

Policies around user identifier collection and usage are shifting sand, more so in the mobile world. Term of use and privacy policies should be clear about what information the company will collect and for what purpose. An app risks rejection if the data collection practice is questionable. In any case, we are prepared to give up data collection in order to preserve the best user experience.

so much to do in mobile - cross device tracking & measurement - part 1

How do you analyze four websites, four mobile phone websites, four tablet websites, and ten apps all together, to find out how your customers are interacting with all or a subset of these products?

How do you then use the data to decide how to market these products, individually or together? What marketing works the best for the product independently or combined? And how spending on one product may have ripple effect on others?

For the companies who started on the web, seeing increasingly traffic coming from the mobile devices, and rushed to build mobile products, these are our challenges.

Customers are oblivious to the technical challenges we face, and expect a coherent experience across all devices. One example of such challenge is a common cart. A customer who placed items in her web cart expects to see the items in the cart when she logs into the mobile app. If she started a custom item on the web, she expects to continue the customization on the mobile devices. Not providing such coherent experience is like splitting her kitchen into many different ones. Each kitchen is well equipped but with different functions. In one kitchen she can only bake, in another she can only stew, in the third she can bake and stir fry but not stew...To make a dinner she needs to remember there is bread baking in number 1, stew in number 2 (or maybe number 3?), and stir fly in number 3. To "help" her to remember, the kitchens start to send her updates. She receives ten messages from each kitchen and quickly loses track.

How do we, the data providers and analysts, build one kitchen for her, and hide all the complexities from the public view?

(to be cont...)

#mobile marketing #mobile tracking #moible measurement #app measurmenet #cross device tracking #cross device measurement #cross device marketing #cross device attribution

powerofdata-blog

While putting together a strategy deck for building a business intelligence system, I started to list the best practices from years working with great teams; designing and implementing systems, databases; resolving operational issues; exploring and persuading the data; pulling and distributing the...

Will you sell the data about youeself?

Wired featured this article "Mad Scientist Sees Future Where We Sell Our Quantified Selves on eBay". I don't think the idea is crazy at all. As a veteran in online advertising, I can testify how much I both loath and love the ad customization using my browsing history. They can be either brilliant (tour recommendation right after I search "Galapagos trip"), or stupid (I do not need another t-shirt after purchased one just minutes ago). I have envisioned a personal data management portal where I go and volunteer the data about me - gender, income, interests, shopping habits, friends circle, profession, desired travel destination, favorite cuisine, hobbies. What I want to get in return is highly relevant information - Google search and recommendation engine combined, delivered with offers like Groupon. Ad networks can use my info to sell advertisement, as long as I earn credit (something like bitcoin) every time the impression is served, ads clicked and actions as a result of using my info. The tracking, counting, and payment progress will be complex but not impossible to build.

A couple of years ago I asked a Facebook Product Management whether they are developing a photo advertising engine. If I have a photo holding a certain product, Facebook can tag me as a potential customer for that brand and serve me an ad. They can also serve the ad to my friends. On both occasions I'll earn credit.

No matter how much we are concerned about privacy, we can no longer live in a world without personalized information, either pulled from us or pushed to us by the information providers. Personalization sucks in most cases because the data providers have to guess who we are and what we really want. Take that guesswork out of picture, users will stop being bombarded by irrelevant information, and publishers save the trouble treading dangerously into the privacy territory.

vnow

Does it take a professional designer to make a comfortable living space and intuitive products? I asked this question while vacationing in Shanghai. Many of the products I used every day are generic and bear no designer’s name. Without the magic hand of a designer, the knowledge accumulated...

we volunteer to be tracked

An interesting article touches upon some the same thoughts regarding voluntary data collection.

Your life as Data: The Rise of Personal Annual Reports

#data collection #tracking

is the whole world trackable?

In a Cleveland clinic, the compliance team watched medical staffs following hygiene guidelines (such as washing hands) and uploaded their observation into a database. Report was generated and presented to the staff, allowing them to see the difference between their true and believed behaviors.

Global Pulse, a UN initiative, aims at collecting and analyzing data points around the world (such as twitter) to recognize the human well-being.

Low cost chips are put into parking meters in San Francisco, collecting data on the city's traffic pattern.

GPS enabled device are attached to children. Parents can monitor their baby sitters through video camera installed at home in real time.

If every interaction point, be it between human, or between human and our environment, becomes a data collection point, imagine the insight we can draw from it! When we meet a friend, count one. When we meet a new person, the device records a new entry, with this person's publicly available information. When we drive, the device records travel time, time stuck in the traffic, and the radio we listen to. When we eat out, the menu is scanned to record our intake. When we eat at home, our fridge keeps track of our food consumption. Each individual who opts for the data collection should own our own data, and choose whom to share with. If the data has commercial value, the data should be priced. Revenue from the data licensing can be donated toward building shared data infrastructure and storage, and funding social, medical and scientific researches.

With the development of technology, more and more of the world will become trackable and our knowledge of it digitized. Will the understanding drawn from such data change human behavior? That is still to be seen.

How Reporting, Analytics and Metrics Affect Outcomes and Save Lives

Showing Them What They Do, Not What They Think They Do

Greg Gillespie Information Management Newsletters, February 9, 2012

#data #tracking #global pulse

from machine back to human intelligence

The question of whether machines are more supreme in intelligence than their human creators keeps on coming up to me. I remember reading an article on computers developing formula based on data so complicated that human cannot possibly recreate the logic in our minds. The researchers need to put their faith into the accuracy of the machine and use the formula without attempting to understand the process how it is derived.

In the business intelligence world, data mining ultimately leads to business decisions - marketing, merchandising, pricing, etc. A few weeks ago I was in a co-founder meetup. A speaker with financial background pitched an idea of real-time social media data crunching. He believes the same algorithm that powers machine trading can be extended to the social data mining. For any autonomous algorithm to work, the machine needs to be told what to find and what to do when the pattern fits. In trading, it might be to locate price point difference across a wide range of locations and stocks. In social media, the questions that we need the powerful machines to help to answer and eventually respond automatically to are not well defined yet. How businesses can gain competitive advantage if they recognize Jeremy Lin is the new NBA star one day earlier than everyone else? If the "formula" derived from "big data mining" is counter-intuitive, should businesses trust its value and make decision upon it anyway? By crunching a larger volume of data within a shorter period of time, are we looking to distribute a localized recognition much faster to the wider population, or are we implying that the results will be most likely surprising and even confusing to our human brain.

http://www.nytimes.com/2012/02/12/sunday-review/big-datas-impact-in-the-world.html

The Age of Big Data

By STEVE LOHR

Published: February 11, 2012

For those who can make sense of the explosion of data, there are job opportunities in fields as diverse as crime, retail and dating.

#big data #data mining;

How big can Facebook be?

There are a LOT discussions around FB's worth as it's approaching the big date. I am truly happy for my friends working there, especially those who had the vision to join a little know company years ago. The IPO will be successful. The market will decide as time goes the "true" value; and it may take years to establish a revenue history. However, in lack of good data, let's we assume FB plays an increasingly important part in three revenue generation businesses:

a) Online advertising (both US and global)

b) Online gaming (both US and global)

c) eCommerce (both US and global)

Global data is harder to find, here are the US numbers (2011 rough estimate)

a) = 32 billion

b) = 100 billion

c) = 190 billion

So how big a pie can FB get from each?

#Facebook #IPO

Data mart vs. data warehouse

I've used the terms for years. This is a good article defining the differences.

"What is a Data Mart?

A data mart is a collection of subject areas organized for decision support based on the needs of a given department. Finance has their data mart, marketing has theirs, sales has theirs and so on. And the data mart for marketing only faintly resembles anyone else's data mart.

Perhaps most importantly, the individual departments OWN the hardware, software, data and programs that constitute the data mart. The rights of ownership allow the departments to bypass any means of control or discipline that might coordinate the data found in the different departments.

Each department has its own interpretation of what a data mart should look like and each department's data mart is peculiar to and specific to its own needs. Typically, the database design for a data mart is built around a star-join structure that is optimal for the needs of the users found in the department. In order to shape the star join, the requirements of the users for the department must be gathered. The data mart contains only a modicum of historical information and is granular only to the point that it suits the needs of the department. The data mart is typically housed in multidimensional technology which is great for flexibility of analysis but is not optimal for large amounts of data. Data found in data marts is highly indexed.

There are two kinds of data marts--dependent and independent. A dependent data mart is one whose source is a data warehouse. An independent data mart is one whose source is the legacy applications environment. All dependent data marts are fed by the same source--the data warehouse. Each independent data mart is fed uniquely and separately by the legacy applications environment. Dependent data marts are architecturally and structurally sound. Independent data marts are unstable and architecturally unsound, at least for the long haul. The problem with independent data marts is that their deficiencies do not make themselves manifest until the organization has built multiple independent data marts.

What is a Data Warehouse?

Data warehouses are significantly different from data marts. Data warehouses are arranged around the corporate subject areas found in the corporate data model. Usually the data warehouse is built and owned by centrally coordinated organizations, such as the classic IT organization. The data warehouse represents a truly corporate effort.

There may or may not be a relationship between any department's subject areas and the corporation's subject areas. The data warehouse contains the most granular data the corporation has. Data mart data is usually much less granular than data warehouse data (i.e., data warehouses contain more detail information while most data marts contain more summarized or aggregated data). The data warehouse data structure is an essentially normalized structure. The structure and the content of the data in the data warehouse do not reflect the bias of any particular department, but represent the corporation's needs for data. The volume of data found in the data warehouse is significantly different from the data found in the data mart. Because of the volume of data found in the data warehouse, the data warehouse is indexed very lightly. The data warehouse contains a robust amount of historical data. The technology housing the data warehouse is optimized on handling an industrial strength amount of data. The data warehouse data is integrated from the many legacy sources.

In short, there are very significant differences between the structure and content of data that resides in a data warehouse and the structure and content of data that resides in a data mart."

History of online marketing - Affiliate

http://en.wikipedia.org/wiki/Affiliate_marketing#Origin

History of online marketing - Internet

http://www.fastcompany.com/magazine/139/click-here.html

History of online marketing - Email

http://www.marketingtechblog.com/infographic-the-history-of-email/

Reflection upon a decade of building business intelligence

· Everything starts with good tracking.

· But don’t let tracking run wild, track the “money pages”.

· Data transfer from and into the company can be painful, collect as much data as you can using your internal systems.

· Utilize every data points you pay for. Retire the data nobody uses.

· Buying the best BI tool does not mean you get the insights automatically. You still need to tell the tool what’s important.

· Your data supports both your current and long-term needs. Organize them thoughtfully.

· The whole company use the same data at various levels – executive reporting, ad hoc analysis, model building, marketing optimization, product design, build into layers of data marts to ensure efficiency.

· Connect data points from various sources. Only then you can discover what’s not obvious.

· Consult experts within and outside the company. They’ve made the mistakes you’re about to make.

· Data quality is ongoing. You need someone to manage it ongoing.

· You need an overarching function to connect the dots. Avoid silos from the beginning.

· Save everyone five minutes in accessing the information they need. And then five minutes more.

· Don’t marry to a BI tool. The industry is always innovating, and gets cheaper.

· Data is a beautiful thing. So design your presentation layer to please your users’ eyes as well as their minds.

· Your organization will always view BI as an expense item, even it drives revenue eventually. Agree on the success metrics before you take the check.

· You are not only building databases, reports, tools, etc., you are also building processes, and a data consumption culture.

Trending Blogs

Recently Viewed Blogs

power of data