Lint Roller? I Barely Know Her
PUT YOUR BEARD IN MY MOUTH
Sade Olutola
I'd rather be in outer space 🛸
KIROKAZE
No title available
No title available
d e v o n
let's talk about Bridgerton tea, my ask is open
Aqua Utopia｜海の底で記憶を紡ぐ
Jules of Nature
No title available
pixel skylines
tannertan36
DEAR READER
Love Begins
wallacepolsom
Cosmic Funnies
2025 on Tumblr: Trends That Defined the Year
Sweet Seals For You, Always
seen from Philippines
seen from Australia
seen from United States
seen from United States
seen from United States
seen from United States
seen from United States
seen from United States
seen from United States
seen from United States
seen from United States
seen from United States
seen from Russia
seen from United States
seen from United Kingdom
seen from China
seen from Portugal
seen from United States
seen from Italy
seen from United Kingdom
@businessintelligenceagility-blog
When the numbers don't add up
Sometimes looking at the data leads to the wrong conclusions. Analysis is going to become the real differentiator here and to see why I pull some information from a world war 2 story here http://www.motherjones.com/kevin-drum/2010/09/counterintuitive-world
See if you would have thought this through and its a relatively simple issue.
Leapfrogging the Tech Giants
With all the advancing architectures and the lowered cost to entry for analytics its interesting to wonder how someone might leverage that. In discussions with others in BI I get the feeling that something interesting might happen and that is seeing a sudden small startup suddenly gain traction and take down one of the giants. Not that it hasnt really happened before but in this case the time is right for some interesting changes. Fortune 500 players have complex architecture -its not really flexible or agile due to organic growth, data demand and data sizes. What could happen if a well funded small player leveraged the cloud for fast launch of analytics and big data architectures for managing scale and being able to look for trends, personalize to a customers key demands and minimize costs. Could an analytics ninja and a supporting architecture be used to a massive competitive advantage and revenue gain so as to allow a small company to leapfrog the big guys? Could this happen? The time is right.
Data Governance in a Big Data world
As we move into a Big Data Architecture what are the things that we need to be aware of the most? Well Governance is a big one, if you dont have a mature process around this you could put yourself at significant risk. Think about it like this if you have not defined customers and products well and have a plan for managing this as new data arrives then bringing unstructured data and trying to get to know the customer 360 personalization becomes hazardous. Big Data brings new sources into play and as such you need to recognize that these are not internal systems. You have multiple apis and changing web formats. From Twitter, Facebook and Instagram there is a changing landscape of data and structure. To manage this needs a specific data steward for each one. Also managing the change in the right architecture is going to be needed. So updates and flexibility are the things you need to think about. Nothing is fixed, its changing and your architecture and processes need to reflect that. Make sure processes are mature before adding Big Data into the mix.
Why not to discuss Big Data with Business
Achieving Intelligence is a business responsibility. It must be driven to, adhered to and leveraged. IT supports this with the right architecture and should be aligned to the same goals. However Big Data is not a goal or a business need, the intelligence is the need, the questions being asked are what are important. Where the connect occurs is when the business realizes it has the capability to ask a question for which an answer can be supplied. Simple enough but really the connect is a little more complex than our first look. Questions are often ad hoc and time boxed. Marketing campaigns need feedback quickly, sentiment analysis requires the ability to measure sentiment sometimes realtime. Timing will then relate to a possible Big Data Architecture or it may be the size of the data set. The business does not care, let them ask the questions be ahead of the curve on what they are asking and have the answers. Right architecture for the maturity of the organization today and also for tomorrow. Tricky balances that everyone is trying to handle and something that we will discuss in future posts.
Customer Centric
The point of a lot of redesign of BI systems is to be able to get a better view of the customer that enables multi channel capability. So what do you do? Shelve everything that came before or integrate into what has been done. To better understand the dilemma it is not just about what you need now but also what you need later. Everything you need to do needs to be flexible and somewhat scaleable. As use cases appear and the rise of ad hoc queries that can make considerable differences in revenue proliferate there needs to be a way to respond. You wont always get the use cases you need right but you need to be able to emulate what others are doing. Flexibility is the key here make sure you have a platform that support current and future needs. Flexible platforms mean many things. Ease of integrating new data, speed of integration, ability to virtualize data quickly, fast automated testing with regression testing are all part of the puzzle not just the choice of hardware and software.
Learning from the top performers
So lately in my discussions with customer executives I noticed an interesting similarity. It was a huge appreciation for what Tesla has done releasing its Patent Technology. Now it may not be why you think but what I learned is that a lot of customers see this as a huge reason Tesla is so well regarded. They see Tesla as a very well run company that is not afraid of competition. Imagine being so confident in your business that you would help others knowing they would not be able to compete with you? The only way to be that company is to have a mature BI capability and strong process driven initiatives that are best in class. Imagine being able to execute with that backbone? Best in class solutions enable this for customers and thats why focus on driving a metrics and data driven organization will eventually payoff, its a backbone to keep you solid as your business expands.
Exposing the trends really driving Big Data
Lets cut to the chase. Whats the big deal with Big Data? What is really driving this trend? What we hear about are the few critical use cases that are beginning to resonate with their respective industry. These are not what is really driving the push here, I would posit that its actually something far more interesting. As the industry has seen Big Data has not really stolen the crown from the standard data warehouse methodology. In fact its more likely that it has solidified the need for the historical contextual and trustworthy data stores. More so the drivers of need here are the need to be agile, the need to be cheaper and the need to compete. It is the time to actionable data window that is critical and the business is always changing. When that is the need then the data warehouse is not the answer, its too process oriented (by design) and hence too slow. Smaller and mid size companies want answers as well and in an age where companies can suddenly generate or beat the market cap of Microsoft within a short time span there needs to be low cost tech that can get the job done. Thus cost containment can be seen as part of lowering the overall IT budget but also as the need for support of smaller to midsize businesses. Expect the innovation and use cases to start showing here where there is more flexibility.
Analytics: Next Level
The rise of companies focused on specialized analytics is going to become commonplace here is an example of some of the fascinating work that is being done. http://darkhorseanalytics.com/blog/breathing-city
Analytics: Next Level
The rise of companies focused on specialized analytics is going to become commonplace here is an example of some of the fascinating work that is being done. http://darkhorseanalytics.com/blog/breathing-city/
Data driven Company
Once architecture is up to scratch what is next. Driving standards around analytics is a good start to getting efficient decision making but beware of over doing it. Consulting for road map development and driving CPM engagements I have seen many of the fortune 500 drown in their own numbers. One company I surveyed used about 200 metrics to measure its business units. Now while this is more performance oriented than other decisions the basic premise that you can run an organization and drive to that many metrics is fundamentally flawed. Take a simple approach, if you cannot do it with five or less then you have the wrong metrics in place. Use a simple philosophy to hone your numbers to a critical few, complexity and sophistication do not necessarily go hand in hand.
Simply collecting Big Data does not unpack its potential value. People need to do that, and those people are hard to come by. Just 3.4 % of CMOs surveyed by McKinsey in 2013 believe they currently have the right talent to fully leverage marketing analytics, and 98.8% describe finding that...
Organizing to Execute
So should you build out a COE of Analysts and Statisticians once you have the architecture settled? For most people the word COE has a pretty negative connotation and for good reason. Usually associated with failure to execute. You do not want a set of statisticians in a separate group for everyone to access as needed, This will not drive the Q&A type of interaction you need. Instead embed analysts into your business units and establish a culture of analytics, Standardize how you review and manage with data. Do not look to data scientists and data mining to solve your problems without setting the organizational construct for success
Do not assume Big Data is a cure for what ails you
As companies revamp architecture look to manage the change with reasonable steps. Leverage Big Data (see previous post on justifications) but do not assume this is the fix. Data sets and Analysis need to be in depth. Architecture is a start but the process of Analytics needs to become endemic to your company. Everyone needs to become an analyst and a process and strategy needs to be put in place to do that. Just because you have data does not make that Intelligence. Intelligence comes from asking questions and getting answers, You have half.
How to prepare for the future with Big Data Architecture
Having trouble bringing in Big data initiatives into the company? Many feel that the use case for this is still out there. Not quite the slam dunk the industry thinks it is going to be. Rest assured that this is just because there is a disconnect between technology and what people think that analytics can do for them. This will resolve itself in various domains when the use cases become clear. While you are waiting for this to happen how can you get Big Data into your bag of tricks? One way to look at this is to bring it in to lower your licensing costs. High cost platforms that license on processing power etc( that store high volumes) can be repurposed to focus on analytics. Archive off the stored data to a big data platform. When the need arises the proactive nature of this approach will gain you precious time.
Business Intelligence as a Self Service
The trend now is to disconnect the business users from the data integration work. Why? Well it enables data integration to happen while the Business user can focus on more than just reading reports. The onus moves to the business to drive BI and leverage the metadata and the middle layer to request data and build reports. The ability to do this relieves IT of having to focus lots of resources on report building when this is really the realm of the business and focus on data delivery. To get to this nexus of BI takes a few things. Process, Architecture and Analytics. You will need to be able to measure the uptake of BI, so a method of seeing what reports are being used, a way of getting meta data on what reports take time, need to be optimized etc and which ones should be used (rationalization) will be required. Also a standard around managing data clarity that is managed by business imperitives needs to be created and managed. Once these process frameworks are available then architecture revamp can occur. More to come...
Decisions and Architecture
As we move our customers into a new architecture we noticed that a lot of IT execs were concerned about how the business might leverage the new architectures. Big Data infrastructure is not necessarily a huge investment to start out but the time and effort to get it all functional is still there. There is also the need to train or pay teams on the new tech. Reluctance from customers to pull the trigger on new capability is driven by the lack of business users asking the questions. Imagine if you will a customer that realizes that there is a new capability in IT; that doesnt necessarily reflect on their ability to think about how to use the capability. Instead look to develop the right use cases for which the answer is Big Data. For instance when you need to focus on sentiment analysis and combine facebook feeds with Neilsen data. This allows you to track users and their opinions based on their viewing habits. Standard databases needing to crunch the large amount of data for an answer will not perform as well as a Big Data solution. However this is the year where the questions are beginning to get asked and our clients that ask the right questions and have the architecture underneath will achieve competitive advantage.
Referential Architectures and Data Concerns
Current technologies reflect the need of the business at certain times. The need started with managing data effectively and Transactional and Relational databases responded. Analytics were required and OLAP became mainstream. Businesses balked at dealing with slow IT and behold Qlikview and Tableau. Cell Phones, Social Media and the Internet made demands of data that were not thought of. Volume and Velocity suddenly become important. There is talk that the data warehouse days are numbered. I dont believe that to be true. They just are not set up to deal with the current need for getting data quickly to a point where analytics can take place. Time to Analytics is key and the standard path looks like this> Data Source sends data to external source. Data Integration takes place, business rules applied. Aggregation happens in the OLAP component and then the business can look at it. There is a production window for this that usually starts when a business ends for the night. The data is offloaded and manipulated as discussed and then sent to the data warehouse and then aggregated in OLAP (not always but assume so for the lesson). At the end its available for analysts to come in and slice and dice it. This usually should happen in the production window before morning. Companies usually underestimate the amount of data that needs to be managed daily.
Assume that within the next few years the amount of data managed by these systems will increase to 30TB for midsized companies. How do you manage that in the context of the current architecture?
Welcome to the Big Data and the New Architectures. Hadoop, Columnar databases and Correlational Databases. A brief introduction will be given here. Note these architectures are complements to the old not replacements. However the lesson to remember is similar to what occurred in India with Cellular Infrastructures. Most of the original hard wiring was so difficult to implement in areas where zoning was impossible to even understand that a lot of people skipped land lines and went directly to the new tech. This is very similar to what Google, Amazon and FB are doing with technologies such as Hadoop which previously did not need to exist.
Small and midsize companies can benefit from this technology and become extremely competitive in a short time. The advantage the larger players will have will relate to their ability to manage their own internal knowledge. Knowledge management is a crucial value to companies that seek to stay ahead of smaller companies that can seemingly now come out of nowhere. Lets look at some of that tech.
Hadoop was brought about by the need to look up information through massive libraries at a fast rate. This can only be done by parceling out the work and letting the servers take on a piece of that work, the intelligence lies in knowing how to split the work across servers and you can maximize this by keeping servers close to each other (in the same rack etc) or understand what the split is of servers that are close and those that are far. This is why the Cloud is not the best architecture for Hadoop since you remove a critical element of the original design which is knowing where the servers are in relation to each other.
Columnar Databases are akin to keeping data in columns instead of tables. The advantage would be that for the most part most sql statements on a relational database would need to go through the entire table to find everyone who lived on the same street. Whereas in a columnar database if you were to keep street data in alphabetical order the search would know where to start and when it had read the last record so it might only need to read 100 records from a very large set. This reduces the work significantly.
Correlational databases are the minimalists of the database, think of it as storing something unique only once. For instance if someone is called James then his name is stored in the name record. Then he is stored in the table as 1. If someone else is called James then you dont save his name again, you just save him as 2 in the table. To illustrate this. In relational you would have James stored twice as String type with a allocated size. In Correlational you would have James, 1, 2. The first as String the next two as integers (possibly a considerably smaller footprint to the original. As the database increases in size the payoff in managing size becomes exponentially better.
So those are the newer types of architectures that you should expect to see and if you are at all concerned about what sort of trouble the business is going to get you into next make sure that you get some POCs done and have some good consultants help you with the vendor selection.
