What's an ODS?

An ODS is a database designed to integrate data from multiple sources for additional operations on the data, for reporting, controls and operational decision support. Unlike a production master data store, the data is not passed back to operational systems.

Think of an EDW as your long term storage and an ODS as your Short term storage.  For Example: an ODS is 3 or less years of Data and your EDW has 7 years of data.  The ODS will aggregate your feeds from source systems and will feed your EDW.

In summary, an ODS can provide benefits that include: Offers scalable and flexible control of data per constantly changing business requirements. Handles large data sets effectively, sourcing from one or multiple systems/databases.

3 ways to move users from Source system reporting to the EDW for reporting

When a company builds an EDW/ODS and sets up that environment, best practices would call for all reporting to be moved to the data warehouse.   How do you move users to this new environment and maintain your BI Manager job?   That can be tricky.    Change is not easy.   Especially for users who have run reports and dashboards the same way for years.   I want to show you 3 ways to move reporting from a source system to the warehouse.

First, way to move folks from the source system to the new EDW is to start with monthly reporting.   Why?  Because the data warehouse is typically a 1/2 day to day behind the source system because the warehouse is based on a cycle.   If the cycle is updated twice a day to every day then the data may seem stale to a user who is used to having the latest greatest data at their fingertips.   As a result, I would move the monthly data to the new platform first.   This would have to be monthly reports for the executive level and not a management level.   You need buy in from the top level who then rubber stamps the move for the levels below.   I would even say that this first level of data should be a dashboard instead of a report.  A dashboard is a way to present data in a summary fashion where leadership can view their KPI's at a glance instead of buried in a report.

Second, most EDW's have a core system where the power users reside.   This is the best way to start your move.   Move the core data to the warehouse and then tackle that data first.   That way you develop momentum to moving the rest of the data.    This should be the data that everyone knows about.    Think about this.   What is the largest bucket of data?  Or perhaps it should be this.  What is the most important bucket of data that will reside in my warehouse.   You have to decide which one is the best move.  Perhaps this is the group that has the most reports or at least the most used reports on the source system.  The key is to win this group over to the move.   You should have a great relationship with this team already and they will be your champion group.   

The first suggestion was to move the monthly reports first, the second suggestion was to move the power users group or the group using the largest data set.  Third is to communicate the positives.   An IT Marketing campaign if you will.  You will benefit by...You fill in the blank.  A positive spin on this with all of the latest tools will be crucial.   Training on new tools can be  a good way to plan for this move.    For example: Qlik training or Power BI training.  Plan for your vendor to offer some new user training close to the cutover using your data.  If the vendor uses your data then the training will make sense to the users.  What tools do you have for the new reporting tool that are missing from source system reporting?  What new functionality will be crucial to your customer?    

I hope that this has given you some practical ideas for your EDW/ODS implementation.   Change is difficult but crucial to moving your business towards best in class Reporting/Dashboard solution. 

The missing team member

I was on an engagement and they had a small BI team there.   The SQL guy.  The ETL guy.   The Qlikview guy and then I was inserted as the BA.   I came in and was asked to document everything they had so far and to add processes to the madness.  So one of my tasks was to understand the data.  I documented everything.  I came in and saved the day.  The existing team was good at developing reports.    I came in and developed functional requirements documentation, technical requirements documentation and then documented report to Source documentation.  

Why do I bring that up?   BA's are sometimes an afterthought, especially for a small company that has limited resources to dedicate to technology.   The role of the BA is extremely important to develop a relationship with the business, to document what the business is asking for scope, and the ensure that the IT/Technology teams deliver on those requirements.   One of the issues with this team was the impact of the no input process.  For example: what were the requirements.   Are the requirements documented?  The last thing your team wants is for the business to start asking questions down the road of where is this data or that data because their request is not only documented but signed off on those requests.   Additionally, sometimes your team gets so interested in pushing out code and reports that they forget to check little things, like spelling, data accuracy, layout. 

The BA can help to find all of those mistakes by writing test cases that make sense.   By documenting those test cases against requirements and then ensuring that the new code didn't break the rest of the code.  It also helps to document what code is currently being used so that in the future, you understand if I change that code it will impact, report a, report b and dashboard 123.   One other bonus is that if the BA is the interface between IT and the business, the development staff can focus on developing instead of meetings, Q&A, etc.

IBM and Watson

Interesting read about IBM and Watson

http://www.computerweekly.com/feature/For-IBM-the-answer-is-Watson-But-what-is-the-question?utm_content=recipe3&utm_medium=EM&asrc=EM_ERU_65309723&utm_campaign=20160929_ERU%20Transmission%20for%2009/29/2016%20(UserUniverse:%202195049)_myka-reports@techtarget.com&utm_source=ERU&src=5560425

What the Top 9 Data Visualization Skills Pay

http://www.information-management.com/gallery/what-the-top-9-data-visualization-skills-pay-10029793-1.html?utm_medium=email&ET=informationmgmt:e7628532:1822140a:&utm_source=newsletter&utm_campaign=daily-sep%2015%202016&st=email&eid=83b1ad123aa52bc4d96f3cc34765ab31

Items for your Data Governance team

In order to maintain a healthy data governance team, they need to tackle the tough questions pertaining to your data security, data quality and protocols.    Lets look at some of the questions to address.

1.  How much reporting if any will happen on your source system?  Remember once you give access to the source system, it will be difficult to pry people away from that data because its "real time" 

2.  Will you allow SQL queries on your source system or just canned reports?  What about your data warehouse or ODS?  SQL queries or canned reports?

3. That begs another question.   How often will you update the warehouse with source system data.  IE. How current is your warehouse data?  Noon and close of business?  Just close of business?  

4.  When will you allow people to export data from your data warehouse?  Not times of day but will you allow your data to be exported to another system?   Remember once your data leaves your warehouse, you have just opened up another can of worms.   Your data warehouse team has just become the middle man in the equation and now you are fielding questions from a third party.  Everything from "I didn't receive all of my data" to "customer 999 is not in the data feed."

5.  How will my organization communicate changes to source system data?  For example, if I add a field to the source system, when will the data warehouse team find out about that change?  If that happens, how will it affect other system downstream from the data warehouse/ODS.?

6.  Which tools will you allow to hit the data warehouse?    Will they have table access using Tableau or will a team build Cognos/Microstrategy reports and feed them to the business to consume.

I hope that these questions have stimulated you to think about your data and placing a framework around that data to ensure data quality and integrity.

Interesting stuff from Synchrony Financial.

https://www.synchronyfinancial.com/Synchrony_Financial_BigData_IT_and_Marketing_Analytics_white_paper.pdf