Showing posts with label ETL. Show all posts
Showing posts with label ETL. Show all posts

Monday, March 23, 2015

Pushing BI to the forefront

Your Data Warehouse is a downstream system meaning that you ingest data from your source systems.  Their changes are your changes.   Even when they don't tell you that the changes are coming.  The data warehouse team needs to work hand in hand with the owners of the source systems and let them know how you are implementing the data warehouse and to remind them that you are there.   Now this might be the point where you may meet some resistance.

For my current client, they had their source systems are the BI team was brought in to create a series of executive dashboards for every part of the company.  These dashboards not only help the company gauge how well they are doing but they are shared with the board of directors on a monthly basis.  Often times the week before the board views this data is filled with anxiety from both the source systems and our BI team.   When you know the numbers for the business and a number that should be static is changing from day to day, you have just pointed out a fatal flaw for one of your "Business Partners."   You have to watch and ensure that your findings don't create a negative relationship for the future.  Your team may win the battle but lose the war.   Keep those doors open.  Maintain a level of openness and transparency.

Here are some tips to keep up a good relationship with your business partners:
1) Get buyin from upper levels of the organization.    BI leadership needs to meet with your business leadership to maintain the importance of your BI roadmap and data quality plan.
2) BI Managers needs to meet periodically with your business partners about the data.
3) Whenever an issue arises, try to resolve it at the lower levels before escalating to your leadership.
4) Communicate deadlines for data issues to be resolved to give them an idea on when you need data issues resolved.  You can use a BI newsletter or just emails.    We created a calendar of dates.   (Dates for board meeting, dates the data is mailed, when your data is needed, signoffs, etc.)

Sometimes its not the business partners that control the changes.  Its your IT partners.   Communicate regularly with your IT leadership such that changes on the source systems need to be communicated downstream.   Perhaps you will want to add BI as a signoff on all change requests.   Remember, always have your etl processes tweeked to reveal any changes to those source system changes.   When all else fails, let the ETL process alarm you of any changes to Database schemas.   Sometimes you can't help it.  Even though you have alerted IT and the business about the need to notify the BI team, you have changes flow through that break your ETL.  It is always better to know about those changes on Day 1 instead of Day 10.   Best of luck to you.

Joe

Wednesday, March 11, 2015

Data Governance

One of the key ways that you can achieve buy in from your business partners is through a Data Governance team.   Who should be on this team?  You will need champions.   You need those folks from your business that run reports, source system owners, perhaps HR data, or another business unit that is involved with PII.  What is PII?


  1. Personally identifiable information (PII) is any data that could potentially identify a specific individual. Any information that can be used to distinguish one person from another and can be used for de-anonymizing anonymous data can be considered PII.
One of my clients was involved in a law suit and to avoid violating PII, we had to monitor everything related to PII.  If you are in the Healthcare sector, you have a similar respect for data and understand the importance of HIPAA and more importantly violating HIPAA.   What is HIPAA?


  1. HIPAA is the federal Health Insurance Portability and Accountability Act of 1996. The primary goal of the law is to make it easier for people to keep health insurance, protect the confidentiality and security of healthcare information and help the healthcare industry control administrative costs.
  2. This is extremely important to a new sector of Patient Safety related to keeping your patients safe.  As a Dr, do you wash your hands regularly?   Was there an issue with an incorrect medication?  Did you misdiagnose an issue?    Not only do you need to keep this data secure from people outside of your organization but also those groups within your organization that don't have the authorization to view said data.  
  3. That's why it is extremely important to keep data issues such as data quality, data security and data accuracy at the forefront of your business.   That's where a data governance team would be an asset to your organization.    One of the key points is to attain buy in from a high level business sponsor.   A Vice President for example, that will chamption this issue and carry the clout necessary to spread the key decisions into the impacted organizations.  This data governance team will make changes to source systems and business processes that will transform your data quality and push for Best in Class Data processes and procedures.