Data lake and
Data Warehouse development

With the advent of Big data due to explosion in data volume, variety, and velocity there is a need to have solutions in place to effectively manage, massage, analyze, and leverage the knowledge in the data with insights. Solutions ranging from modern BI analytics platforms, Data Warehouses, and Data Lakes are available to address various use cases. eQube-BI is a modern A/BI platform that addresses multiple analytics use cases for many personas (such as: a regular end user, a power-user with software background, a citizen data scientist, a data scientist, a manager or an executive, etc.). In addition, using eQube-MI as the core platform, efficient and comprehensive solutions can be built for Data Lake and Data Warehouse initiatives.

Data Lake: Data from multiple sources (files, sensors – products and machines, sentiment – social media, databases, and COTS applications) can be ingested in a Data Lake using the eQube platform. eQube-BI and eQube-DP solutions are used for data discovery and preparation during the ingestion process orchestrated by eQube-MI while eQube-TM maintains the knowledgebase of the data augmenting the semantics information related to the data. eQube based approach provides a framework for efficient ingestion of raw data in a 'Cold store' and enables orchestration for populating 'Warm store' and 'Hot store' for rapid analysis. eQube Connectors for file systems, COTS applications, databases, NoSQL systems, and Big data stores can be leveraged to develop and maintain the Data Lake solution on-premise or in the Cloud. Once a Data Lake is a part of it, multiple business use cases are addressed efficiently and comprehensively.

 


Data Warehouse: Data from multiple sources (text files, databases, and COTS applications) can be transformed and populated into a dimensional model in a Data Warehouse using the eQube platform. eQube-BI and eQube-DP solutions are used for data discovery and preparation during the ETL process orchestrated by eQube-MI while eQube-TM maintains the knowledgebase of the data augmenting the semantics information related to the data. eQube connectors for file systems, COTS applications, and databases can be leveraged to develop and maintain a Data Warehouse solution on-premise or in the Cloud. Due to eQube's innovative architecture, using eQube for the development and maintenance of a Data Warehouse is far more efficient than traditional approaches and tools.
Data Warehouse as a solution is most applicable for the following types of business use cases:

I. Decision-making and business performance reporting based on past performance:
          1. Financial reporting and analysis
          2. Sales and customer satisfaction analysis
          3. Business performance analysis

II. Descriptive analytics: Predominantly focused on past performance data.

 


If 'stale-data' is unacceptable and security/access control are critical, eQ recommends using its modern A/BI platform, eQube-BI. eQube-BI aggregates data by directly connecting to SORs for providing real-time or near-real time analytics honoring the underlying applications' security / access control rules. In this approach, data is kept where it belongs and the need for a Data Warehouse is eliminated. However, if a Data Warehouse already exists and data from it also needs to be aggregated with data from certain SORs, then eQube-BI can mash-up data from both the SORs and the Data Warehouse on-demand. Thus eQube-BI can leverage the investments made in a Data Warehouse or Data Mart(s).
 


Top

 Resources
 
 Learn More