Scope
- Creating a robust and scalable Business Intelligence/Analytics (BI) solution powered by a cloud-based data lake.
- Providing a BI solution that will run across multiple departments and present different KPIs according to user roles.
- Creating a data lake that will store the large amount of data generated from various sources in real-time, providing that information to the real-time dashboards for KPI monitoring and decision-making
- Creating a data lake that will be supplemented with various individual data marts for their respective departments’ data segregation and separate dashboards.
Challenges Involved
The client has been running the operations for many years, and that too at an enormous scale. As a result, the client deployed multiple legacies. We had to implement new systems internally to handle the various processes.
- Different systems generated different types and formats of data at different places.
- The top management had difficulty understanding the vast amount of data at a combined level. This impeded the decision-making and overshadowed the opportunities to improve further.
- The client’s IT team was not updated with tech advancements to provide access to the various sources, considering the updated frequency of data.
- Data access is a significant problem with the PCI data security protocol.
- Keeping the data lake and corresponding data marts uniform and updated with new data.
- Efficient data models to store large volumes of data in an optimized format.
- Keeping the overall solution cost-effective for the client.
- Multiple software systems for different operations and departments make a massive mess of the collected data.
- Not getting a better understanding of the business due to data being saved in different places and with changes of manual error as well.
- Lots of manual effort and chances of human errors and data inconsistencies leading to misinformation.
