AI & Data
Manufacturing

Modernizing DataOps for Strategic Growth with Azure

Trility was hired to help the client’s data and analytics team support the company's vision. To complete and meet the client’s business objectives, they required timely, accurate, and comprehensive data. This necessitated setting up a DataOps team and data platform architecture, tooling, and capabilities by establishing standards, automating testing, and unifying development platforms.

Problem Statement

The client had embarked on various transformational efforts, including multiple acquisitions to expand market reach, migrating its service software from Oracle to Salesforce, developing new eCommerce that integrates with branch locations and consumer platforms, and building intelligent production operations. These initiatives depend on timely, accurate, and comprehensive data to compete and meet business objectives, necessitating new data platform architecture, tooling, and capabilities required to provide it. 

Key challenges included inconsistent processes and standards for data modeling and upskilling team members to properly model data. The data and analytics team had yet to adopt GitHub, which was a company-wide initiative to enhance security, provide version control and CI/CD, and improve developer experience. 

The data and analytics team also fielded support tickets from across the enterprise. Following up and communicating status and progress required custom manual intervention that impeded the team from making progress on initiatives.

Solution Approach

Trility developed a comprehensive strategy to create data modeling standards and processes, including a case study designed to guide users – from novices to experts – through best practices. 

Next, Trility helped the data and analytics team migrate Azure DevOps (ADO) to GitHub to implement version control and enhance security. The migration included redesigning branching strategies and continuous integration/continuous deployment workflows, which enabled the client to incorporate automated testing for the different environments. As an added benefit, built-in security tools were enabled to increase deployment quality. 

After the GitHub migration, Trility implemented an automated continuous testing proof of concept (POC), using the Great Expectations testing suite to validate data between the landing zone and bronze layers. Test results were easily readable through HTML reports.

Trility also helped identify and implement a solution to streamline the support ticket process using ServiceNow, a system already in place and used across the company.

Outcomes

This client now benefits from a DataOps team and a unified data platform, which enhanced collaboration, security, and accessibility across the enterprise. Key outcomes included:

  • Empowered users at all skill levels to adopt proper data modeling practices, improving data consistency and accuracy.

  • Migration from ADO to GitHub Enterprise for version control, integrated CI/CD, enhanced security, and improved developer experience – aligning with the company’s strategy.

  • Enhanced branching strategies, deployment workflows that prioritize automated testing for higher-quality outputs.

  • Implemented and enforced automated continuous testing POC for higher quality output. Separate development and production environments were established to enable smoke testing and full-scale migration efforts – streamlining processes and improving overall data integrity.

  • Created a streamlined support ticket process using ServiceNow to free up the team to focus on new initiatives. 

Project Attributes

  • Reduced COA
  • Reduced COO
  • Reduced Risk
  • Reduced Technical Debt
  • Accelerate Delivery
  • Increased Automation
  • Increased Scalability
  • Reusable Patterns
  • Increased Capabilities
  • Increased Security
  • Coaching
  • Documentation
  • Learning Sessions
  • Paired Programming
  • Videos

Technologies Used

  • Azure OpenAI
  • Google Data Studio
  • Azure Cognitive Search
  • Microsoft SQL
  • Databricks
  • Informatica
  • Azure DevOps
  • GitHub
  • Azure Synapse