Eliminating ETL bottlenecks and reducing data costs by 80%

Federal IT leaders are under pressure to modernize aging data platforms that deliver mission-critical insights while maintaining strict compliance, security, and cost controls. A large federal health program partnered with ICF to migrate its legacy data system to Snowflake. The result: increased efficiency, lower costs, and secure, enterprise-scale data sharing.

Challenge

Our client faced three critical challenges:

• Performance bottlenecks: Data duplication and lengthy ETL processes slowed dashboards and delayed access to critical insights.
• Rising costs: The program spent a significant amount of money hosting a replica of the parent agency’s integrated data repository (IDR) in AWS Redshift.
• Strategy misalignment: Because the parent agency uses Snowflake, the client did not have direct access to IDR data, requiring duplicative ETL processes.

Solution

To modernize the data environment with minimal risk, we supported a phased migration from Redshift to Snowflake that aligned the program with the agency’s enterprise data strategy. Working closely with stakeholders, our team addressed data integrity, security, and performance challenges while eliminating redundant ETL processes and enabling direct access to the agency’s enterprise IDR. The migration was completed with zero downtime and no disruption to end users, preserving continuity for mission‑critical reporting.

Results

As a result of this work, the program established a low‑risk approach to data modernization that aligns with the agency’s broader enterprise data strategy.

Already, our client is seeing tangible results from its Snowflake migration, including:

• Improved performance: Reductions in ETL time from 12+ hours to 3 hours and elimination of redundant ETLs, enabling more timely insights
• Significant cost savings: Monthly costs reduced from ~$30,000 to ~$6,000 after eliminating its IDR replica in Redshift DB
• Proactive strategy alignment: In step with the agency’s enterprise-level move to Snowflake
• Improved security and compliance: Implemented role-based access control and Snowflake-native controls for compliance

This new scalable, secure, and efficient data-sharing solution in Snowflake empowers faster, better decision-making for the client’s leadership and improved data-sharing capabilities with partners across the parent agency.