NCS is a big-data analytics firm that offers a SaaS platform for the monitoring, compliance and enforcement needs of financial institutions and government agencies charged with managing industries that operate in complicated legal and regulatory environments. When one of NCS’s financial institution clients was acquired by a much larger company, NCS was asked to validate its data model to ensure that it complied with the new rules and regulations the client was suddenly subject to. NCS turned to the Armanino contract and compliance team and in just over three months, the Armanino team was able to complete the labor-intensive exercise of validating NCS’s reporting.
When CEO Adam Crabtree co-founded Denver-based NCS Analytics (NCS) with CFO Mark Nassi in 2014, Colorado was one of only two states in the U.S. to have legalized cannabis use and possession for recreational purposes. Legal at the state level but not legal at the federal level, the nascent industry presented challenges to the numerous entities charged with monitoring and regulating it. To address this issue, Crabtree and his NCS colleagues set about developing a big-data platform that financial institutions and regulatory agencies could employ to better manage the industry.
Supporting the monitoring, compliance and enforcement needs of institutions and government agencies, the SaaS platform’s use now extends to a variety of industries that operate in complicated legal and regulatory environments. Today, NCS uses the patent-pending predictive and behavioral analytics at the heart of its platform to analyze more than $192 million in monthly sales, processing upwards of 89 million data records per week in order to monitor more than 16,000 business licenses. As the company grows, its customers are growing too.
So, when one of NCS’s financial institution customers was acquired by a much larger entity, NCS found itself facing a new challenge. Explains Crabtree, “Bank requirements change based on asset size. Thus, when this financial institution got acquired, they moved into a new asset class. As such, they wanted to have their data model validated to ensure it complied with the new rules and regulations they were now suddenly subject to.”
For Crabtree, Nassi and company, the challenge was a welcome one. While they were confident in NCS’s data models, they knew that getting an independent party to confirm its performance would solidify NCS’s position as a leader in its space.
“We’ve historically strived to lead the industry when it comes to obtaining external accreditations or validations of our services,” says Crabtree. “So, when this financial institution customer asked us to validate our analytical models, it just seemed like the next iteration. We want to continue to be on the cutting edge — proactively setting expectations rather than responding to others’ expectations of us.”
The difficult part was finding a firm with the right knowledge and expertise to carry out the validation work. For starters, there are very few big-data consultants. And even among those, NCS found it challenging to identify a credible firm that could carry out the validation process as the NCS team envisioned it.
Then came Armanino LLP — one of the 20 largest independent accounting and consulting firms in the United States. Not only did the contract compliance team at Armanino have the technical expertise and analytical skills NCS required for its validation, they were also excited to take on the challenge.
“We deal with consultants of all sorts,” says Crabtree. “And much of the time, it seems like they’re being forced to do the job. In contrast, the team at Armanino was genuinely excited and curious, which made it easier for us to sign off on moving forward with them. That same attitude also made the whole engagement a lot more collaborative.”
According to CFO Nassi, the Armanino team was more than up to the challenge presented to them. “For us, the hardest part of the model validation process was explaining the complexity of our data and providing an understanding of the terminology and nuances of the industries we serve,” he says. “But Armanino listened, asked the right questions and then hunkered down with the data.”
NCS and Armanino had an ambitious initial timeline of four months to validate NCS’s models. To do this, Armanino needed to understand the nuts and bolts of the model, including NCS’s procedures, policies, underlying analytical code, controls environment, model operations and reported findings. Working closely, NCS and Armanino accomplished all these objectives in three months, shaving a full month off of the timeline.
Perhaps the best news was that Armanino’s validation process confirmed NCS’s assumption that it was reporting accurate data to its customers. After a line-by-line review of code and an evaluation of the model’s data feeds and outputs, only one small, isolated glitch in the code was discovered. Armanino detailed the problem in its final report to NCS, and the analytics company quickly resolved the issue.
The thing that was great about Armanino’s validation was that it was so granular. Knowing that they put our data model through its paces and that [the model] performed not just adequately but phenomenally allows me to finally step back and take a deep breath.
- Adam Crabtree, CEO and Co-Founder, NCS Analytics
For CEO Crabtree, the biggest benefit of having Armanino validate NCS’s data model has been peace of mind.
“Public records show that various governments doubled or tripled the taxes they were able to collect within a year of deploying our platform,” says Crabtree. “Records also show that our platform revealed hundreds of businesses that didn't have licenses. These are all fun real-world validations of our data model. But the thing that was great about Armanino’s validation was that it was so granular. Knowing that they put our data model through its paces and that the model performed not just adequately but phenomenally allows us to finally step back and take a deep breath. It’s no longer just our word. We now have third-party proof that our data model is solid.”
As NCS looks to bring larger institutions into its customer fold, this kind of third-party validation will be essential. Explains Crabtree, “As we go through due diligence with institutions that have billions in assets (rather than the tens of millions it was back when we started), these organizations derive a lot of comfort from the fact that we’ve expended the time and resources — and gone above and beyond what we’re required to do — to get this type of validation from Armanino. This really sets us apart and serves as a huge selling point.”
The biggest risk we've mitigated by taking advantage of Armanino’s validation services is the potential loss of clients. If we stop being the first organization in our space to earn an accreditation or to accomplish something, we suddenly become the followers. And that’s a position we never want to be in.
- Adam Crabtree, CEO and Co-Founder, NCS Analytics
In the complex industries that NCS supports, regulations are constantly changing. In the cannabis industry, for example, each state has its own set of rules, and NCS’s data models must be built out and adjusted to accommodate all of them. Going forward, NCS will continue to turn to Armanino for validation.
“As we expand, our need for validation only increases because we must continue to show that we aren't under any misconception that there’s a one-size-fits-all solution to a problem this complicated,” says Crabtree. “The biggest risk we've mitigated by taking advantage of Armanino’s validation services is the potential loss of clients. If we stop being the first organization in our space to earn an accreditation or to accomplish something, we suddenly become the followers. And that’s a position we never want to be in.”
Contact our contract compliance experts to learn more about how third-party validation can help you maximize your financial benefits while mitigating non-financial risks with critical business partners.
