California DoJ Stop Data Collection DataStax Architecture and Production Readiness Review

The California Department of Justice is a statewide investigative law enforcement agency and the legal department of the California executive branch, under the elected leadership of the California Attorney General. It carries out complex criminal and civil investigations, prosecutions and other legal services throughout the state.

The DOJ chose DataStax Enterprise Cassandra as the scalable storage platform for its Stop Data Collection system, built to meet the requirements of the California Racial and Identity Profiling Act of 2015. A statutory reporting system has a release date that does not move, which shapes how you review it.

DataStax brought RhinoSource onsite to review the DataStax Enterprise system architecture, network, security, expected workloads, data models, code, disaster recovery and operational procedures — with the goal of accelerating development and preparing the RIPA Stop Data Collection platform for release.

After a deep dive into the architecture, data model and code, RhinoSource made recommendations to improve cluster and query performance, and addressed tombstone issues discovered in several of the production tables.

We also provided guidance and best practices for using OpsCenter to manage and monitor the DataStax Enterprise clusters, covering backup and recovery configuration and the ongoing cluster repair process.

Finally, RhinoSource showed the team how to use Solr and Spark Analytics to improve application response times, recommended a process for testing cluster tuning changes safely, and delivered a role-based training plan to build working DataStax knowledge across the DOJ team.