![](\"\/wp-content\/uploads\/2017\/06\/CloudSecurity.jpg\")
<\/p>\n<\/p>\n
Don\u2019t Let Hadoop and Cloud Security Fears Derail Your Path to Value<\/strong><\/p>\n As data practitioners, we have spent the last four years creating an enterprise data science environment for a life sciences client. We\u2019ve taken the project from an initial pilot on AWS (swiped a credit card!) to an analytic hub that houses 100TB of internally-owned data as well as data procured from syndicated data providers. The data sets are still growing, and they constitute the foundation of our client\u2019s analytics today. The toughest part of our big data journey was not standing up the different machines, dealing with the sources, generating the analytics, or running the updates. It was, and continues to be security.<\/p>\n Hadoop from its inception was not designed with security as its primary mandate. To lock our environment up tight we needed to work fluently with AWS security groups and administration, Microsoft Active Directory, Cloudera Sentry, MIT Kerberos, encryption-at-rest, and transport layer security using SSL certificates. Oh, and we also had to do it all in an ecosystem of Amazon Workspaces that could communicate with the cluster.<\/p>\n In a perfect world, the company\u2019s users would have accessed the environment from their own network but the challenges were too daunting. This means that all analytic tools (R, SAS, Tableau, etc.) and the Active Directory had to reside within the VPC, and have to integrate across all of the worker nodes. There are no external IPs\/firewalls into the environment, so we traded usability for security. While we solved all of the core security concerns from IT, we constrained the ways in which the environment can be used.<\/p>\n We don\u2019t believe our experience accommodating all of the various security concerns was unusual. Yet, this security framework took two full-time Hadoop \/ network security engineers four months to design, setup, and test. It now takes one full time person to manage issues and grant\/change the permissions required for the growing user base and data.<\/p>\n When thinking about a big data environment, security has to be addressed first. It has surpassed quality assurance as the number one concern. It is a tough, yet necessary nut to crack, and it will make or break a project given that all the patient level information these days has to pass rigorous compliance tests, and any hint of a breach will shut a project down indefinitely and immediately.<\/p>\n There are many ways to address the compliance and end user needs and all will require a high degree of skill and experience. This begged us to ask this simple question: do we want to tolerate the lag and friction inherent to standing up a new Hadoop environment, or do we want to stay laser focused on delivering new insights and analytics to our clients vis-\u00e0-vis our data science sandboxes? We choose the latter \u2013 and we chose Cazena! <\/p>\n![\"Sentier](\"\/wp-content\/uploads\/2017\/06\/64e320_c3da147686164600bfbba68478dc6d9c~mv2_1.png\")
<\/a>
\nGuest blog from Rich Sokolosky & Chael Christopher of Sentier Health Informatics<\/a>.<\/em><\/p>\n
\n