{"id":2530,"date":"2017-07-05T00:00:00","date_gmt":"2017-07-05T00:00:00","guid":{"rendered":""},"modified":"2020-12-14T21:29:15","modified_gmt":"2020-12-14T21:29:15","slug":"video-dan-stairs-odsc-east-2017-presentation-building-near-real-time-data-pipeline-cloud","status":"publish","type":"post","link":"https:\/\/cazenasite.com\/?p=2530","title":{"rendered":"Video: Dan Stair&#8217;s ODSC East 2017 Presentation, &#8220;Building a near-real-time Data Pipeline in the Cloud&#8221;"},"content":{"rendered":"<p class=\"rtecenter\"><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/qrBX46gaOWk\" width=\"970\" height=\"546\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p>In this video from the Open Data Science Conference in 2017, Cazena Senior Engineer Dan Stair shared a technical talk about real time data pipelines. For more recent insight, <a href=\"https:\/\/cazenasite.com\/category\/blog\/\">please visit the Cazena blog.\u00a0<\/a><\/p>\n<p>&nbsp;<\/p>\n<p style=\"font-family: proxima-nova, Arial, sans-serif; font-weight: 400; font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal;\"><span style=\"font-size: 16px;\"><strong>Abstract from Dan&#8217;s 2017 talk:<\/strong>\u00a0Learn more about how we built, tested and delivered a near-real-time data pipeline using Apache Spark in the cloud in two weeks &#8212; and still saw our families. We faced a looming deadline, and real-time analytics requirements. Using a cloud-based platform with Spark and Impala running on Microsoft Azure, and armed with a few hundred lines of Python code, we designed, tested and deployed an end-to-end data pipeline and analytics infrastructure in two weeks. The project had its challenges, both technical and operational; learn what we learned and our tips for success.<\/span><\/p>\n<p style=\"font-family: proxima-nova, Arial, sans-serif; font-weight: 400; font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal;\">\n","protected":false},"excerpt":{"rendered":"<p>In this video from the Open Data Science Conference in 2017, Cazena Senior Engineer Dan Stair shared a technical talk about real time data pipelines. For more recent insight, please visit the Cazena blog.\u00a0 &nbsp; Abstract from Dan&#8217;s 2017 talk:\u00a0Learn more about how we built, tested and delivered a near-real-time data pipeline using Apache Spark [&hellip;]<\/p>\n","protected":false},"author":9,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[65,61],"tags":[18,46],"class_list":["post-2530","post","type-post","status-publish","format-standard","hentry","category-videos","category-webinar","tag-azure","tag-technical"],"_links":{"self":[{"href":"https:\/\/cazenasite.com\/index.php?rest_route=\/wp\/v2\/posts\/2530","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cazenasite.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cazenasite.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cazenasite.com\/index.php?rest_route=\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/cazenasite.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2530"}],"version-history":[{"count":2,"href":"https:\/\/cazenasite.com\/index.php?rest_route=\/wp\/v2\/posts\/2530\/revisions"}],"predecessor-version":[{"id":3846,"href":"https:\/\/cazenasite.com\/index.php?rest_route=\/wp\/v2\/posts\/2530\/revisions\/3846"}],"wp:attachment":[{"href":"https:\/\/cazenasite.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2530"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cazenasite.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2530"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cazenasite.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2530"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}