Today many people recognize that data in the 21st century is like oil was in the 18th century; an immensely untapped treasured asset. WITI Orange County's September panel event focused on digitization of enterprise and, it is clear, that today's digital world is just too complex to run on intuition. Data to the rescue! … and the topic of WITI OC's October event was born. Data is a company's most valuable asset and through insights on data businesses, they can get to competitive differentiations. There have been many advances in technologies like Big Data, NoSQL, and the Cloud to provide companies the best database technologies for their data needs. This event looked at the latest trends and discussed what skills can help us unleash and reap rewards.
We were honored to have Zillow Group (http://www.zillowgroup.com/) sponsor and host the event in their impressive Orange County offices. We could not wish for a better venue to embody the overall theme of the event. Zillow is transforming real estate through big data and machine learning and Qian Zhu, Engineering Lead at Trulia, was one of the panelists. The other panelists included Lenka Vanek, Senior Director of Engineering, Quest Software, and Barbara O Forth, Architecture and Data Governance expert. The event was moderated by Terry Dear, Vice President of Software Development at BMS.
Given the diverse background of panelists, we were able to explore a whole range of topics - from big data processing pipelines, to Database as a Service (DBaaS), to upcoming General Data Protection Regulation (GDPR). We learned how Zillow processes well over 1.2 Terabytes of data daily. How big is your data pipe and data lake? We also had an overview of GDPR that will be enforced by the European Union (including England) by May 25, 2018. Data is now global, so think about how your organization handles data, especially if you have any European data information. While we did not go into the bowels of data security (not enough time), GDPR did surface this point; all data should to be encrypted in the pipeline as well as Data at Rest (DAR). Companies need to take, for example, the "right to be forgotten", seriously, otherwise GDPR fines are scheduled to follow.
Is there one perfect database for all your needs? No, there is not. What used to be an easy choice is a complex decision with significant implications today. One key take-away from the event, is that it is not RDBMS vs. NoSQL - it is RDBMS and NoSQL. Both types of technologies are relevant, and innovation continues on many fronts. The new data landscape is composed of systems such as HBase on one side and Oracle Autonomous database on the other.
How does one become data driven? No silver bullet here, either. Panelists discussed how technology can help us drive change in behavior, how we can take actions from analytics, and how we can improve data governance processes. Although SQL is still very much relevant, we also need to recognize that some of the latest technologies, like JSON format or MongoDB for example, make developers very productive and enable them to build new applications very fast.
And yes, we did talk about job opportunities and various data job roles, including what is a data scientist and chief data officer. Skills like machine learning are very much in demand, at the same time, one must be willing to roll up their sleeves and do "good old" data cleansing first. On the database side, recommendation for database administrator (DBA) type of job is to explore new DevOps roles where DBA's skills to script everything will come handy. We would have loved to have you at Zillow as we discussed technical skills along with soft skills needed for data jobs in the future. Check out our event photos here.
Today, DB-Engines.com ranks popularity of 310 database systems with various special-purpose database models. The world of data and databases is complex like never before. The Clash of Data Titans is not over. As data evolves, WITI Orange County hopes to see you next year as we explore the evolution of data technologies. Hopefully your Data Lake is NOT a Data Swamp!