Friday, May 27, 2022

Data Scientist vs. Machine Learning Engineer

hi everyone welcome my name is sandia simhan i'm the director of marketing at deeplearning.ai welcome to our expert panel data scientist versus machine learning engineer this event is presented to you by deep learning dot ai and forth brain .

Fort brain is a company that creates accessible and flexible pathways to ai careers by training candidates with valuable practical and interpersonal skills you can learn more about their programs on their website forthbrain.ai scientist and machine learning engineer are both highly sought after roles in .

The tech industry when it comes to the difference between these roles and the responsibilities there has been much confusion so in fact we've received many questions from regarding this topic from our community today we hope to provide you with some inspiring insights through the discussion .

With our amazing panel of speakers you can find a list of topics we'll be covering if you scroll down to check out the event description after the panel discussion we will be taking questions from the community via slido if you have signed up for the slido ticket check your email for the access .

Link to post and upvote questions there and now without further ado let's welcome our panelists today they are aishwarya sreenivasan ai and mml innovation leader at ibm kate strz and steve nori head of data science of the australian computer society welcome everyone let's start off with a .

Brief intro uh i forget you want to go first hey thank you so much um i'm really really glad to be here i can't tell you how many uh like positive comments and like dm's i have got often after we released uh the poster for this event i'm super excited to be here and .

Amazing uh set of panelists with me kate and steve uh been uh like in touch with them uh in the recent past and they they're doing amazing work so yeah currently as you mentioned i'm currently uh ai and ml innovation leader at ibm so um previously i'm coming from a data .

Science background i have a computer science and uh data science masters and um i have been working hands-on on building machine learning models so i can speak better for what data science requires and how can you build your career up in the field so um .

From being a data scientist i've been moving up the ladder in more of a leadership position where i am still working hands-on because that's something i always wanted to do so i made sure that that's part of my role where i'm still working on hands-on machine learning models apart from which i'm working very closely with .

Product management teams and clients so where we try to understand that what are the best kind of products and services we can build in ibm cloud and how can it benefit the best for uh for the clients this also involves me working closely with ibm research so that we can bring up the latest state-of-the-art .

Models from there and integrate them into our products so that's that's what i do briefly awesome okay hello hello everybody thank you so much for having me here really really excited to be part of this panel with aishwarya and steve and sanja great to meet you all i know we've been kind .

Of watching each other on linkedin and maybe some other social medias are really fun to spend the next hour together here um my name is kate strashe i'm the founder of dedicated which is really dedicated to data so i love all things data i started my background in really finance and .

Investments and sort of went into risk management then took a little detour and spent some time in data analytics at a consulting company and then realized that my passion is really for talking to people and building communities and this is what i really do with dedicated now .

Now i'm not a an expert data scientist nor machine learning engineer but i do have conversations with several companies that either specialize in the space or people that have taken this path and journey to becoming a data science and machine learning engineer so really hoping to share .

Kind of what i'm seeing across the community and take questions from the audience when the time is right it's a little bit weird for me to be part of a panel versus moderating i typically on the other side of the show you know asking the questions uh but it's definitely fun to be on the other side here so thanks for having me .

You're very welcome we're excited to have you here and last but not least steve thank you very much sandia it's great to be here among this awesome panelists i know all of them from linkedin and we interact and i guess we're all passionate about data .

My background is software engineering i was an i.t um a graduate a long time ago when data science was not i guess a word back then i was working with data a lot querying sql and other stuff but then um i found that my passion is .

In actually finding interesting insight in the data from re report building and um back then we didn't call it but it is more of an analysis analyst job so i then transitioned into an analyst role about um 12 years ago and learned machine .

Learning from andrew wing's video on youtube which is uh awesome i loved it back then it was a little bit different more lengthier and more interesting i guess 12 years ago but then um yeah i'd learned a lot from deep .

Learning that deep learning ai courses i transitioned into head of data science at australian computer society where i um help different functions to deliver data science and analytics projects and also i am an ai expert at the .

Iso international standard organization helping uh to um contributing to standards in ai trustworthiness and data sharing apart from that i sometimes write for forbes and help people learn about ai and and data science on linkedin .

Awesome man of multi-hats so thank you all for joining us today um so for the next 40 minutes or so we're going to be discussing topics like especially the main differences between these two roles how you might shift from one to the other and so on so to start off for the audience who isn't familiar with what a .

Data scientist or machine learning engineer does uh can each of you use one sentence to describe um what these roles do in your own words uh maybe i should start us off so one of the simple explanations that i can think of is as a data scientist you need you not .

Only have to have good grasp on coding and math and statistics but you also need to have a very good hold on subject matter expertise in whichever industry you're working whereas in machine learning engineering your focus is more towards coding and as well as math and .

Statistics but you need not really know so much of subject matter expertise i'm assuming i'm next sandhya since that's the order we took so i'll just jump in here so essentially i'd say a data scientist is someone that can apply the scientific method to data right they conduct exploratory .

Data analysis they get insights they as uh ashwarya mentioned they interface with the business whereas a machine learning engineer is basically someone that takes the models that are built by the data scientist and makes them work in a production environment a lot of times at scale so those are the main differences there .

That's one answer and that's steep awesome uh yeah you guys are making my life so easy on the last and i i pretty much agree with um all of it and as as um kate and ashley mentioned so um i guess data scientist is more of a scientist and uh the scientists do more .

Experiments is is more of a creative uh i guess role and the other one is more of an engineer which engineers are trying to maintain and operate things and uh that would be a quick one fair enough um why do you guys think that the concept or .

Description of these two roles is still a little vague in the community you know it's been almost 10 years that we've these these terms are have been you know used in the industry why do we think it's still a little bit vague steve um yeah so the reality is uh many jobs right now they evolve and .

Change it's not only about the data science or ai roles they based on the new innovations the customer needs and all other parameters around the job itself they would evolve and that's the same with these roles when they started .

It it was more of a generic terminologies and then a lot of complexity was added into these roles many other i guess other roles came out of the data science term like the data professional terminology and it evolved to cater for for the new world .

And it will change again as well also the complexity of each one of them uh just makes it weak because then um each company based on the size based on the um i guess the needs and and the industry that they're performing they might need a little bit of different .

Capabilities or different requirements for from each one of these roles um it's not like iso will come up with some sort of standard for the name or for the terminology and everybody gonna follow it so um i totally agree still it nobody knows exactly what each one means .

Fair enough it's definitely one of those things that's evolving as the you know industry itself evolves fair enough um can you give us uh can one of you give us some examples of what each role does in a typical workday maybe ishrea sure so um just adding on to what .

Steve mentioned that there's this um little blurriness in 10 years is probably not a lot of time when you think about like a new field emerging and various different companies are having their own sort of definition for machine learning engineer and data scientists .

Some of the some of the companies are requiring you to have research experience for being a data scientist and some of them aren't right some of the companies for example my previous uh role in data science was a mix of both uh data scientist and machine learning engineering where i was also like working on productionizing the .

Models so if i were to um compare it in like a ideal situation what data scientists would be working on is building the models getting their hands dirty with data understanding how are they going to formulate the business statement how are they going to help .

Uh in a particular use case and um how would how would they like you know dig deeper into data make sense out of it see if there's any data quality issues see if there's any bias existing in the data basically work put most of their time on the data understanding it and .

Choosing the right algorithms optimizing the algorithms and building those and testing them as a proof of concept that that's what typically a data scientist would be working on so that's inclined towards research inclined towards knowing deep into mathematics and statistics whereas .

Machine learning engineer uh in a typical sense would mean that you're picking up what the data scientists have built and you're trying to scale it you're trying to productionalize it you're trying to get it into an organizational workflow how does the models that are being built .

By the data scientist reach to your tips in your phones or your computers where you're actually using them so they are the ones who are getting it into the into the uh you know um workflow on ontology of an organization where companies or other users can actually .

Make use of these models all right so do you see data scientists sort of mining the gold and machine learning engineers to the one who make it usable to others yeah use it absolutely yeah fair enough um could you chat a little bit about what you think the typical .

Career progression for each of these roles is having seen so many of these folks in your community yes absolutely happy to talk about that so i think in most roles including in the data space there are typically two ways that you can take your career right there's a management path and then there could be an .

Individual contributor path and i see the same for the data scientists for example where you typically start as an entry-level data scientist or maybe even an intern right and then eventually you grow through the ranks and you become a senior data scientist at this point when you're in that position .

You can really start to think about do you want to become a manager of data science teams or do you want to remain like this highly specialized data scientist where you can focus on your individual contributions now as you go up from the path from that entry level to that senior .

Level data scientist you tend to develop more skills right you get more responsibility where you used to be maybe managing a specific process in that whole data science process maybe now you're looking at things more end to end and more importantly you're also learning kind of how to work with that .

Business and solve business problems now for machine learning i think you know you typically start out with learning a programming language and thinking python would be popular in uh with our audience here um learning some fundamentals of computer engineering like data structures algorithms and then .

You know it's really helpful if you're starting out with an undergraduate degree in something like computer science computer programming i know back in the day we didn't have things like data science but now it's really really available for us and then some roles that basically .

Machine learning engineers start with include you can start as a maybe software developer or computer engineer and similar roles and then you can progress into a machine learning engineer now individuals in this space that also possess strong leadership skills they can also .

Look into management positions such as those that i mentioned in the data science space and in addition to getting a job for both in the you know data scientist and ml space i think because their demand is so high you don't have to commit to just working for an organization .

I think there are plenty opportunities there to do consulting freelance or you can actually teach the next level you know the next generation of data scientists and machine learning engineers so plenty of hope out there for those who are looking to learn this this stuff .

Awesome um i mean many of our audience say you spoke a little bit about sort of you know the initial phase i would say like many of our audience members are you know thinking about starting a career in either data science or machine learning trying to figure out which one is more suited for you know their interests and .

Their passion so to help them figure out which track is a better fit there are some critical skills that we think are required but you know we'd love to understand how those skills are you know required for each role so i'm gonna name a couple skills and if you guys want to pipe in and chat a .

Little bit about you know how where you think you know is more relevant so uh how much math and software engineering do you think is actually required um so i i can answer this one so um you know the the reality is a lot of people ask me about which one of these roles are um more .

Interesting or more important which one is suitable for me so first of all the answer is it's based on each individual people's performance skills uh plans for the future there is no answer about which one is a better um job .

No such thing available okay so if you want to choose either of them um spend some time like listening to these kind of um events going to help you to understand what entails and if you are up for for the challenge secondly um the the industry that you might want to join .

Is also important they there are different requirements in different industries um definitely uh i guess if you're more techy person you you would like to go to um these techie companies that will do so some more hardcore stuff and if you have some background knowledge about .

Specific industry um like finance or um biomedicine then you might want to out opt for one of those uh industries which they need some business understanding like heavy business understanding um apart from that the technical skills or the skills that we can um just uh quickly mention about each .

One of these roles i would say um as we already defined a little bit of each one of them we can see that data scientists probably need more of uh hands-on with data maybe understanding sql is one of the major skills i know that um .

My friends all always share about sql on linkedin and everybody loves it so that's um sequel is one of the major ones uh it's not dead uh believe me if you are starting it spend really good time on um uh i guess um this particular one but then mass and .

The stats is something that probably uh safety across both of them um and maybe heavier in the data science side where the the science comes to the i guess uh to to play the role and uh and the visualization part is also important more important for the data scientists .

I guess many of these skills are important for everyone for example visualization is something that no matter what you do you want to present some sort of data in an understandable way you need to have some skills but then maybe a deeper understanding is needed for a data scientist but for .

On the other hand and machine learning engineers as um schweiner mentioned it's more of a software engineering kind of uh skill that you need to make sure that you understand object oriented programming a lot more uh deeper than the data scientists and uh .

Maybe um you know hardcore coding skills and uh this would be a major difference for me github is also something that you need to work with it a lot and also using ides that are more relevant for software engineers versus that probably data scientists would use .

Jupiter notebooks more awesome great ideas yeah i love how you're distinguishing between even just the tools that the individual roles use um kate could you chat a little bit about what you think comes from you know like sort of minimal uh educational qualification you know if one role is .

More oriented around research and another is more oriented around say business insights like how how do the educational qualifications play into that yeah i think you know the answer used to be you have to have a programming degree a computer science degree or physics statistic something of that sort .

But i think we're slowly moving away from that where you're you'll see some of these big tech companies saying you don't need to have an undergraduate degree in anything relevant or even a degree to begin with because they value experience so much more over .

Your actual credentials right can you get the job done that's that's literally what most companies care about i think you'll still see companies using those credentials to maybe weed out applicants because if they have a huge influx of people applying to a job .

They'll need a reason to kind of narrow that list down and i think that's that's where that might become useful but i think the more you can highlight your project experience and showcase exactly what you can accomplish in the skill sets that you have i think that becomes a lot more powerful than .

Maybe a degree i'm definitely not swaying people against getting a degree i think it does become a lot easier if you if you have that foundational background where you don't have to relearn all this stuff on your own but we are living in an age where you can you can take boot camps you can .

Read textbooks you can go online and google information right which is what most people end up doing is googling information uh when they're stuck so i do think there's hope for for those that might not have a technical background yeah the one thing that every engineer .

Needs to know stack overflow right yes exactly um uh so what i think one of the distinguishing aspects of engineering is not just sort of let's say creating the uh or manipulating the program that you have but sort of you know putting it into production so the sort of the ml .

Ops as we call it uh the operations uh skill set of scaling product of putting it into large uh you know uh real world situations so aishwarya could you chat a little bit about how much you think a data scientist versus versus an mle needs to know about that ml skill set .

Absolutely uh again like pointing out to the fact that every individual company has very different kind of roles and responsibilities for a data scientist versus an mle for example in my team when i was working as a data scientist we didn't have .

Separate people doing data engineering or data visualization or ml ops so a data scientist was another word for like a full stack engineer is um is a unicorn in data science where the same person is able to do the entire pipeline of data science and also has the subject matter expert .

Expertise in different fields so um as kate was mentioning ml ops is something which need not be necessary for any uh for any particular role unless and until it is specifically mentioned in their job description but what's what's sort of important in certain data science roles is that subject matter expertise for .

Example like um in the recent times where we have seen multiple companies facing issues with respect to ethical ai we have more awareness being created that on the team where we are building these ai systems where we are building these machine learning systems .

We need people from diverse backgrounds we need philosopher we need uh we need people who are expert in psychology so that they can give us a better perspective on what could be an ill effect of this kind of a system and uh that's where i that that's where i wanted to say that you know ml ops is something which is very specific and it .

Could or could not be absorbed in a role like data science in certain companies it is absorbed in certain companies they have specialized people for data engineering they have specialized people for doing ml engineering they have specialized people for even doing database .

So yeah it's again like i would encourage people that whenever you're applying for a data science position do not just look at the title of the role and dig a little deeper into okay what's the job description go on look for people who are working in same team or similar team in the same company on linkedin and reach out to .

Them talk to them about what is their day-to-day look like do they engage in ml ops if you are interested then you can go ahead if they are not like it's it's your decision then no that's a really fair point um we know that there's a lot of uh people .

Who come from non-traditional backgrounds like some non-cs non-engineering right people like myself with psychology who are like hey how can i be perhaps not a machine learning engineer but still in this space in this field helping develop these sorts of ai ml based products so you know how would .

You advise someone like that like who's coming from that sort of non non-technical background uh to sort of get their feet wet and kind of enter the industry absolutely so just an as an example like people who are developing web applications it's not just about the back end that that you're working it's not just how you .

Create the website it's also about how do users interact with it how attractive is it how easy it is for you to use so all that thing come in a front end or like a ux designer similarly we have various different components when it comes to like building .

Data science or building machine learning models so when when you were saying that you know um people who are coming from different background compared to cs even in my team there are people who are coming from uh operations research people who are coming from healthcare especially .

Who create a whole lot of difference for us for example me i'm coming from a computer science background and when i was working on a healthcare project we had this other person coming from a healthcare background from a neuroscience background and she could give us so much more .

Perspective on what the data means one of the one of the things which andrew has been recently like pushing a lot on is having data science data centric approach and i cannot emphasize more on that on how important that is because we are trying to you know stitch something up without knowing .

What's what's what's the content in it and that is one of the major uh reasons of some of the failures that we have seen in in ai systems that we don't understand the data well and that's where it's super important that every company does engage people who are coming from a .

Certain background in in a different industry if the company is you know working on building automobiles we have people who are coming from mechanical background and not just data scientists building building these uh self-driving technology because yeah like .

As a data scientist i could build something without knowing that what could be uh what could be a thing that might go wrong in the mechanics of it i would not know like how long does uh does the card take to stop when i'm breaking it at a certain a certain pressure versus the other so things like that um require people .

Coming from a subject matter expertise and people who are experts or like who have backgrounds in these various uh specialized fields are most welcome in the data science field if not they might not be building models in python r or like other coding languages but they are absolutely very essential to be on the .

Table no that makes a ton of sense um we we do a number of these types of events and so recently uh earlier this year we did an event with um our sister company workfiera who which talked about you know at like taking your industry expertise and adding ai .

In order to sort of you know move your career into the next phase and obviously uh fourth brain uh was really founded around this idea that you know people come from such various different backgrounds and that enabling more of them to more readily access the type of knowledge getting them to .

The point where they can take a data scientist role and let's say bring their expertise in you know retail management to an actual retail company that is now trying to adopt ai practices like that is how you know we're really going to increase adoption of ai in a safe and you know ethical way to your con .

To your point so yeah i really love that yeah our definitely big thing big uh concept that we think about at ai fund at deep learning and in our ecosystem overall um so uh i mean obviously there's you know a ton for getting people who don't do coding into .

You know and kind of introducing them to ai but there's a large number of people who already do have some of that technical background you know maybe they're a computer software engineer or something like that um that are wanting to transition into data science specifically related roles and so in your experience steve i'd love .

To hear what do you find is the most critical missing skill or missing knowledge of the candidates who you know consider such a transitionist you know is it statistical expertise is it more on actually the software engineering side you know uh you know what do you find is is the .

Gap yeah that's a that's a great question because um ai and data science are becoming more interesting and trendy these days a lot of people are thinking about upskilling transitioning or adding them into their own skills right .

Now so what is happening is uh some of them are jumping a little bit too fast into this and i can see um based on the background um they're missing some of the elements because they want to make sure that they quickly get up and running and uh they find his job in it uh relevant to .

To data science so um they want to fast track things right if they have done four or seven years of education in something else they don't want to miss another um five and uh seven years to to learn data science from scratch going to university what happens .

Sometimes they pick the shiny things like uh machine learning may um you know connecting a machine learning algorithm to some data and get some results and um that's that is something that actually is misleading i remember i had a um i had a class at .

University i used to be a lecturer and um i had a lot of uh students from different you know backgrounds just uh jumping into advanced machine learning course like the the the subject to learn machine learning um on the final subject of the the master of data science course because .

They thought that's pretty much uh what is needed and i remember i was asked once that like um like is that it like we just connect some data set to um an algorithm and get it trained and that's that's called data science and why even .

That that is a thing and the the reality is it's not like i i just told them like this the class is showing you some of the stuff that you might see but the reality is we are not showing them the the real world problems the data the datasets are not actually that clean .

That easy to to find to put together and they miss a lot of this when they're learning so what i would say based on their background maybe they missed some of the um you know uh daughter related skills like the cleaning the the .

Munching and and the etl part if they're coming from um very um i guess strong programming background they they might miss the the stats and math to understand what is happening in the algorithm where they might think oh that's a black box we just need to connect it to the data and wait to see .

What's coming comes out of it and then we might want to connect it to kind of auto ml to um optimize the algorithm to to become more accurate and that's that's something that i would see a lot within uh the the new cohort generation jumping into data science .

That's really interesting yeah um so let's say for someone who's currently a data scientist then and you know maybe they maybe do a little bit of machine learning modeling but then they want to explore that potential career path in you know proper machine learning engineering what would be the road map or maybe the .

Resources that you would recommend then to make that type of a transition so it's like if i already know how you know the data manipulation part how do i learn the rest um that's also depends heavily on people so as i said like i i share a lot of learning resources on linkedin and i always say that .

Uh um there is no one-size-fits-all like some people love reading books i don't i personally would rather um watching andrewing's uh videos on youtube and coursera that's that's my preference but that's the that's the that's the reality like people have different preferences .

Um some of them would like to listen to podcasts um some some of them might want to read um different notes and and that's that's the reality but uh what i would i would say that um you need to make sure that you get the uh latest and and uh easy to understand information and .

Resources uh from wherever it's possible at this point it is mostly available on um courses that are um available on different platforms and also the videos because it's a little bit more uh like a faster way to generate these .

Resources if you think about books it it takes time and and sometimes you might uh miss the relevancy and uh you know it some of them might become a little bit up outdated um i guess blogs are also very interesting i personally like blogs they are much more .

Quicker to to read and pick up and finally i would say um if you know if you just search under google you might find a lot of results and you might get puzzled it's better to find some people like people that already know about it and get some guidance or follow .

You know people that share these kind of interesting insights that are hand-picked and curated on different platforms like social media yeah i think that you add to that yeah i just wanted to add i definitely agree steve everyone best have their own um methods of learning .

And i think for a lot of people it might be a combination of every all of the above of what you just listed right i think the flexibility of reading a book i actually have a book here on machine learning engineering i happen to have andrew burkov's new book that i was .

Skimming through uh there was an interesting uh forward by um like cassie kashurkov there but anyways there are so many ways to learn and i think right now people who are trying to break into the space are not at a lack of resource right .

We're not looking for oh man i wish i had a book or i wish i had a course it's more of the the oh my god i have a thousand or more courses that are reputable that i can choose from so i think at this point it's really about going after the method that works for you but also .

Looking for maybe a program that is structured around getting you to your personal goal as quickly as possible so if your goal is to be hired in the next you know four to six months maybe find a program that says you know we'll have the structured program here's what we're going to teach you and we're going to .

Help you find a job or help you you know maybe spice up your linkedin profile to to really prepare you for for jobs and interviews so i think you really have to ask yourself what is it that you want do you want to become the best data scientist the best machine .

Learning engineer do you want to get to the job as quickly as possible do you want to just start slowly transitioning while you still have you know maybe another job in in the business or in another um space so it all starts with finding out what you want and then .

As steve mentioned there are plenty of plenty of resources i know uh matt gore from fourth brain would would appreciate the plug here but fourth brain brain.ai does have a a cool-looking 16-week program so you know i encourage you to check that out i was looking at it myself actually just .

Just for fun to see what's covered there and it looks like they do give you some hands-on projects and some of that support so find programs like that i think that's really helpful yeah for sure fourth grade does a great job of sort of uh they have one program that you know focuses on sort of the let's say the fundamentals of machine .

Learning and kind of what that means but then uh they're also launching a new program this summer um that is oriented around ml ops so for the folks that you know do already have the data knowledge but really are trying to learn uh the scalability and putting into production and sort of those .

Those engineering skills if you will um and so uh that's a new course that they're launching i believe next month or maybe later this month you can learn out you can check out more at fort brain dot ai if you're interested in that um so uh moving on to our next question so .

With the lines blurring right between data scientists and moes sort of you know as i show you i said sometimes a data scientist is is the equivalent of a full stack engineer uh so you know because of these blurring roles or blurring lines uh do you think that someone could you know .

Potentially be both of these or do you think that they still have you know relatively distinct uh you know positions that you don't see kind of combining later on kate yeah i think they're i think they're still distinct enough i know uh you mentioned that they might might .

Have been around for about 10 years now i think that's not a very long time um 10 years i guess it could be for in in some areas i think they're still distinct enough where i do see two separate roles and interestingly i have a fun example of you know of another area that we can .

Think through where we can see these differences so if we think about a nuclear scientist and a nuclear engineer right they have very distinct roles for example a nuclear scientist has to know the science behind the atom right they have to write the recipe for how do we actually .

Extract energy from the atom they have to know the specific and various interactions but then you have the nuclear engineer who who doesn't have to know all of that right they have to take that recipe carry it to the world and you know they have limited knowledge of the actual atomic physics .

But they have other knowledge such as you know the materials that are used economics and everything else that it takes to build that nuclear power plant so i do see them as similar to that to that space not that i'm a nuclear you know expert .

By any means but i think it helps when you have an analogy and you think through that um so yeah i'm gonna go with two separate roles but who knows what happens in the future right for sure and i think as the field evolves you'll also see you know the nature of roles change i .

Mean 10 years ago social media marketing wasn't a thing you know and and for that matter neither was content marketing and now those are both you know entire departments at some large companies now so definitely something to think about as you know uh .

The specialization uh will that will happen as the industry matures um but this also begs the question you know will these the the tools and the skills that we think of as data scientists skills do we think that they will become so fundamental to the way that we .

You know produce technology and produce software and programs and platforms that you know they'll get dissolved into you know what software engineers do it at one point you're a software engineer or you're a software you are a software engineer who did cloud things and now you know that's those are no longer a distinction those .

Are the same thing you know if you're a software engineer you can't do cloud everyone kind of looks at you like what you know what time what time uh travel device did you hop out of so you know do we think that data scientist roles will get dissolved into software engineering roles in the future i assure you absolutely yes uh so one of .

The one of the things that i see is you know um probably back when in 2013 where when i was doing my undergrad software engineering itself was like a specialization people used to you know uh not everybody knew how to develop software's not everybody knew how to code in python or et cetera .

Whereas now i i am appalled to see like books and like courses being published you know uh software engineering or like coding for kids and all and it does like you know uh surprised me a lot that people these days want their kids at the age of five to learn how to code .

So that's that's a great example where we are trying to see that now we have when we are seeing this technology grow and we have all the all the kids and like people in their junior school using phones and laptops and tablets and what and what not other gadgets they might as well know how how that .

Works right so that's why we have transformed on seeing how um like software engineering itself or like coding itself have become has become mainstream and people from very different backgrounds now do they they do know how to code and the only place where i see .

Data science becoming merged into software engineering is where we have more and more packages like scikit learn like um like uh stats models etc like where we have all these models uh sitting inside these python libraries or like other libraries which is .

Now available to you at your fingertips it takes you two lines of code to run a model on a data set so that's how the ease of use has become and that's where i feel like data science is going to be integrated in software engineering because none of the industries are spread from data science like almost every industry .

Has been using data science similar to what what we see with cloud every industry is transforming their businesses into cloud similarly every industry is transforming their businesses into data science so what might still be intact is the people who are building these models because there still has to be research .

Going on on building a better models building optimized models as in when we see challenges with respect to scalability um we might see such challenges coming as well in future like where we move from data to big data to cloud that's that's how we'll .

Probably move towards some other challenge and we'll have a new technology to tackle that that's where uh like data scientists who are building the optimized models who are building the uh the core of it are still going to be there but .

Use of data science is going to be mainstream like everybody is going to start using data science in like pretty much uh in in their like day-to-day jobs awesome that's a that's a that's a really interesting feature like you know what the the difference between the .

World before everyone got a smartphone in their hands versus after has you know dramatically changed just the way we go about our day-to-day lives we'd love to see what happens when you know ai is sort of enabled in the larger uh larger public um now with uh pivoting a little bit more um .

With model building now becoming more automated right you spoke to more advanced packages um more complicated systems that are available for use is you know do do you guys feel that data science and machine learning are moving to become more focused on say business .

Intelligence rather than uh technical involvement so you know are we uh do you feel that we're gonna move towards you know specialized machine learning practitioner so you know maybe a data engineer and who focuses on on on data processing and then somebody else .

Who focus you know a data scientist that focuses on the specific types of research and then software developer or maybe you know ml ops person who you know focuses in that space or do you feel that we're going to move towards you know end-to-end generalization in the sense that .

You know there's going to be someone who's kind of expected to take it from hey i have a small idea all the way through to hey this is how you know the model now runs and provides insights to my you know business for uh steve i know you've you've thought about this a lot would you like to share your perspective .

For sure sandia so um to be very honest i would say both of them and that's that might look a little bit crazy uh because it as a general rule of thumb when um kind of special technology advances you might have more specialized people to deal with that .

Because of the breadth and depth of the knowledge right so if you think about it uh probably doctors used to be just all um all doctors and then um maybe a couple of hundred years ago they become specialized and then now more specialized in .

The subset or or very specific part that is just very complex so that would happen similarly to data science we have a lot of questions we have a lot of problems that we still don't know how to solve it i can just give you some examples based on the .

Work that we are doing at iso explainability ethics they're so complex believe me we're still um we have no idea how we're going to solve these challenges and um from one perspective it's like you connect these uh you connect these things .

These algorithms to specific um libraries and they will give you some ideas about how these models are being built and how they come up with prediction but that's not the case and it's not the reality it's a lot more uh deeper than this and we need a lot of people to actually just think about those problems but then .

With the use of lots of um auto ml and platforms we might end up with some sort of citizen data scientist or uh software engineers that will be able to leverage those automated tools to deliver end-to-end results for smaller organizations like if you are a startup .

You want to quickly uh deliver something a poc or mvp that is also possible using some tools it might be similar to um making a website these days like you can use wix to quickly make uh some sort of website but then you might want to write it .

From the scratch for some very complex um projects no that's really interesting i mean i've always thought of industries as they mature they become orchestras you all you always have specialists right you have your violinist and your .

Violist and your cellist and you know et cetera but at the end of the day you also need some generalist you need a conductor who kind of knows a little bit about everyone's thing to sort of go oh okay this is how we're going to make music together um so definitely see the see the need for votes so that's that's a that's a .

Really i love that perspective um so our very last question before we head off to the slider so we're going to do this rapid fire what industries do you see the most need for and growth for in the use of machine learning engineers and data scientists well .

I sure you want to give us your top one or two for me i think the two most important ones are healthcare because that's that's one of the crucial places where we are playing with uh lives and that's where we need most research that's where we need most uh most um like better um .

Technology being served in that space the other one is again like coming out of one of my experiences with united nations where we were working on uh like helping the climatic change and helping on how we can use the data and set up timelines for ourselves and .

Pull up more initiatives which can help um you know help us with the sustainable development goals so these are probably two of the areas which is like i am really fond of healthcare and climate change excellent steve i would say the the ones that have more .

Data would be probably um more in the most need in future uh healthcare education and finance and then kate yes so i don't think there will be any industries that are left untouched by the impact of data but i will i definitely keep my eye on financial .

Services because that's where my background is and i kind of have most of my domain expertise there but i definitely agree um with both stephen aishwarya that healthcare is one of the most important areas because we can literally save lives so i'm definitely keeping my eye on that sounds good um and if you guys are .

Interested in hearing more about the intersection of ai plus healthcare we had a recent event uh where uh andrew our founder spoke with fife lee from the stanford hai team um and had a fantastic conversation about you know the sort of how ai can actually help in healthcare so if that's something that you're .

Interested in please be sure to check that out as well um so now we're going to head over to our slide-out questions from our audience uh so the first one is from edgar do you think that having a phd or academic experience is an advantage or maybe even a disadvantage in the .

Sense that you might be overqualified for an industry job that's all right sorry um yeah i can have a quick one for this one because um yeah um as a lecturer i used to get a lot of these questions like are we in the right place to to learn data science .

Is universities still relevant so the answer is um yes and no again sorry for be being like this i don't want to make it complex but the reality is you will get a lot of soft skills at university that you will not get it online just looking at your camera right you need to work in a cross-functional .

Team that's kind of a learning plus um i guess job experience at the same time so if you're doing phd and you want to be a data scientist you definitely are learning uh technical and software skills at the same time that will help you it's not 100 necessary to start with that but i i don't dislike having um kind of an .

Academic experience that will bring a lot of other things more than just the understanding of the technical um so this is a dilemma which i had when i finished undergrad and i wanted to pursue my for like higher education i was confused between .

Going for a phd versus a masters and it was a safe decision for me to choose a master's because that was something i could extend and convert that into a phd so that that's what i went with and while i was in my school i realized that i am sort of a person who likes uh brick who likes varied applications .

Of data science who likes to explore a lot and read something about reinforcement learning read something about federated learning try multiple different use cases in different perspectives and that's what motivates me and again like as as steve said the answer is yes and no because it depends on what you want .

Like there are people who really want their focus on say natural language processing or computer vision or robotics and i have friends who have been you know dedicatedly working on robotics to um build arms uh for disabled people and .

That's that's where you have to find your motivation on what drives you is is it something which uh depth drives you or does breath uh drive you uh and i i think like my attention spam is probably of a fish and that's that's how i get excited with everything around me so when i'm .

When i am on linkedin i'm just super excited to read about like so many different things that people keep posting about and i i love uh learning from there and somewhere i feel the experience that i have gained in the last two years as a working professional is much more .

Worth for me what compared to what i could have gotten if i had gone for a phd so that is just my personal experience on um how my learning curve works so yeah that's my answer to that um next up we have a question about um something that's a little off our .

Main topic today but uh could one of you explain what you think is the difference between a data engineer and a data analyst i mean in the simplest terms i'd say a data engineer is someone who lays down the pipelines right of data someone who can understand where the data is flowing from and .

Making sure that everything is structured perfectly well it's never perfect right but as close to perfect as possible and i'd say a data analyst is someone on the very other side of the spectrum who actually takes that data and analyzes it maybe pulls some insights built some data .

Visualizations i'd say that's probably the the simplest answer on that question um um thank you for that um so one of the biggest uh stumbling blocks in order to start a career in this space is you know coming in without prior experience so you know let's say someone's you know learn the materials study it etc .

But hasn't quite had the opportunity to you know apply it in a professional setting how would you recommend that they sort of take that step kind of getting that hands-on practical experience so as an example like when i was in school i have i had done a couple of internships or you know .

Capstone projects with companies and one thing that i realized that there is only so much of real world experience that you can get with these uh internships or with with respect to like projects that you're going to do on kaggle etc uh because there's a whole lot of a different uh ball game when you get into .

An industrial setting and where you're actually tackling with situations where you have either too much data you don't have relevant data you have bad quality data and some of the situations where you don't even have data like you don't even know where where .

Where to pull that data from so those kind of situations i think you have to get into an industrial setting for you to get that exposure but companies do not really expect you to have that exposure before getting into an industry like if you're a fresher out of grad school i think having enough um experience .

With uh use cases which are closer to real life use cases is good enough where for example you can say that hey i'm working on a social media data and i'm trying to extract certain sentiments to predict something or or i'm working on a prescriptive analytics projects etc if your .

Uh thinking process is something which is business oriented which is problem solving oriented i should say not business oriented problem problem-solving oriented that is enough like where you're trying to understand that okay what am i trying to solve .

So then you reverse engineer on how you how you're gonna solve it the first question as a data scientist should be like what am i trying to solve what am i trying to achieve out of it before digging into like how do i go about this data yep you got to know where you're headed .

Before you try to map it makes complete sense um so one next question that we have from the slido audience that we're going to skip is about the difference between emily and emma lops we actually have an event with andrew as well a number of our instructors of the ml ops .

Program coming later this month uh so please audience keep an eye out for that if you're if you want to stay up to date on the latest in our events um make sure you sign up to the batch as well as uh other communication from us on our website deep learning dot ai um but we will .

Uh you know we'll we'll happily address that question in that in that session um so now moving on to another next question is which of these two career paths do you think allows for more creative freedom i i can i can kick things off here i i think i'm more of a creative person than .

A technical person and i think the data science role probably allows the individual to be a bit more creative and explore different areas right because we get to work with different data sets you get to visualize your data not sure .

If you can get even more you know i don't think what i don't know what gets more creative than visualizing data that's that's literally my passion is telling stories of data and visualizing data and i think the main reason is because you get to be creative you get to pick how do you visualize this what charts do .

You use which colors do you use and which methods are the most you know impactful and effective at telling those stories so i'll definitely go with data scientists but i'd love to hear from others um which one you think is more creative 100 agree with you kate i do believe .

That data scientist is going to be more creative and um playing with the data is just something that you need to have that curious mindset and creativity to deliver some results absolutely i would agree to that um one of the examples that we saw for .

Machine learning engineer where you're where you're productionizing it and that sort of becomes monotonous after a point because um you're not getting too much into how how the data is working or like how is it going to effect impact you uh in a in a modeling perspective but you .

Are mostly concerned about the packaging of it so i would also like go with data science where you are more creative on how you're creating uh the models how you are developing how you can extract more and more information out of the existing .

Data that you have compared to like uh doing the packaging out of it makes sense yeah um i i wouldn't say that i can speak to whether one is more creative than another but i do think that one of the hallmarks of sort of career progression is your ability to be creative within the guidelines or limits of your .

Particular role um you know you could be an exceptionally quantitative you know type role in marketing but there's still ways to be creative um and similarly you know i would encourage folks to think about themselves like if you know if you want to make something .

If you want to be creative about how you do it like even an engineer does have an opportunity they can't quite you know say a mechanical engineer can't quite defy the laws of physics but uh you know they can definitely think about new ways to approach problems and so i love that attitude .

Um so uh what we will do is uh wrap this up today so thank you all for your insightful discussion and thank you speakers for joining us today this brings us to the end of today's event uh we hope you enjoyed it we're going to set up a follow-up email with a survey for all of our attendees today we'd love .

To hear about how to make our future events better we're also giving out a 50 discount code for any of our online courses to up to 200 people who submit qualifying responses so keep an eye out and make sure to send in your feedback and thanks again to our co-organizer for brain for their support .

For today's event we'll be collaborating with them for more future events so if you don't want to miss them make sure you sign up via the link in the description box below and don't worry if you can't make a live event we always record and make you can always find these .

Recordings on youtube so we hope you walk away today feeling inspired about your next career step and we look forward to seeing you next time and as always keep learning thank you so much thank you

RELATED ARTICLES

Most Popular