Our chief in Charge of Data Science muses recruiting into the difficult Big Data World
Recruitment is a funny old thing
(Bear with me on this one, I promise it’ll get more Data Science/Big Data related in a second).
It’s one of those love/hate relationships – one minute you’re creating matches between a candidate and client that you know were meant to be together, the world seems wonderful, and you’re walking down the road singing to yourself like it’s a terribly cheesy American box office feel-good film in a moment of very naff self-congratulation. (It may have happened, I won’t confirm or deny).
At other times however it doesn’t quite click, the world is terribly bleak, and you’re left scratching your head so much your colleagues fear that you have head lice. (Same again, but for the record I have tremendously clean if astonishingly unstylish hair).
If you’re reading this, it’s more than likely that you’re working within the Data Science/Big Data field in some capacity – practitioner, leader, possibly even fellow recruiter. I think one common thread between us all, that is to say in each of our jobs, is an element of logic and perspective. So just as with successes, it’s important to look for what we can learn from relative failures.
When we break down the reasons for the recruitment process not working out, there are a couple of themes that come to the top of my thoroughly hygienic head;
1. There were better candidates that applied through another channel (I’m grounded - it happens!).
2. A lack of communication over the process meant that the best person for the job went to a different company.
3. Sometimes nobody gets hired – budget cuts, loss of project work, etc.
4. The candidate, having seen the company and heard about the work available in person, doesn't see a future in the role.
and of course many others. We’re not selling storage space here; we’re dealing with real lives and monumental personal decisions in the context of a business environment. Looking at the most common problems, the most obvious solution is to understand the human element in more detail in relation to massive variety of commercial projects being dealt with. So, what do I mean by that, and why is it particularly important within Data Science and “Big Data”?
I thoroughly enjoy meeting my clients and candidates when I get the chance to (that’s why Skype is a Godsend for all my global contacts to a certain degree – niall.w.xcede if you want to add me there). This is because intelligent, personable people are the most interesting part of my job, and even more importantly because it helps prevent the aforementioned problems.
Since I started researching and subsequently recruiting for the Data Science (analytics)/ Big Data (engineering) market at the back end of 2012, the space has been growing rapidly, particularly in the UK and across wider Europe (the key reason for setting up my European focused “Data Science Collective Group on LinkedIn – check it out here
). During one of the many discussions and topics raised there it became clear that for my next blog people were really curious to find out exactly which sectors were getting involved and taking advantage of the talent currently available. To me this ties in entirely with my desire to put the human context of the recruitment process into perspective within this market.
A Variety in Data Science Uptake
Clearly at Xcede Recruitment Solutions
, along with a very select few others, we managed to begin building a desk early in a pretty green field. This has exposed us to a huge number of companies, ranging from multinationals (think top 10 companies in the world), right down to start-ups at seed-funding level / conception stage. No matter who it is, everyone struggles to find good people in this area.
Here’s a few examples of consistent client areas then to answer the Collective’s
need for companies to explore (!) and to add a little colour to what I’m talking about:
- We’ve dealt with a number of Publishers over the last few years looking at data sources that they’ve never had access to previously. With the rapid market transition from traditional paper literature to E-Books/digi-reading taking hold, my clients are getting a constant feed of information coming through directly to them to explore for internal business verticals. One of the key projects has been linked to their pricing processes. Getting this data first hand has provided them with the opportunity to revise their pricing strategy entirely; rather relying on information from front line vendors/high-street book stores they can now make informed decisions without the middle man. There’s so much more planned for these departments that their scientists will be busy for a good few years yet (you can guess, I’m not entirely at liberty to say here exactly which future avenues they’re exploring though).
- Gaming. Oh, gaming. Let’s split this up for a start too, because whilst there is a general field to address, we’re talking social gaming platforms that have made the headlines with the rise of mobile/app data, but also gambling companies that have been leading innovation in this field right from the early days looking exclusively at online/web analytics. Some of the most important innovations in the Data Science sector (predictive modelling/ automated algorithm based analysis in real time) came from this space, so it’s important not to rule it out. One of the key aspects of monetising gaming apps comes from customer retention and predicting drop-out points – if your free product (app) isn't causing people to stick around for the paid extras, what hope do you have?
- Ad-Tech is proving another source of key Data Science/Big Data technological turning points. One of the most interesting problems for companies in this sector has been to try and find a way to harness the data that they’re getting in a near constant stream/ on a massive scale. I’ve seen a clear trend in architectural progress and experimentation here in this area (commercial Spark/Storm/Vertica implementation projects).
- One area that has been slightly slower in catching up but can be seen making improvements right across the board is the Retail sector, whether that be in pure e-commerce or the biggest supermarket names around. The digital transformation these guys and girls are going through is tangible on a number of levels. Those who in the early 2000’s broke new ground by basing some of their business on offline analytics (utilising loyalty data through package tools like SAS, etc) are now moving to more dynamic open source environments that ensure that the job can be done no matter what tool is used. Marketing, online and offline customer relationship, supply chain management – each area is being explored to ensure that they’re running at an optimal level.
- Advertising companies/campaigns are taking a good hard look at unusual sources of data that have been presented to us over the past few years. Social media analytics is one of the fastest growing areas in Data Science for good reason – rather than relying solely on social analysts using tools like Radian6, etc, teams are integrating Data Scientists who may have a long term background in text mining (for example), in order to analyse sentiment towards a product or multi-million pound campaign with far greater scope for improvement.
There are so many more we could get into (banking, insurance, recommendation/ranking websites, Venture Capitalists, utilities/energy, mailing groups, delivery companies, media companies (television/radio), lending companies, charity houses, aggregators, telecoms, fine-art/ auction house investment, even recruitment software!) but this could go on for 10 pages as I’m typing, if not more. The best thing to do if you’d like to find out about other areas is to get in touch with me directly as an interested potential candidate, hiring manager/division director, or even a company owner. Even if you don’t see yourself as any of these yet and are just curious to find out more feel free to say hi – my mobile/ skype profiles are available before/during/after work due to nature of the beast!
What Can We Learn From This?
At the end of the day, you can see why I find it so important to meet my clients and get as much information as possible from them about their work and ideal type of person – how is a process to work if companies expect a “template data scientist” candidate to be sent over without considering the nuances of their technical ability, personality, and career goals in relation to the sector and size of company that they’re potentially committing years of their life to? Take this article by Michael Li for a good general look at this - it's not always the case, but it's usually important to ask yourself if the company/ potential hire is 'producing analytics for humans or machines
It’s really not just about ‘X’ amount of experience as a ‘Data Scientist’ or ‘Data Engineer’ – people may be doing the relevant work under a different guise (I’ve noticed a particular disparity across Europe about job titles in relation to skill set right across analytics in general).
Nearly two years into my job, I still feel that I’m learning the trade to a large degree – like anything else good recruitment takes time to establish (I’m sitting across from two of the best consultants in the country who I’m lucky to count as slightly bonkers mentor figures with 15 and 7 years of experience respectively) despite having enjoyed a relative amount of early success. What I personally can offer compared to other recruiters (I hope) is a level of understanding about the subtleties of each scientist and engineer that I deal with in terms of their technical abilities and personality traits. It’s all I do and all I know. I don’t pretend otherwise.
For my strength to come into play, I rely on a total understanding of who I’m dealing with and what they need – with such variety in Data Science/Big Data projects out there, candidates need the same level of understanding about their options as their employers do about the talent pool they’re searching within.
I’ll be interested to hear all of your thoughts about the market as it continues to grow. Are you working within a company operating in a sector that I’ve failed to mention here? I'm sure anyone looking at this would appreciate examples of your team's work!