Do Your Job... And How a Reference Check Provides Insight to how a Candidate can do Their's
Do your job! Whether discussing a professional sports team or complex corporate structures, successful groups epitomize this manta. In the case of Super Bowl XLIX, the team famous for “doing one's job” eventually came out on top, even if it did require just a little bit of goal-line luck.
In recruiting, the entire process circles around the very same idea. Job ad copy, resume keyword search, interviews, assessments and screening all help hiring managers drill down to the candidate that can best do the job. Except anyone who has made a bad hire or missed out on a good one knows the process is far from perfect.
The problem according to Marcus Buckingham is that most HR data is bad data. Whether in peer reviews, interview assessments or reference checks, the Idiosyncratic Rater Effect plays a major role in the variance of the data. The Idiosyncratic Rater Effect is the degree to which the rater’s personal biases apply to the rating of the ratee. Studies reveal that 61% of a performance rating is a reflection of the rater, not the ratee. For that reason, one person’s score of "10" is another’s "7" for the same criteria. In fact, the actual overall performance of the ratee doesn’t make up for more than 20% of the review score!
In the space of reference checks, biases of hand-picked references skew responses positively and give little objective insight to how the candidate will perform. That is true, unless you ask the right questions.
To better know whether a candidate can do the job, we have re-engineered our questionnaires to extract objective and actionable feedback from references without the need for back-channeling or hiring a private investigator. Based on the qualified references responses, our algorithm distributes a capped number of points to a set of attributes to give an accurate representation of the candidate’s strengths and weaknesses.
In the example above, the candidate was rated highly for being both confident and motivated but came in well below average for communicative. That’s not to say that the candidate is not communicative at all. Rather, the results indicate that communication may be an area for improvement to become a more balanced candidate. Finally, references can add some color to the results by elaborating on perceived strengths and weaknesses in the extended answer section. With this objective feedback, hiring managers are better able to answer the question, "Can this candidate do the job?"
Want us to put together a report for you for free? Request an invitation to our beta.
Dor is CEO of Rillate. Have a question or comment for him? Shoot him a note at [email protected].
Tying the Knot between Robots and Recruiters with Automated Reference Checks
Since the Industrial Revolution, there has been a constant and widely misconstrued notion that humanity is trending towards a dystopian world ruled by robots and software. While it is true that jobs and communication have been displaced as of the result of evolving technology, the overall volume of opportunities has increased, allowing us to connect on even more personal levels.
The “human touch” still and will always play a crucial role for recruiters and hiring managers. Since online job boards and social networks have made it easier for candidates to find jobs, the importance has shifted from discovery to engagement. Or more specifically, it gives the advantage to the hiring manager who develops a genuine relationship that extends all the way through an accepted offer. To stay up late, to forge the bond with both candidate and the hiring manager, to exaggerate shamelessly and most importantly, to get it done. All things an automated search can't do.
As we revolutionize the reference check to feed the recruiters intuition rather than confirm it, we must maintain principles that can establish that trust while objectively bringing to surface information not present on a resume or within an interview. Information that could be used to make better hiring decisions from the start of the recruiting process. Information such as:
3rd Party Perspective on a Mix of Strength and Weakness - On a scale of 1-10, references provide their input on a mix of both practical and cultural attributes regarding the candidate. For example, we ask their opinion on the extent to which the candidate is intelligent and autonomous but also communicative and a leader.
Specially Crafted Open-Ended Questions- Questions like “If you were to start a company, would you hire the candidate?” and “What question should we ask the candidate before we hire them?” provide a 3rd dimension to the sourcing and interviewing process. Answers to “What is the candidate passionate about outside of work?” can provide an advantage to recruiter that can connect relevantly to the recruit from the start.
Kathleen Gaile, HR manager at Samasource and an early product advisor, stressed to us the importance of marrying our robot with a human. Whereas a robot can crunch ridiculous amounts of information and standardize data to account for the inaccuracies of human feedback, it still has a hard time saying "Hi."
Want to tie the knot with our robot? Start here to create a report for your next hire.
Dor is the CEO at Rillate. Have a question or comment for him? Shoot him a note at [email protected].
10 Ways Rillate is Making Reference Checks Less Terrible
At Rillate, we believe the key to building a successful startup is to connect bold ideas to the real-life practices of potential customers. Since our mission is to improve trust between strangers online, over the last month or so, I connected with dozens of recruiters and HR managers to learn about the ways candidates establish trust with employers. Through all the conversations, one thing always stood out when it came to discussing the recruiting and the onboarding process. As one HR manager so eloquently put it, “reference and background checks suck! I hate them!”
This got us thinking, what makes references “suck” so much? What are the ways we could help recruiters and HR managers not only do these checks more efficiently but also solve for problems they couldn’t the old fashioned way.
Below is a list of 10 components of a reference check we improve with Rillate reference checks.
If their job is to disqualify candidates, they don’t do a good job at that. Less than 1% of candidates are disqualified as a result of a reference check. Because it happens so late in the recruiting process, sometimes even a bad reference won’t stop a candidate from getting a job because the hiring team is in a crunch to hire someone or are overruled by a more senior member.
None of the feedback can be used in the interview process. Since it is used as one of the last steps in the process, none of the information collected can affect the preliminary phone or in-house interviews.
They are a time suck. On average a conversation with each reference takes 20 minutes. Since there are 3 references, a check can take over an hour including the time for scheduling and recording the information.
The data is trapped. Once HR managers go through the painstaking process of recording that information, it becomes trapped, sometimes even literally when stapled to the back of the resume and filed in a cabinet.
Reports are text heavy. Most reviews are paragraphs of text. It not only takes a lot of time to prepare the information, but also makes it difficult to consume. Which leads us to #6.
No standard questions. Since the conversations with references are free-flowing, you might not get to ask the same exact question every time. These slight inconsistencies and recording of answers makes it difficult to evaluate candidates equally.
References themselves are not qualified. Collecting the information is great, but what makes the references qualified to provide a review? Would you then do a reference check on a reference? And then a reference on the reference’s reference?
It publicizes that the candidate is changing jobs. As soon as a HR manager requests a reference check, it discloses that the candidate is on the way out. Since candidate’s would prefer not to make it public until the change in jobs is 100% certain, they won’t ask the most qualified reference, their current manager, for a reference. Not to mention that it makes for a painfully awkward exit interview if that job offer was rescinded after the reference check.
Most reference checks are general. Very generalized questions on the candidates strength and weaknesses and are difficult to tailor to the specific job opening and company culture.
Education verification isn’t included. For those recruiters that want to perform education verification, it would require a completely different set of calls to the university registrar office.
Our reference check reports address each of the issues above by creating a balanced report including both quantitative and qualitative information while keeping the human touch. We help recruiters and HR managers find more qualified candidates faster and with more accuracy than an an average reference check.
To get a sample of our questionnaire and report, request an invitation to our beta.
Dor is the CEO at Rillate. Have a question or comment for him? Shoot him a note at [email protected].
Growing Collaborative Economy Marketplaces through Trust
Qualities for maintaining a healthy marketplace rest on a strategic balance of 3 important pillars: 1) supply and demand, 2) information, and 3) trust. Below is a screenshot of how Amazon executes on these factors in their results page. But imagine for a moment if one of these elements was missing. Upon performing a search for a new iPhone 6 case, you would run into the issue of not having the item in stock, not knowing specifics about the product like color or features, or not believing if and when you would receive the product after you pay. Notice how Amazon addresses each of these elements on the results page. Also, it is no coincidence that the icon for Prime includes a check mark to evoke thoughts of confirmation to strengthen trust of purchase.
These same principles apply in collaborative economy marketplaces as well, but with the added complexity of a human layer. The practice of acquiring, presenting, and delivering services naturally varies from experience to experience and provider to provider. Trust, as a result, carries a bigger burden to complete the transaction. On the spectrum of purchasing a homemade good on Etsy to staying in a stranger’s house on Airbnb, a different but necessary threshold of trust must be obtained before any engagement takes place.
When it comes to making a decision, our brains assess a bevy of factors we assume to be true, but subconsciously question, before making a final decision. User reviews, profile descriptions, pictures, services provided, and price all are given decision making weight to tip the scale and make a decision of whether or not to trust an individual. Airbnb does a great job of anchoring these factors by normalizing user reviews with regression analysis, sending professional photographers to verify the look of the space and allowing the host to verify themselves in many ways. Had they acquired a traveller, connected to the perfect host, but not created an environment where the traveller could trust the host, their model would not work.
In the last year, the collaborative economy has grown into a $26B market with 25% annual growth. $26B represents remarkable growth in a relatively short time but still just a nominal sum in comparison to market sizes of traditional competitors in the hospitality, transportation and service industries. So what factors are holding back online trust from catching up to and surpassing traditional markets?
Nightmare but all too common cases of a Manti Te’o being “catfished” through an online dating scheme or Airbnb horror stories dominating media attention.
Users realizing that Facebook and Twitter profiles are just not enough. Though Facebook is trying to enforce the use of legal first and last name on its platform, it still doesn’t answer the question, “Can you complete this specific task for me?”
Traditional institutions were not involved in this online shift. Professional associations have long dominated the offline world, but they have now been replaced with amateur micropreneurs.
Background checks, once necessary to attain a salary job, have not evolved to meet the technical requirements of online platforms nor do they cater to the audience consuming the information.
Data, anchored by relevant and reliable sources, tips the scale in favor of trust and growth for the marketplace. Data that is not only relevant to the decision making process, but also is owned by the ecosystem and does not exist solely in the vacuum of a specific marketplace. Rillate aims to extricate data from walled-garden environments and allow users to “BYOT” (Bring Your Own Trust) so they can arrive at marketplaces with competitive and accurate history of their accomplishments wherever they land on the World Wide Web. At the same time, we have the opportunity to reimagine the background check as a whole and shed the puritan stigmas associated with criminal and financial background checks. Together, we have the ability to infuse marketplaces with data individuals are proud of to decrease the barrier of engagement for new suppliers and tip the scale for consumers who have yet to engage.
APIs are happening. Driven by the apps and mobile connectivity explosion on one end, and enterprises gearing up for faster, more effective way to do business on the other, APIs are becoming the lifeline for the internet powered economy. The gold rush into the API marketplace does not mean that every API is destined for success - there are a plethora of factors involved on top of just converting your business into HTTP packets. Making the API discoverable, developer friendly, secure and scalable are each critical to engage consumers of your API. After initial investment of effort to design for success, supporting a thriving community around the API is ongoing work.
If you're a small startup with the objective of getting to market quickly and efficiently, solving for these needs can be overwhelming. Enter API Management solutions (also known as API Proxies, your API’s best friend). At Rillate, we decided an API proxy was what we needed to bring our verification API to market. With the following checklist in hand, we set out to evaluate offerings in the API proxy market:
discovery: API marketplace gives our API additional visibility in front of developers ready to consume APIs.
monetization: building pricing and billing into our product is a nontrivial effort. We wanted to be able to use external provider for those to speed up our time to market.
consumability: features like built in documentation and testing, client libraries to minimize the zero to production time for the developer.
support: efficient way for developers to communicate with us about any questions or issues.
scalability and stability: features like rate limiting, uptime and latency monitoring at proxy level.
security: a core feature for API Proxy, we looked to make sure our design meets standards so developers don’t have to re-learn and re-implement authentication code.
analytics: anything worth doing is worth quantifying.
cost: last but not least, we wanted all the above features at a cost within the stage of our business
The quest for API began with identifying the key players. We found this API comparison article by API Evangelist aged quite well and it served as a pretty accurate starting point to narrow the candidates.
Mashery is the best known and most mature API proxy out there, trusted by enterprise market and is well suited for the needs of large scale, sophisticated business by offering high availability, on-premises and cloud deployment options, personalized API certification process and guidance on best practices. For our set of objectives, this was a higher point on features and cost that we looked for.
Apigee's offering is as solid and diversified, with similarly rich featured enterprise support. They do fantastic job advocating the API economy, promoting best practices and reaching out to developer community. Apigee has tiered services and pricing structure, and we realized that monetization capability comes in a price range that wasn't yet feasible for us.
3scale is set apart by its architecture - API management is done by NGINX proxy deployed alongside your app. We liked the idea of simplicity, greater control, flexibility and improved performance that this approach enables, and we loved the open source aspect of the technology. Monetization feature with clear pricing structure was also a plus. Operational overhead of installing and managing additional components in our production environment ended up being a deterrent.
Apiary, a relative newcomer, has attractive clean philosophy of iterative API design with automated testing and documentation generation, based on their emerging standard of API Blueprint. They're doing great work and are definitely worth another look in the future. However, our requirement of monetization feature had us continue our search.
Mashape ended up being our final choice. We found it to be on par with others when it came to developer centric features such as the developer portal and communication tools, fully interactive API documentation pages. Mashape biggest differentiator is its developer community and API Marketplace, which aligned with our objective to have our API discoverable by developers who need it. Setting up API endpoints was easy and intuitive and the resulting documentation/test page was easy to use - we soon found ourselves preferring Mashape's API Test/Documentation page to our internal tools for API testing. Mashape provides open source Unirest libraries in most popular languages enabling the developers to integrate our API in just a few lines of code.
A huge factor in our decision was how Mashape approaches monetization: we found it to be aligned with the needs of an early stage startup like ourselves:
no upfront cost: Mashape takes a cut of the price paid by developer to API provider. It costs nothing to start having your API consumed via Mashape.
flexible pricing structure: Mashape's price plans are oriented toward straightforward subscription pricing model; but Mashape also gives the API provider greater control over the cost billed for each call. This is done by setting the 'billing' header on the response to the desired amount, allowing the application to own the billing logic, but still delegate execution to Mashape. Our choice of pricing structure is closer to pay-per-use model, with a friendly twist - the caller only pays for checking someone's school enrollment if the call returned a positive response. Not only there's no upfront subscription cost to pay, we removed the upfront risk of integrating with us (such risk could come from unknown distribution of end user data, running pilots or exploratory projects, A/B testing). Even with the pricing structure so different from the typical subscription model, we were able to make it work with little code added to our backend.
Last but not least, we felt great cultural fit with Mashape - our questions and requests for support were answered quickly and with great deal of knowledge and understanding. It is obvious Mashape is still growing and adding features, but they got the key values right.
Natalya is the CTO at Rillate. Have a question or comment for her? Shoot her a note at [email protected].
Thank you so much for your interest in our blog. We here at Rillate are proud to announce our first verification product that will allow developers to verify their users' stated affiliation to over 4,500 academic institutions in the United States.
When Natalya and I launched Rillate in February, our biggest challenge was putting together a reliable dataset with wide coverage. Today, through direct partnerships with organizations and institutions, the data used for your verifications is the same one used for graduate school applications across the United States. Whether it is affiliation to a public, private or professional institution, there is a 98% chance we can vouch for it.
To make it easy to get started, we have partnered with Mashape to publish our API. In the meantime, we would love to hear from you! Let us know what you are working on or just say hi. We are available anytime at [email protected].
