I’ve been a jack of all trades web developer for nearly 15 years and moved to working full-time in hiring in September 2020. At Automattic, we’re hiring as fast as we can. This has given me a chance to work with and evaluate hundreds of software engineers over the past eight months.

The Day-to-Day: A Bit of Context

As a full-time hirer, my first responsibility is to kindly assess, mentor, and guide candidates through our hiring process to determine if they would be successful at Automattic. On the surface, it sounds like all I do is run interviews, grade code tests, and guide trials all day. 

While that may be my primary responsibilities, it comes across less as “grading” and more as:

• Digging into the why behind someone’s code.
• Becoming a PR Review expert.
• Delivering kind, constructive feedback.
• Implementing new ideas on how to accurately and more quickly evaluate candidates. 
• Formalizing/automating processes to make hiring more efficient.
• Working on interesting technical challenges to improve our hiring toolkit.
• Improving my own problem solving and designing abstractions skills by challenging my assumptions of how code can and should be written.
• Changing someone’s life by letting them know they got the job. 🎉

The Human Side of Engineering

Most engineering roles at Automattic are focused on coding in order to help people. In hiring, it’s a bit more meta: we are focused on identifying the best people to write code in order to help people. 

Engineering hiring is an interdisciplinary role that allows you to think deeply about how to best evaluate and mentor your candidates. Even when a candidate might not be ready to be an Automattician today, most are eager, kind, and smart. Our goal is to help them learn through useful feedback and mentorship on how they can grow to be successful their next time around.

If you’re like me and have a hard time with giving direct feedback, then working in hiring is also a huge opportunity to grow as a person. I’ve learned more about how to confidently give effective, constructive, useful feedback in the last eight months than I have in my whole life.

Giving Effective Feedback

This should be its own post to dive into the details. Here are a few quick lessons learned:

• Be positive, and focus on their ability to grow to meet the challenge.
• Be direct. If the feedback isn’t clear and actionable, you can’t be sure it was understood. And if you can’t be sure it was understood, you didn’t really give any feedback.
• Give feedback continuously. Don’t save it all for one big review at the end. The “official” feedback shouldn’t be a surprise – they should already have heard it before. This also allows people a chance to correct problematic areas before they become a bigger problem. It also shows you how they are at receiving and addressing feedback.
• Set clear expectations. What does successfully addressing the feedback look like? How will they know they’re doing it better now?

What Makes a Good Developer?

This is something I’ve spent a lot of time thinking about recently. I think it’s different for each organization. For example, you can be a really technically skilled engineer and not get a job at Automattic. And you can be a great engineer for Automattic, and not be able to get a job at another big organization. I don’t think I have the technical skills to pass more algorithmic-focused code tests, but I’m a great match for the kinds of problems that need solved at Automattic.

So, more accurately, the question should be, “What makes a good developer at [insert organization name here].”

For us, a brief list looks like:

• High self-motivation and drive
• Technical ability
• Fast learner
• Great written communication
• Can adapt and solve problems from many different areas, even if they’re not familiar with the language or codebase
• Good self-management and dependability
• Can give and receive feedback well, allowing for a lot of positive growth

There’s plenty more, but the main takeaway here is that technical ability is one of many pieces. It’s OK to not have the strongest technical ability as long as you have many of the others and, especially, if you can learn whatever you need to learn.

How do you Evaluate that?

Having a list of things we’re looking for is one thing, but how do you fairly and accurately determine someone has those qualities?

This is much harder to do than it seems, and relying on your own intuition of, “I know it when I see it,” is a great way to have a super-biased, toxic, homogenous company.

I wrote about Qualities of Expertise earlier this year as part of my learning around evaluating expertise. It’s hard, as the problems that require expertise to solve are open-ended, contextual, and don’t have a clear right/wrong answer.

The best way we’ve found to evaluate this is to have a 25 – 40 hour, paid trial where we give candidates a close-to-real-world codebase with some basic direction and let them solve it however they feel is best. It’s like a paid-mentorship and code camp where you might get a job offer in the end.

The great thing about the trial is that there are no right answers. There are multiple, valid approaches to everything. It’s less about the direction you choose, and more about:

• How you arrived at your decision.
• How you communicate your decision.
• How you solve/implement it.
• Does it work? Does it work well? How do you know?

There’s no accidentally passing the trial. If you receive an offer, it’s because you’ve been thoroughly vetted and deserve the offer.

---

The fast-paced learning and self-improvement is keeping me highly engaged in Engineering hiring. My next big step is learning more data and rubric development. I doubt I’ll run out of things to learn anytime soon.

If you’re a software developer looking to grow in these areas and have the opportunity to work in hiring, I highly recommend it.

Thanks to Thuy Copeland, Derek Springer, and Boro Sitnikovski for edits and feedback.