How do you know if you have somebody good at tech, or if they are just good at faking it? This is a frequent problem many tech managers are faced with. If you aren't technical, how can you tell the quality people from the fakers?

In many other industries, quality is easy to measure. The difference between a 15-year short-order cook who hasn't advanced beyond burgers and frozen hashbrowns and a master chef who has worked 15-years leading multiple restaurants becomes very apparent with a taste test of their presented meals.

Experiential Learning

Quality comes through the forge of experiential learning, or basically learning by experience (on-the-job). Through this we are continually challenged and pushed out of our comfort zone by trying new things, and this often is driven even more when changing jobs.

This on-the-job training becomes even more critical because technology changes so rapidly, creating a fear environment. Furthermore, the demands of a production environment simply are not learned in the conventional rote / didactic training of a school setting.

There is an ongoing observation about how long people stay at their jobs in tech, which averages around two years. Many find this alarming, but I suggest it can be a good thing.

To leverage experiential learning, we must embrace four elements:

be active - you engage and are part of the experience
introspect - you take time to reflect on the experience
analyze - you use analytical skills to conceptualize the experience
leverage - you use decision making and problem-solving skills to leverage the new ideas

The Danger of Titles

A small company of three people can have four VP's (yes, multiple hats), where one of them is the "VP of Engineering"; yet a company of hundreds can have a single "Director of Engineering." So who is the better candidate?

To understand how either job helped each individual in their skills, ask some questions: How many direct and indirect reports did you have, and for how long? What did you think of that experience? How did you handle the disagreements of tech decisions? How did you hire good talent?

The other challenge of titles is when different things are combined into one title, such as duties, experience, and authority — a Software Engineer III might have five years experience and be a team lead, yet be the worst at developing and leading. This misuse of titles is an anti-pattern, yet there is a continual push to associate "rank," or some form of experience, as being equivalent to capability, and then use a measuring stick of a job title to denote authority.

Don't get caught up on titles. If you need to define authority structure, do that, but don't make it about expertise or experience, make it about duties and responsibilities; and don't assume just because somebody lists a title, that they have any domain knowledge around what that title may mean to you.

Are years of experience misleading?

Somebody with 15 years of experience at one location may be amazing, but they could also be stuck in a rut and haven't been challenged as much as they would have been if they had moved on. Ask the right questions to learn how the candidate has been challenged, if they are working at the same place for a longer duration.

Measure by Aptitude, Expertise, and Empathy

Knowing we are faced with a hard challenge, focus instead on Aptitude and Expertise. Aptitude is our ability to self-motivate, decompose a deliverable into tasks to produce a result, and to understand the idiomatic way of our engineering discipline.

Aptitude doesn’t quantify Expertise, however, nor non-engineering soft skills like leadership.

Expertise is different from Experience; it is a way to determine how much an engineer may have been challenged beyond their usual routine — or how much they may have been exposed to experiential learning. The result is it may be a positive thing for somebody to have moved jobs every few years, while also expanding to different languages and domains of knowledge, and a negative to stay at the same place for too many years.

Empathy is harder to measure. Consider doing an exercise where the candidate pair-codes with you driving the keyboard, but they are providing the direction on a simple task. Learn from this interaction how well they empathize with you.

Rubrics

I have created two rubric's which you can use, either as a capability maturity model to be a guide to advancing your career or as a way to help understand candidates and your team.

In this rubric, I use tiers, where the higher the tier, the more advanced the individual. They are also written so you can self assess. Then look upwards at the next tier and identify what you can do to improve.

Aptitude

Tier A5 - Driven

You are proficient in multiple programming languages and understand the strengths, weaknesses and idiomatic patterns of each.
Personally explore new things - You continually improve your skills outside of work, with side projects and personal education steps.

Tier A4 - Deliverable

You understand the idiomatic behavior of the language, and what are proper/improper forms.
You can write tests for your own code.
You can work off deliverables, not individual tasks -- somebody can ask for something, and you can break it into individual steps without requiring a lot of oversight.

Tier A3 - Taskable

You are proficient in at least one language and can write functionally and logically correct code, with only a few anomalies.
Other people write tests for your code.
You are good at completing work if somebody else outlines the tasks for you.

Tier A2 - Needs Support

You can write code that is functionally correct if somebody else gives the technical direction & oversight.

Tier A1 - Things to work on

Your code cannot compile or does not work as expected.

Expertise

Tier E5

10+ years of production experience, 3-5+ companies and 5+ domains/languages

Tier E4

6-10 years of production experience, 2+ companies and 3+ domains/languages

Tier E3

3-5 years of production experience, 2+ companies or 3+ domains/languages

Tier E2

1-3 years of production experience (where you worked on a product that was in actual use, by actual paying customers, not just a prototype or project.)

Tier E1

You are still learning (intern / schooling level)