Being Chief Technology Officer: Lessons learned in my first year

Happy New Year!

2020 was a difficult year for most of us, as we fought with COVID-19 and came to terms with the remote way of working. It was a year when we had a lot more time in our hands as all of us were locked in our houses with almost no travel. The things that we took for granted were taken from us and there was a constant fear of losing the loved ones. 

I hope in 2021 we regain our freedom to live freely. But, this time with a sense of responsibility and awareness. 

It is now little over a year since I became Chief Technology Officer (CTO) in my current organization. And, I thought it will be a good time to do a quick retrospective on the lessons learned in my first year as CTO. The journey has been tough but deeply rewarding for me. There were occasions when I thought the leadership role was not for me and I should go back to being an individual contributor. But, with support from my organization and learning (books, blogs, observation), I have started to enjoy the role and its challenges.

Before I talk about the lessons I learned, let’s look at my software engineering journey.  

My software engineering journey

• 2005-2008: Software Engineer
• 2008-2012: Senior Software Engineer
• 2013-2014: Principal Technology Evangelist
• 2014-2019: Principal Engineer/Architect (during this period, I was in the role of Director, but I was neck-deep in writing code, building systems, and technology consulting)
• 2020-present: Chief Technology Officer (my first leadership/management role)

Yes, this is my first leadership and management role. The leap from an individual contributor to a CTO was as much nerve-wracking as it was ambitious in terms of responsibilities.

Of course, along with this opportunity came its share of pros and cons. Nonetheless, I took the plunge and went through this transition without any preconceived notion.  

Defining the CTO role

When I was told that I will be the next CTO, I immediately looked for a playbook that would describe and provide best practices and guide me on the rules of becoming a good, successful CTO. I wanted to be ready for the role and study the entire CTO literature available on the web. Guess what? There was no such playbook. 

At the end of the day, it is always about learning on the job and not making the same mistake twice. 

The first thing that I read was what people usually define the role of Chief Technology Officer as. Apparently, it is one of the most confusing C-level roles, and each CTO plays a bit differently. The best description of this role that I found was in an essay by Matt Tucker. According to Matt, to better understand the role, you should break into five flavors.

I think the above framework gives a good mental model on how to think about the role of a CTO. Mostly, they come with a combination of two or more aforementioned flavors. 

Let me share what my role, as a Chief Technology Officer, looks like. In my view, the framework (above) shared by Matt Tucker is from a product organization’s perspective. I am a CTO of an IT services organization, where this role gets diluted, fragmented and challenging because of the following additional factors:

• People are an important asset in any organization, but for an IT service organization, growth is directly proportional to the number of people. If you want to remain a niche service provider, then the story could be different. 
• You have to make modern technology accessible to large enterprises that are not ready for the change. In the last 5 years or so, there is a push towards digital transformation in most organizations. In my experience, organizations are now more open toward trying new technologies (React, Golang, Flutter, Cloud, Kubernetes) and architecture styles (Microservices, Event-driven, Serverless), more than ever before. This is a great news, but very few organizations understand the complexity introduced by these modern technology stacks. They are not doing the groundwork required to become the next Google in their domain. You can read my post 11 Reasons Why You Are Going To Fail With Microservices 
• There are not many good software engineers in the market with real experience in these modern technologies. Good software engineers are expensive and their financial and work aspirations are more aligned with product organizations. For an IT service organization, it is not possible to pay at the scale of product organizations. 

There is not much written about/by the CTOs of IT service organizations. It is not clear who should be your role model, so I will take Matt’s framework and define it for myself.

Here, you will see Matt’s framework modified by virtue of the flavors comprising my role. I have given a rough estimate on the time I had spent on each activity in the last year. 

As you can see in the above table, I am everywhere. Luckily for me, I had less overhead in context switching. The main reason was I made sure that I never get involved in more than 2 tasks at a time. 

In my first year as the CTO, I have built some level of delegation hierarchy. Hopefully, in the second year, I will be more focussed on a few of these flavors.

Lessons learned in the first year

So far, I have shared about my journey and the CTO role definition. Next, let me walk you through the lessons that I have learned in the first year as CTO.

Lesson #0: You have to believe in yourself and ask for the role

Most software engineers dream of becoming a CTO one day. This is how some of us define a successful engineering career. Someone will not make you the CTO just because you are the most capable software engineer/architect in your organization. You really need to have the hunger. 

It took me close to 2-3 years before I was sure in my mind that I am ready to become a CTO. One of the reasons I thought I was not ready for a leadership role was Peter Principle. 

The Peter Principle is an observation that the tendency in most organizational hierarchies, such as that of a corporation, is for every employee to rise in the hierarchy through promotion until they reach a level of respective incompetence.

I always took pride in being a competent software engineer/architect. The fear of becoming incompetent one day made me stick to the hands-on individual contributor role.

One thing that I realized was if I am not going to do the role, someone else will. Since, I knew the organization long enough and I have figured out my engineering leadership style, so why not give it a try. The most difficult part for me was asking for the role. The sad part is if you don’t ask people, they don’t give you what you deserve. 

Now, I’ve actually always found something to be very true, which is that most people don’t get those experiences because they never ask. I’ve never found anybody who didn’t want to help me when I’ve asked them for help – Steve Jobs [1]

Lesson #1: Schedule time for yourself

A couple of weeks back, a colleague asked me how you are able to do deep work when you have to attend so many meetings. I discovered that people will add you to a meeting as soon as they find a free meeting slot in your calendar. I struggled with this for the first half of 2020.  I was in meetings most of the day. Most of the thinking work that I did during this period was either after office hours or on weekends. 

I changed my way of working in the second half after realizing that I can also schedule time  with myself. Now, everyday I schedule a couple of hours to half a day with myself and do one deep work task. This way I am able to manage between maker schedule and manager schedule [2].

Another way, I avoid becoming a slave to my calendar, is by ensuring with the organizer whether my presence in the meeting is critical. At other times, I decline meetings when someone from my team is already attending. 

Lesson #2: Getting things done without doing them

This is the challenge that most individual contributors face when they take up managerial/leadership roles. You know you can do the task better and faster. Thereby, you prefer to do the task yourself. This does not scale and you quickly become the bottleneck. And I bet you already know the answer. 

The best way to scale yourself is through delegation. There are two parts in the delegation: what to delegate and which delegation level to apply.

What to delegate

In How to Decide Which Tasks to Delegate [3], Jenny Blake categorizes tasks into 6 categories which she calls 6 Ts.

• Tiny: Tasks that are so small they seem inconsequential to tackle but they add up.
• Tedious: Tasks that are relatively simple probably are not  the best use of your time.
• Time-consuming: Tasks that, although they may be important and even somewhat complex, are time-consuming and do not require you to do the initial 80% of research.
• Teachable: Tasks that, although complicated-seeming at first and possibly comprising several smaller subtasks, can be translated into a system and passed along, with you still providing quality checks and final approval.
• Terrible At: Tasks that not only do not fall into your strengths, but an area where you feel unequipped.
• Time sensitive: Tasks that are time-sensitive but compete with other priorities; there isn’t enough time to do them all at once, so you delegate an important and time-sensitive task so that it can be done in parallel to your other project-based deadlines.

Tasks like organizing internal tech talks, operating internal infra, code reviews, junior engineer hiring I have delegated to others in my team. 

Once you know which tasks to delegate, you have to use a delegation level that gets the best job done. I learnt about 7 levels of delegation at Management 30 website [4].

1. Tell: I will tell them 
2. Sell: I will try and sell it to them 
3. Consult: I will consult and then decide 
4. Agree: We will agree together 
5. Advise: I will advise but they decide 
6. Inquire: I will inquire after they decide 
7. Delegate: I will fully delegate

At the end of the day, you have to use just enough management to get things done.

Lesson #3: Reduce the chaos

Whether you like it or not, there are chaotic situations that happen in every organization. The chaotic situation could be a software delivery gone wrong, a critical bug that took the customer system down, system performance issue, or any people related issue. When you, as a leader, enter such a meeting, then people expect that at the end of the meeting, things will be more clear and there will be a thorough plan that the team can follow. 

Your job as a leader is to reduce the chaos and bring more clarity. An example playbook that I have use for one such situation is as follows:

• Simplify the problem by asking questions. This usually involves going to the guts of the problem and removing all the unnecessary information associated with it. Your goal as a leader is to make people think. So, you have to ask deep and broad questions like why do you choose this design? What is this metric going to tell us? What do you think the user is thinking at this moment?
• Creating a small todo list with clear actions. For example, for a performance fix, this was the todo list:
  • Observe the performance metrics in the APM tool to ensure there is an issue
  • Reproduce the issue in the lower environment
  • Write a test if it does not exist to understand the functionality we have to performance tune
  • Fix the issue making sure test still runs
  • Test in the lower environment and see impact of the performance fix
  • Deploy to the production environment
  • Observe the APM tool metrics to be sure fix correctly worked
• Assign owners to the tasks
• Document in the wiki

Another related thing I have observed is that many people can’t work with abstract ideas. Most people find it easy to incrementally add value once someone has done the initial effort.  You have to distill things done to concrete tasks that people can work on. This means you have to do a lot of thinking before you can hand-over tasks to the people. As a leader you learn with time how much upfront thinking is required. 

Lesson #4: There are times when you have to react and show your displeasure

As a leader it is expected that you don’t react to things emotionally. I agree that 90% of the time a leader should not react. But, there are times when you have to react and show your displeasure at things. The bar at which you react should be so high that it couldn’t be reached easily. Someone should really screw things up badly that you have to react. 

Let me share an example, one of our senior developers (close to 10 years of experience) posted a message on the Teams channel that they are stuck since the previous day because they were unable to call the REST API from Java but the same API is working from Postman. This was weird behavior so I called up on Teams to look at the problem. He showed me first the Postman and then the Java code. The problem was that the JSON body was expecting username as the field but the developer was passing userId in the Java code. In Postman they were using username so it worked fine. I know this looks like a trivial example and most developers have made such a mistake. The reason why I was disappointed If Postman is working and your code is not working, then developer should not think something is wrong with the API without checking what request they are making through code. You can’t call off your day and in your next daily standup tell to the client that there is something wrong with the API. Clients are not that forgiving. There are automated ways to generate REST client using tools like Insomnia that can help validate your assumption.

Also, when I say showing displeasure it does not mean hurting someone. It only means setting clear expectations and letting them know that you expect them to figure these things out. Also, at the same time teach them ways they could debug such problems.

Similarly there are other situations where you could react or show displeasure but as I said the bar should be higher that this situation does not happen often. 

Lesson #5: Learn from mistakes of others

It is common to hear that you should learn from your own failures. If you are observant you can limit your failures and learn from the failure of others. In other words, you can learn at the expense of others. This is very powerful and it can help you reach your goals faster. But, it requires you to observe other people closely and understand their traits and context under which they failed. Once you understand the people around you and know why they failed at something then you can avoid making the same mistake. It is common in organizations that multiple leaders try to bring similar changes. Understanding why a previous leader failed can help the current leader understand how they can succeed this time.

Lesson #6: You will not have all the answers. And, it is ok.

I put a lot of pressure on myself to provide answers to questions people ask. I was debugging bugs, thinking solutions to their problems, and doing a lot of research for others. 

The more I tried to solve the problems, the more problems I got from people. I was expecting people to learn and start owning their problems. But, people were looking for complete answers from me. 

I realized part of the problem is my approach to handling such issues. I wanted to be a good leader who has all the answers. We all know this is not true but when you are starting as a leader there is a tendency to be a hero. So, I changed my approach. I started giving people a Google document template which they have to fill before I can work with them to solve their problem. The document required them to clearly define the problem, their understanding of the problem, what all they have tried, and what has not worked. Three things happened: few people were able to solve their problem themselves; with few others, I worked with them being the owner; and the remaining I never heard back (I don’t think they ever worked on the document).

As a leader your job is not to find answers for the people but to help them find the answer.  Saying I don’t know and letting other people take ownership is a sign of a mature leader.

Lesson #7: You will build people and they will leave

In the last few years, I have hired many engineers with potential. I paired with them, mentored them, and gave them opportunities. But, they decide to leave. For most good engineers, often product companies are their dream job. This is sad but true.

There are days I feel that one of the jobs of a good IT service organization is to build engineers for product companies. 

I think there are good reasons why engineers want to join good product organizations:

• They themselves want to a start a company one day so product startups are a good learning environment for them
• There is less switch between projects. You can work on one team for years.
• You can build depth in one domain.
• They pay much more than IT service organizations.

In my professional career so far, I have mostly worked with IT service organizations. One reason I can give, in favor of an IT service organization, is that: 

You can work with the breadth of technologies and domains in a short time. I have built DevOps tools, built a low latency pricing engine, built multi-tenant SaaS apps, built a social platform, helped banks in their digital transformation (architecture, DevOps practices) journeys, and many other things.

Lesson #8: It is fine to code. But, don’t make promises

If you read on the web on whether an engineering manager or a  CTO should code, you will find that most of them advise against coding [5]. The two valid reasons to avoid doing code yourself are:

• You will have multiple competing priorities and coding will take a back seat often. You may not be able to finish a task on time and this could result in slowing down your team.
• The danger with giving in to ‘the love of coding’ is losing the business goals under the desire to build ‘amazing’ technology and reinforcement of confirmation bias.

In my first year as CTO, I have worked with our delivery teams and shipped code. This is what I have done for three critical projects where I was involved:

• Made it clear to the delivery team that I am not the owner of tasks. I will ship code if my time allows but they should not consider it in their estimates. This means each project should have a technical owner apart from me
• Always pair with a team member who can take the task forward in case I become unavailable
• Provide value by doing code and engineering design document reviews rather than writing code
• Made priority to unblock others in case they are stuck.

I enjoy coding. I continued to work on a side project after office hours to keep my coding skills warm.

Lesson #9: Only few customers really value quality

Most customers want their vendors to build:

1. a high quality product 

2. with agile project requirements 

3. within fixed cost and timeline

We all know it is difficult to build a product with all these three constraints in place.[6]

Out of all the constraints, quality is the first thing that goes out of the door when teams are under time and cost pressure. Quality costs time and money and it is one thing that is given less priority over features. 

Customers believe in the factory model of software delivery where number of people in the team is directly proportional to feature velocity. In my experience, the factory model of software delivery is flawed and you can’t build good products using this model.

Lesson #10: I like you but I don’t have to speak your language

This has happened multiple times during my first year as CTO that people expect you to speak their language if you happen to have a good working relationship.

My realization is that as you move up the ladder it gets more lonely. It is difficult to keep personal and working relationships in balance. When you like someone, it is difficult to question them without hurting them.

Lesson #11: Pick your battle wisely

I have realized that there are only a few problems that are worth fixing. You can’t fix all the problems with all your best intentions. I found this flowchart in the Camille Fournier post [7] that gives a good framework to decide which battle you should pick.

Lesson #12: Keep moving things

This is an important lesson that a leader should keep in mind. You can’t become a bottleneck in the system. There are two ways you can become one:

1. Trying to fix a broken system by halting it and fixing it in one go. This does not work. You will end up overwhelming yourself and make more enemies than friends. You have to do it in a piecemeal manner – one small step at a time keeping people aligned with your end goal. You don’t even have to tell your end goal. Keep making small tactical changes and keep large strategic goals in mind.

2. Not taking decisions quickly. If you are not used to taking decisions that impact others then it will be stressful and overwhelming when you are put in such a situation. Most people want to delegate decisions or not own them completely. As a leader, you have to understand that not all decisions are equal and not taking a decision is also a decision. I like Jeff Bezos, the founder of Amazon mental model on decision making. He says you should ask yourself whether a decision is reversible or irreversible.

Some decisions are consequential and irreversible or nearly irreversible – one-way doors – and these decisions must be made methodically, carefully, slowly, with great deliberation and consultation. 

If you walk through and don’t like what you see on the other side, you can’t get back to where you were before. We can call these Type 1 decisions. But most decisions aren’t like that – they are changeable, reversible – they’re two-way doors. 

If you’ve made a suboptimal Type 2 decision, you don’t have to live with the consequences for that long. You can reopen the door and go back through. Type 2 decisions can and should be made quickly by high judgment individuals or small groups.

As it turns out, most decisions that we have to take are reversible. So, we can take decisions faster and keep things moving.

Lesson #13: What you say matter to people and you will be held accountable

Don’t let people pressure you into decisions you don’t believe in. They’ll hold you responsible for them later, and they’ll be right. Decisions are your responsibility.

References

1. Steve Jobs on Failure – Link
2. Maker’s Schedule, Manager’s Schedule – Link
3. How To Decide Which Tasks To Delegate – Link
4. Delegation and Empowerment – Link
5. Back away from the keyboard — should CTOs code? – Link
6. Triple Constraints of Project Management – Link
7. OPP (Other People’s Problems) – Link

Thanks AD for reviewing the draft.

You can also support me and my work by following me on https://twitter.com/shekhargulati. Thank you