more info ⬇

@amattn

SW engineering, engineering management and the business of software

subscribe for more
stuff like this:

2020 10 26

When to Hire Your First DevOps, SRE, Security or Data Engineering Specialists in a Startup

Engineering Specialists Aren’t Pokemon

When you are managing a very small team (single digit number of engineers), you essentially want generalists. They don’t have to be full stack, but they have to be able to stretch into adjacent areas. Typically this means you are hiring backend folks who can do a passable job at infrastructure or front-end. Sometimes it means front-end or mobile folks who can moonlight as backend until the team grows. It’s very natural for a small eng team to be mostly generalists and large teams to have mostly specialists.

But when does this change happen? As an engineering manager advisor and consultant, I answer that specific question a lot. The high-level answer is: “When stuff breaks.”

Let’s go role by role:

Infrastructure, DevOps and SRE

For the sake of this post, these are fairly similar roles.

Typically you need this investment around the 10-15 engineers mark, because that is when infrastructure management starts breaking. Too early and you are investing in tooling you don’t need and will likely throw away. Too late, and you’ve built up a lot of cruft, fragility and unreliability into your systems.

Infrastructure fragility is partly the systems you use and don’t use as well as the people and process around testing, deployment, monitoring and debugging (prod or otherwise)

Do you not have any production users? Do each of your devs work on completely separate systems that don’t interact? Are you still figuring out product market fit? You may not need infrastructure engineering at all.

Does it take over two weeks to get a simple change into prod? Does stuff work in dev and test, but not prod? Do you have significant downtime or lots of alerts, false or otherwise? Do changes to one repo, system or component break others? You’ve probably waited too long to hire infrastructure engineering.

Security

This one depends. If you are in a highly regulated space (healthcare, finance, etc.) or are dealing with sensitive information of any kind (privacy, secrets, etc.) you should be investing early. Probably around team size five to ten engineers or so.

Are you mostly early with no or few users? Cat pics or other non-sensitive data? You can wait.

Investment in Security Engineering is a constant gamble. You should be constantly weighing the risk (chance of breech or data loss) against the consequences of those risk outcomes (blog post apology? sink the company?). Your risk profile is high if a small chance has potentially severe outcomes (such as sinking the company) or you have a large chance of incident due to tech, vertical (finance in particular), size of customer base, etc.

If your risk profile (roughly chance times outcome severity) is high, you should have already hired security engineering professionals.

Data Engineering

There’s no typical team size. This is more an issue of data and data processing workload. My rule of thumb here is to make sure to avoid premature optimization. Mostly companies don’t need fancy stream processing or Hadoop style cluster operations.

Basically, if your data and workload doesn’t fit on the beefiest RDS or other similar managed SQL database, not including best practices like read clones, caching layers, etc. you probably don’t need data or pipeline engineering specialists.

Things that make the cut? Petabytes of data. Hard real-time response requirements. Ultra complex, multi-join statements (sometimes). Millions of concurrent write events. stuff like that.

Things that don’t make the cut? Terabytes of data. Millions of users. (I promise. they won’t be concurrent for most all businesses.) Most Transactional Business logic.

I can’t count the number of startups that I have personally seen spin up Hadoop or Spark or Kafka workflows and then just throw them away because they never scaled, added too much complexity or the company pivoted far before they ever needed those systems.

“When stuff breaks.”

Again, the common theme is breakage or fragility that slows down engineering velocity.

You may be asking yourself, “Self, what if I don’t want stuff to break?”

Think of it as avoiding premature optimization. If the phrasing makes you or your boss uncomfortable, think of it as when stuff gets too fragile as to negatively affect velocity.

If you hire a specialists too early, you are potentially slowing your team down. they are putting tooling and process in place where it may be unnecessary and wasteful. Importantly, you are likely spending resources (money, headcount, meeting and training hours) in things that aren’t likely useful to an early stage company that hasn’t likely hit product-market fit. It’s far more likely your product requirements will change before your infrastructure investments pay off.

Similarly if you wait too long for any specific specialist, you are also slowing velocity. You will need CI/CD eventually. You will probably have a security incident of some kind at some point. Waiting too long means that your teams are struggling when a key hire could help alleviate some internal pain points.

Exceptions

There are always exceptions. Every combination of market, company, industry and team is unique. Is your company focused on mobile, data or security? Then by all means, hire those folks early.

Non-Engineering specialists?

Broadly speaking, the similar advice applies to non-engineers. When do you need a Finance manager or accountant? when the money bits get too complicated for the CEO or other co-founder who is probably doing it part time. When do you need your first marketer? When the marketing workload starts holding back key people from doing their jobs (typically co-founders or sales people) or you have evidence that the absence of competent marketing activity is negatively affecting growth.

Engineering Specialists aren’t Pokemon

Be thoughtful about when to hire. Don’t collect engineering specialists like they are Pokemon. Try to find the magical time where that key hire will add to rather than be a detriment to everyday engineering productivity. Ask your team or your peer network to think about both the pros and cons of a specialist hire. You probably won’t get it perfectly right, but you can improve your odds of not being detrimental.

2020 10 30

Plant Based Burger Review Update: Fall 2020

This is not a food blog. My one indulgence into the culinary milieu however is my ongoing search for great burger replacements. I originally wrote up a blurb on plant-based burgers a couple of years back.

For context, you can roughly devide plant burgers into two styles: Meat Imitation/Replacement and Veggie Patties. Back during my original blurb, there were really two viable chices in the meat-replacement category, Impossible Burger and Beyond Burger. Furthermore, in the grocer, only Beyond burger was available for making at home. Today, there are a lot more. All options below are available at grocers (at least in California, your region may differ).

Plant based burgers are still more expensive than the meat based originals. For example, Burger King sells the Impossible Whopper for seven and change, but the you can get two regular Whoppers for five. Hopefully this changes over time. It probably will remain this way for a while, given how the industry has spent so much on R&D on these products.

Fall 2020 Update: Plant-based Burger Rankings

  1. Impossible burger:
  2. Open Nature Plant Based Patties
  3. Beyond Burger
  4. Sweet Earth Awesome Burger
  5. Lightlife
  6. Morning Star’s Incogmeato
  7. Laura’s Plant Based burgers
  8. Trader Joe’s Protein Patties

Most of the remaining alternatives I’ve tried since the original 2018 post were either veggie patties or more like veggie patties than meat replacement.

Summing it all up

Impossible still rules the roost. Open Nature and Beyond represent a great second tier of choices.

There’s a large middle class of decent, but not great brands, including Lightlife, Incogmeato, and Laura’s.

If possible, stick to the top 3 or 4 on the list. There’s a bit of a drop off after that.

Still to try:



the fine print:
aboutarchivetwittertwitchconsulting or speaking inquiries
© matt nunogawa 2010 - 2020 / all rights reserved
back ⬆