SW engineering, engineering management and the business of software |
|
This is a set of bullet point lists about variously things I’ve done, experienced or come across in 2020. Items listed in no particular order.
Inspired by Fogus.
Past editions: 2019 Year in Review
I like very much all of the above technologies. Elixir in particular, I’m very happy with. It is relatively new to me though, so I may be in a honeymoon phase.
My most popular content of 2020 According to Analytics:
Some of those are ancient. There are 2011, 2012 and 2013 era technical tutorials still getting SEO hits for some reason.
Apparently, my engineering management or recruiting content doesn’t rate.
It doesn’t get any SEO juice, but I’m proud of the management content I write.
Also I’m quite proud of what I stream at https://twitch.tv/amattn It’s mostly Elixir content these days, but hoping to do more around engineering management and recruiting.
So I got sick the second week of March. I don’t know if it was Covid-19 or just a flu because there was no access to testing. The whole family got it, but recovered in about 4-5 days.
Mental health much better
During shelter in place orders, I tried to go for 30-60min walks (keeping the 6ft distance) with the family. Did this most days.
The habit tracking was particularly useful. I also used to to build of five days/week for six week meditation streak. Next year, I’m planning to go longer.
Roughly a 5% increase. This isn’t really indicative of learning however as I’ve been using the Notes.app built-in to MacOS and iOS far more lately. as of today, 907 note entries. around 300 of them were created or edited in 2020.
The best stuff I come across I will typically post on twitter.
Also there was a quote by James Clear that I can’t find right now. Paraphrasing:
To learn rapidly, learn from others. To learn deeply, do it yourself.
Overall number of books read in 2020: ??? across paper, kindle, iBooks.
Disney+ is killing it with content suited to my tastes. The family prefers Netflix, but I’ll likely be a D+ subscriber for a long time.
One of those little round robo-vacuums.
Inspired by this reddit thread, I think this could be a great question to ask to learn something quickly:
What feature of X is used very often by experienced programmers/users/etc, but not so much by a newbie?
Obviously, not a big year for travel.
Obviously, not a big year for conferences.
The 2020 Todos were originally published in my 2019 Year in Review
Done:
Partial Credit:
I love many things about go. One of my favorite things is that cross-compilation is an absolute breeze. It takes just seconds and it’s especially easy with a tool like gox. After that, deployment is even easier. Just rsync or scp your executable binary and maybe a secret file or two. Boom💥 done. It’s essentially the easiest possible deployment story of any langauge or framework I know of.
So as a long time go developer (nearly a decade!) every other ecosystem pales in comparison.
Elixir is among them. Elixir itself is nearly a decade old, so there has been a lot of community work to make this better over time. As someone relatively new to this lovely language, I’m super grateful for all the hard work. Deploying basic Phoenix apps isn’t golang-easy, but’s it’s not painful.
This post details how I deploy a phoenix webapp, developed on a Mac to a single linux cloud instance.
First off, as you read about deploying elixir apps, you’ll see two primary release tools.
The first is an older tool that predates mix release. As I understand it, Distillery was essentially a community tool created to help with production deployments. mix release is realtively new, and is bundled with Elixir 1.9. Most of the new development and tooling is built around mix release. At least one long time Elixir dev I talked with seemed to think that mix release is the future.
I didn’t spend much time with Distillery, but mix release is a nice little tool. It essentially is able to package everything you need to run an Elixir app into a single directory. the layout looks like this:
/my_app-1.2.3
/bin
my_app
/erts-1.11.5
/bin
/lib
/releases
/1.2.3
The /my_app-1.2.3/bin/my_app is a simple shell script that you can use to start, stop, check version, launch in daemon mode, or even eval or call rpc expressions. /erts-x.y.z is the erlang VM. /lib is your app and all it’s dependencies. releases is a bunch of misc helper files, scripts, etc.
These release directories are essentially self-contained and can simply be scp’d or rsync’d to a remote cloud instance. rsync is preferred as most of it doesn’t change between releases. I typically see 100x or more speedups using rsync.
For this post, I added the following definitions to mix.exs’ project definition:
def project do
[
app: :my_app_,
version: "0.3.2-a.2",
<snip>
default_release: :my_app,
releases: [
my_app: [
include_executables_for: [:unix],
include_erts: true,
strip_beams: true,
quiet: false,
steps: [:assemble, :tar]
]
]
]
end
Assuming you are deploying a webapp, you will definitely want to start with the Phoenix release documentation:
https://hexdocs.pm/phoenix/releases.html
You are going to want to exit any files the above link tells you to edit. In particular, the stuff about prod.secret.exs and release.ex. The actuall steps are automated with scripts I’m detailing below.
For release.ex I had to add a line to make it work. Specifically, I needed to start :ssl in the load_app function:
defp load_app do
Application.load(@app)
Application.ensure_all_started(:ssl)
end
Additionally, the mix release hexdoc page has more details that help explain what is going on under the hood:
https://hexdocs.pm/mix/Mix.Tasks.Release.html
Definitely read the first, and I recommend reading both.
With Elixir, cross-compiling is theoretically possible, but it’s not tested and not supported in any way.
Update: I’ve since learned that cross-compilation may not be as dangerous as I first assumed. See here: https://github.com/LostKobrakai/criss_cross
For the common case of building on a Mac or Windows machine and deploying to a linux cloud instance, you need a way to build on linux. Your basic strategies come down to:
For this post, I’ll walk thru the first.
For the second, just do a git checkout or rsync your project to the cloud instance and mix compile && mix release. You’ll have to make sure that elixir and any dependencies you need are installed on the cloud instance.
For the third, see the Phoenix documentation here.
With CI tools, reference your CI tool of choice to see if existing documentation exists or try to implement one of the above strategies via the CI tool.
First credit and thanks to Kai Wern Choong and their post about deploying Phoenix. It was super helpful to as I was learning about how to deploy with docker.
Here is the script I use to build a Phoenix app inside of a linux docker container, then spit out a tarball:
#!/bin/sh
set -o nounset
set -o errexit
#######
# build project in a dockerfile then output a tarball to sync with external server
# primarily using this post as a resource:
# https://kaiwern.com/posts/2020/06/20/building-elixir/phoenix-release-with-docker/#build-image
export DATABASE_URL=`cat postgres.secret`
export SECRET_KEY_BASE=`mix phx.gen.secret`
docker build --build-arg DATABASE_URL --build-arg SECRET_KEY_BASE -t my_app_server .
APP_NAME="$(grep 'app:' mix.exs | sed -e 's/\[//g' -e 's/ //g' -e 's/app://' -e 's/[:,]//g')"
APP_VSN="$(grep 'version:' mix.exs | cut -d '"' -f2)"
BASE_NAME=${APP_NAME}-${APP_VSN}
TAR_FILENAME=${BASE_NAME}.tar.gz
id=$(docker create ${APP_NAME}_server)
docker cp $id:/app/${TAR_FILENAME} .
docker rm $id
mkdir -p _staging/${BASE_NAME}
mv ${TAR_FILENAME} _staging/${BASE_NAME}/.
cd _staging/${BASE_NAME}
tar -xf ${TAR_FILENAME}
rm ${TAR_FILENAME}
It will simply create a directory called _release and put any release it builds in there.
The SECRET_KEY_BASE env variable is used for things like session tokens. In this script we regenerate everytime, but that will essentially log out all your users every release. SECRET_KEY_BASE should be persisted accross releases if you don’t want that behavior.
Here’s the Dockerfile I use: https://gist.github.com/amattn/037e3ef59c02140efe60b8aa1c00b687
Walking thru the Dockerfile we are essentially doing the following:
Next I have a simple deploy.sh script that deploys a version to an environment (dev|test|staging|prod). It’s not the most elegant script. But it’s my script and I love it:
https://gist.github.com/amattn/ce12a89ceebe402e464ef3c43147f930
This script does assume you have ssh keys or whatever setup.
Here’s what is going on in there:
release.ex like the docs say you should.Lastly, a fairly typical systemd service file (that is rsync’d by the deploy.sh script):
[Unit]
Description=myapp
After=network.target
[Service]
ExecStart=/home/USERNAME/my_app/bin/my_app start
ExecStop=/home/USERNAME/my_app/bin/my_app stop
ExecReload=/home/USERNAME/my_app/bin/my_app restart
WorkingDirectory=/home/USERNAME/my_app
StandardOutput=inherit
StandardError=inherit
Restart=always
User=USERNAME
[Install]
WantedBy=multi-user.target
I use caddy in front of the Phoenix app to get https working.
A v2 Caddyfile directive will look like this:
# myapp.com reverse proxy
myapp.com {
reverse_proxy localhost:4000
encode zstd gzip
}
Caddy is the easiest way to get a Let’s Encrypt cert. It happens automatically. It’s basically a miracle compared to the hoops we used to jump thru before Let’s Encrypt was a thing.
My time to automate deployment for my first Phoenix app was about a day and half.
The final steps would look like this:
./build_docker_release.sh./deploy.sh 0.3.2 prodA good chunk of it was just understanding what was going and debugging the Dockerfile (esp those ENV variables). My Docker-fu is a bit rusty these days.
Conceptually, nothing really hard is going on here. Using the above scripts and files as a template, that time to deploy should be much much shorter.
All the hard work was done by the Elixir community members who worked on the mix release functionality. Their work makes it so that I can deploy to a cloud instance without having to worry about what version of what package is installed on that cloud instance. I’m too spoiled to go back to the days where your release artifact has system dependencies. That way lies madness.
All in all, Elixir’s deployment story, like Elixir itself, has a bit of a learning curve, but ends up rather pleasant after you figure it out.
Editor’s note: This is part one of a short series. You can find part two here.
I’ve spent the last month and change working on a little side project.
The interesting thing about this one is that it’s written completely in Elixir. This is a post about my experience doing so.
Elixir is a language that came out of the heyday of Ruby on Rails about a decade ago. It was originally billed as “Ruby on the Erlang VM”, trying to blend the best of both worlds. Spoiler, I believe it has largely succeeded.
It’s still got quite a bit of Ruby influence, but overall, it’s much more functional. You will see maps, filters and reduce’s all over your code base.
From the Erlang side of the family, you’ll see a lot. There’s no mutability. Variables cannot be changed, you can only create new ones from combining or transforming existing. Pattern Matching is everywhere and it’s still quite powerful. You can trivially call down to Erlang primitives whenever you need to.
The Ruby influence is more in style and syntactic sugar. You’ll see your def’s and end’s and ->’s. You can skip parenthesis around function arguments if it doesn’t introduce ambiguity. There’s no early return, you always return the last expression. Map and list syntax will be fairly familiar. The @ prefix denotes attributes / decorators. Atons.
It’s got some quirks. The native list primitive is a classic linked list, not an array. This means that prepend is cheap, but count, insert, append is expensive. It also means you can do head/tail car/cdr operations quite easily. Keyword lists as a weird list/map hybrid.
The |> operation is a particularly good example. To most newcomers, it’s a weird, inscrutable symbol whose operation isn’t easily intuited.1 Yet once you get used to it, there’s quite a bit of elegance to it, and even historical reference to the pipe operation in the shell.
For those new to the language it works like this: take the input of the pipe operation (x below) and pass it as the first argument of the target function of the pipe operator.
x
|> function(y, z)
is equivalent to
function(x, y, z)
The elegance comes from chaining these together like so;
x
|> function(y, z)
|> other_function(a, b)
|> yet_another_function(c)
If this is the last expression in a function you return the results of the final yet_another_function
You can also bind it to a variable like so:
final_output =
x
|> function(y, z)
|> other_function(a, b)
|> yet_another_function(c)
The pipe metaphor breaks down as you start with x, do a ton of transformation on it and your eyes have to scan way back to the beginning to see what variable you are assigning the final results to.
It kind of micro-example of the entire macro-experience of using Elixir:
To see an actual code snippet with pipes in action:
collected_errors =
Date.range(start_date, yesterday)
|> Enum.to_list()
|> Enum.filter(fn _ ->
random_number = :rand.uniform(100)
random_number < 75
end)
|> Enum.map(fn date ->
make_map_of_attrs(date, user, some_bool, some_int)
end)
|> Enum.reduce([], fn entity_attr, all_errors ->
case create_entity_with_attr(entity_attr) do
{:ok, _entity} ->
all_errors
{:error, err_msg} ->
[err_msg | all_errors]
end
end)
If the above looks like moon runes, it’s probably just because the syntax is new to your eyes. After a week or so with the language, it reads very clearly. Walking thru that snippet, we start with the second line:
create_ prefix means you are creating and object and often storing it in some persistence layer.all_errors accumulator during the reduce operation.collected_errors variable.I’m very, very fond of go, but a typical implementation of equivalent functionality would typically be at least 5x longer and certainly more irritating to write.
So, back to that macro experience of using Elixir:
That last bullet point, in my relatively short experience with Elixir, I’m fairly certain the overall developer productivity is quite high. Likely in the top quartile or better of all development environments (possibly even top 10%). There are others who think so as well.
The combination of the last two bullets is what I refer to as sublime efficiency in the title of this post. This is a rare combination. I believe go to be a great language for “IDE to prod” developer productivity, but I don’t believe it is a particularly elegant language. Lisp and it’s variants can be quite elegant but you will run into feature velocity problems at some point (deployment, ecosystem, hiring…).
Developing in a modern functional language, with great safeguards (pattern matching, guards, type annotations, strong testing libraries, etc.) somehow allows developers to be efficient, write elegant code and also achieve high feature velocity.
It’s clearly not a perfect language or ecosystem. The deployment story is not as good as go.2
But it is one that I could see a lot of companies using as a competitive advantage with respect to time to market of getting product out.
All you have to do is get over the learning curve.
Authors’s Note:
This post is the first of a two part series. You can find the second part here.
Lastly, I do live code streaming about Elixir, Phoenix, and LiveView on twitch.tv. You should follow me there.
Footnotes:
This is part two of a short series titled “The Sublime Developer Efficiency of Elixir, Phoenix and LiveView”. You can find part one here.
To recap the initial thoughts on Elixir:
The above is a succinct summary of my experience using Elixir. My overall impressions of the language, tooling and broader ecosystem are quite positive.
The Phoenix web framework is (as far as I can tell) the crown jewel of the aforementioned broader Elixir ecosytem.
It’s an opinionated web framework, with a bit of magic and convention in it. Just like Elixir was strongly influenced by Ruby, Phoenix’s ancestral roots point to Ruby on Rails.
It’s a great framework. Like Elixir itself, it is quirky & has a learning curve & elegant & you can be quite productive with it. If you are coming from an environment where everything is explicit, the magic and convention steepens the learning curve a bit, but that’s part of the productivity tradeoff.
It is based on the classic MVC paradigm. Elixir’s template language, Eex, essentially compiles down elixir code. It’s fast enough. You can scale vertically, horizontally with a load balancer or with any fancy cluster management solution. The Ecto ORM and Postgres integration quite well done. Ecto itself is, you guessed it quirky, learning curve, etc.
The thing about Phoenix however, is that it’s just an opinionated MVC web framework. Plain old Phoenix isn’t enough for me to radically adjust how I think about creating new web apps or change how I staff up engineering teams.
Ruby on Rails was a tremendous productivity booster during a good chunk of it’s early life. You could make reasonable and often convincing business cases for developing Rails. However, it is 2021; many other web app ecosystems have caught up or surpassed its vermillion tracks.
LiveView is a relatively new extension to Phoenix. The first release, 0.1.0, was published in August of 2019 and I believe work started in 2018.
Spoiler: LiveView is more than enough for me to consider transitioning a large amount of work to the Elixir/Phoenix platform.
The way it works is tremendously clever. There’s some liveview javascript you include in your html. Then you defined your live view templates like you always do, but with a special .leex suffix (LiveView Eex).
Your typical template will look like this
<html>
<head>
<meta charset="utf-8"/>
<%= live_title_tag assigns[:page_title]%>
<head>
<body>
<h1>
User Details
</h1>
<ul>
<li>
username:<%= @user.name %>
</li>
<li>
email:<%= @user.email %>
</li>
</ul>
</body>
</html>
What Liveview does (conceptually) is takes all those embedded elixir tags, and creates a map like so:
{
0: "User Details Page",
1: "Jane Doe",
2: "jane.doe@example.com"
}
When the page is sent to the browser for the first time, those elixir tags are filled in server side. you will see the right values even if javascript is turned off.
The magic happens next. LiveView sets up a websocket to the server. The client can send events like phx-click="checkbox_toggle", phx-capture-click="close", phx_change: "validate", phx_submit: "save", etc.
The server can then do any backend processing, such as change a users email, save it to a database and email a notification. Maybe calculate a new estimate of shipping time for updated contents of a shopping cart. Anything a server needs to do really.
Then, instead of a 302 redirect to reload the entire page to refresh the content (email, cart, whatever), LiveView will send just the data over the websocket like so:
{
0: "User Details Page",
1: "Jane Doe",
2: "janes_work_email@example.com"
}
There’s a bit of javascript logic that rerenders just that part of the DOM with new values. in this case, the <li> with the updated email.
What you get is the speed, responsiveness and UX performance exceeding that of a React/Vue/javascript front end framework against a server API, but you haven’t written a line of javascript.
This is a magical feeling.
Another similar-in-spirit technology is Hotwire. Hotwire however, sends little bits of HTML over the socket instead of tiny chunks of data, but conceptually, they are quite similar.
A lot of productivity gains of the Elixir/Phoenix/LiveView stack1 is based on the fact that you can build modern browser experiences without having to write any javascript whatsover.
Phoenix does have a webpack dependency, but it comes preconfigured and the javascript glue code that makes LiveView work is all written for you and included with the framework.
LiveView’s primary advantages:
I can’t stress how important the first point above is. The way to faster development cadence is rarely “add more developers”. Usually, it’s make them write less code to do good work.
CI/CD makes developers faster, because more of the testing and deployment is automated, drastically reducing human error, reducing code review time, human time on keyboard per deploy, and importantly reducing risk.
Good Project Management improves feature velocity because developers are typically better coordinated with design, product, marketing, etc. They end up writing the more important bits first.
Good Customer Development or User research improves developer productivity because they build features that people use instead of stuff that doesn’t move the needle on business goals.
LiveView makes it so you don’t have to write javascript.2 A poor metaphor: This is a like saying a car repair shop doesn’t need to paint or do body work anymore. All there customers are totally satisfied with having a running enginer.
Obviously, you still have to do HTML and CSS. LiveView is amazing, but it’s not a miracle. All logic is written in elixir and executed on the server. There is no javascript logic required whatsoever.
If you do need some fancy Javascript functionality, there are hooks built-in to the framework that allow arbitrary javascript to send events to LiveView and vice versa.
LiveView, like the rest of the broader ecosystem, continued the streak: Quirky, Learning Curve, Elegant, High Feature Velocity.
Again, Elixir and Phoenix are quite good, but the productivity gains of LiveView are great enough for me to strongly favor it for all new projects and seriously consider rewriting certain existing projects in that stack. The business case for using LiveView is that strong. It changes the total headcount and makeup engineers I need for a given web app team.3
So that title up at the top? that part-cheesy, part-aggrandizing line about Sublime Developer Efficiency? That’s LiveView. After you get past the quirks and the curve, it’s pleasant to work with and there’s a strong business case to be made that you or your team are going to be really productive with it.
Authors’s Note:
This post is the second of a two part series. You can find the first part here.
Lastly, I do live code streaming about Elixir, Phoenix, and LiveView on twitch.tv. You should follow me there.
Footnotes:
node_modules, dependency hell (leftpad), reliance on node and npm with almost no alternatives, etc.↩