willfullyobtuse.com/one.org

• You're being willfully obtuse
• HTTPS @ Homelab
• Multi-room audio setup
• vi-editing everywhere
• AWS Cloudwatch Metric Filters
  • Structed and passively collected metrics via AWS CloudWatch
• Simple CSS frameworks
• Let people fail
• The problem with large companies
• Just what is it you do here?
  • Stated title
  • Actual responsibilities
    • Business goal prioritization
    • Writing stories and technical plans
    • Iteration planning
    • Ensuring timelines get met
    • Keeping production from breaking
    • Software design
  • Expectations (from others)
    • Discuss story details, expectations, changes, etc
    • Adjudicate technical disagreements
  • Expectations (on myself)
    • Maintaining technical quality
    • Primary interrupt
• Managing Expectations

You're being willfully obtuse

Here's what I'm thinking…

HTTPS @ Homelab

I run a lot of services at home.

This includes, but isn't limited to

• ArchiveBox
• VaultWarden
• Navidrome
• Plex
• LibrePhotos
• This blog

and a lot more.

Pretty much anything that's served up over HTTP is always nice if not necessary to have behind TLS.

LetsEncrypt long ago brought free certs to the masses and there are a lot of tools for automating that nowadays.

My preferred approach for getting all the unnecessary nonsense I self-host at home behind TLS is Caddy.

I have a super straight forward setup, generally:

• Run Caddy in a docker container
• Create a wildcard CNAME record in my DNS pointing at my home's (effectively) static IP
• Add an entry in my Caddyfile for each services I'm running at home on its own subdomain
• If it's a service then I add it with a reverse_proxy block
• If it's a static site (like this) then there's a block for
• If it's something I want only accessible on my home network then I put a block like

      @local_network {
        path *
        remote_ip 
      }

in the directive. And voila.

Then tell Caddy to reload the config and I'm done.

Multi-room audio setup

I've put my home audio solution together out of the following components.

• Snapcast
• MPD
• Librespot
• Shairport-sync
• A mini-PC in my closet running the above software
• Two Raspberry Pi 4s
• Four Raspberry Pi Zero Ws
• Some desktop speakers and some Bluetooth speakers (wired to the Pis)

Each of the Raspberry Pis is in a room or porch attached to a speaker.

Snapcast lets me take an audio source and synchronize it across multiple clients. Each of the Raspberry Pis are running a snapclient instance and play whatever the snapserver instance tells them to.

Snapcast is setup to send whichever of the streams (MPD, Spotify, Shairport-sync/AirPlay) is playing audio to each of the clients that are connected to it.

This lets me or anyone else on my WiFi network play directly on one or more of the speakers - each named for the room that they're in using either Spotify, AirPlay, picking from my own music collection or by pointing at a URL (like to a podcast episode).

This works out great and we've used it at home for the past year.

I'd like to get the podcast experience to a more seamless place but it's pretty OK right now using AirMusic on my phone to play audio to the speakers over AirPlay.

vi-editing everywhere

For my sake, I prefer to have Vim bindings in as many places as possible.

Most shells can be configured to use Vim bindings by putting set -o vi somewhere in your shell startup script.

If you're using ZSH then you'll probably want an additional binding to restore CTRL-R reverse history search.

bindkey '^R' history-incremental-search-backward

For CLI tools that use the readline library then you can configure its input mode using a .inputrc file in your $HOME directory.

This affects REPLs like ghci and tools like psql.

set editing-mode vi
$if mode=vi

set keymap vi-command
# these are for vi-command mode
Control-l: clear-screen

set keymap vi-insert
# these are for vi-insert mode
Control-l: clear-screen
$endif

AWS Cloudwatch Metric Filters

Structed and passively collected metrics via AWS CloudWatch

AWS is a vast and sprawling set of services. It can be hard to find the hidden gems like this one so I wanted to point this one out.

Structured metrics are very helpful to monitoring the health and function of an software system.

• Do you want to know how long a particular transaction typically takes?
• How fast your database queries are?
• How long external APIs take to respond?
• Fire an alert when a particular function on the site happens too many times? Or too few times?

…plus a million other things specific to whatever system you're working on.

There are a lot of great tools for doing this and one that you might not know about is AWS CloudWatch Metric Filters. If you're already on AWS then you should consider these because it requires only that your application logs to CloudWatch.

If you're on ECS then the awslogs log driver for Docker gets you that nearly for free. By "free" I mean that your application itself can have zero dependencies on AWS services and not require any AWS credentials or libraries to start pumping out metrics that you can visualize, alert on and record over time.

The AWS docs themselves offer the canonical reference for configuring these so I won't go into detail here.

However, the gist is that for a log filter you define the following properties

• A filter pattern for extracting a discrete metric value out of a log entry
• A metric name to store the value in
• An optional dimension for sub-classifying the value
• And finally a log group to extract the metric values from

After that you just run the application and as the logs roll in the metric values get pumped out. Then you can define alarms for alerting on them, graph them, define autoscaling rules from them and more.

To conclude - AWS is big and hairy. While there are benefits to staying platform agnostic, some AWS services don't require much or any coupling of your application code to take advantage of. Cloudwatch Metrics is one of those services and you can get a lot of value out of it with not much effort.

Simple CSS frameworks

I really like simple drop-in CSS resets like the one I use for this site.

At the time of writing, I'm using Pico but I also considered tacit

The idea is that they provide nice default styling of HTML elements out of the box without the need to reference any specific classes.

The idea works well for sites that are much more content than layout - like this one.

Using tacit is a matter of incluing this link tag in the page's HEAD element:

  <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@picocss/pico@2/css/pico.min.css">

Let people fail

Warning: This, like most things, will involve a fair bit of projection.

Effective and enjoyable collaboration with other people requires mutual trust.

I believe that for someone to feel trusted by another person then they need the space to fail.

I think this is obvious when considering what not having the space to fail looks like.

Not having the space to fail means your collaborator is doing one of two things:

1. Directing every action you take a.k.a. micromanaging
2. Coming behind you and redoing all of your work

Both of these are attempts by the other person to minimize risk (or simply cases where they're failing to manage their own anxieties).

These actions are counter productive to fostering trust and should be avoided unless failure is too costly.

I'm not saying all collaboration requires building trust. There are times when you simply can't afford failure or mistakes.

What I am saying is that people frequently misjudge the value in deliberately giving others the space to fail for the sake of fostering trust.

Building trust is important and we should do it deliberately.

The problem with large companies

Organizing people is a difficult problem which only gets more difficult as youmore people need to be organized.

The larger a company is the more of its internal structures, rules, policies, history, etc are devoted just to organizing people.

For me, realizing this was like the first time you hear a flourescent light buzzing in an otherwise quiet room.

Reasonable people can differ on this point, but for my own sake I'd much rather avoid all the people-organizing baggage that comes with large companies.

I don't have a hard-and-fast rule about the size of a place I want to work but the larger a place is then generally the more reason I need to want to be there.

Of course, this is all kind of theoretical at this point, as Flipstone is my forever home.

TODO Just what is it you do here?

I've never liked working at large companies. Mostly because I think they complicate things, but some things are more complicated at small companies.

Specifically, leadership roles tend to be fluid in their definition where they grow organically over time. Also, as I'm learning, old responsibilities don't tend to get pruned on their own. So you have to be diligent about shaping your role over time or you'll end up over-burdened and unable to do a good job at any of your responsibilities.

Here's an outline of my job;

Stated title

Director of Engineering (formerlly technical lead)

Actual responsibilities

Business goal prioritization

Providing technical input to and vetting of business goals.

This is basically a combination of saying

• "X will take (days|weeks|months|years)"
• "Y will be (easier|better|less risky) if we do it after X"
• "We can get 90% of the benefits of A if we do B - at half the cost - instead"
• etc…

Writing stories and technical plans

Many (but not all) of our "tickets" "cards" "stories" what-have-you end up getting written directly by me. This is, in some sense, an easy job to delegate out but it's risky to do so because getting this part wrong can lead to a lot of re-work amongst other costs.

Iteration planning

Deciding what the team will actually do in a given week.

Ensuring timelines get met

While we don't have a lot of "hard dates" on deliverables compared to large companies, we have them sometimes and it's my responsibility to either ensure we hit them or to understand why we didn't so we can better manage expectations in the future.

Keeping production from breaking

At the end of the day, if the system is on fire then I have to make sure it gets fixed.

Fortunately, our system is fairly resilient and we have a rotating "support" role that everyone gets a turn at - so I am not personally responding to every issue that comes up.

However, if we do have a big enough or hard enough problem then I need to be able to provide support. And that usually means the situation is urgent so I have to know the details of the system well enough to resolve issues quickly.

Software design

Expectations (from others)

Discuss story details, expectations, changes, etc

A user story is a promise for a conversation. Very often I am the one keeping that promise and this puts me in the middle of a lot of conversations.

TODO Adjudicate technical disagreements

Expectations (on myself)

TODO Maintaining technical quality

Primary interrupt

This responsiblity is a hold-over from my tech lead days and it's one I need to get rid of.

Being the primary interrupt means protecting the team from interruptions and allowing them to focus on executing the current plan of work (i.e. the stories in the iteration)

TODO Managing Expectations

I'll figure this out one day. Until then I'll just keep saying yes and burning myself out making everyone happy.