The Blog

page 2 of 3
Career

Employment Update

I just started working for a consulting firm called Fidelity Solutions. In the past I’ve worked with some folks who are there now, so that made it easy to join. I’m really looking forward to working with them again on projects larger than what I could take on solo.

I’m grateful that the new job allows me a schedule to continue doing volunteer work. Time is a more precious commodity than we often realize.

I was trying out UpWork but it never really worked out, maybe because I put very little effort into it. Still, I’ve taken down the links to UpWork from the home page and support page, likely permanently. My time is now devoted to the new job and volunteer work.

Search

Pagefind

I have developed and written about my own static site search library for Astro. It was fun and instructive. I learned more about search, Astro’s inner workings, and practiced publishing an NPM package.

I wanted to provide something as easy to configure and use as the original VuePress static search. It should be possible to whip up a markdown-driven static site with a decent search without begging Algolia for help. The problems I faced with this solution were:

  • It was very Astro-specific, deeply tied to Astro’s rendering pipeline.
  • I never provided a plug-and-play UI component, the documentation was just “here’s an idea, good luck.”
  • It was going to be a ton of work to keep updated.

I found an integration with a new library called Pagefind through the Astro integrations library. In many ways it is technically superior. It indexes the static output HTML after the build, so works with any SSG, not just Astro. There’s a default UI that works Just Fine after a little CSS tweaking. You can easily customize how content is indexed. Unlike my library, it actually displays highlighted content excerpts. There’s a lot to like, and I appreciate CloudCannon’s work on this.

My main complaint is that it does not split documents by heading. I have some long-ish documents and the search only helps you know the query is somewhere in the page. However, this feature appears to be on the roadmap, so I’ll keep an eye out.

Will I continue to maintain my own astro-minisearch library? Time will tell. For now, I recommend you try out Pagefind and see if it works for you.

Astro

Astro v2.0

In my continuing Astro saga, here’s the latest chapter. This post marks the completion of this site’s Astro v2 upgrade. Rather than explain how to upgrade because that already exists, this is more of my thoughts and impressions on how the upgrade went.

Since that last post where I said “I should work on a plugin for Astro + MiniSearch”, I went ahead and did that. At the end of January 2023, Astro v2 launched. One of the headline features was Content collections, something similar to Nuxt Content. If you consider .md files to be data and .astro files to be presentation, then keeping them separated makes tons of sense.

Although I didn’t have to use the new content collections, part of this site’s raison d’être is to understand and extend Astro, so I went for it. If it wasn’t for my search plugin, upgrading probably would have been fairly smooth. I think the Astro team did a phenomenal job once again of documenting how to do it and keeping breaking changes to a minimum. In particular, the “Migrating from File-Based Routing” section was clutch.

For the most part, things Just Worked. Things I like about Astro v2:

  • Markdown and MDX plugin configuration is improved.
  • Using content collections is just cleaner than file-based routing – for example, you don’t have to specify layout in the frontmatter of every blog post anymore.
  • Upgrades to Vite 4, which increases my confidence that Astro will continue to stay modern.

Not so great:

  • I had a lot of trouble porting my search indexer endpoint. I got the cryptic error ReferenceError: Cannot access '_page9' before initialization at first, and the “generating static routes” step would just die silently. I finally figured out that all content fetching code had to exist inside of the get() function, not just at the top of the file like before.
  • Relative links and images in markdown files that were migrated to content collections were initially broken.
  • If you use schemas but you don’t define a particular property, when you try to output the undefined property in a component, Astro will hide it instead. That is the Zod library’s default behavior and can be overridden.
  • A weird error from an old workaround in astro-icon took a while to find and remove.
  • Content collections schemas get very odd if you color outside the lines.

More detail about that last one. It’s cool to define collection schemas, but they are sort of generated at build time I guess? The documentation is still a little sparse, or maybe my use case is extreme. For this site, I have two content collections, blog and articles that I want to index for search. Both collections have a mandatory title attribute and an optional description. The documentation describes CollectionEntry class but it’s not clear how to use it.

In the end I sort of figured it out, but it’s not ideal. I used CollectionEntry<any> in my search indexer and defined a base search item schema with Zod. I used Zod’s extend method to “inherit” from a common set of searchable document fields. Instead, I wish there was a ContentEntry<"search"> type without requiring an empty “search” collection. Since I’m ignoring the type anyway there’s little purpose to the fancy schema. That may be a microcosm of TypeScript’s overall failures though.

Ideas

Fair Tickets Follow-up

My post about selling tickets fairly generated a lively discussion at Hacker News with over 330 comments. As of this moment my analytics shows 10.9k unique visitors from that page, the majority (6.8k) from HN. The HN crowd is generally quite thoughtful and experienced, so there is lots of good stuff there.

The top comment was from user buro9, a technologist with inside music industry experience. I’m grateful for the time he spent adding his valuable perspective to the discussion. I especially appreciated his link to red ocean vs. blue ocean strategy, it’s worth a read. At the risk of paraphrasing poorly, I understand the primary objection to be that a descending price auction is in nobody’s interest except for wealthy fans. Promoters and venues prefer sales to be finalized as early as possible. Meanwhile, artists want all the seats filled with their most ardent (but possibly poorer) fans. He says a Dutch auction is antithetical to Taylor Swift’s core brand and would be “flat-out” rejected. Would it, though?

As far as the needs of promoters and venues, I’m tempted to shrug who cares about them. Prioritizing The System over The Fans has led to the present rotten mess. But, there’s no escaping that promoters and venues are stakeholders. Would a Dutch auction be materially worse for promoters and venues somehow? I wasn’t convinced by any reasons put forth in that thread. The sale might take a different shape than what they’re used to, but that doesn’t make it worse. Regardless of a theoretical revenue difference, detailed pricing information from the auction would be invaluable. Unless scalpers are somehow sharing their sales numbers, that data is currently unavailable. Information that unlocks future revenue is likely more valuable than one-time revenue.

But let us fantasize briefly about an alternate reality where a superstar artist could dislodge Ticketmaster’s chokehold (all historical evidence to the contrary). I believe a Dutch auction would be a big upgrade in ticket purchase fan experience, and that’s something any artist should be interested in. In a world of rapid inflation, the descending price aspect would be a fan-friendly move artists could brag about. Superfans could plausibly score dirt-cheap tickets in the nosebleed seats at the last minute. How is that possible today?

Dutch auction or not, the final ticket price should be all-inclusive. Not charging extra fees and taxes is crucial for a pleasant checkout experience, and the dynamic pricing of an auction can more easily enable that. The current system’s bait-and-switch dark UX patterns, obnoxious fees, and unpredictability leave them feeling anxious and ripped off. Are artists 100% sure that their fans’ animus is only directed at Ticketmaster/LiveNation? Anyway, a descending price auction is but one tool for the difficult and interesting problem of improving ticket sales.

There were objections to my using the word “fair” to describe the Dutch auction system. The idea that the people with all the money always seem to hoard the nice things seems deeply unfair. Agreed. I don’t know how an equitable or charitable ticket sale would work, but that wasn’t my focus. I meant “fair” in the sense that as a consumer, I expect to be able to make a purchase with minimal shenanigans. I don’t want to feel like I’m trying to win a game that’s rigged against me. I will continue to try to improve my word choice.

There was a Twitter thread by Ian Hogarth (co-founder of Songkick, sad story): “A few thoughts on Ticketmaster and @taylorswift13”. That spurred another interesting HN discussion related to this topic.

Ideas

How To Sell Tickets Fairly

Update: there’s a lively discussion at Hacker News and a follow-up post.

Earlier this week, millions of people tried to purchase Taylor Swift concert tickets at the same time. It was a complete fiasco. Ticketmaster crumbled under the high demand and everyone was angry.

Of course Ticketmaster bears primary responsibility, not just for this incident, but also for monopolizing a broken marketplace they are unmotivated and unlikely to fix.1 One big problem is in the structure of the sale itself: all tickets go on sale simultaneously.2 This creates the conditions for digital line cutting and stampedes, which then fuel a predatory secondary market. You could hardly create a more chaotic and user-hostile system if you tried.

On the other hand, selling tickets with a slow descending-price (or “Dutch”) auction would be much better for everyone except ticket scalpers. This was apparently how tulips used to be sold. Maybe this hasn’t caught on yet because auctioning concert tickets was protected by a patent that just expired in 2020.

Rob Croes for Anefo, CC0, via Wikimedia Commons

Here is how I imagine the live concert ticket auction would work:

  1. Tickets go on sale for a high price per seat (say $2,000) six months before the concert date. The bare minimum ticket price is set at $20 (Kurt Cobain would have liked that). Every concert date and venue would be a separate auction.
  2. If tickets go from $2,000 to $20 linearly over 180 days, then every day the asking price for a ticket would go down by $11. Each hour ticket prices would drop by $0.46 ($1,980 / 4,320 hours); each minute, nearly a penny. The ticket sales page would show the current ticket price and the number of tickets left. What you pay for a ticket depends on when you buy it.
  3. 14 days before the concert the ticket price would be $174 (all fees and taxes included3). Someone buying 4 tickets on that day would first pre-authorize their credit card for $696. If approved, they could choose some available seats and make the purchase.

Two buyers might try for the same exact seats at the same moment, but that is much less likely in a slow single-bid auction like this. True, the deepest pockets would win the best seats, that is no different than now. Concerts in affluent areas might sell out earlier, but people living in areas with less might pay less too. The fair price for each seat in each venue would be determined naturally by the local market, an economist’s dream scenario. Scalping tickets or running bot farms to snatch tickets early would be risky and unprofitable.

Rather than a mad crush of millions on day one, the system would handle a reasonable volume of transactions per hour. Lower peaks would translate into lower costs. The platform could embrace automation and offer an open API rather than fighting an anti-cheating arms race. Fans would be happier for not going through “several bear attacks” for their tickets, and artists would keep more of what belongs to them. I’m not naïve though; the most serious barriers to this becoming a reality are probably not technical.

Shameless plug: I understand how to make and launch exactly this kind of system, at least technically. If you want it built, please contact me.

Footnotes

  1. Ticketmaster would rather that you blame “evil bots”, hilariously. In their help article “Why am I getting a ‘blocked’, forbidden’, or ‘403 error’ message” it says:

    Sorry, you’ve been temporarily blocked from buying tickets because you may have refreshed your browser too frequently. When that happens our system thinks it’s a bot, an evil automated program trying to scoop up tickets. And we automatically block bots.

  2. For popular events, Ticketmaster uses something they call the Smart Queue. Even in this recent case, there was an attempt to spread out the tsunami wave with a “Verified Fan Presale”. These measures were, shall we say, inadequate.

  3. People hate fees and it makes them feel ripped off. It also makes pre-authorization work as expected. Since the price is absolutely flexible, then you could calculate the pre-tax “price” after the fact. This is all paid for by money that would have otherwise gone to scalpers and the “evil bots”.

Docker

Node.js + Docker Development

A page about how to Dockerize a Node.js application exists on the Node.js site. There is a corresponding page about using containers for development on Docker’s site. Those guides are fine in theory. In practice, it can be a little tricky to use a dev server like Vite or Astro on your local machine with Docker Desktop. For example, you may find that HMR appears to work when you save a file, and yet the browser window doesn’t refresh automatically.

Partly to record how to do it for myself, I created a demo repo on GitHub for Astro + Docker. Here’s what’s most important. The Dockerfile looks like this:

FROM node:18-alpine
WORKDIR /app
COPY package*.json .
RUN npm ci --audit=false --fund=false
CMD ["npm", "start"]

…and the docker-compose.yml file looks like this:

services:
  web:
    build: .
    ports:
      - "3000:3000"
    volumes:
      - "./:/app"
      - "/app/.tscache"
      - "/app/dist"
      - "/app/node_modules"

Notice that the usual COPY statement to copy the source files appears to be missing. That is intentional. When docker compose runs, our files are mounted as a volume from the local file system and shared with Docker. Thus the files we change in the editor are the same as what the Node.js process running inside the container is monitoring, not a copy. Voilà.

Even so, I prefer to shut Docker down and run my code directly Node.js on my local machine whenever possible.

Astro

Astro Update

Since I first mentioned this in April, I’m continuing to use Astro. It has experienced some churn between then and now, and I guess that was the price of trying out pre-beta software. I think it was worth it, though. Here’s my updated astro.config.mjs file for v1.1:

import { defineConfig } from "astro/config";
import vue from "@astrojs/vue";
import tailwind from "@astrojs/tailwind";
import mdx from "@astrojs/mdx";
import remarkGemoji from "remark-gemoji";

export default defineConfig({
  site: "https://barnabas.me",
  build: {
    format: "directory",
  },
  integrations: [vue(), tailwind(), mdx()],
  markdown: {
    extendDefaultPlugins: true,
    remarkPlugins: [remarkGemoji],
  },
});

The font bug that affected me (requiring a weird workaround) was fixed. Compared to the original config, the markdown configuration has improved. Most notably, Astro has embraced MDX wholeheartedly, which meant a bit of migration for my sites. Honestly NBD. They continue to iterate quickly and are on v1.1.5 as I write this.

VuePress and VitePress still have the edge in out-of-the-box documentation experience, particularly for having a simple built-in client-only search function. I suppose I should work on a plugin for Astro + MiniSearch or Lunr.js or something instead of complaining about it.

Web Platform

Web Platform 2022

I have launched a few websites over the years and have tried many different companies and services. There are many services to set up to have a full online presence. If this isn’t your day job to deal with, it can be a little overwhelming.

Consider my list below an essential checklist of establishing a low-cost online presence in 2022. None of these are affiliate links, but if you find this list useful, you can thank me in other ways.

Domain: Cloudflare Registrar

Cloudflare sells domain names at cost, so it literally can not be any cheaper. I always liked to put my sites behind their free DDoS protection anyway, so registering the domain itself with Cloudflare skips that step.

Runner-up: Hover.

Hosting: Render

If you’re deploying a static site or a web application, Render is straightforward and powerful. It feels like the heir apparent to Heroku. The free plan goes a long way. Runners up:

  • Fly.io. I love what they’re doing, awesome free tier.
  • Cloudflare Pages. Powerful but gets complicated if not 100% static.

Email: Fastmail

What good is a website without a way to contact you? Fastmail is an excellent privacy-friendly value at $5/month, with additional users for $3, 30-day trial. Runners up:

Analytics: Plausible Analytics

Simple, lightweight, as privacy focused as analytics can be. $9/month, 30-day trial.

Code: GitLab

The free version of GitLab comes with 5GB of storage and 400 CI/CD minutes per month, more than enough to build the average static site several times a day. You can host your site directly with GitLab pages, but there are also integrations with Render, CloudFlare Pages, Vercel, Netlify, and others.

Runner-up: GitHub.

In all, this could be $15 out of pocket to launch and $15/month for email and analytics.

Meta

Asking for Support

While putting together DBaaS Review I’ve been wondering how to sustain it. At some point the bills must be paid. I began evaluating some monetization options.

I considered and rejected display ads. There are privacy concerns, and my target users are likely to be banner-blind and/or blocker-savvy. That said, I have enabled privacy-friendly Plausible Analytics on both that site and this one here. One can’t understand or improve on what one doesn’t measure.

Inspired by content producers like Jason Kottke and John Gruber, I’ve decided to try direct sponsorship. Jason uses a service called Memberful, but I’m going to start small and see how it goes.

I’d like to give people the opportunity to sponsor or work with me personally too. That’s why I added the “Support” button in the footer. Here are some income/monetization options I researched:

DBaaS Review

TL;DR: go check out DBaaS Review and let me know what you think.

A couple of years ago I started tracking DBaaS prices in a spreadsheet. Besides AWS RDS, it just seemed like there were more and more good options being announced each month. Finally in 2019 I cleaned up my data and turned it into an interactive calculator on this website.

There were a few limitations with the simplistic approach, though. I was originally interested in the sub-$50/month market segment, but many interesting offerings are well above that. When you add more dimensions (like regions, replicas, and commitment) the number of potential options balloons. It was no longer feasible to keep the data in a static single page application.

I bought dbaasreview.com and deployed new static site to Cloudflare pages, later to Render. I investigated using Cloudflare workers for the API, but for now deploying a separate API from a separate repository was more straightforward. I’m using Render’s free tier, so the main drawback is that the API shuts off after a few minutes of inactivity.

The vast majority of the work is importing all of the different product plans into a single unified database that powers the API behind the calculator page. I spent two days puzzling over Azure’s pricing API, I’m not looking forward to doing the same for AWS and GCP. Since the data is pretty small (< 5MB) and read-only, the API uses a SQLite database deployed alongside the application, ironically. If and when the project outgrows this, I will scale up API replicas.

Anyway, it’s been fun and interesting to build and maybe will turn into a consulting gig or something, who knows.