Adam Leventhal's blog

Close this search box.

I started the year thinking that we had done some neat work in the Rust / API / OpenAPI ecosystem and I wanted to talk about it. I found my people at the Nordic APIs Austin Summit; it’s a conference that had been on my radar for a while, but it was my first time attending. I was pleasantly surprised by how interesting the sessions and conversations were… and was pleased to share some stuff we’ve been up to.

The Oxide API-a-matic universe

I presented (slides) our experience with Rust as API server and client, generating an SDK, and generally how terrific all that has been. It’s worked out better—if differently—than I expected. Joining Oxide, I had some experience with OpenAPI. I thought we’d be able to avoid a bunch of work, relying instead on the strength of the ecosystem. Faith in the ecosystem turned out to be misplaced. However, the investment we made in our own web API framework (dropshot), and SDK generation (progenitor) turned out to be incredibly and surprisingly valuable.

SDK: The Next Generation

Given my poor experience with open source SDK generators, I thought we might be alone in seeing the value of SDK generation. Instead, it was a discernible theme at the conference.

Microsoft was there (in numbers) talking about Kiota, their open source SDK generator (and TypeSpec—also interesting). It’s very cool, very post-lost decade Microsoft. Part of the thesis (as far as I can tell) is that first-party API producers may not need to ship their own SDK; instead, they can share OpenAPI (or TypeSpec) and let consumers use Kiota to generate clients. Further, they can generate specifically the clients they need: not just by picking the relevant language, but the relevant subset of the API, or even by picking subsets of several APIs to have a consistent client interface for all the APIs they want to talk to. You don’t need various vendors to make consistent SDKs—you can generate the SDK you need for the integrations you want to build. It’s a neat concept I hadn’t seen before.

TypeSpec is pretty cool; it jives with some of my opinions on OpenAPI. TypeSpec is a higher-level, terser, opinionated, more expressive way to define an API. It’s a new TypeScript-familiar language that you can translate into OpenAPI. At Oxide, we define our APIs with Dropshot as annotations in the code itself, but it’s a similar idea. In both cases humans aren’t writing JSON by hand (Yes! Leave machine formats to the machines!). There can be some lossiness, e.g. TypeSpec allows you to express pagination semantics that can’t be expressed in OpenAPI (and doesn’t seem to be on the radar for the next OpenAPI release).

I was blindsided by SDK generation being a thesis for a whole batch of startups. APIMatic and liblab were at the conference. I came across Speakeasy previously; and since discovered Stainless, Fern, and Konfig. Except for APIMatic all of these seem to have been founded in 2022 with at least $65m invested by my count. This is wild to me… here we are just giving away our Rust SDK generator… like suckers! (Just kidding! Open source is central to our mission at Oxide and we’re happy to work almost entirely in the open. We don’t ask for copyright assignment that might allow for the source-available maliciousness we’ve seen proliferating.)

Each of these companies charges $100s/month for SDK generation. I guess the pitch is something like: you, a large company, drank the API Kool-Aid and now you have N internal APIs. At a minimum you’re exerting N x (number of languages in use) effort, but in all likelihood, that work is being repeated in a bunch of places where groups aren’t aware or don’t trust the SDKs made by others. So with SDK generation, you just call our API with your latest API specs and out pops SDKs in all the languages you and your customers care about. It will be interesting to see if this turns out to be a sustainable model.


Can you go a tech conference in 2024 where AI / ML isn’t a theme? I doubt it. Lots of uses of AI in the API space on display. Use of generative AI APIs, writing docs through gen AI, training AI on docs, even writing SDKs through gen AI. Gen AI all the things.

In-person Conferences… still a thing!

I had not been to a conference in … a while. It was surprisingly great. In particular, it was a chance to go deep on topics I don’t usually discuss. I work with a ton of great folks at Oxide, but I think none of them cares—for example—about the draft proposal for the next OpenAPI release or has opinions about it. A bunch of folks who do care were there in person, and we could debate topics I’d been chewing on. I got to hear about how others are approaching similar problems in ways that’s not always obvious from the talks that come out of conferences or online discussion.

I wasn’t sure if in-person conferences were still going to be valuable. It was for me; it might be for you.

A bit over two years ago, I started work on typify, a library to generate Rust types from JSON Schema. It took me a while to figure out it was a compiler, but I’ll call it that now: it’s a compiler! It started life as a necessary component of an OpenAPI SDK generator—a pretty important building block for the control plane services at Oxide. Evolving the compiler has become somewhere between a hobby and an obsession, trying to generate increasingly idiomatic Rust from increasingly esoteric schemas.

Get in, loser; we’re writing a compiler

Why did I start building this? I came to Oxide with a certain amount of OpenAPI optimism (from my previous company), optimism that was in some cases well-founded (it has earned its place as the de facto standard for describing HTTP-based APIs), and in other cases profoundly misplaced (the ecosystem was less mature than expected). On the back of that optimism, Dave and I (but mostly Dave) built a server framework, dropshot, that emits OpenAPI from the code. We gave a pretty good talk about it in 2020 about using the code as the source of truth for interface specification.

As we built out services of the control plane we wanted service-specific clients. Ideally these would be derived from the OpenAPI documents emitted from dropshot. We couldn’t find what we wanted in the ecosystem (read: we tried them and they didn’t work) so we built our own. Before we could invoke APIs and understand their responses we needed to generate types. Since OpenAPI uses JSON Schema to define types**, I started there.

**: sort of; and it’s actually quite annoying but I’ll save my grousing for later.

Sum types

Pretty uncontroversial take for Rust programmers: sum types are great. We use enums a bunch in the API types because they let us express precise constraints. (They do make for tricky SDK generation in languages that don’t support sum types, but that’s not important here.) How are enums represented in JSON serialization or JSON schema? The answer, with some irony, is “variously”. The ubiquitous Rust de/serialization framework gives 4 different choices (that I’ll show below).

My woodworking mentor as a kid observed that I start projects in the middle. That’s exactly what happened here. Reconstructing enums from their generated schemas seemed tricky and interesting, so that’s where I started. Generally, an enum turns into a oneOf construction (“data must conform to exactly one of the given subschemas”). I try to apply heuristics that correspond to each of the serde enum formats:

 let ty = self
    .maybe_option(type_name.clone(), metadata, subschemas)
    .or_else(|| self.maybe_externally_tagged_enum(type_name.clone(), metadata, subschemas))
    .or_else(|| self.maybe_adjacently_tagged_enum(type_name.clone(), metadata, subschemas))
    .or_else(|| self.maybe_internally_tagged_enum(type_name.clone(), metadata, subschemas))
    .or_else(|| self.maybe_singleton_subschema(type_name.clone(), subschemas))
    .map_or_else(|| self.untagged_enum(type_name, metadata, subschemas), Ok)?;

Externally tagged enums have this basic shape:

  "<variant-name>": { .. }

Internally tagged enums look like this:

  "<tag-name>": { "const": ["<variant-name>"] },
  … other properties …

Externally tagged enums:

  "<tag-name>": { "const": ["<variant-name>"] },
  "<content-name>": { .. }

Unlike other formats, the final format, “untagged”, doesn’t include any indication of the variant name—it just dumps the raw type data (and one needs to be careful that the subschemas are mutually exclusive).

Seeing enums traverse JSON Schema and turn back into the same Rust code was very satisfying.While I basically got enum generation right, there are a couple of JSON Schema constructs that I really screwed up.


In JSON Schema an “allOf” indicates that a data value needs to conform to all subschemas… to no one’s surprise. So you see things like this:

  “title": “Doodad",
  “allOf" [
    { “$ref": “#/$defs/Thingamajig" },
    { “$ref": “#/$defs/Whosiewhatsit" },

Serde has a #[serde(flatten)] annotation that takes the contents of a struct and, effectively, dumps it into the container struct. This seemed to match the allOf construct perfectly; the above schema would become:

// ⬇️ This is wrong; don’t do this ⬇️
struct Doodad {
    thingamajig: Thingamajig,
    whosiewhatsit: Whosiewhatis,

This is wrong! Very very wrong. So wrong it often results in structs for which no data results in valid deserialization or serializations that don’t match the given schema. In particular, imagine if both Thingamajig and Whosiewhatis have a fields of the same name with incompatible types.

Perhaps more precisely the code above is only right under the narrow conditions that the subschemas are all fully orthogonal. In the wild (as we JSON Schema wranglers refer to its practical application), allOf is mostly commonly used to apply constraints to existing types.

Here’s an example from a github-related schema I found:

"allOf": [
  { "$ref": "#/definitions/issue" },
    "type": "object",
    "required": ["state", "closed_at"],
    "properties": {
      "state": { "type": "string", "enum": ["closed"] },
      "closed_at": { "type": "string" }

The “issue” type is an object with non-required properties like:

  "state": {
    "type": "string",
    "enum": ["open", "closed"],
    "description": "State of the issue; either 'open' or 'closed'"
  "closed_at": { "type": ["string", "null"], "format": "date-time" },

The result of this allOf is a type where state is required and must have the value “closed” and “closed_at” must be a date-time string (and not null). (closed_at was already required by the base type, so I’m not sure why the allOf felt the need to reassert that constraint.)

This is very very different than what #[serde(flatten)] gives us. Originally I was generating a broken type like this:

struct ClosedIssue {
    type_1: Issue,
    type_2: ClosedIssueType2,

struct ClosedIssueType2 {
    state: ClosedIssueType2State; // enum { Closed }
    closed_at: String,

Wrong and not actually useful. More recently I’ve applied merging logic to these kinds of constructions, but it’s tricky and opens the door to infinite recursion (one of the many sins the JSON Schema spec condemns albeit with merely its second sternest form of rebuke).


I got allOf wrong. I got anyOf much wronger. AnyOf says that a valid value should conform to any of the given subschemas. So if an allOf is just a struct with a bunch of flattened members then it would make sense that an anyOf is a struct with a bunch of optional members. It makes sense, especially if you don’t think about it.

// ⬇️ This is wrong; don’t do this ⬇️
struct Doodad {
    thingamajig: Option<Thingamajig>,
    whosiewhatsit: Option<Whosiewhatis>,

But if you do think about it even briefly, you realize that a type like carries only the most superficial relationship with the JSON Schema. For example, at least one of the subschemas needs to be valid and this type would be fine with an empty object ({}) turning into a bunch of Nones.

So what’s a valid representation of anyOf as a Rust type? In a way I’m glad I went with this quick, clever, and very very wrong approach because a robust approach is a huge pain the neck! Consider an anyOf like this:

  “title": “Something",
  “anyOf": [
    { “$ref": “#/$defs/A" },
    { “$ref": “#/$defs/B" },
    { “$ref": “#/$defs/C" },

Bear in mind, my goal is to allow only valid states to be represented by the generated types. That is, I want type-checking at build time rather than, say, validation by a runtime builder. Consequentially, I think we need a Rust type that’s effectively:

enum Something {
    A ∪ B,
    A ∪ C,
    B ∪ C,
    A ∪ B ∪ C,

You need the power set of all sub-types. Sort of. Some of those are going to produce unsatisfiable combinations (i.e. if the types are orthogonal). We’d ideally exclude those. And we need to come up with reasonable names for the enum variants (AAndBAndC?). Ugh. It’s awful. While I’ve cleaned up allOf, typify’s anyOf implementation is still based on that original, wrong insight.

JSON kvetching

I used to abstractly dislike JSON Schema. My dislike has become much more concrete. With a big caveat: I’m considering only the use cases I care about, which assuredly bear little-to-no resemblance to the use cases envisioned by the good folks who designed and evolve the standard. By way of a terrible analogy here’s the crux of the issue: I think about product concept documents (PCDs) and product requirement documents (PRDs) which are vaguely common product management terms (that I’ll now interpret for my convenience). A PCD tells you about the thing. What is it? How’s it work? How might you build it? A PRD provides criteria for completion. Can it do this? Can it do that? JSON Schema is much better at telling you if the thing you’ve built is valid than it is at telling you how to build the intended values.

What I want is a schema definition for affirmative construction, describing the shape of types. What are the properties? What are the variants? What are the constraints? JSON Schema seems to have a greater emphasis on validation: does this value conform?

As an example of this, consider JSON Schema’s if/then/else construction.

  “if": { “$ref": “#/$defs/A" },
  “then": { “$ref": “#/$defs/B" },
  “else": { “$ref": “#/$defs/C" }

If the value conforms to a schema, then it must conform to another schema… otherwise it must conform to a third schema. Why does JSON Schema even support this? I think (but am deeply unsure) that this is equivalent to:

  "oneOf": [
      "allOf": [
        { "$ref": "#/$defs/A" },
        { "$ref": "#/$defs/B" }
      "allOf": [
        { "not": { "$ref": "#/$defs/A" } },
        { "$ref": "#/$defs/C" }

In other words, { A ∪ B, ¬A ∪ C }. Perhaps it’s a purely academic concern: I haven’t encountered if/then/else in an actual schema.

More generally: there are often many ways to express equivalent constructions. This is, again, likely a case of my wanting JSON Schema to be something it isn’t. There’s an emphasis on simplicity for human, hand-crafted authorship (e.g. if/then/else) whereas I might prefer a format authored and consumed by machines. The consequence is a spec that’s broad, easy to misimplement or misinterpret, and prone to subtle incompatibilities from version to version.

Typify to the future

As much as it’s been a pain in the neck, this JSON Schema compiler has also been a captivating puzzle, reminiscent of the annual untangling of Christmas tree lights (weirdly enjoyable… just me?). How to translate these complex, intersecting, convoluted (at times) schema into neat, precise, idiomatic Rust types. I feel like I kick over some new part of the spec every time I stare at it (dependentRequired? Who knew!). There are plenty of puzzles left: schemas with no simple Rust representation, unanticipated constructions, weirdo anchor and reference syntex, and—to support OpenAPI 3.1—a new (subtly incompatible) JSON Schema revision to untangle.

In 2021 Twitter rolled out Spaces. Listen to people you follow! Talk to followers! It sounded awful–the neologism “webinar” uncomfortably close. So when Bryan asked if I’d hang out with him to give it a shot, I set aside my skepticism–at the very least it would break up the mid-pandemic monotony!

And it was great! The next week illumos dropped support for SPARC so we had Tom Lyon join us (now frequent OxF contributor) and we jury rigged a recording system for a great discussion (… and eulogy: SPARC had been pretty important to us, early in our careers in particular). We wanted to share the recording in the lowest effort (cheapest) way possible so made a YouTube channel.

The (mostly) weekly show became a thing we do We were learning and having fun… and so were listeners. Social audio turned out to be a new (low-effort) way to share technical experience, wisdom, and stories. Bryan gave a great talk about it last year:

We only sporadically introduce guests and topics. We’ve evolved running jokes (“for the light-cone!“, “millennial podcaster audio quality”) with pretty narrow appeal. The Simpsons is referenced like bible verses. Arguably, we laugh at our own jokes too much? But bear with us, because the technical conversations are always pretty interesting.

Oxide and Friends is now a podcast–we just posted our 100th episode (Predictions 2024!). I imagine we’ll keep going as long as we’re having fun.

DTrace’s User-land Statically-Defined Tracing (USDT) was… kind of an accident. Bryan has (kindly) retconned the genesis of USDT as a way to understand dynamic languages[citation needed]. Indeed, it’s been essential for that, but its origins were much less ambitious or prescient.

Way back in the 1990s(!) Jeff Bonwick had created a program called lockstat(1M) for live instrumentation of kernel locking primitives (“live” as distinct from “dynamic” in that while the instrumentation could be turned on and off, the data payloads were pretty much static). This was incredibly useful! What locks were hot? Where were they contended? New observability led to new performance fixes. When we built DTrace, we incorporated its instrumentation as the lockstat provider.  After building user-land tracing with the pid provider, it seemed like an obvious step to build a plockstat(1M) command to understand user-land locking primitives. So I built it. And, my goodness, was the first iteration of that a disaster. A total mess. Special cases on special cases with unholy knowledge sprinkled everywhere. We yanked that out of the first integration of DTrace and went back to the drawing board. What we came up with was USDT, the first provider of which was plockstat whose probes are consumed by the eponymous command.

Bryan and I touched on this somewhere in the 2+ hours of DTrace history we recorded back in September:

Years passed as they do, and USDT turned out to be very very very useful. At Oxide, we wanted that usefulness in the Rust code we’re writing, so my colleague, Ben Naecker, and I built a usdt crate. While we have probes in lots of places, knowledge had passed more or less word-of-mouth. Shocking! To remedy this, at the last Oxide all-company event, Ben and I put together a little slide deck on inserting USDT probes in Rust code, using those probes (spoiler: here’s a new cause for frustration with async Rust), and an exercise to try it out. Enjoy!

(tl;dr add USDT probes where you have log statements and you’ll probably thank yourself later.)

When Apple announced their new file system, APFS, in June, I hustled to be in the front row of the WWDC presentation, questions with the presenters, and then the open Q&A session. I took a week to write up my notes which turned into as 12 page behemoth of a blog post — longer than my college thesis. Despite reassurances from the tweeps, I was sure that the blog post was an order of magnitude longer than the modern attention span. I was wrong; so wrong that Ars Technica wanted to republish the blog post. Never underestimate the interest in all things Apple.

In that piece I left one big thread dangling. Apple shipped APFS as a technology preview, but they left out access to one of the biggest new features: snapshots. Digging around I noticed that there was a curiously named new system call, “fs_snapshot”, but explicitly didn’t investigate: the post was already too long (I thought), I had spent enough time on it, and someone else (surely! surely?) would want to pull on that thread.

Slow News Day

Every so often I’d poke around for APFS news, but there was very little new. Last month folks discovered that APFS was coming to iOS sooner rather than later. But there wasn’t anything new to play with or any revelations on how APFS would work.

I would search for “APFS snapshots”, “fs_snapshot”, anything I could think of to see if anyone had figured out how to make snapshots work on APFS. Nothing.

A few weekends ago, I decided to yank on that thread myself.


I started from the system call, wandered through Apple’s open source kernel, leaned heavily on DTrace, and eventually figured it out. Apple had shipped snapshots in APFS, they just hadn’t made it easy to get there. The folks at Ars were excited for a follow-up, and my investigation turned into this: “Testing out snapshots in Apple’s next-generation APFS file system.”

Snapshots were there; the APIs were laid bare; I was going to bring fire to all the Mac fans; John Siracusa and Andy Ihnatko would carry me on their shoulders down the streets of the Internet.


On the eve that this new piece was about to run I was nervously scrolling through Twitter as I took the bus home from work. Now that I had invested the time to research and write-up APFS snapshots I didn’t want someone else beating me to the punch.

Then I found this and my heart dropped:


Skim past the craziness of MFS and the hairball of HFS, and start digging through the APFS section. Slide 49, “APFS Snapshots” and there it is “apfs_snapshot” — not a tool that anyone laboriously reverse engineered, deciphering system calls and semi-published APIs — a tool shipped from Apple and included in macOS by default. F — .

Apple had secreted this utility away (along with some others) in /System/Library/Filesystems/apfs.fs/Contents/Resources/

What To Do?

The article that was initially about a glorious act of discovery had become an article about the reinvention of the wheel. Conversely, vanishingly few people would recognize this as rediscovery since the apfs_snapshot tool was so obscure (9 hits on Google!).

We toned down the already modest chest-thumping and published the article this morning to a pretty nice response so far. I might have happier as an FAKE NEWS Prometheus, blissfully unaware of the pre-existence of fire, but I would have been mortified when the inevitable commenter, one of the few who had used apfs_snapshot, crushed me with my own boulder.

I had been procrastinating making the family holiday card. It was a combination of having a lot on my plate and dreading the formulation of our annual note recapping the year; there were some great moments, but I’m glad I don’t have to do 2016 again. It was just before midnight and either I’d make the card that night or leave an empty space on our friends’ refrigerators. Adobe Illustrator had other ideas:

Unable to set maximum number of files to be opened.

I’m not the first person to hit this. The problem seems to have existed since CS6 was released in 2016. None of the solutions was working for me, and — inspired by Sara Mauskopf’s excellent post — I was rapidly running out of the time bounds for the project. Enough; I’d just DTrace it.

A colleague scoffed the other day, “I mean, how often do you actually use DTrace?” In his mind DTrace was for big systems, critical system, when dollars and lives were at stake. My reply: I use DTrace every day. I can’t imagine developing software without DTrace, and I use it when my laptop (not infrequently) does something inexplicable (I’m forever grateful to the Apple team that ported it to Mac OS X).

First I wanted to make sure I had the name of the Illustrator process right:

$ sudo dtrace -n ‘syscall:::entry{ @[execname] = count(); }’
dtrace: description ‘syscall:::entry’ matched 500 probes
pboard 1
watchdogd 2
awdd 3
... 7065
Google Chrome He 7128
Google Chrome 8099
Adobe Illustrato 36674

Glad I checked: “Adobe Illustrato”. Now we can be pretty sure that Illustrator is failing on setrlimit(2) and blowing up as result. Let’s confirm that it is in fact returning -1:

$ sudo dtrace -n 'syscall::setrlimit:return/execname == "Adobe Illustrato"/{ printf("%d %d", arg1, errno); }'
dtrace: description 'syscall::setrlimit:return' matched 1 probe
CPU     ID                    FUNCTION:NAME
  0    532                 setrlimit:return -1 1

There it is. And setrlimit(2) is failing with errno 1 which is EPERM (value too high for non-root user). I already tuned up the files limit pretty high. Let’s confirm that it is in fact setting the files limit and check the value to which it’s being set. To write this script I looked at the documentation for setrlimit(2) (hooray for man pages!) to determine that the position of the resource parameter (arg0) and the type of the value parameter (struct rlimit). I needed the DTrace copyin() subroutine to grab the structure from the process’s address space:

$ sudo dtrace -n 'syscall::setrlimit:entry/execname == "Adobe Illustrato"/{ this->r = *(struct rlimit *)copyin(arg1, sizeof (struct rlimit)); printf("%x %x %x", arg0, this->r.rlim_cur, this->r.rlim_max);  }'
dtrace: description 'syscall::setrlimit:entry' matched 1 probe
CPU     ID                    FUNCTION:NAME
  0    531                 setrlimit:entry 1008 2800 7fffffffffffffff

Looking through /usr/include/sys/resource.h we can see that 1008 corresponds to the number of files (RLIMIT_NOFILE | _RLIMIT_POSIX_FLAG). Illustrator is trying to set that value to 0x7fffffffffffffff or 2⁶³-1. Apparently too big; I filed any latent curiosity for another day.

The quickest solution was to use DTrace again to whack a smaller number into that struct rlimit. Easy:

$ sudo dtrace -w -n 'syscall::setrlimit:entry/execname == "Adobe Illustrato"/{ this->i = (rlim_t *)alloca(sizeof (rlim_t)); *this->i = 10000; copyout(this->i, arg1 + sizeof (rlim_t), sizeof (rlim_t)); }'
dtrace: description 'syscall::setrlimit:entry' matched 1 probe
dtrace: could not enable tracing: Permission denied

Oh right. Thank you SIP. This isa new laptop (at least a new motherboard due to some bizarre issue) which probably contributed to Illustrator not working when once it did. Because it’s new I haven’t yet disabled the part of SIP that prevents you from using DTrace on the kernel or in destructive mode (e.g. copyout()). It’s easy enough to disable, but I’m reboot-phobic — I hate having to restart my terminals — so I went to plan B: lldb.

First I used DTrace to find the code that was calling setrlimit(2): using some knowledge of the x86 ISA/ABI:

$ sudo dtrace -n 'syscall::setrlimit:return/execname == "Adobe Illustrato" && arg1 == -1/{ printf("%x", *(uintptr_t *)copyin(uregs[R_RSP], sizeof (uintptr_t)) - 5) }'
dtrace: description 'syscall::setrlimit:return' matched 1 probe
CPU     ID                    FUNCTION:NAME
  0    532                 setrlimit:return 1006e5b72
  0    532                 setrlimit:return 1006e5b72

I ran it a few times to confirm the address of the call instruction and to make sure the location wasn’t being randomized. If I wasn’t in a rush I might have patched the binary, but Apple’s Mach-O Object format always confuses me. Instead I used lldb to replace the call with a store of 0 to %eax (to evince a successful return value) and some nops as padding (hex values I remember due to personal deficiencies):

(lldb) break set -n _init
Breakpoint 1: 47 locations.
(lldb) run
(lldb) di -s 0x1006e5b72 -c 1
0x1006e5b72: callq  0x1011628e0     ; symbol stub for: setrlimit
(lldb) memory write 0x1006e5b72 0x31 0xc0 0x90 0x90 0x90
(lldb) di -s 0x1006e5b72 -c 4
0x1006e5b72: xorl   %eax, %eax
0x1006e5b74: nop
0x1006e5b75: nop
0x1006e5b76: nop

Next I just process detach and got on with making that holiday card…

DTrace Every Day

DTrace was designed for solving hard problems on critical systems, but the need to understand how systems behave exists in development and on consumer systems. Just because you didn’t write a program doesn’t mean you can’t fix it.

Recent Posts

April 17, 2024
January 13, 2024
December 29, 2023
February 12, 2017
December 18, 2016