Looking for writing-related posts? Check out my new writing blog, www.larrykollar.com!

Thursday, September 08, 2022 No comments

Cobbled together

Wow, has it really been four years since I discovered the peach tree next to the composter? Not like there was much more to say. The peaches were hard, and the critters stole the entire bunch every year as soon as they started getting ripe. We never did get around to moving it, either.

But, after the clearing out we did, it started getting more sun. I figured there wouldn’t be any this year; the tree started blossoming during a warm spell in January, and you can guess what happened a week later. But later on, as I went out to dump scraps in the composter, I saw it had about a dozen hard fruits on it.

As August rolled around, we moved Chief (the outdoor dog) next to the tree to keep critters away. I started checking every couple of days, felt them begin to soften, saw them start to turn. One fell off, and I took it inside to ripen up in the kitchen window.

A couple weeks ago, the composter run showed me a batch of ripe and nearly ripe peaches! I pulled up the front hem of my T-shirt for an impromptu basket, and harvested the lot before the critters could.

Wife stuck them in the fridge, with assurances that she would make them into something. She was leaning toward peach ice cream, using goat milk so Charlie could have some, but goat milk prices are getting to the point where it might be worth having our own goats. Then Mason got the ro, then the wife did. I considered freezing them (i.e. the peaches), so they would be at least usable for something when we got to them.

Last weekend, she got them out and got going on some peach cobbler. It wasn't huge, because we only had about 10 to work with, but there was enough to go around.

Tastes as good as it looks!

The tree itself is still rather spindly, so I’m still thinking about transplanting it (assuming we get a winter that’s more than Long November again). And I should have grabbed some vanilla ice cream to go with the last of the cobbler.… but it’s pretty darn good on its own.

So even FAR Manor is capable of pleasant surprises. Too bad they’re so rare.

Monday, September 05, 2022 No comments

Adventures of a #techcomm Geek: Go API chapter 2, “No ReST for the weary” (edit#2)

Image source: openclipart.org

Last time I had to deal with an API, it was pulling a vendor’s documentation into our own system. Now, I have to document our own APIs.

OpenAPI, formerly known as Swagger, is quite popular in the ReST API universe these days. And why not? One source builds a website, the hooks, documentation, and everything. At least online. If you want to provide a PDF document describing the API, though, there’s a little more to it.

  • First, all those definition and summary strings need some attention. Where developers involve the technical writers in the process makes a huge difference in effort (at least on the writer side).
  • Second, there’s more to documenting an API call than the definition and summary strings. There are path variables, query variables, examples, and the list goes on.

Fortunately, there are several utilities that extract documentation from an OpenAPI file. For my purposes, Widdershins works best—it produces a complete Markdown file—although it’s nowhere near ideal.

  • One issue was definitely not the fault of the tool. The developers told me of a dozen categories (or tags in OpenAPI parlance) that didn’t need to be documented for customers. Widdershins groups all API calls with the same tag under the same section, and that helps a lot.
  • The second issue could be either Widdershins or my personal preference. I didn’t like the order that Widdershins presented data for each method. There were some other minor issues as well.

I had a big wad of text that was my nail, and awk once again is my hammer. I started pounding. I did consider using a YAML parser for a brief time, but realized Widdershins did a lot of busy work for me. It actually does a pretty good job of building a Markdown document, describing all the method calls and schemas. If only there was a way to fix the presentation order, it would be perfect.

My first goal was to reshuffle the internal sections of each method to get them in the order I wanted. Deleting the unneeded groups, I reasoned, was a one-time thing that I could deal with myself.

My script worked the first time, but scrambled a bunch of things on the second attempt. Worse, doing the search-and-delete on those unneeded sections took more time and care than I’d anticipated. I needed a re-think.

Fortunately, a Computerphile interview with Brian Kernighan (the “K” in awk) came around, right when I needed it. It gave me… if not the key to my problem, a map to the key. In a nutshell, Dr. Kernighan advocates against large, monolithic awk scripts. His 1986 paper Tools for Printing Indexes describes his approach as:

…a long pipeline of short awk scripts. This structure makes the programs easy to adapt or augment to meet the special requirements that arise in many indexes.  

This approach can also be easier to debug, as you can replace the pipeline with temporary files and verify that the output of one stage is correct before feeding it to the next stage. Each stage refines the input further.

So I split the monolithic script into two medium-size scripts:

  • Stage 1a (weed) fixes headings, weeds out unneeded HTML markup (mostly <a name="x"/> tags), and gets rid of those unneeded sections. Having less cleanup already makes this approach worth the effort.
  • Stage 1b (shuffle) re-orders the remaining method descriptions. I learned that the input order is important for making this work; so if future versions of Widdershins move things around, it could break the script and I would need to fix it again.

It takes maybe a second to process the raw Markdown through both stages under Cygwin, which is noticeably slower than a shell under a native POSIX system. I expect my 8 year old iMac would be nearly instantaneous.

Now that I’ve cracked the code, so to speak, more stages are coming.

  • Stage 2 throws out the schema definitions that none of the remaining methods refer to. A pair of scripts identify which schemas needed to be kept, then weeds out the others.
  • Stage 3 fixes cross-references (mostly to schemas). The monolithic Markdown file uses a URL of the form #schemaxyz. Since the ultimate goal is to split the single Markdown file into topics, those URLs need to point to the eventual file name instead. A trio of scripts create file names that correspond to the URLs, replace the original #xyz name with the file name, then shuffle the schema’s description to the top of the topic.

These stages take another second to process… so 13,000 lines of YAML to monolithic Markdown file is about two seconds. The mdsplit script, that splits the methods and schemas into topics and builds a Lightweight DITA (LwDITA) bookmap, takes less than ten seconds to complete. So I’m now at the point where it’s easier to regenerate the entire document if I run into a scripting issue, instead of pushing through the problem. Uplifting the LwDITA to full DITA takes maybe a minute. After the uplift, another script fixes the extensions, changing .md to .dita, and fixing the cross-references.

At this point, I can focus on adding value: adding metadata, grouping related schema definitions, and the like. If I need to regenerate this thing again, I need only run the shell scripts that conduct the Geek Chorus.

Going forward, I’ll need to be able to compare versions, so I can replace topics with actual content changes, or add new topics. At that point, I could hand things off and intervene only when the input changes enough to make a difference. Or, we might decide to ditch the PDF entirely, and that would make things far easier on everyone.

Techcomm geeks never worry about automating themselves out of a job, by the way. There’s always a new presentation format, or new source document formats, or many new ways to streamline workflows. Handing off a system is a triumph; it means we have more time to focus on the next thing.

Edited 5 Sep: I didn’t realize I’d pasted this out of Logseq before going through it and fixing some things. Now we’re up to date.

Edited 7 Sep: All the tweaks have been made, and I now have a turnkey system.


Related Posts Plugin for WordPress, Blogger...