Author Archives: heckj

Using Combine – first edition available

I just finished my first edit pass of the content of Using Combine, and am incredibly pleased. Sufficiently pleased, in fact, that I am going to call this version the “first edition”.

It is certainly not perfect, nor even as complete as I would like, but a significant enough improvement that I wanted to put a stake in the ground and get it out there.

I’m not planning on stopping the development work, there are more examples, more details, and useful tidbits that will be developed. I have continued to receive wonderful feedback, and plan to continue to accept updates, as all the content, example code, and sample project pieces are available in the github repository swiftui-notes.

The content will continue to remain available for free in a single-page HTML format, hosted at http://heckj.github.io/swiftui-notes/. The ePub and PDF version is available through gumroad.

SwiftUI and Combine – Binding, State, and notification of changes

When I started the project that became Using Combine, it was right after WWDC; I watched streamed WWDC sessions online, captivated like so many others about SwiftUI. I picked up this idea that SwiftUI was “built using the new framework: Combine”. In my head, I thought that meant Combine managed all the data – notifications and content – for SwiftUI. And well, that ain’t so. While the original “built using Combine” is accurate, it misses a lot of detail, and the truth is a bit more complex.

After I finished my first run through drafting the content for Using Combine, I took some time to dig back into SwiftUI. I originally intended to write (and learn) more about that. In fact, SwiftUI is what started the whole segue into Combine. I hadn’t really tried to use SwiftUI seriously, or get into the details until just recently. I realized after all the work on examples for Combine and UIKit, I had completely short shifted the SwiftUI examples.

Mirroring a common web technology pattern, SwiftUI works as a declarative structure of what gets shown with the detail being completely derived from some source of truth – derived from state stored or managed somewhere. The introductory docs made is clear that @State was how this declarative mechanism could represent a bit of local state within a View, and with the benefit of Daniel and Paul’s writing (SwiftUI Kickstart and SwiftUI by Example), it was also quickly clear that @EnvironmentObject and @ObservedObject played a role there too.

The Combine link to SwiftUI, as it turns out, is really only about notifying the SwiftUI components that a model had changed, not at all what changed. The key is the protocol from Combine: ObservableObject (Apple’s docs). This protocol, along with the @Published property wrapper, does the wonderful work of generating a combine publisher – the default type of which is represented by the class ObservableObjectPublisher. In the world of Combine, it has a defined output and failure type: <Void, Never>. The heart of that Void output type is that the data that is changing doesn’t matter – only that a change was happening.

So how does SwiftUI go and get the data it needs?Binding is the SwiftUI generic structure that is used to do this linkage. The documentation at Apple asserts:

Use a binding to create a two-way connection between a view and its underlying model. For example, you can create a binding between a Toggle and a Bool property of a State. Interacting with the toggle control changes the value of the Bool, and mutating the value of the Bool causes the toggle to update its presented state.

You can get a binding from a State by accessing its binding property. You can also use the $prefix operator with any property of a State to create a binding.

https://developer.apple.com/documentation/swiftui/binding

Looking around a bit more while creating some examples, and it becomes clear that some handy form elements (such as TextField) expect a parameter of type binding when they are declared. Binding itself works by leveraging swift’s property getters and setters. You can even manually create a Binding if you’re so inclined, defining the closures for get and set to whatever you like. Property wrappers such as @State, @ObservedObject, and @EnvironmentObject either create and expose a Binding, or create a wrapper that in turn passes back a Binding.

My take away is the flow with Combine and SwiftUI has a generally expected pattern: A model to be represented by a reference object, which sends updates when the data is about to change (by conforming to the ObservableObject protocol). SwiftUI goes and gets the data that it needs based on what was declared in the View using Binding to get to the underlying data (and potentially allowing the SwiftUI view to update it in turn if that’s relevant).

Given that SwiftUI views are also designed to be composed, I am leaning towards expecting a pattern that state will need to be defined for pretty much any variation of a view – and potentially externalized. The property wrappers for representing, and externalizing, state within SwiftUI are:

  • @State
  • @ObservedObject and @Published
  • @EnvironmentObject

@State is all about local representation, and the simplest mechanism, simply providing a link to a property and the Binding.

@ObservedObject (along with @Published) adds a notification mechanism on change, as well as a way to get a typed Binding to properties on the model. SwiftUI’s mechanism expects this always to be a reference type (aka a ‘class’), which ends up being pretty easy to define in code.

@EnvironmentObject takes that a step further and exposes a reference model not just to a single view, but allows it to be used by any number of views in their own hierarchy.

  • Drive most of the visual design choices entirely by the current state

But that’s not the only mechanism that is available: SwiftUI is also set up to react to a Combine publisher – although not in a heavily predetermined fashion. An interesting aspect is that all of the SwiftUI views also support a Combine subscriber: onReceive. So you can bring the publisher, and then write code within a View (or View component) to react to what it sends.

The onReceive subscriber acts very similarly to Combine’s sink subscriber – the single-closure version of sink (implying a Combine pipeline failure type of Never). You to define a closure within your SwiftUI view element that accepts that data and does whatever needs doing. This could be using the data, transforming and storing it into local @State, or just reacting to the fact that data was sent and updating the view based on that.

From a “What is a best practice” point of view, it seems the more you represent what you want to display within a reference model, the easier it will be to use. While you can expose a publisher right into a SwiftUI view, it tightly couples the combine publisher to the view and all links all those relevant types. You could (likely just as easily) have the model object encapsulate that detail – in which case the declaration of how you handle event changes over time are separated from how you present the view. This is likely a better separation of concerns.

The project (SwiftUI-Notes) linked to Using Combine now has two examples with Combine and SwiftUI. The first is a simple form validation (the view ReactiveForm.swift and model ReactiveFormModel.swift). This uses both the pattern of encapsulating the state within the model, and exposing a publisher to the SwiftUI View to show what can be done. I’m not espousing that the publisher mechanism is a good way to solve that particular problem, but it illustrates what can be done nicely.

The second example is a view (HeadingView.swift) that uses a model and publisher I created to use the built-in CoreLocation framework. The model (LocationModelProxy.swift) exposes the authorization as a published property, as well as the location updates through a publisher. Within the built-in Cocoa framework, those are normally exposed through a delegate callback. A large number of the existing Cocoa frameworks are convertible into a publisher-based mechanism to work with Combine using this pattern. The interesting bit was linking this up to SwiftUI, which was fun – although this example only taps the barest possibility of what could done.

It will be interesting to see what Apple might provide in terms of adopting Combine as alternative interfaces to its existing frameworks. CoreLocation is such a natural choice with its streaming updates, but there are a lot of others that could be used as well. And of course I’m looking forward to seeing how they expand on SwiftUI – and if they bring in more Combine based mechanisms into it or not.

programming tools – start with sketching and build to fully specified

A couple of years ago I was managing a very distributed team – or really set of teams: Shanghai, Minneapolis, Seattle, Santa Clara, and Boston. Everyone worked for the same company, but culturally the teams were hugely divergent.

In one region, the developers in the team preferred things to be as loosely defined as possible. Their ideal state was “tell me how you’re going to rate me” and then you got of their way as they min-max’d based on whatever measurement rules you just set in place. When they were in charge of a specific element of the solution, there was a great deal of confidence it’d get done and effectively. But when they needed to be a bit more of active coordinator with a growing solution, they struggled quite a bit more – communication overhead really took a steep hit.

The opposite end of the spectrum was a region where the developers preferred for everything to be fully specified. Every detail defined, every option laid out and determined, how things work, and how things fail. Lots of detailed definition, to a level that extremely exhaustive – and would frankly annoy nearly all the developers from the first region.

I would love to make a hasty generalization and tell you that this was a “web developer” on one side and an ex-firmware developer on the other, but these folks were *all* firmware-level developers, stepping into a new team we had merged from a variety of pieces and places. Everyone was dealing with new development patterns, a new language – quite different from what they were used to historically, and a different business reality with goals and intentions that were in a high degree of flux.

What is surprising to me is that while I started out sympathizing with the first region’s culture far more, the influence of the spectrum – and the far side that preferred the specificity is what has stuck with me. I had previously left a lot more unspecified and open to interpretation – and not surprisingly that bit back upon occasion. Now with programming, regardless of the language, I want the tools to help me support that far more detailed, specified version of what I expect.

In the last year, I’ve actively programmed in C, C++, Javascript, TypeScript, Swift, Python, and Objective-C. When I programmed in dynamic languages (javascript, python) I just *expected* to need to write a lot of tests, including ones that a type system would otherwise do some overlapping validation with. A lot of time it was tracking null values, or explicitly invalid inputs to functions and systems to make sure the responses were as expected (errors and failures). As I followed this path, I gained a huge amount of respect for the optional concept that’s been embedded into Swift. It has been an amazing eye opener for me, and I find I struggle to go back to languages without it now.

Last fall I diverted heavily into a C++ project, where we explicitly pulled in and used an optional extension through a library: type_safe. Back in December, Visual Studio went and exposed a fatal compiler error bug in the Microsoft compiler update that just exploded when attempting to use this library, much to my annoyance. But even with the type system of C++ and this library working with me, I can’t even *imagine* trying to write stable C++ code without doing a ton of tests to validate what’s happening and how it’s working – and failing.

I’m still tempted to delve into Rust, as I keep hearing and seeing references to it as a language with very similar safety/structural goals to swift (quite possibly with stronger guarantees – I don’t have a great way to judge the specifics there though). Bryan Cantrill keeps pimping it in the background of his conversations at Oxide Computing’s On The Metal podcast. If you’re into some fascinating history/interviews about the software-hardawre interface, it’s amazing!

The end result is that I’ll start out sketching in ideas, but want my tools and languages to help me drive to a point where I’m far more “fully specified” as I go through the development process and iterate on the code. In my ideal world, which I don’t really expect, the language and software tooling helps me have confidence in the correctness and functionality of the code I’ve written, while allowing me the expressiveness to explore ideas as quickly as possible.

Just like the cultures and regions I navigated with that global team, it is a matter of sliding around on the spectrum of possibilities to optimize for the software challenge at hand.

CRDTs and lockless data structures

A good five or so years ago Shevek, a friend I met while building cloud-generating appliances at the (now defunct) Nebula, spent an afternoon and evening describing the benefits of lock-free data structures to me. He went deep into the theoretical aspects of it, summarizing a lot of the research thinking at the time. While I tried to keep up to retain it all later; I really didn’t manage it. The (predictable) end result was that I got the basics of what it did, why it was valuable, and some basic examples of how it was used, but I missed a lot of the how can this be more directly useful to me. A lot of that conversation was focused on multithreaded code and making it efficient, and what could be done (or not) to avoid contention blocks.

Jumping back to today, when I’m not writing away on Using Combine, I am helping some friends with a project that includes real-time collaborative editing. The goal is the same kind of thing that you see in Google Docs where multiple people are editing a document at the same time – you see each other’s cursors, lives updates, etc.

The earliest form of this kind of functionality that I used was originally a mac program called SubEthaEdit, and a couple years later a browser-based “text editor” called EtherPad, which I used extensively when working with the open source project OpenStack. (This was also around five years ago – I haven’t been actively contributing to OpenStack in quite a while).

Google adopted this capability as a feature, and has done an outstanding job of making it work. Happily, they also talk about how they do such things. In the details I was able to find, they often used the term “operational transformation” to cover most of their recent efforts. That was also the key technology behind Google’s (now dead effort): Wave. Other systems have done the same thing: ShareJS, Atom’s Xray, and the editor Xi.

I spent some time researching how others had tackled the problem of how you enable this kind of feature. The single best cohesive document I read was a blog post by Alexei Baboulevitch (aka “Archagon“) entitled: Data Laced with History: Causal Trees and Operational CRDTs. The article describes his own code and implementation experiments, conveniently available on github as crdt-playground. It also includes a tremendously useful primer into the related topics and links to research papers. It also helped (me) that all his code was done with Swift, and readily available to try out and play with.

Data Laced with History: Causal Trees and Operational CRDTs is absolutely amazing, but not an easy read. While I highly recommend it if you want to know how CRDT works, expect to need to read through it several times before the details all sink in. Alexei writes clearly and very accurately, and it is extremely dense. I have read the whole article many times, as well as dug through all that code repeatedly, to gain an understanding of it.

While I was buried in my own learning of how this works, I had an epiphany one evening: the core of a CRDT is a lock-free data structure. Once I stepped back from the specifics of it, looking at the whole thing as a general algorithm, the pieces I picked up from Shevek clicked into place. Turns out that conversation (which I had mostly forgotten and thought I lost) had a huge amount of value for me, years later. The background I gleaned after the fact gave me some ideas to understand how and why CRDTs work, and ultimately  proved incredibly useful to understand how to effectively use them.

Human Voice

When I joined twitter, it was because my friends were talking about it. Conversations that I could normally only participate in during conferences or meetups became available to me. I tried to follow it slavishly at first, and then I had an epiphany that it was more like chatting with some friends at a restaurant or bar – people are coming and going, and you chat with whomever is around and available. Facebook was similar, but family and friend focused – keeping up with what my friends are doing after I moved 1000 miles away.

Fast forward a decade, and the human voice has been nearly extinguished in both mediums. I still have accounts in both systems, but it’s more like turning on a constant advertising stream. I ceased being able to rely on either for even slight recency of human voices, let alone the friends and conversations that I used to have. They have become the modern noise-on-the-TV that I grew up with – nothing good on. Perhaps worse, because so much of it is emotionally strident – “this one small trick”, “you’ll be shocked and amazed”, etc. So much bullshit.

Fortunately I’ve (re)found a place where I can get that human voice again:

NetNewsWire

2019 in review

2020 Roadmap

I’ve gone back to checking my RSS feeds first for reading instead of hitting twitter or facebook, which makes a huge difference. That dopamine hit isn’t the same – RSS isn’t a never-ending stream of potentially interesting content that keeps you addicted like a manic crackhead, but the content that is there tends to be pretty darned good.

I’ve gone back to curating – looking for mentions and links, and following those back to sources. The “X Weekly” curated newsletters are equally good for finding new people to read, as well as friends of friends. It takes some effort, but that is also making it more real. If someone goes to wonky, I can easily ignore them for a bit, or drop their feed from my set – no shaming or cancel notification, just stepping away to more of what I’m interested in.

If you want to pipe up and join in the conversation, you can easily host your writing at Micro.blog, WordPress, or Medium. Micro.blog and WordPress are $5 and $8 per month and Medium is no direct cost.

Remember if you’re not paying for a service, you are likely the product…

I have used WordPress for years, so I stuck there, but honestly the easy to get started option is very much micro.blog. Write about whatever you want, as much as you want. A sentence, paragraph, or longer – there’s no limit, no “right way/wrong way”, and you don’t need to torture your words into some small number of characters.

Using Combine – reference content complete!

I’m thrilled to be announcing that an updated version of Using Combine is now available!

It has taken me nearly 6 months to draft it all, reverse engineering and writing tests for all the various publishers, operators, and pieces in between – and documenting what I found. The end result is 182 pages (in US PDF format) of reference documentation the way I’d generally like to have it.

While the live site is updated automatically, updated PDF and ePub versions are now available on Gumroad. If you purchased a copy previously, you can go to Gumroad and get an updated, DRM free, content in either PDF or ePub formats.

This updates finishes the largest swath of reference updates, creating tests to verify all the various operators and writing the documentation reference sections for the following:

There was also an update for Xcode 11.3 and associated iOS 13.3 and macOS 10.15.2, which included some subtle changes to the throttle operator behavior, which I recently wrote about in some detail.

With this update, the majority of the core content is now complete, but the work is by no means finished!

The next steps for the book are review and editing. On the to-do list are refining the descriptions of the reference sections, reviewing all the patterns now that we have had Combine for a few months, and seeing the updates as the API changes and refines. There are some diagrams now, but more are likely needed in some sections – both in the patterns and reference sections.

As before, this continues as a labor of love and for the community. Meaning that the content will continue to be free, available on the live site, with updates being made available as I make them. The work has been financially supported by 116 people as I’m writing this, as well as a number of people providing pull requests to fix typos and grammar flaws.

If you want a DRM-free digital copy in either PDF or ePub format, please consider supporting this work by purchasing a copy at gumroad.

Combine: throttle and debounce

Updated March 2020 with more thoroughly accurate timing diagrams, after vetting against iOS13.2, iOS 13.3, and iOS13.4 beta.

Combine was announced and released this past summer with iOS 13. And with this recent iOS 13 update, it is still definitely settling into place. While writing Using Combine, I wrote a number of tests to verify and generally double-check my understanding of how Combine was working. With the update to iOS 13.3, the tests showed me that a few behaviors changed once again.

The operator that changed and trigged my tests was throttle. Throttle is meant to act on values being received from an upstream publisher over a sliding window. It collapses the values flowing through a pipeline over time, choosing a representative value from the set that appeared within a given time window. It also turns out that throttle has slightly different behavior when you’re working with a publisher that starts out sending down an initial value (such as a @Published property).

While I was poking at throttle to understand how it changed, I also realized that debounce was acting differently than I had originally understood, so I took some time to write some additional tests and make a more explicit timing diagram for it as well. Since debounce didn’t change behavior between 13.2 and 13.3 (that I spotted anyway), I’ll describe it first.

debounce

When you set up a debounce operator, you specify a time-window for it to react within. The operator collects all values that come in from the publisher, and most notably it resets the starting point for that sliding time window when it receives a value. It won’t send any values on until that entire window has expired without any other values appearing. In effect, it’s waiting for the value to settle. The marble diagrams show this really well.

Without a break that lets the sliding window expire, a single value is returned.
With a break that lets the sliding window expire, two values are propagated.

When using debounce, you will see a delay invoked between when the event was published and the time it is received by your subscriber that is at least the value that you are processing the debounce across (0.5 seconds in the above example).

throttle

Throttle acts similarly to debounce, in that it collects multiple results over time and sends out a single result – but it does so with fixed time windows. Where debounce will reset the start of that window, throttle does not – so it doesn’t collapse the values entirely, but sort of “slows them down” (and that matches the name of the operator pretty well).

Which value from the set that arrive that’s chosen to be propagated is influenced by the parameter latest, which is set when you create the operator. In general, latest being set to true results in the last value appearing getting chosen, and latest being set to false results in the first value that appears. This is one of those items that is a lot easier to understand by looking at a marble diagram:

Throttle (latest=false), under iOS 13.2.2
throttle (latest=true) under iOS 13.2.2

The notable behavior change is how it handles initial values. Initial value seems to be a bit “flexible” in what is specifically initial though. When I first wrote my tests, I was using a class with a @Published variable, which sends a value upon subscription, and then updates when it is changed. To illustrate the 13.3 behavior change better, I re-wrote and expanded those tests to use a PassthroughSubject, so there wasn’t an automatic initial value.

throttle (latest=false) under iOS 13.3
throttle (latest=true) under iOS 13.3

So under iOS 13.3, the initial value (which is sent out roughly 100ms after the creation of the pipeline in my test), is always propagated, and then the sliding window effect begins immediately after that.

When using throttle, you will see almost no delay between when the event was published and the time it is received by your subscriber. In the tests where I work the timing, using throttle with latest=true shows almost no delay, and throttle with latest=false with a very short delay: 20 to 40 ms in my test environment. debounce, by comparison, will always include a significant delay.

If you want to see the underlying tests that illustrate this, check out the following bits of code from the Using Combine project:

Using Combine (v0.8) update available!

A new version of Using Combine (v0.8) is now available.

The live HTML site for Using Combine is updated automatically, and the PDF and ePub versions are now available on Gumroad.

This version has a number of additional notes and changes, primarily from reader feedback, and some references to Combine’s changes with the release of IOS 13.2. A few more issues have been noted in the book, along with references to feedback reports sent to Apple where they may represent bugs or unexpected behavior in the current implementation.

This release also includes SVG based diagrams – so the original ASCII art diagrams are now gone, which should make that content far more accessible in the ePub format.

In addition, I added a section on marble diagrams, specifically in how to read them and how they apply to the code and examples illustrated in this book. I originally planned on generating all the marble diagrams, but after repeated efforts at that I backed off that idea and am creating them by hand, with the help of OmniGroup‘s wonderful tool OmniGraffle. And yes, the source for this is also in the github repository.

For the next release, I am planning on getting back to detailing out the as-yet-unwritten section on a number of operators. 

The project board at https://github.com/heckj/swiftui-notes/projects/1 also reflects all the various updates still remaining to be written.

A huge thank you to all who have supported this book and my efforts to provide it!

Japan

A torii gate at Danjo Garan in Koyosan

We recently returned from our first visit to Japan. Karen and I have been wanting to go for quite a while, and the time was right – so most of the month of October was dedicated to that endeavor.

I loved our trip, and we were fortunate enough to even be in Kyoto during on of the festivals – Jidai Matsuri.

Our trip and tour included Kushiro, Tokyo, Nagoya, Hiroshima, Meijima Island, and Kyoto – as well as a number of places in between that will take a while to unblur in my memory.

Hiroshima Memorial

We took a tour through Samurai Tours – Best of Japan, which I highly recommend if you want a good sampling of a wide swath of Japan. Their guides were excellent and informative, and I can’t thank Charlie enough for his instruction in how to navigate the signage for the Tokyo trains and subways, which more or less held us in stead through the entire trip. I do wish we could have stayed longer in some areas, but another trip in the future will have to do.

Yours truly, visiting a Studio Ghibli store in Kyoto
The Golden Shrine, in Kyoto
Looking around from the top of Miyajima Island, near Hiroshima

Tokyo subway map – yep, it’s complicated – but amazing!
Pigment, a dangerous store for artists…
giant buddha at Kamakura Daibatsu
The ryokan we stayed at in Takayama

The sheer artistry in every day life in Japan was amazing, and I miss it already being back in the US. Another trip in the future, I tell myself…

It is OK to test the framework

When I started to write the book Using Combine, I was learning the Combine framework as I went. There was a lot I was unsure about, and especially given that it was released with the beta of the operating system, the implementation was changing between beta releases as it firmed up. I chose to use a technique that I picked up years ago from Mike Clark – write unit tests against the framework to verify my understanding of it – while writing the book. (yes, I’m still working on it – it’s a very lengthy process)

While listening to a few episodes of the Under the Radar podcast, I heard a number of references to the idea of “make sure you’re not testing the framework”. It is generally good advice, in the vein of “make sure you’re testing your code first and foremost”, but as a snippet out of context and taken as a rule – I think it’s faulty. Don’t confuse what you are testing, but reliably testing underlying frameworks or libraries, especially while learning them or they evolve, can easily be worth the effort.

I have received a huge amount of value from testing frameworks – first in verifying that I understand what the library is doing and how it works. More over, it has been a very clear signal when regressions do happen, or intentional functionality changes.

If you do add tests of a framework or library into your codebase, I recommend you break them out into their own set of tests. If something does change in the library, it will be far more clear that it is a change from the library and not a cascading side effect in your code.

Most recently, this effort paid off when I stumbled across a regression in the Combine framework functionality with the GM release of Xcode 11.2. While I’ve been coming up to speed with the various operators, I’ve written unit tests that work the operators. In this case, the throttle operator – which has an option parameter latest – changed in how it operates with this release.

Throttle is very similar to the debounce operator, and in fact it operates the same if you use the option latest=true. They both take in values over time and return a single value for a specific time window. If you want the first value that’s sent within the timeframe, theoretically you should use latest=false with the throttle operator. This worked in earlier releases of Combine and Xcode – but in the latest release, it’s now disregarding that path and sending only the latest value.

You can see the tests I wrote to verify the functionality at https://github.com/heckj/swiftui-notes/blob/master/UsingCombineTests/DebounceAndRemoveDuplicatesPublisherTests.swift, and right now I’m working on a pull request to merge in the change reflecting the current release and illustrating the regression. And before you ask, yes – I have submitted this as a bug to Apple (FB7424221). If you are relying on the specific functionality of throttle with latest=false, be aware that the latest release of Xcode & Combine is likely going to mess with it.

If you are more curious about all the other tests that were created to support Using Combine, then feel free to check out the github repository heckj/swiftui-notes – the tests are in the UsingCombineTests directory, and set up as they’re own test target in the Xcode project. There are more to write, as I drive down into the various operators, so I do expect more will appear. I won’t assert that they’re all amazing, well constructed tests – but they’re getting the job done in terms of helping me understand how they work – and how they don’t work.