There’s been a lot of motion in the last four months of the evolution of the Swift programming language that I’ve been wanting, waiting, and hoping for. The language maintainers are tackling concurrency as a first-class construct in the language. I’m following along with the language evolution proposals in the forums, and so far have mostly been able to keep up with the details – although I’m sure I’m missing a lot of the fine-grained implications. Reading the forum pitches where people are talking them through has been fascinating.
One of the interesting take-aways is that the terms “safe” and “unsafe”, or at least the specific implications of when they’re used in the swift language, are broadening what they cover with the upcoming changes. You could start to see it as early as last October when the Swift Concurrency Roadmap was published, but the wording wasn’t fully in place, more of just conceptual frameworks. The details of the broadening of the definition didn’t hit home for me until I caught up with the recent discussion on the pitch for task local values.
Prior to these language extensions, “safe” implied something pretty narrow and specific: memory safety. It started with some swift-language specific guarantees about variables being guaranteed to be initialized before use. There’s a Swift.org blog post from back in 2015 that calls this out, and far more detail in The Swift Programming Language book’s chapter on Memory Safety. There’s an even better explanation and detail in the presentation Unsafe Swift from WWDC 20. The heart of it being that the “safe” APIs have more preconditions and guarantees wrapped around them.
Across the recent pitches and proposals, some of the language terms that use safe are now being used to imply concurrency safety, somewhat independently of memory safety. The goal looks to be to provide APIs that have some guarantees about thread-safe access and updates. And along with the safe versions, there are some potential “unsafe” variants to use when you need the escape hatch and are willing to take on the thread safety guarantees yourself.
This article isn’t a how-to so much as a debugging/dev diary entry for future-me, and any other soul who stumbles into the same (or similar) issues.
Let me provide the backdrop for this story:
I’m working on a private C++ language based project, previously written to be cross platform (Windows, Linux, and Mac). It has a number of C++ library dependencies, which it’s managing with vcpkg, a fairly nice library package manager solution for C++ projects. It happens to align well with this project, which uses CMake as its build system. One of the dependencies that this project uses is grpc, which in turn has a transitive dependency on OpenSSL.
With the M1 series of laptops available from Apple, we wanted to compile and use this same code as an M1 arm64 native binary. Sure – makes sense, should be easy. The good news is, it mostly has been. Both vcpkg and openssl recently had updates to resolve compilation issues with M1/arm based Macs, most of which revolved around a (common) built-in assumption that macOS meant you were building for an x86_64 architecture, or maybe, just maybe, cross-compiling for iOS. The part that hasn’t been so smooth is there’s an odd complication with vcpkg, openssl, and the M1 Macs that ends up with a linker error when the build system tries to integrate and link the binaries created and managed by vcpkg. It boils down to this:
ld: in /Users/heckj/bin/vcpkg/installed/arm64-osx/debug/lib/libcrypto.a(a_strex.o), building for macOS, but linking in object file built for iOS
For anyone else hitting this sort of thing, there’s two macOS specific command-line tools that you should know about to investigate this kind of thing: lipo and otool.
lipo does a number of things – mostly around merging various archives into fat libraries, but the key part I’ve been using it for is to determine what architecture a library was built to support. The command lipo -info /path/to/library.a, tells you the architecture for that library. I stashed a copy of vcpkg in ~/bin on my laptop, so using the command lipo -info ~/bin/vcpkg/installed/arm64-osx/lib/libcrypto.a reports the following:
Non-fat file: /Users/heckj/bin/vcpkg/installed/arm64-osx/lib/libcrypto.a is architecture: arm64
Prior to OpenSSL version 1.1.1i, that reported an x86_64 binary.
As I’ve learned, architecture alone isn’t sufficient for C++ code when linking the library (at least on macOS). When the libraries are created (compiled), the libraries are also marked with information about what platform they were built for. This is a little harder to dig out, and where the command line tool otool comes into play. I found some great detail on Apple’s Developer forum in the thread at https://developer.apple.com/forums/thread/662611, which describes using otool, but not quite all the detail. Here’s the quick summary:
You can view the platform that is embedded into the library code directly using otool -lv. Now this generates a lot of output, and you’re looking for some specific patterns. For example, the command
And as far as I’ve been able to discern, if you see LC_VERSION_MIN_IPHONEOS in the output, it means the library was built for an iOS platform, and you’ll get the linker error I listed above when you try to link it to code built for macOS.
Another library which did get compiled “correctly” shows the following stanza within its output:
Spotting LC_BUILD_VERSION and then the details following it shows the library can be linked against macOS code build for version 11.0 or later.
After a number of false starts and deeper digging, I found that OpenSSL 1.1.1i included a patch that enabled arm64 macOS compilation. The patch https://github.com/openssl/openssl/pull/12369 specifically enables a new platform code: darwin64-arm64-cc.
I grabbed OpenSSL from its source, and started poking around. A lot of that poking and learning the innards of how OpenSSL does its builds wasn’t entirely useful, so I won’t detail all the dead ends I attempted to follow. What I did learn, in the end, was that the terminal – being native arm64 or rosetta2 emulated x86_64 – can make a huge difference. For convince, I made a copy of iTerm and ran it under rosetta so that I could easily install homebrew and use all those various tools, even though it wasn’t arm64 native.
What I found is that if I want to make a macOS native version of OpenSSL, I need to run the compilation from a native arm64 terminal session. Something is inferring the target platform – not sure what – from the shell in which it runs. I manually configured and compiled OpenSSL with an arm64 native terminal, and was able to get an arm64 library, with the internal markers for macOS.
From there, I thought that perhaps this was an issue of how OpenSSL was configured and built. That’s something that vcpkg controls, with these little snippets of cmake under the covers. vcpkg does a nice job of keeping logs, so I went through the compilation for openssl (using --triplet arm64-osx in case you want to follow along), and grabbed all the configuration and compilation steps. I grabbed those logs and put them into their own text file, and edited them so I could run them step by step in my own terminal window and see the status of the output as it went. I then adapted the steps to reference a clean git repository checkout from openssl and the tag OpenSSL_1_1_1i, and updated the prefix to a separate directory (/Users/heckj/openssl_arm64-osx/debug) so I could inspect things without stepping into the vcpkg spaces.
What I found was the if I ran this code under a terminal running under rosetta, I’d get the results that indicated the code was built against an iOS platform. And when I ran it under a native arm64 terminal, it would correctly report macOS as the platform.
This is a major insight, but I haven’t yet figured out how to apply it…
I originally installed and compiled vcpkg using the rosetta terminal, and it was running as an x86_64 binary, so I thought perhaps that was the issue. Unfortunately, not. After I installed vcpkg with the arm64 native (and verified the binary was arm64 with the lipo -info command), I made another run at installing openssl, but ended up with the same iOS linked binary.
Prior to getting this far, I opened issue 13854 as a question on the OpenSSL repository, which details some of this story. However, I now longer think that’s an issue, as I was able to get an arm64 native binary when I manually compiled things. There might be something OpenSSL could do to make this easier/better, but its build setup is incredibly complex and I get lost pretty darn quickly within it.
So to date, I’ve trailed this back to some interaction that vcpkg, and x86_64 emulated binaries, are having on the build – but that’s it.
The story ends here, as I don’t have a solution. I have filed issue 15741 with the vcpkg project, with a summary of these details.
For anyone reading until the bitter end, I’d love any suggestions on how to fully resolve this. I hope that someone stumbles across the issue at some point with more knowledge than I and has a solution in the future. In the meantime, this blog post will hopefully record the error and how you can diagnose the architecture that a library is compiled for, even if it doesn’t solve the end problem of getting you to a final resolution.
A new year’s wander through a machine learning research paper led me down a really interesting rabbit role. Somewhere down the hole, I found a reference and have since been reading Thinking Fast and Slow, by Daniel Kahneman. The book was referenced by several AI/ML researchers, which is what got me started there.
Chapter 4, titled The Associated Machine, is an interesting example of this book. It talks about the psychological effect of priming, and then goes on to illustrate it in the book with simple examples you can practically do while you’re reading the chapter. That kind of example brings the concept home, makes it super concrete and far easier to understand. That’s a delightful mechanism to find in a book where you’re reading to learn.
That chapter, both its content and how it presented the topic, spurred a wacky question in my head:
Are there priming techniques that could be used in technical documentation, either subtly or directly, that makes it easier for the reader to read and retain the content?
I don’t know if there is such a thing. From the examples in the book, I can see where there’s definitely priming that could be done that could work against taking up details, but I’m not coming up an inverse. The closest I could imagine was a relatively simple predictive puzzle of some sort, just something to get you in a puzzle-solving frame of mind, and then tackling the learning. Of course, if you’re one that hates puzzles that’s is likely going to “crash and burn” as a technique.
I’ve no grand conclusions, only more questions at this stage. Although I do highly recommend the book if you’re interested in the processes of cognition, associating, and how they relate to our everyday brain capabilities.
I have been following the bare outlines of building, and using, machine learning models in Apple’s software ecosystem for a while. Most of my learning and personal research has been with foundational technologies – following some of the frameworks (TensorFlow, PyTorch, SciKit-Learn) and some of the advances in models and their results. Until this holiday, I had not applied myself to seeing the latest evolution of Apple’s machine learning frameworks and tooling. I scanned through websites last summer during Apple’s WWDC, but didn’t really clue in to the changes that were coming, and that are now available, with Big Sur (macOS 11).
Aside: The web site Papers With Code has been a great consolidated reference point for digging into the academic research behind quite a variety of machine learning advancements over the past couple of years, and includes both the research papers (often linked to ArXiv) and frequently references to the code matching the papers.
First off, the tooling to create simple models with CreateML has received quite a boost. Matched by the documentation, there are a large number of new models that can be easily generated with you “just” providing the data. Models to classify sounds, motion data, and tabular data – including regressors (predictors) as well as classifiers. Prior to macOS 11, they had models for classifying and tagging words and sentences, as well as image classification. Those all still exist, and Apple provides some ready-to-use models built-in to frameworks such as Natural Language.
The goal I chose to explore with was focused on natural language processing – more on that later. Apple has long had a pretty good natural language library available, the earlier stuff being a bit more focused on Latent Semantic Mapping, but recently turning to – and leveraging – quite a bit of the natural language processing advances that have happened using more recent machine learning techniques.
The most interesting win that I immediately saw was the effectiveness of using transfer learning with CreateML. When you’re making a model (at least the word tagger model), you have the option of using a CRF (conditional random field) or applying the data with a transfer learning over existing models that Apple has in place. They don’t really tell you anything about how these models are built, or what goes into choosing the internals, but the bare results from a few simple experiments are positive, and quite obviously so.
I made sample models that mapped language dependency mapping from some publicly available datasets provided by Universal Dependencies. Dependency maps specify how the words relate to each other, and the part that I was specifically interested in was leveraging the ability to identify a few of those specific relationships: nsubj:pass and aux:pass. These are the tell-tales for sentences that use passive voice. I did multiple passes at making and training models, and transfer learning was clearly more effective (for my models) in terms of the reported accuracy and evaluation.
Training a Model
One of my training runs used approximately 5000 labeled data points (from https://github.com/UniversalDependencies/UD_English-GUM). About 4200 of those points I allocated for training, and another separate file with 800 data points for testing. In this case, a “data point” is a full or partial sentence, with each word mapped to an appropriate dependency relationship. The CRF model topped out at 81% accuracy, while the transfer learning model reached up to 89% accuracy.
I ran these experiments on one of the new M1 MacBooks with 16GB of memory. This training set took just over a minute to train the CRF model, and 7 minutes to train the transfer learning model. A similar run with more data (27,000 data points) took 20 minutes for the CRF model, and 35 minutes for the transfer learning model. Transfer learning takes longer, but – in my cases – resulted in better accuracy for predictions.
Evaluating the Model
Once trained, CreateML provides an evaluation panel that gives you precision and recall values for each value that in your tagging set. This information is ideal for understanding how your data actually played out versus what you were trying to achieve. Using it, you can spot weak points and consider how to resolve them. One way might be gathering more exemplar data for those specific classifications. For example, in the following data table, you can see that the recall for “csubj:pass” was 0%. This showed that I simply didn’t have any examples in the test data to validate it, and possibly only a few samples in my training data. If that was a tag I was interested in making sure I could predict from input text – I could find and improve the input and testing data to improve that accuracy.
Previewing Predictions from your Model
Probably the most fun of using CreateML is the preview mode. Since I was making a word tagging model, it provided a text entry area and then applied the tagger against the data, showing me an example of the predictions against what-ever I typed.
Since I’d done two different models in the same project, I could flip back and forth between them to see how they fared against this sample, showing me the predictions from the model and the associated confidence values of those predictions. (and yes, I have considered locking my brother into a kitchen, he’s a damned fine chef!)
Exporting a CoreML model from CreateML
CreateML includes an Output tab that shows you the tags (also known as class labels) that you trained. It also gives you a preview of the metadata associated with the model, as well as kinds of inputs and outputs that the model supports. This makes it nicely clear on what you’ll need to send, and accept back, when you’re using the model in your own code.
One of the details that I particularly appreciated was including a metadata field explicitly for the license of the data. The data I used to create this model is public, but licensed to by-nc-sa-4.0 (An attribution, non-commercial, share-alike license). Sourcing data, both quality and licensed use, is a major element in making models. I think it’s super important to pass that kind of information along clearly and cleanly, so I’m glad it’s a default metadata attribute on models from CreateML.
Notes on CreateML’s Rough Edges
While CreateML was super easy to use and apply, it definitely has some issues. The first is that the user interface just doesn’t feel very “Mac” like – things I expected to be able to “click on and rename” didn’t smoothly operate as such (although I could control-click and choose rename), the window doesn’t easily or cleanly resize – so the app dominates your screen wether you want it to or not. On top of that, a queueing feature, and related window, for lining up a bunch of training was quite awkward to use and see the results as it was progressing, and after it was done. I had a couple training runs fail due to poor data, but the queuing window didn’t show any sort “well, that didn’t work” information – it just looked like it completed, but there was no training on those models.
The other awkward bit was dealing with messy data through the lens of CreateML. I can’t really say what it did was “wrong”, but it could have been so much better. In one of my experiments, the data I had chosen for training had issues within it: missing labels where the training system expected to see data. That’s cool – but the error was reported by a small bit of red text at the bottom of a screen saying “Entry # 5071 has a problem”. Finding entry #5071 of a structured JSON data set is, bluntly, a complete pain in the ass. When I’d parsed and assembled the data per the online documentation for making a word tagging model, I’d dumped the data into a monster JSON, single-line data structure with no line breaks. That made finding a specific element using a text editor really rough. In the end, I re-did my JSON export to include pretty-printed JSON, and then also used VSCode’s “json outline” functionality (scaled up, since it defaults to 5000 items), to track down the specific item by position in a list. I found the offending data, and then looking around, noticed a bunch of other areas where the same “partially tagged data” existed. In the end I dealt with it by filtered it out if it wasn’t fully tagged up. It would have been much nicer, especially since CreateML clearly already had and could access the data, if it could have shown me the samples that were an issue – and notified me that it wasn’t just one line, but that a number were screwed up.
The evaluation details after you train a model aren’t readily exportable. As far as I can tell, if you want to share that detail with someone else, you’re either stuck making a screenshot or transcribing what’s in the windows. It seems you can’t export the evaluation data into CSV or or an HTML table format, or really even copy text by entry.
The details about the training process, its training and validation accuracy, number of iterations used, are likewise locked into non-copyable values. In fact, most of the text fields feel sort of “UIKit” rather than “AppKit” – in that you can’t select and copy the details, only get an image with a screenshot. This is a “not very Mac-like” experience in my opinion. Hopefully that will get a bit of product-feature-love to encourage sharing and collaboration of the details around CreateML models.
I filed a few feedback notices with Apple for the truly egregious flaws, but I also expect they’re known, given how easy they were to spot with basic usage. They didn’t stop the app from being effective or useful, just unfortunate and kind of awkward against normal “Mac” expectations.
I do wish the details of what CreateML was doing behind the scenes was more transparent – a lot of what I’ve read about in prior research is starting with descriptions of ML models in PyTorch, Keras, or Tensorflow. If I wanted to use those, I’d need to re-create the models myself with the relevant frameworks, and then use the CoreML tools library to convert the trained model into a CoreML model that I could use. By their very nature, the models and details are available if you take that path. It’s hard to know, by comparison, how that compares to the models created with CreateML.
Creating a Model with CreateML Versus a Machine Learning Framework
My early take-away (I’m very definitely still learning) is that CreateML seems to offer a quick path to making models that are very direct, and one-stage only. If you want to make models that flow data through multiple transforms, combine multiple models, or provide more directed feedback to the models, you’ll need to step into the world of PyTorch, Keras, and Tensorflow to build and train your models. Then in the end convert the trained models back to CoreML models for use within Apple platform applications.
Where the raw frameworks expose (require you to define) all the details, they also inflict the “joy” of hyper-parameter tuning to train them effectively. That same tuning/choosing process is (I think) happening auto-magically when you build models with CreateML. CreateML chooses the iterations, while paying attention to convergence, iterations, and epochs of applying data. It also appears to do a good job of segmenting data for training, evaluation, and testing – all of which you’d need to wrangle yourself (and ideally not screw up) while making a raw machine learning model. That’s a pretty darned big win, even if it does end up being more of a “trust me, I know what I’m doing” rather than a “see, here’s what I did for you” kind of interaction.
Final Thoughts on Learning CreateML
I’m still very much in “the early days” of learning how to build and apply machine learning models, but CreateML has already been an immense help in learning both what’s possible, and providing hints as to how I might structure my own thinking about using and applying models within apps.
The first is simply that they provide a lot of good, basic models that are directly available to use and experiment with – assuming you can source the data, and that’s a big assumption. Data management is not an easy undertaking: including correctly managing the diversity of the sourcing, data licensing, and understanding the bias’ inherent within the data. But assuming you get all that, you can get good – mostly interactive – feedback from CreateML. It gives you a strong hint to answer the question “Will this concept work, or not?” pretty quickly.
The second is that showing you the outputs from the model makes the inputs you provide, and what you expect to get out, more clear. I think of it as providing a clear API for the model. Models expose a “here’s what the model thinks it’s likely to be – and how likely” kind of answer rather than a black-and-white answer with complete assurance. Not that the assurance is warranted with any other system, just that exposing the confidence is an important and significant thing.
If I were tackling a significant machine learning project, I’d definitely expect to need to include some data management tooling as a part of that project, either browser/web based or app based. It’s clear that viewing, managing, and diagnosing the data used to train machine learning models is critical.
I’m also looking forward to see what Apple releases in the future, especially considering the more complex ways machine learning is being used within Apple’s existing products. I imagine it to include more tooling, advances to existing tooling, and maybe some interesting visualization assistance to understand model efficacy. I would love for something related to data management tooling – although I suspect it’s nearly impossible to provide since everyone’s data is rather specific and quite different.
The new Apple Macs with an M1 chip in them is finishing a job that started a few years ago: changing my assumption that commodity hardware would always win. Having worked in the technology/computing field for over 30 years, you’d think I know better by now not to make such a broad assumption, even internally. For years, I thought the juggernaut of Intel/x86 was unstoppable, but now? Now I’m questioning that.
Apple has always prided itself on its deep integration of hardware and software. And sometimes they’ve even made good on it. The iOS (and iPadOS) integration has been impressive for the last fifteen years, and now they brought it to their laptop lineup, spectacularly so. It’s not just Apple doing this – Samsung and some other manufacturers have been down this road for a while, merging in computing silicon with sensors at a deep foundational level, changing what we think of as components within computing systems. Some of the deeply integrated cameras are effectively stand-alone systems in their own right. Lots of phones and digital cameras use that very-not-commodity component.
I still think there’s a notable benefit to leveraging commodity hardware – shoot, that’s what this whole “cloud” computing market is all about. But it’s also pretty clear that the guarantee that the win that commodity gives you won’t necessarily outweighs the commercial benefits of deep hardware/software integration.
One of the interesting things about the M1 system-on-a-chip isn’t the chip itself, but the philosophy that Apple’s embracing in making the chip. That pattern of behavior and thought goes way beyond what you can do with commodity stuff. The vertical integration allows seriously advanced capabilities. Commodity, on the other hand, tends to be sort of “locked down” and very resistant to change, even improvements. Then pile on top of that the tendency for these chip designs to be far more modular. They’re aggressively using coprocessors and investing in the infrastructure of how to make them work together. I think that’s the core behind the unified memory architecture. What Apple has done, or started to do, is invest in the ways to go even more parallel and make it easier for those co-processors to work together. In a commodity (Intel) system, the rough equivalent is the PCIe bus and the motherboard socket you plug cards into. Only that doesn’t solve the “how you share memory” or “who talks to who, and when” problems. In fact, it kind of makes it worse as you get more cards, sockets, or components.
I may be giving Apple’s hardware engineering team more credit than they’re due – but I don’t think so. I think the reason they talked about “Unified Memory Architecture” so much with this chip is that it IS a solution to the “how to get lots of co-processors to work together”, while not exploding the amount of power that a system consumes while doing so. (If you’re not familiar, memory is a “sunuvabitch” when it comes to power consumption – one of the biggest sinks for power in a modern PC. The only thing that’s worse than memory are ethernet network ports, which was a serious eye-opener for me back in the day.)
There are other computing changes that aren’t trumpeted around so much that are going to contribute equally to the future of computing. I was introduced to the world of NVMe (Non-volatile memory) a few years back, when it was just hitting it’s first iteration of commercial introduction. The “holy crap” moment was realizing that the speed at which it operated was equivalent to those power hungry memory chips. Add that into the mix, and you’ve got lots of compute, persistent memory, and far lower power requirements in the future for a heck of lot of computing. It’s the opposite direction that Intel, and nVidia, are charging into – assuming they’ll have power to spare, and can happily burn the watts to provide the benefits. Truth be told, only individual consumers were still really oriented that way – the cloud providers, over a decade ago, had already clued in that the two most expensive things in running cloud services were 1) power and 2) people.
Bringing this back to the M1 chip, it’s tackling the power component of this all very impressively. I’m writing this on an M1 MacBook, and loving the speed and responsiveness. Honestly, I haven’t felt a “jump” in perceived speed and responsiveness to a computer with a generational gap like this in over 15 years. And that’s WITH the massive power reduction while providing it.
I do wish Apple was a little more open and better about providing tooling and controls to help solve the “cost of people” equation of this situation. Yes, I know there’s MDM and such, but comparatively it’s a pain in the butt to use, and that’s the problem. It needs to be simpler, more straightforward, and easier – but I suspect that Apple doesn’t really give much of a crap about that. Maybe I’m wrong here, but the tooling to make it really, really easy to combine sets of systems together hasn’t been their strong point. At the same time, groupings and clusters of compute is just the larger reflection of what’s happening at the chip level with the M1 SOCs (For what it’s worth: SOC stands for “system on a chip”).
I think you can see some hints they might be considering more support here – talking about their AWS business interactions a bit more, and on the software side the focus on “swift on the server” and the number of infrastructural open-source pieces they help lay out over the past year that are specific to capturing logging, metrics, tracing, or helping to dynamically create clusters of systems.
I think there’a s lot to look forward to, and I’m super interested to see how this current pattern plays over of this coming year (and the next few). Like many others, I’ve raised my expectations for what’s coming. And I think those expectations will be met with future technology building on top of these M1 chips.
This has been a right 🤬 of a year, and that probably applies to most anyone else on this globe. In the US, the stress of this presidential election was extreme, and acerbated by COVID pandemic that we pretty much failed to get any sort of handle on. I’ve managed to stay tucked down and safe, but my day to day life from a year ago is completely different now. My favorite “office” coffeeshop, El Diablo, is now gone – 20 years after it started. The loss has an oversized impact on my because it was also my social hub connecting me to friends in my community. Like a lot of others, my world is currently collapsed down to a pretty tiny bubble.
The downsides of this year are undeniable and clear, but there have been a number of silver linings, and since we’re heading into Thanksgiving, it seemed a good time to reflect and celebrate the positives for the year. It doesn’t remove the horror and pain, but to me – it helps offset it.
Right as the pandemic was sweeping up, I managed to get a longer term contract gig that’s been really useful for me, working on technical writing. I’ve been programming, and managing programmers, devops, QA, and the whole software lifecycle kit for multiple decades, and one of the skills that I’ve been trying to cultivate has been communication – specifically writing. Last year around this time, I self-published what was primarily a labor of love – Using Combine – a book/reference doc combination on Apple’s Combine framework. The year before I’d published a different book through Packt, Kubernetes for Developers (A google search on that phrase now directs you to more of offline training and certifications, but there’s a book in there too!). I’d been searching for editors to work with, that really dug into the work to help me structure it – not just the fluffy top level stuff that so a number of publishers stick to.
It’s the combination of these things that ends up being what tops my give-thanks list this year. In the past eight months, I’ve had a chance to work closely with a number of truly amazing editors, helping to refine and improve my writing skills. While I’m still crappy at spotting passive voice in my own writing, the feedback I’ve gotten from Chuck, Joni, Susan, Colleen, Mary Kate, and Paul has been amazing. Everything from basic grammar (that I had just never really did well) to structure, narrative flow, complexity. Top that off with some wonderful writers, Ben, Liz, Dave, and Joanna, willing to answer the odd question or just chat about the latest garden recipes on a video call, and really these past months of contracting have been a terrific experience.
I was super fortunate to find a gig when everyone else seemed to losing them. I’m sure the troubles aren’t even close to over – there’s so much to rebuild – but it’s a bit of light against the backdrop of this year.
Just before last weekend, the folks on the Swift Core Team provided what I took to be a truly wonderful gift: a roadmap and series of proposals that outline a future of embedding concurrency primitives deeper into the swift language itself. If you’re interested in the details of programming language concurrency, it’s a good read. The series of pitches and proposals:
If you have questions, scan through each of the pitches (which all link into the forums), and you’ll see a great deal of conversation there – some of which may have your answer (other parts of which, at least if you’re like me, may just leave you more confused), but most importantly the core team is clearly willing to answer questions and explore the options and choices.
When I first got involved with the swift programming language, I was already using some of these kinds of concurrency constructs in other languages – and it seemed to be a glaring lack of the language that they didn’t specify, instead relying on the Objective-C runtime and in particular the dispatch libraries in that runtime. The dispatch stuff, however, is darned solid – battle honed as it were. You can still abuse it into poor performance, but it worked solidly, so while it kind of rankled, it made sense with the thinking of “do the things you need to do now, and pick your fights carefully” in order to make progress. Since then time (and the language) has advanced significantly, refined out quite a bit, and I’m very pleased to see formal concurrency concepts getting added to the language.
A lot of folks using Combine have reached for it, looking for the closest thing they can find to Futures and the pattern of linking futures together, explicitly managing the flow of asynchronous updates. While it is something Combine does, Combine is quite a bit more – and a much higher level library, than the low level concurrency constructs that are being proposed.
First up, pitches and proposals such as these are often made when there’s something at least partially working, that an individual or three have been experimenting with. But they aren’t fully baked, nor are they going to magically appear in the language tomorrow. Whatever comes of the proposals, you should expect it’ll be six months minimum before they appear in any seriously usable form, and quite possibly longer. This is tricky, detailed stuff – and the team is excellent with it, but there’s still a lot of moving parts to manage with this.
Second, where there’s some high level conceptual overlap, they’re very different things. Combine, being a higher level library and abstraction, I expect will take advantage of the the lower-level constructs with updates, very likely ones that we’ll never see exposed in their API, to make the operators more efficient. The capabilities that are being pitched for language-level actors (don’t confuse that with a higher level actor or distributed-actor library – such as Akka or Orleans) may offer some really interesting capabilities for Combine to deal with it’s queue/runloop hopping mechanisms more securely and clearly.
Finally, I think when this does come into full existence, I hope the existing promise libraries that are used in Swift (PromiseKit, Google’s promises, or Khanlou’s promise) start to leverage the async constructs into their API structure – giving a clear path to use for people wanting a single result processed through a series of asynchronous functions. You can use Combine for that, but it is really aimed at being a library that deals with a whole series or stream of values, rather than a single value, transformed over time.
The async and concurrency proposals are goodness, not replacement, for Combine – likely to provide new layers that Combine can integrate and build upon to make itself more efficient, and easier to use.
This post is specific to Swift the programming language and SwiftUI, Apple’s newest multi-platform UI framework. If you’re not interested in both, probably best to skip past this…
I was working on visualization code that leverages SwiftUI to see how that might work, and ran into a few interesting tidbits: playgrounds with SwiftUI works brilliantly, right up until it doesn’t and then you’re in a deep, dark hole of WTF?!, and the SwiftUI layout engine, especially with nested and semi-complex container views, do some sort of iterative solver technique. The less interesting tidbit is a classic aphorism: It’s the things you think you know that aren’t true that bite you.
When you’re fiddling with adding in your own constraints in SwiftUI, you might be tempted to use the range operator – at least my thinking was “Oh hey, this looks like a super convenient way to check to make sure this value is within an expected range”. It works stunningly well for me, as long as I’m careful about creating it. I started creating ranges on the fly from variable values, and that’s where playgrounds, and my bad assumptions, bit me.
If you create a range that’s ludicrous, Swift can throw an exception at runtime. So if you’re working with ranges, you’ve got the possibility that passing in a blatantly incorrect value will give you a non-sensical range, and that will result in a crashing exception. When you stumble into this using Playgrounds, you get a crash that doesn’t really tell you much of anything. When you kick that same thing up in Xcode, it still crashes (of course), but at least the debugger will drop into place and show you what you did that was wrong. I love using SwiftUI with Playgrounds, but the lack of runtime feedback when I hit an exception – about what I screwed up – makes it significantly less useful to me.
And debugging this in Xcode was where I learned that closures you provide within SwiftUI layout, such as alignmentGuide or a method of your own creation working with a GeometryReader don’t get called just once. Sometimes they’re called one, but other times they are called repeatedly, and with pretty strange values for the view’s dimension. I think underneath the covers, there’s an iterative layout solver that’s trying out a variety of layout options for the view that’s being created. Sometimes those closures would be invoked once, other times repeatedly – and in some cases repeatedly with the same values. Interestingly, sometimes those values included a ViewDimension or GeometryProxy with a size.width of 0. The bad assumption I made was that it would be sized quite a bit larger, never zero. Because of that, I attempted to build an incorrect range – effectively ClosedRange(x ... x-1) – which caused the exception.
Even with my own assumptions biting me, I like the use of range and I’m trying to use it in an experimental API surface. Lord knows what’ll come of the experiment, but the basics are bearing some fruit. I have a bit of code where I’ve been porting some of the concepts, such as scale and tick, from D3 to use within SwiftUI.
I participated in NaNoWriMo once before, in 2016 – having watched my partner get more deeply involved in previous years. I haven’t really been back, but this year with all the … yeah, I decided to give it a shot again. I’ve got my profile set up, at least the basics, and now I’m stumbling around trying to figure out what my story is going to encompass.
Last time through, I made a great start, but didn’t have much of a plan, so it fizzled out pretty hard about half-way through. So this time, I thought I’d try something a bit different and do a little planning. Last weekend, the local NaNoWriMo group hosted a session for planning and plotting, which I’d not tried previously. I’m going to try out using a beat sheet this time, to provide some constraints and structure for the overall story arc.
Earlier this week, James Dempsey asked on twitter about who else was actively trying to build macOS apps using SwiftUI. I’m super interested in SwiftUI. A year ago, it spawned my own side-project into writing my own reference docs on Combine. Originally I had a vision of writing about Combine as well as SwiftUI. Combine alone was hugely, so I stopped there with the notes, especially as SwiftUI is still massively maturing. Some pretty amazing updates came just earlier this year. While clearly not finished, likely not even close to finished, it’s now far enough along in it’s maturity that you can at least consider using it for full apps. Or parts of your app if you like – macOS, iOS, watchOS, or tvOS.
While I’m been keeping track of the framework, I’ve also been keeping track of people who are using it, writing about it, struggling with it, etc. There’s two implementations of full applications, all open source (and hence completely visible), that I’ve been following as super interesting examples of using SwiftUI: NetNewsWire and ControlRoom.
I’ve contributed a bit to NetNewsWire, only a tiny amount (and mostly around continuous integration), but I’ve been using it since the earliest days and from it’s original inception and through multiple owners to it’s current state as an open-source project that Brent Simmons is leading. The code is available online, ever evolving, at https://github.com/Ranchero-Software/NetNewsWire. The recent work to embrace SwiftUI is on it’s main branch with a lot of the SwiftUI code under the directory multiplatform/shared. Take a deep dive and dig around – there’s some gems and interesting questions, and you can see some really fascinating examples of integrating SwiftUI and UIKit or AppKit where SwiftUI isn’t quite up to some of the tasks desired by the project.
The other app I’ve been watching is ControlRoom, an app that Paul Hudson referenced in a demo capture on twitter. ControlRoom’s code is on Github at https://github.com/twostraws/ControlRoom, released earlier in SwiftUI’s lifecycle, and showing an integration not of the new SwiftUI app architecture pieces, but more of “classic” macOS AppKit integration. Like NetNewsWire, I found a number of really great gems within the code, often having “light-bulb” moments when I understood how the app accomplished some of its goals.
There are easily other apps out there, that I’m unaware of – but not too many folks are openly sharing their development like the two projects above. I did find a list of open-source IOS apps on GitHub that includes a subset listing SwiftUI, that might be interesting.
I have a few of my own experiments, but nothing as polished and effective as these two, and I don’t think I solve any problems in novel ways that they haven’t. In a bit of test and benchmarking code, I was creating SwiftUI interfaces across macOS, iOS, and tvOS – which turns out to be a right pain in the butt, even for the simplest displays.
I hope, but don’t expect, more apps to become available – or to be more visible down the road. Having the open sharing of how they solved problems is invaluable to me for learning, and even more so for sharing. Apple has their sample code, well – some of it anyway – but seeing folks outside of Apple use the framework “in the wild” really shows it’s working (or where it isn’t).