Second Guessing Yourself

I’m working through the book: Crafting Interpreters. The author — Bob Nystrom — used Java for the first half of the book, a lovely choice, but I wanted to try it using the Swift programming language. Translating it on the fly was a means to exercise my programming and language knowledge muscles, and that does come with challenges. Some translation choices I made worked out well, others have helpfully provided a good learning experience. This post is about happened was while I was hunting, and resolving, programming mistakes this past week.

Recursive functions have long been something that I found difficult. They are a staple of some computer science tasks, and the heart of the techniques that are used in scanning, parsing, and interpreting as shown of this book. It was because I found them difficult that I wanted to do this project; to build on my experience using these techniques.

It shouldn’t surprise you that I made mistakes while translating from Java examples into Swift. There wasn’t anything super-esoteric at the start, a little miss here or there. My first run through implementing a while loop didn’t actually loop (oops). The one that really frustrated me was a flaw in not properly isolating how I set up environments before invoking enclosing blocks or functions, which meant a little Fibonacci example blew up – values oscillating instead of growing. It was kind of my worst nightmare, interpreting recursive code within a bunch of compiled recursive code. I made similar errors in the parser, small subtle things, that meant some of the examples wouldn’t parse.

Testing and Debugging

So I backed up, and started writing tests to work the the pieces: scanner, parser, and interpreter. I probably should have built in testing from the start. I think more importantly — I did it when I needed it. The testing provided a framework to repeatedly work sections of code so that I could debug them and understand what was happening. I started out using the debugger in Xcode to work through the issues, but the ten-plus layers of recursively called functions exploded beyond my head’s capacity to track where I was and what was going on.

I reverted back to more direct tools. I annotated my parser code (and later the interpreter) a flag variable named omgVerbose, printing out each function layer as I stepped into it. With that in place, I could run the test, view the (often quite long) trace, and walk it through for the example. I traced my code in an editor and compared what I thought it should do to what showed in the trace. I even added a little indention mechanism, so that every layer deeper in the recursive parsing would show up as indented. It was what you might call “ghetto”, but it got the job done.

Do I really know what I know?

The second guessing hit me while I was debugging the interpreter. It was the worst kind of bug; the kind where the test sample seems to work — it doesn’t crash or throw an exception — but you get the wrong output. The code snippet was advanced, working the recursive nature of this language I was implementing:

fun fib(n) {
    if (n <= 1) return n;
    return fib(n - 2) + fib(n - 1);
}
for (var i = 0; i < 20; i = i + 1) {
    print fib(i);
}

I added my omgVerbose variable to show me the execution flow within the interpreter. The output from that short little function was incredibly overwhelming. That was the point at which the metaphorical light-bulb 💡 went off for me, making it clear and immediately obvious WHY people created debuggers in the first place. The printed execution trace was overwhelming, and a debugger could let me step through it, piece by piece, and inspect. I didn’t think I was up for making a debugger for my interpreter right then though, so I just fortified myself with a long walk to clear my head, some fresh coffee, and tucked in to trace it through.

I read through the trace, scrolling through my editor with the underlying code, seeing what was executed and what happened next, and trying to nail down where the failure was occurring. While going back to my implementation in Swift, I stared at it so long that the words lost meaning. I struggled so much with finding this error, with my own frustration for not understanding why it was failing, that I started to second guess how even the Swift language was working. For at least an hour, I’d managed to confuse myself with exceptions and optionals. At one point, I’d nearly convinced myself that try before a statement implied that the statement returned an optional value (which it very definitely does NOT). I did ultimately find the issue — it had nothing to do with either optional values or error handling. I had made a mistake in scoping when invoking a function within my interpreter – effects were bleeding between function invocations, but only showing when the same function and parameters were called from the same context. The recursive function I was testing happened to highlight this extremely well.

The point isn’t the flaw I made, but that anyone can get to second guessing themselves. I’ve been programming in one language or another for over 30 years, and programming using the Swift language for over five years. I am not the most amazing programming I know, but I’m comfortably competent. Comfortable enough to ship apps written in it, or reach for the Swift language to explore a problem without worrying about having to learn it first. Even still, I second guessed myself.

Recovering From Brain Lock

I suspect it may be more common to encounter this situation when you’re trying to learn something new. I wasn’t using this to learn Swift, but to learn techniques about how to build a programming language and interpreter. It is puzzle solving – exploring a space, making some assumptions, and following the path of where that might go. When you find a mistake, backtrack a bit and take another path. You’re already in the mode where you’re questioning your assumptions, so it’s pretty easy to have that slip back to questioning broader or more baseline knowledge.

This is when it’s more important than ever to step back and “get out of your head”. The good ole ‘rubber ducky’ approach isn’t bad: explain what you think is happening to the rubber duck. Saying it aloud externalizes it enough to illuminate a poor, or mistaken, assumption or prediction. Same thing goes when explaining it to another person – typing, calling, or whatever – better even if they have a different perspective from yours and can ask questions you’re not.

Just as importantly, step away from the problem for a bit. It doesn’t get easier by just pushing forward when you’re frustrated and/or confused, it gets harder. Step outside and go for a walk, away from the keyboard and problem. Listen to music, watch a show you enjoy, or the go for the best recovery mechanism of them all: get a good night’s sleep before coming back to the problem if you can.