Not perfect.

A blog about overcoming perfectionism and other stuff

My (mis)adventures in calling C++ code from Go

2019-07-26 23:30

Recently I've been tasked with moving some parts of an internal client-only desktop app into a server, for the purpose of having the inputs and the completed work stored in one centralized place, decouple processing data from displaying it, introduce action history and access rights, all that good stuff.

Now, it should be noted that the program in question performs quite a lot of signal processing, so C++ was my language of choice for it — quite an obvoius one.

However, when deciding what language to use for the server I really didn't want to use C++, and Go came to my mind as an easy answer: some people describe it as compiled Python, and I think they are kinda right. It has simple syntax and all the batteries needed for quick development. Also, it would be much easier to find a Go developer, if the need comes, to read and improve this code. I'm not into that kind of job security.

So! I've spent about a week learning the basics of Go and writing some simple stuff with it, preparing myself for what I've set out to do.

Also, been thinking whether I should finally learn Rust and use it instead.

Everything was kinda okay, apart from some frowning at those huge Go binary sizes. Whatever.

And then I stumble upon Go performance benchmarks. Welp…

"Not the right time to give up! What's Go's way of FFI?" I thought, so here comes Cgo. Looks kinda weird, but okay, maybe it'll work. I'm spending some more days reading up on the intricacies of Cgo, discovering not that much information honestly: it's all bits and pieces all over the Stack Overflow, but eh, somewhat manageable.

So I've decided to write a post that's called "Calling C++ code from Go", outline it a little bit, give up for a couple of days (that's just how it is), and then find this post in my RSS feed: Passing callbacks and pointers to Cgo.

Wow, I thought, I like Eli's articles, and now he is stealing my idea! Dammit. :D

And then he drops another one: Faster XML stream processing in Go, form which I discover that the overhead of calling C functions from Go is just too damn high. In Eli's case, it's like 4-5x difference in processing time.

And I need to call a lot of C functions to process a huge file as well. Oh no. Don't really want to make the app four times slower for nothing. Developer speed is important, but not that important. I need the app to perform the same way it did when it was just a desktop app. And this is not a problem you could throw more servers at to make it perform better (it's a useless approach most of the times anyway) — it needs to read a huge file in chunks, perform some DSP on those chunks then give the results. Change the parameters in real time and observe recalculated results as soon as possible. Nice task for a nice multi-threaded CPU: more servers would only make it more difficult for no gain.

This tweet from Jonathan Blow was also a good reminder and finally killed my motivation in writing this server in Go.

Now I'm writing that server in C++. How hecking funny. Wish me luck.

Discuss this post on Twitter