The growable array type std::vector<T> is least impacted by these archaic choices out of the tools in the box you're likely to reach for. So it will make sense very often to choose this type first.
For example, since I allowed for objects to be shared between threads, I decided to use struct of arrays so the reference count, metadata, and value would be stored in separate cache lines. This ended up hurting me because object initialization touched three separate cache lines (obvious in hindsight, but the advice of using SoA failed me here). I also heard that you want to pack your values as tight as possible, so I used a packed string index, but then I ended up with integer division to unpack the string (also a mistake, but again the advice failed me). I used a custom allocator to avoid indirection with lists (list items were allocated directly after the list head), but then I had heap fragmentation and the implementation complexity exploded.
Anyways, I am now happily using two to three levels of indirection in my data structures, large structs, and malloc for individual objects, and it's still been faster in my end to end testing. So maybe this is unique to interpreters, and maybe I could have done it better, but the suggestions don't automatically apply in my experience.
I mean this is kind of what happens with any advice that has nuance to it, that's not carried with the advice.
E.g. if you have a point in 3D space with x, y, z coordinates. Array points as SoA of individual dimensions makes sense only if you do a lot of averaging and such on the individual dimensions.
If you mostly use the 3 coordinates together, SoA will have bad caching behavior.
So the better advice would be to try to keep things that are used together in the same cache line, whether it's on dimension or all 3. Usage makes the difference.
Beyond the low-hanging fruit like ensuring you aren't creating O(n^2) complexity by accident, I think C++ is fast enough/has mature-enough compilers that by the time you're worrying about cache hits materially affecting performance, you're probably also sufficiently staffed and capitalized to pay people to A/B test that performance.
2. Virtuals are, with the exception of PGO, mostly a black box i.e. you get a hard optimisation boundary, no inlining at all.
3. The C++ standard library is usually comically slow (yes, even compared to Java/C#/the likes) so if your project uses std::vector and the such instead of specialised libraries, you've already lost at the beginning.
4. If you don't pay attention to performance from the get-go, the approximate amount of autovectorisation you'll get is close to zero. Some compilers are better than others (Clang>MSVC for example) but I've seen codebases with 8 figures of LoC where the number of vectorised divides/multiplys was like less than ten when you dumped the object listing. In the whole program.
5. Since aliasing and other optimisation barriers (you didn't use restrict or manually hoist, did ya?), it's not uncommon for large C++ programs to spend a third of their runtime doing atomic increments because shared_ptr is supposedly cheap and who cares about lifetimes anyway.
6. If you're targeting Windows, the default new operator / malloc is also comically slow. Luckily that one is fairly easy to fix with installing mimalloc and deploying the hijack dll, but the negative effects on cache by the fragmented allocations is also significant.
Surely nobody outside microsoft is doing serious work targeting Windows any more are they? Isn't that a dead platform? I read somewhere a while back they're now below 60% market share.
I work in game development and for the last six years I've spent most of my time specifically on optimization. A lot of that effort has been focused on cache behaviors. Not because it's fun, but because it's often the difference between being able to ship the game on weaker hardware (e.g. Nintendo Switch) or not.
As a contrived example: there are specific cases when a particular non-quicksort algorithm is optimal. In almost all real world scenarios, though, you're just going to say fuck it and use quicksort until profiling determines that the sort is the bottleneck.
Unless you already have specific knowledge that your data comes in a particular shape, defaulting to quicksort is good design (IMHO). Worrying about pathological sorting before you've seen benchmarks is premature optimization.
I think it's more like: prioritize cache locality over big O compexity.
Run your own benchmarks on your own data of course. Also map is not considered the best key value store.
This likely won't be true in a real application with a non-trivial allocation pattern.
Still a custom map that allocated a bunch of nodes would be a useful optimization.
And then, on the other hand - I really doubt GP's map beats a vector, with all of those pointers bins and stuff, in a non-contrived benchmark with 10 elements.
Finally - it's not either-or: There are better hash maps whose memory is sequentially allocated and/or are otherwise cache-aware. And there are data structures geared towards parallel execution on multiple threads; and towards SIMD; etc. etc.
If you’re down to that sort of decision-making, you have to measure.
I feel like the DoD movement is a slow-moving, but big, change through how systems programming is done, but that there's still insufficient material for how to do this in different scenarios. I would really like to apply this more to my areas of work, which are also in C++, but there seems to be a gap between what they're presenting and how it can be applied.
More specifically, I'm using C++ to build a dynamic programming language runtime for a Clojure dialect. That runtime is required to be garbage collected, type-erased, and highly polymorphic. So I surely can't just SoA or AoS everything. Yes, I can pack my data, and I can avoid the GC whenever possible, both in compiler/runtime code and in generated code via escape analysis. But what about everything else, which is the 80% or more of the system? It could be that this runtime is too far at odds with DoD, but I generally see things as a gradient rather than black and white.
Or were you referring more to all the intermediate allocations that aren't the object heap? V8's zones are interesting in this area, because they're like an arena, except that they're only partially reset when a zone ends, so zones can nest inside each other.
High-performance programming is a big topic. The scope is far too broad for a single blog post, which naturally gives only cursory discussion of C++ and computer architecture. The article isn't bad considering, but I do think it's the wrong format. A blog series, or even a book, would be more fitting.
What you've written mostly makes sense to someone who already has a solid understanding of SIMD and of C++ (although I can't say I follow all of it), but the target audience is people who don't. For them, each point needs a much lengthier explanation.
A quick restrict example:
#define fn __attribute__((used))
fn void copy1(int* to, const int* from, const int size)
{
for(int i = 0; i < size; i++)
to[i] = from[i];
}
fn void copy2(int* to, const int* from)
{
constexpr int size = 1024;
for(int i = 0; i < size; i++)
to[i] = from[i];
}
fn void copy3(int* restrict to, const int* restrict from)
{
constexpr int size = 1024;
for(int i = 0; i < size; i++)
to[i] = from[i];
}
gcc test.c -c -O3 && objdump -d ./test.o
This is a good starting point for self teaching. The impact of your TLB, L1, and overall instruction count (with IPC) can further be measured with `./perf stat -d -d -d ./a.out`. If you want a quick rule of thumb, no instructions are fast instructions.
creata's comment [0] mentions the works of Agner Fog, which seem very good, and are freely available.
I haven't read C++ High Performance [1] but it looks like it covers the sorts of topics you'd expect, although it looks like it doesn't cover computer architecture in detail e.g. branch prediction. There are books on that too, of course.
[0] https://news.ycombinator.com/item?id=49868657
[1] https://www.packtpub.com/en-us/product/c-high-performance-97...
If your device has enough resources to power V8, modern GUIs are certainly very pleasant and snappier than a more minimal GUI like HN. Otherwise they are horrendous and very laggy.
QtQuick/QML/JS is very pleasant and I do wish more people would use it, but from I've seen it's 25-50% the resource use of electron, not some multi-order-of-maginute improvement, do I understand why many people still prefer electron for portability in this case.
Even if you write them in hand-optimized assembly they would still clamor for more speed.
Note that we started our project before Rust was an option. These days I would certainly look at rust to see if that would cover our 5% of the needs but now we have a lot of C++ and mixing rust with C++ is a pain.
It is less pain than for most other languages, except for C. The pain is in exposing a C API for your C++ code. Then you build a library and you're set - because basically every language has the ability to call C code. Python, Rust, Java, etc. etc.
The painful part is to have to go through a C API (modern languages can express much richer APIs and of course there are different constraints on the different runtimes, e.g. GC).
The annoying part is that each language adds overhead (its runtime). I wouldn't call it painful (I don't have much to do about it), I say "annoying" just because I would rather minimise the amount of code I ship.
Similarly I like to do video stuff in C just because I call gstreamer/ffmpeg directly in C, rather than having to bridge everything.
The next step of going SOA benefits from all of the above, it just further unlocks you packed quad and oct instructions (AVX256 and 512 depending if you buy AMD or not).
GCC 11 (2021) std::visit was slower than virtual dispatch.
GCC 12 (2022) optimized std::visit so it can be faster than virtual dispatch.
https://shubhankar-gambhir.github.io/posts/your-stdlib-imple...
https://devblogs.microsoft.com/oldnewthing/20060731-15/?p=30...
https://learn.microsoft.com/en-us/archive/blogs/ricom/perfor...
https://web.archive.org/web/20250201145327/https://users.ece...
The kicker is, in my case I chose C++ because templates allow me to reuse most of the code in the rendering pipeline _regardless_ of whether I go for AoS or SoA layout. I leverage operator overloading to do vector by matrix multiplication which is implemented in both variants. I do have to specify the desired variant during building, but I've profiled and for Intel x86 AVX in my case SoA is something like twice as efficient because I process ("shade") 8 vertices with 4-5 instructions instead of 1 vertex at a time (still shaded with vectorisation -- just "rotated", i.e in the pipeline axis and not vertex buffer axis).
TL;DR; C++ gives you plenty fast by default, but it's not always enough. The difference between 5 and 15 frames per second, well, makes all the difference -- our eyes are only fooled once the frames-per-second rate goes sufficiently up, anything below an acceptable threshold and it's completely different experience. You then either sacrifice resolution or level of detail etc, or decide to squeeze more from the language by helping the compiler.
https://youtu.be/jsdwRf3JvZM?si=0tysoCsvaWl0R_PZ
40’000 NPC in game with collision avoidance, steering, on 8yo hardware.
In (soft) realtime audio programming, your audio callback might only have a time budget of 1.3 milliseconds. Everytime you exceed that limit, you'll hear a dropout. That's when you'll start to optimize the hell out of your program :)
How 100Gbit NICs could your filter through your stateful firewall at line speed? And with 64 byte packets?
So many complex, esoteric, and difficult to maintain incantations that used to be required for efficient code generation are no longer necessary.
I find it crazy that popular systems languages didn’t have an explicitly valid way to deal with all of these ambiguous ownership and lifetime issues around memory until relatively recently.
Or take constexpr - it permits to move computations to compile time that are complex and in older versions either had to be done at runtime, or an ugly workaround had to be used (e.g. assigning a mysterious literal pre-computed in another run or by hand).
What C++23 feature allows that?
[0]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p27...
[1]: https://herbsutter.com/2025/11/10/trip-report-november-2025-...
There are so many things that are expressible in C++ now that could not be without writing much more code or using per-compilation tools back then. The ability to run code at compile time that is not run at runtime is huge, #embed lets us make other tools output available without linker scripts or compiler specific tools that.
Also, most of the code from the past still works(from 10 years ago definitely works)
On one hand, you have consteval and stuff, letting you FINALLY initialize data at compile time (hey, 20 years late but still!)
on other hand, it is done in most non-debuggable way possible. try setting breakpoint or adding print to constexpr function that causes your requires clause to fail...
so no, newer C++ the language is not possible to use for low level work. The dialects that compiler makers support are. We will see for how long
"Premature optimization is the root of all evil"
Actually, we don't know that. The meaning of volatile is rather subtle
So, if you are thinking about the sort of “business logic” that’s often Python, but the performance comes from the parts that are usually not.