When I bought my 64 MB Duo they were $3!
Then for a long time they were $5 for the 64 MB, $7 for the 256 MB, and $10 for the 512 MB.
Sadly, like everything else, they've gone up considerably this year.
Note that the application-processor 1 GHz C906 and Arm A53 are either/or, you can't run both at the same time — similar to the Arm M33 and RISC-V Hazard3 core pairs on the Raspberry Pi RP2350.
And I hope it stays that way, I don't want MCU shortages...
I'll give a specific example apropos of TFA. Computer vision models were never run on MCUs because they were constrained to the point of being useless for this task, but then someone tried the impractical, and now it's trivial[1]
Regarding MCU shortages, you should be worried about the supply chain, but I don't see the impact really being from running LLMs on ESP-32s, of all things.
I think most of my embedded projects aren't that useful, but they've taught me a lot.
In my book, impractical means "I built a cuckoo wristwatch". Beyond impractical: "I built a cuckoo wristwatch but there was no room for a working mechanism".
I enter most of my learning project from the assumption that I could just buy/install whatever I am building and save time/money.
Please consider seeing things from a different perspective than "your book".
Not saying this to antagonise you, but because a lot of time reading comments like this makes other posters less willing to share their impractical efforts, and would love to read more of those, not less.
You're just extremely biased in what you consider to be a useful ML model. For example, for some strange reason you think only LLMs exist. The model must be as big as possible or else it is pointless.
Training custom non-LLM models for specific tasks so they run on a resource constrained device? You must be insane.
It's almost never worth it to buy the cheapest chip unless you're making a million of something, but there are very good ones around $1-$2, and $5 is the upscale stuff.
Used in the cheapest devices you wouldn't even imagine to contain any software - eg. To control a touch sensitive on-off switch.
Iirc an important caveat was that it was a one-time programmable (OTP) part. So you buy a bunch of them, programming failure or firmware-under-test doesn't work? -> toss the part. Of course that isn't an issue for a $0.03 part. But it can be an issue in terms of a board you want it on. Either that means discarding (breakout) boards too, or for development you'd need some kind of adapter to put bare ICs in.
Such annoyances only make sense for high-volume, low cost applications. Which is eactly where parts like that go in.
it's fun reading all the reactions to it (like "it’s cheaper to program a Padauk PMS150C to be a logic-level converter than to just buy a logic level converter"), but the magic is gone more-or-less
but some ideas inside it and padauk's ecosystem as a whole seem impressive https://jaycarlson.net/2019/09/06/whats-up-with-these-3-cent...
edit: PMS152 seems to still be 0.03$ !
Heck, that one would have no issue running Xerox PARC OSes.
Naturally there are still use cases were a PIC with 4KB would be the best option, but lets not behave as if there isn't anything better in most cases.
What are you going to do with 120 of them?
The arm experiment ran its course though; the power one vendor had in the marketplace started to be abused for the benefit of the IP holder and detriment to others. Now RISC-V is going one step further with a completely open ISA and also completely open designs. This is an excellent development in the nick of time.
I wonder how Apple feels about arm64 and RISC-V now. They could have probably bought ARM at any point but maybe never considered it to avoid anti-monopoly blowback.
Also you can reach out to ESP32 as alternative, plenty of maker projects using them.
Imagine a world where your toothbrush could give you tips about dental hygiene - or advertise toothpaste. What a time to be alive!
They already can. Philips' higher end models have Bluetooth connections and provide feedback through the app.
> advertise toothpaste
God please no. I don't want to have to look for an adblocker for a toothbrush AI.
PSRAM, Flash, and even SD cards may not have the best bandwidth individually, but they can reach quite impressive rates when you have a shitton of them running all at the same time.
The large scale dedicated hardware systems will still have the edge for performance per watt, but the low entry level and slow incline does make these things quite appealing.
I can see that being cheaper to bitbang with PIO than to actually compute.
There's certainly some latency stack up, but throughput should be remarkably good.
They have a single cycle double multiply per core, and the interpolators give you a heap of ability
The PIO can be awkward, but you can run a bunch of them at once. Going from MCU to MCU you don't even need to involve the CPU cores, PIO to PIO Comms via pins
You are obviously not going to get big TOPS from it because a Trillion is a ridiculous amount anyway. But never underestimate the power of controlling the whole pipeline.
Ultimately none of the other things I'm doing with MCUs are practical, why would this to be any different.
As someone who has done a reasonable amount with PIO, I do not think this is possible. However, that should not stop you. If you get it to work, please ping me.
I think they got something like 10 _seconds per token_ (not tokens per second).
EDIT: it was 25GB of ram and up to 20 seconds per token! https://github.com/JustVugg/colibri
As someone with a healthy amount of RAM, but just a 16GB GPU, I am wondering what kind of work I could queue up for overnight runs. I thought the best models were fully out of reach, but the 128GB CPU only test had a 1.8 tokens/second. While not speedy, you could probably do something with that given extensive coffee breaks. This speed simulator[0] demos what it looks like.
I could see where you could set up some coding task and let it churn all night.
This microcontroller is a beast, currently using it to do dev work on a pi4.
Having two usb ports with one otg lets you do some neat things that would cost $100+ otherwise
This works today with tinyusb and pico-pio-usb, but I'm also playing with a Rust port which I'm hoping will have higher performance.
In the small parameter regime there is no room for benchmaxxing, so instead of leaderboards becoming useless, their utility was merely reduced to establishing ever smaller models with similar performance on the benchmarks, forcing compression or redundancy to be recognized and eliminated at the modeling level.
Flash wears out from writing and the answer is in the tens to hundreds of thousands of writes per cell.
It's write and/or erase cycles that cause wear.
https://milkv.io
The duo has up to 256MB of memory, and a 1TOPS@INT8 TPU. They run Linux and are $5. I bought 5!